Git over an rclone mount of a Storj bucket

Developer Discussions troubleshooting
1

I am learning about Storj, generally really like it. But running into a small problem.

I have an rclone mount via the Storj-S3 gateway and that is all working fine. But, when I do a git init in a directory it doesn’t work properly and I get an error:

2022/04/11 12:21:07 DEBUG : /Sandbox/tt/.git/HEAD.lock: Rename: newPath="/Sandbox/tt/.git/HEAD"
2022/04/11 12:21:07 ERROR : Sandbox/tt/.git/HEAD.lock: Dir.Rename error: Fs "S3 bucket <my bucket>" can't rename files (no server-side Move or Copy)
2022/04/11 12:21:07 ERROR : Sandbox/tt/.git/HEAD.lock: Dir.Rename error: Fs "S3 bucket <my bucket>" can't rename files (no server-side Move or Copy)
2022/04/11 12:21:07 ERROR : IO error: Fs "S3 bucket <my bucket>" can't rename files (no server-side Move or Copy)

So, I suppose this is happening as the “full API” isn’t implemented for the Storj gateway (?).

So, I tried it in the rclone native-Storj. The first time it didn’t work but is likely due to my previous issue. I just tried it again and it actually did work using the native-Storj backend for rclone, sigh.

So… given my connection (Verizon FIOS) doesn’t seem to be fast enough (?) for using the native backend with rclone, is there some way of fiddling to get the S3 gateway working for things like git?

tbh, not sure if this is a Storj question or rclone question and I hope I am interpreting things correctly.

I am using as the rclone parameters.

        --vfs-cache-mode=full \
        --dir-cache-time=30s \
        --vfs-write-back=2m \
        --cache-info-age=60m

thanks…

2

I know server side copy is a very recent addition to Storj. I think it’s only there since version 1.50.4 (not even sure if it’s complete yet, but there were a few code changes merged to support it in that version). So it might be they just haven’t gotten around to supporting it fully in the S3 compatible gateway yet.

2 Likes
3

That is correct @BrightSilence. Some of the plumbing is in place but nothing for edge services is complete or released. I’m not completely sure if it is working via uplink in the released code but you’re more than welcome to try. Below is an example that allows you to copy between buckets.

uplink cp “sj://$BUCKET/big-upload-testfile” “sj://$BUCKET/copied-big-upload-testfile”