S3 sync does not work

It just won’t skip the files already present in my storj bucket. It copies them every time over the network.

I’ve used: aws s3 --endpoint=https://gateway.eu1.storjshare.io sync --size-only --delete /mnt/foo s3://bar/bar/

No matter if I use --size-only or --delete. I will transfer the file every time.

Yes, it’s not preserve the modification time, so the object will be always new.

I would recommend to use rclone sync instead. The rclone preserve the modification time unlike AWS CLI.
If you would choose the s3 protocol again, then you would need to use rclone --no-update-modtime sync.

Great, it is working! Thank you.
But I used S3 (again) over rclone, because it eats 1:1 data on upload.

Now unchanged files are skipped.

1 Like

This script synchronizes local directory with storj directory with uplink
It does not add, remove, or update any file in the local directory
It does not remove or update any file in the remote directory .
It just copies new files from local directory to remote storj directory
It just check filename and filepath, not file size or file date creation

python3 ./synchronize_storj.py --local_dir /dir6565/dir773 --remote_dir sj://bucket_845/dir747/dir334 --mode_test_or_real real --show_new_files_yes_or_not not

1 Like