Rclone failed to copy error

I have a 76MB file that I can’t seem to copy to storj using rclone. I can copy smaller files just fine but this one file keeps failing with this error:

2022/12/21 02:11:50 Failed to copy: uplink: stream: ecclient: successful puts (25) less than or equal to repair threshold (35)

The rclone copy works to numerous other supported cloud storage platforms, just not storj. The number of puts (25 in the above attempt) changes from run to run. Is something going on with the storj infrastructure at the moment? I’ve never had any problems copying files (even larger ones) in the past.

Hello @Frank ,
Welcome to the forum!

This error means that connections were dropped during upload either because of timeouts (your router dropped connections) or because your upstream bandwidth is too low (less than 20 Mbit) and nodes dropping slow connections to serve more fast ones.
It also possible that you have a restrictive firewall, which doesn’t allow to connect to hundreds nodes across the globe.

In case if you have a restrictive firewall, you need either disable all outgoing rules (not incoming!) or add an additional rule to allow connection from your host to any host and any port in the internet.

If your router is not capable to handle hundreds parallel transfers, you can try to reduce number of parallel transfers with --transfers option (it’s 4 by default) or use a wired connection or reboot your router (but likely it will be overloaded again after a while).
In case of low upstream bandwidth or router problems you can use an S3-compatible integration instead of native - it opens much less connections and would work better in this case.

Thank you very much for the response. My upstream bandwidth is 1000Mbit and I have been doing a much larger backup copy to storj once a week for a little over a month now and never had a single error before now. Pretty much all of those backup files were larger than this one I am getting errors with too. If it was my bandwidth or the firewall one would imagine I would have seen issues before now (especially on the much larger files).

If no one else is seeing any problems I’ll chalk this one up to bad luck and monitor future uploads to see if it happens again. Thanks again!

In that case I would try to reboot the router, perhaps it collected errors too much and now drops connections.
What is your parallelism, chunk size and number of transfers?

See Hotrodding Decentralized Storage

By the way, what is satellite?

I am pretty sure the router was not the problem as the copy worked after a few hours without rebooting. I’m using all the defaults currently, I haven’t needed to tweak anything but if I get more errors in the future I can look into that. Thanks for the link.

I assume us1 but I’m not positive. I know I initially set it up for the US region. I just run this command:

rclone copy backup.tar.gz storj:backup

As stated previously that has been working without a single failure until this post. It failed numerous times for a few hours straight and then finally worked (with no changes on my end). I monitor resources on the router, firewall, my bandwidth from ISP and I saw no anomalies at all, everything looked normal except that the copy was failing and then it wasn’t.

I do the rclone copy to storj but I also do one to other cloud storage platforms (multiple copies in many locations just to be safe) and none of those rclone copies failed, only the storj one. That is why I assumed something might be going on with the infrastructure. Regardless it seems to have fixed itself for now so I don’t want to take up any more of your time but I greatly appreciate all of the suggestions provided.

Thank you for details! I’ll pass your feedback to the team.

I want to make sure - you used the native integration, didn’t you?