Special upload/download mode for large transfers

Large transfers, both uploads and downloads, will take a lot of time regardless of bandwidth. Tools like rclone can do many parallel transfers, so for large transfers tail latency cancellation matters little.

Would it make sense to have a special mode in libuplink that disables tail latency uploads/downloads, essentially making it only send 80 chunks at upload (and not 110), or 29 chunks at download (and not 39)? Or maybe a tunable “tail latency cancellation strength”, which would go from 0% (meaning: 80/29) to 100% (meaning: 110/39)? Then a smart client that rclone probably is could adjust the parameter to start from 0% and end with 100% to leverage tail latency cancellation for the last bits of data.

This kind of setting would reduce the client’s overhead of partial transfers from/to slower nodes, making the transfer a bit faster. Also could reduce the impact of tail latency cancellation on storage nodes.

2 Likes