I have just finished a test upload of a single file sized 1 GB.
Altogether it took 1 hr 34 m to upload.
Main reason for that long duration that 2 times the upload restarted after it terminated with the error message that the number of successful puts were less than the threshold.
When it finally worked, the upload took around 50 minutes to complete. For 1 GB.
Altogether this is not really a good performance.
I do not think it would be possible to solve the problem with low upstream bandwidth, too many connections, each of them is too slow for storagenodes and many of them just drop slow connections, several reported as overloaded (when nodes rejecting requests, see Should I change max-concurrent-requests? >> Note: The implementation has changed. You shouldn't use this setting any longer. Leave it commented out) as result you experiencing the issues.
I would not expect a quick solution for this problem.
With low upstream bandwidth using a Gateway should work better.
Perhaps only rclone could help a little bit - it has an ability to retry upload automatically.
I am just trying to see this from a customers view. And if I try to upload a 1 GB file and the connection resets 3 times and restarts from zero always so that it takes 90 minutes to upload then I am not sure if I would use this.
I believe most users can understand connection issues and dropping connections, but what makes this so bad is, that the upload always starts from the beginning. This would be the main thing to fix in my view. Because this gives the user a very very bad impression.
Maybe we should make it as a feature request.
The free FileZilla should be able to retry upload from the point where it is stopped last time.
This could help to finish uploads even on low upstream bandwidth with a native connector.
Yes right. As a Storj customer I would go mad at some point if my transmissions (up- and downloads) always fail and start from the scratch.
Edit: I don’t know how other integrations handle these cases, of course this would be true for any other integration as well.
Any integration based on libuplink would have the same behavior. The multipart upload is implemented in Gateways, libuplink should have them too as soon as possible.
We just still have a lot of work here and there.
Another approach would be to implement some form of congestion avoidance in the node selection process. There have been some suggestions in the past around that topic.
The interesting part is that this would solve the problem for customers, but is also a feature often requested by SNOs as it would prevent their nodes from being overloaded.
Multipart uploads will help, but especially if traffic levels increase, it is merely a retry and doesn’t prevent the problem from occurring to begin with. Smarter node selection might.
Mutually linking:
Another mutual link to the use case cinema: The digital cinema as potential Tardigrade use case? - #103 by jammerdan
Because if Storj DCS was to be used in such a manner of transferring files from a source to a target, it has to just work and not losing connections and start over and over again attempting to downlad a file.
What I don’t underst and is that we are speaking of a spcific piease in such a case.
So the uploader/downloder could send a request to exchange this node into another before fully giving up. And then let’s say the satellite passes several more addresses the uplink can try to upload/download from these instead of terminating the download completely.
Well it could be several pieces, but regardless what you suggest could help. However it still feels a bit like just retrying until it works and those piece transfers still need to start over. Ideally the satellite would select nodes that wouldn’t reject pieces in the first place. This would be a whole lot easier if the satellite had more immediate feedback of success for each transfer. But I’m not sure it works like that. If there is a message being sent back to the satellite from the uplink, that message could include failed transfers (so not slow ones, only failed ones) so that the satellite can remove that node from selection cache and try it again when the new cache gets built. That would be a relatively simple thing to implement I think. It still means that you have the risk of this problem popping up when the node selection cache just got refreshed, but hopefully by then most troublesome nodes have gotten some time to deal with the remaining transfers and will be able to process new ones.
And yeah, as a fallback it would be great if the uplink could ask the satellite for a few more nodes for the same upload. But I don’t think that should be the primary strategy as it would require the node to wait for a timeout from all remaining nodes, then ask the satellite for more and start the transfer of the missing pieces over. This will make that piece transfer still take twice as long, if not more.
I really don’t mind what the solution could be. But it is extremely annoying.
I am doing another test here. Since 8 am I am trying to upload a 14 GB file via Filezilla onto Storj DCS.
There are several re-attempts recorded in the logs with error message such as upload failed: stream error: ecclient error: successful puts (79) less than success threshold (80) and after that it the upload restarts from zero.
It just happend again when the file total upload was at 99%. Now I am back at 1,5%. After more than 10 hours.
This is inacceptable.
Please provide a debug log and ill sent it off to FileZilla right away. In the near term I could walk you through rclone which is very fast and reliable.
Thank you,
-Dominick
I’ll leave it running for a while and see if or when the upload succeeds.
But it is just testing so I am not looking for an alternative.
What I am saying is, if Storj advertises Filezilla as successful integration then it should work.
I mean this is one of the fundamental core functionalities: Up- and downloading of a file.
And especially for large files it needs to be resilient as I can’t imagine anything more annoying than a restart just seconds before the upload would have been finished.
Do you have any indication how often this happens? This seems like a pretty bad experience especially for big files. And while the ability to retry just a single segment or request more nodes for upload would cause a hiccup in speed, it’s a hell of a lot better than having to start a 14GB file all over again. Will multipart support fix this automatically?
I have let it run all the night, and still the file has not been uploaded.
It is still trying and trying, currently at 34%.
Maybe additional information first:
Here is the log but i have it still running:
|08:30:55|Status:|Connecting to us2.tardigrade.io...|
|08:30:55|Status:|Retrieving directory listing of /...|
|08:30:55|Command:|list /|
|08:30:57|Error:|Failed to retrieve directory listing|
|08:30:57|Status:|Disconnected from server|
|08:30:57|Status:|Connecting to us2.tardigrade.io...|
|08:30:57|Status:|Retrieving directory listing of /...|
|08:30:57|Command:|list /|
|08:30:58|Error:|Failed to retrieve directory listing|
|08:31:00|Status:|Connecting to us2.tardigrade.io...|
|08:31:00|Status:|Retrieving directory listing of /...|
|08:31:00|Command:|list /|
|08:31:01|Error:|Failed to retrieve directory listing|
|08:31:01|Status:|Disconnected from server|
|08:31:01|Status:|Connecting to us2.tardigrade.io...|
|08:31:01|Status:|Retrieving directory listing of /...|
|08:31:02|Status:|Directory listing of / successful|
|08:31:02|Status:|Retrieving directory listing of /...|
|08:31:02|Command:|list /|
|08:31:02|Error:|Failed to retrieve directory listing|
|08:31:02|Status:|Disconnected from server|
|08:31:02|Status:|Connecting to us2.tardigrade.io...|
|08:31:02|Status:|Retrieving directory listing of /...|
|08:31:02|Command:|list /|
|08:31:03|Error:|Failed to retrieve directory listing|
|08:31:04|Status:|Connecting to us2.tardigrade.io...|
|08:31:04|Status:|Retrieving directory listing of /fireplace...|
|08:31:05|Status:|Directory listing of /fireplace successful|
|08:31:26|Status:|Connecting to us2.tardigrade.io...|
|08:31:26|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|08:31:26|Command:|put PATHTO\Fireplace 10 hours full HD.mp4 /fireplace/Fireplace 10 hours full HD.mp4|
|09:56:44|Error:|upload failed: stream error: ecclient error: successful puts (79) less than success threshold (80)|
|09:56:44|Error:|File transfer failed after transferring 2.080.374.784 bytes in 5117 seconds|
|09:56:44|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|09:56:44|Status:|Retrieving directory listing of /fireplace...|
|09:56:44|Command:|list /fireplace|
|09:56:44|Command:|put PATHTO\Fireplace 10 hours full HD.mp4 /fireplace/Fireplace 10 hours full HD.mp4|
|18:06:02|Error:|File transfer failed after transferring 14.159.970.304 bytes in 29357 seconds|
|18:06:02|Status:|Connecting to us2.tardigrade.io...|
|18:06:03|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|18:06:03|Status:|Retrieving directory listing of /fireplace...|
|18:06:03|Command:|list /fireplace|
|18:06:04|Command:|put PATHTO\Fireplace 10 hours full HD.mp4 /fireplace/Fireplace 10 hours full HD.mp4|
|18:21:32|Error:|File transfer failed after transferring 67.108.864 bytes in 927 seconds|
|18:21:32|Status:|Connecting to us2.tardigrade.io...|
|18:21:42|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|18:21:42|Status:|Retrieving directory listing of /fireplace...|
|18:21:42|Command:|list /fireplace|
|18:21:43|Command:|put PATHTO\Fireplace 10 hours full HD.mp4 /fireplace/Fireplace 10 hours full HD.mp4|
|18:35:07|Error:|File transfer failed after transferring 134.217.728 bytes in 804 seconds|
|18:35:07|Status:|Connecting to us2.tardigrade.io...|
|18:35:07|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|18:35:07|Status:|Retrieving directory listing of /fireplace...|
|18:35:07|Command:|list /fireplace|
|18:35:08|Command:|put PATHTO\Fireplace 10 hours full HD.mp4 /fireplace/Fireplace 10 hours full HD.mp4|
|21:22:39|Trace:|CStorjControlSocket::OnTerminate without error|
|21:22:39|Trace:|CControlSocket::DoClose(64)|
|21:22:39|Trace:|CControlSocket::ResetOperation(66)|
|21:22:39|Trace:|CStorjFileTransferOpData::Reset(66) in state 4|
|21:22:39|Error:|File transfer failed after transferring 5.368.709.120 bytes in 10051 seconds|
|21:22:39|Trace:|CFileZillaEnginePrivate::ResetOperation(66)|
|21:22:39|Trace:|CControlSocket::SendNextCommand()|
|21:22:39|Trace:|CStorjConnectOpData::Send() in state 0|
|21:22:39|Status:|Connecting to us2.tardigrade.io...|
|21:22:39|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:22:39|Response:|fzStorj started, protocol_version=3|
|21:22:39|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:22:39|Trace:|CControlSocket::SendNextCommand()|
|21:22:39|Trace:|CStorjConnectOpData::Send() in state 1|
|21:22:39|Command:|host us2.tardigrade.io:7777|
|21:22:39|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:22:39|Trace:|CControlSocket::SendNextCommand()|
|21:22:39|Trace:|CStorjConnectOpData::Send() in state 3|
|21:22:39|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:22:39|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:22:39|Trace:|CControlSocket::ResetOperation(0)|
|21:22:39|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:22:39|Trace:|CControlSocket::SendNextCommand()|
|21:22:39|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:22:39|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:22:39|Trace:|CStorjListOpData::Send() in state 0|
|21:22:39|Status:|Retrieving directory listing of /fireplace...|
|21:22:39|Trace:|CStorjListOpData::Send() in state 2|
|21:22:39|Command:|list /fireplace|
|21:22:40|Trace:|CStorjListOpData::ParseResponse() in state 2|
|21:22:40|Trace:|CControlSocket::ResetOperation(0)|
|21:22:40|Trace:|CStorjListOpData::Reset(0) in state 2|
|21:22:40|Trace:|CStorjFileTransferOpData::SubcommandResult(0) in state 0|
|21:22:40|Trace:|CControlSocket::SendNextCommand()|
|21:22:40|Trace:|CStorjFileTransferOpData::Send() in state 1|
|21:22:40|Trace:|CControlSocket::CheckOverwriteFile()|
|21:22:40|Trace:|CStorjFileTransferOpData::Send() in state 4|
|21:22:40|Command:|put PATHTO\Fireplace 10 hours full HD.mp4 /fireplace/Fireplace 10 hours full HD.mp4|
|21:57:23|Trace:|CStorjControlSocket::OnTerminate without error|
|21:57:23|Trace:|CControlSocket::DoClose(64)|
|21:57:23|Trace:|CControlSocket::ResetOperation(66)|
|21:57:23|Trace:|CStorjFileTransferOpData::Reset(66) in state 4|
|21:57:23|Error:|File transfer failed after transferring 1.342.177.280 bytes in 2082 seconds|
|21:57:23|Trace:|CFileZillaEnginePrivate::ResetOperation(66)|
|21:57:23|Trace:|CControlSocket::SendNextCommand()|
|21:57:23|Trace:|CStorjConnectOpData::Send() in state 0|
|21:57:23|Status:|Connecting to us2.tardigrade.io...|
|21:57:23|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:57:23|Response:|fzStorj started, protocol_version=3|
|21:57:23|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:57:23|Trace:|CControlSocket::SendNextCommand()|
|21:57:23|Trace:|CStorjConnectOpData::Send() in state 1|
|21:57:23|Command:|host us2.tardigrade.io:7777|
|21:57:23|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:57:23|Trace:|CControlSocket::SendNextCommand()|
|21:57:23|Trace:|CStorjConnectOpData::Send() in state 3|
|21:57:23|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:57:23|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:57:23|Trace:|CControlSocket::ResetOperation(0)|
|21:57:23|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:57:23|Trace:|CControlSocket::SendNextCommand()|
|21:57:23|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:57:23|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:57:23|Trace:|CStorjListOpData::Send() in state 0|
|21:57:23|Status:|Retrieving directory listing of /fireplace...|
|21:57:23|Trace:|CStorjListOpData::Send() in state 2|
|21:57:23|Command:|list /fireplace|
|21:57:33|Error:|Connection timed out after 10 seconds of inactivity|
|21:57:34|Trace:|CControlSocket::DoClose(2050)|
|21:57:34|Trace:|CControlSocket::ResetOperation(2114)|
|21:57:34|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:57:34|Trace:|CControlSocket::ResetOperation(2114)|
|21:57:34|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:57:34|Error:|File transfer failed|
|21:57:34|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:57:34|Trace:|CControlSocket::SendNextCommand()|
|21:57:34|Trace:|CStorjConnectOpData::Send() in state 0|
|21:57:34|Status:|Connecting to us2.tardigrade.io...|
|21:57:34|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:57:34|Response:|fzStorj started, protocol_version=3|
|21:57:34|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:57:34|Trace:|CControlSocket::SendNextCommand()|
|21:57:34|Trace:|CStorjConnectOpData::Send() in state 1|
|21:57:34|Command:|host us2.tardigrade.io:7777|
|21:57:34|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:57:34|Trace:|CControlSocket::SendNextCommand()|
|21:57:34|Trace:|CStorjConnectOpData::Send() in state 3|
|21:57:34|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:57:34|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:57:34|Trace:|CControlSocket::ResetOperation(0)|
|21:57:34|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:57:34|Trace:|CControlSocket::SendNextCommand()|
|21:57:34|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:57:34|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:57:34|Trace:|CStorjListOpData::Send() in state 0|
|21:57:34|Status:|Retrieving directory listing of /fireplace...|
|21:57:34|Trace:|CStorjListOpData::Send() in state 2|
|21:57:34|Command:|list /fireplace|
|21:57:44|Error:|Connection timed out after 10 seconds of inactivity|
|21:57:44|Trace:|CControlSocket::DoClose(2050)|
|21:57:44|Trace:|CControlSocket::ResetOperation(2114)|
|21:57:44|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:57:44|Trace:|CControlSocket::ResetOperation(2114)|
|21:57:44|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:57:44|Error:|File transfer failed|
|21:57:44|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:57:44|Trace:|CControlSocket::SendNextCommand()|
|21:57:44|Trace:|CStorjConnectOpData::Send() in state 0|
|21:57:44|Status:|Connecting to us2.tardigrade.io...|
|21:57:44|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:57:44|Response:|fzStorj started, protocol_version=3|
|21:57:44|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:57:44|Trace:|CControlSocket::SendNextCommand()|
|21:57:44|Trace:|CStorjConnectOpData::Send() in state 1|
|21:57:44|Command:|host us2.tardigrade.io:7777|
|21:57:44|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:57:44|Trace:|CControlSocket::SendNextCommand()|
|21:57:44|Trace:|CStorjConnectOpData::Send() in state 3|
|21:57:44|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:57:44|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:57:44|Trace:|CControlSocket::ResetOperation(0)|
|21:57:44|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:57:44|Trace:|CControlSocket::SendNextCommand()|
|21:57:44|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:57:44|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:57:44|Trace:|CStorjListOpData::Send() in state 0|
|21:57:44|Status:|Retrieving directory listing of /fireplace...|
|21:57:44|Trace:|CStorjListOpData::Send() in state 2|
|21:57:44|Command:|list /fireplace|
|21:57:55|Error:|Connection timed out after 10 seconds of inactivity|
|21:57:55|Trace:|CControlSocket::DoClose(2050)|
|21:57:55|Trace:|CControlSocket::ResetOperation(2114)|
|21:57:55|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:57:55|Trace:|CControlSocket::ResetOperation(2114)|
|21:57:55|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:57:55|Error:|File transfer failed|
|21:57:55|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:57:55|Trace:|CControlSocket::SendNextCommand()|
|21:57:55|Trace:|CStorjConnectOpData::Send() in state 0|
|21:57:55|Status:|Connecting to us2.tardigrade.io...|
|21:57:55|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:57:55|Response:|fzStorj started, protocol_version=3|
|21:57:55|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:57:55|Trace:|CControlSocket::SendNextCommand()|
|21:57:55|Trace:|CStorjConnectOpData::Send() in state 1|
|21:57:55|Command:|host us2.tardigrade.io:7777|
|21:57:55|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:57:55|Trace:|CControlSocket::SendNextCommand()|
|21:57:55|Trace:|CStorjConnectOpData::Send() in state 3|
|21:57:55|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:57:55|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:57:55|Trace:|CControlSocket::ResetOperation(0)|
|21:57:55|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:57:55|Trace:|CControlSocket::SendNextCommand()|
|21:57:55|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:57:55|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:57:55|Trace:|CStorjListOpData::Send() in state 0|
|21:57:55|Status:|Retrieving directory listing of /fireplace...|
|21:57:55|Trace:|CStorjListOpData::Send() in state 2|
|21:57:55|Command:|list /fireplace|
|21:58:05|Error:|Connection timed out after 10 seconds of inactivity|
|21:58:06|Trace:|CControlSocket::DoClose(2050)|
|21:58:06|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:06|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:58:06|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:06|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:58:06|Error:|File transfer failed|
|21:58:06|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:58:06|Trace:|CControlSocket::SendNextCommand()|
|21:58:06|Trace:|CStorjConnectOpData::Send() in state 0|
|21:58:06|Status:|Connecting to us2.tardigrade.io...|
|21:58:06|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:58:06|Response:|fzStorj started, protocol_version=3|
|21:58:06|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:58:06|Trace:|CControlSocket::SendNextCommand()|
|21:58:06|Trace:|CStorjConnectOpData::Send() in state 1|
|21:58:06|Command:|host us2.tardigrade.io:7777|
|21:58:06|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:58:06|Trace:|CControlSocket::SendNextCommand()|
|21:58:06|Trace:|CStorjConnectOpData::Send() in state 3|
|21:58:06|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:58:06|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:58:06|Trace:|CControlSocket::ResetOperation(0)|
|21:58:06|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:58:06|Trace:|CControlSocket::SendNextCommand()|
|21:58:06|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:58:06|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:58:06|Trace:|CStorjListOpData::Send() in state 0|
|21:58:06|Status:|Retrieving directory listing of /fireplace...|
|21:58:06|Trace:|CStorjListOpData::Send() in state 2|
|21:58:06|Command:|list /fireplace|
|21:58:16|Error:|Connection timed out after 10 seconds of inactivity|
|21:58:17|Trace:|CControlSocket::DoClose(2050)|
|21:58:17|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:17|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:58:17|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:17|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:58:17|Error:|File transfer failed|
|21:58:17|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:58:17|Trace:|CControlSocket::SendNextCommand()|
|21:58:17|Trace:|CStorjConnectOpData::Send() in state 0|
|21:58:17|Status:|Connecting to us2.tardigrade.io...|
|21:58:17|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:58:17|Response:|fzStorj started, protocol_version=3|
|21:58:17|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:58:17|Trace:|CControlSocket::SendNextCommand()|
|21:58:17|Trace:|CStorjConnectOpData::Send() in state 1|
|21:58:17|Command:|host us2.tardigrade.io:7777|
|21:58:17|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:58:17|Trace:|CControlSocket::SendNextCommand()|
|21:58:17|Trace:|CStorjConnectOpData::Send() in state 3|
|21:58:17|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:58:17|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:58:17|Trace:|CControlSocket::ResetOperation(0)|
|21:58:17|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:58:17|Trace:|CControlSocket::SendNextCommand()|
|21:58:17|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:58:17|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:58:17|Trace:|CStorjListOpData::Send() in state 0|
|21:58:17|Status:|Retrieving directory listing of /fireplace...|
|21:58:17|Trace:|CStorjListOpData::Send() in state 2|
|21:58:17|Command:|list /fireplace|
|21:58:27|Error:|Connection timed out after 10 seconds of inactivity|
|21:58:27|Trace:|CControlSocket::DoClose(2050)|
|21:58:27|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:27|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:58:27|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:27|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:58:27|Error:|File transfer failed|
|21:58:27|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:58:27|Trace:|CControlSocket::SendNextCommand()|
|21:58:27|Trace:|CStorjConnectOpData::Send() in state 0|
|21:58:27|Status:|Connecting to us2.tardigrade.io...|
|21:58:27|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:58:27|Response:|fzStorj started, protocol_version=3|
|21:58:27|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:58:27|Trace:|CControlSocket::SendNextCommand()|
|21:58:27|Trace:|CStorjConnectOpData::Send() in state 1|
|21:58:27|Command:|host us2.tardigrade.io:7777|
|21:58:27|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:58:27|Trace:|CControlSocket::SendNextCommand()|
|21:58:27|Trace:|CStorjConnectOpData::Send() in state 3|
|21:58:27|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:58:27|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:58:27|Trace:|CControlSocket::ResetOperation(0)|
|21:58:27|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:58:27|Trace:|CControlSocket::SendNextCommand()|
|21:58:27|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:58:27|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:58:27|Trace:|CStorjListOpData::Send() in state 0|
|21:58:27|Status:|Retrieving directory listing of /fireplace...|
|21:58:27|Trace:|CStorjListOpData::Send() in state 2|
|21:58:27|Command:|list /fireplace|
|21:58:38|Error:|Connection timed out after 10 seconds of inactivity|
|21:58:38|Trace:|CControlSocket::DoClose(2050)|
|21:58:38|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:38|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:58:38|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:38|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:58:38|Error:|File transfer failed|
|21:58:38|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:58:38|Trace:|CControlSocket::SendNextCommand()|
|21:58:38|Trace:|CStorjConnectOpData::Send() in state 0|
|21:58:38|Status:|Connecting to us2.tardigrade.io...|
|21:58:38|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:58:38|Response:|fzStorj started, protocol_version=3|
|21:58:38|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:58:38|Trace:|CControlSocket::SendNextCommand()|
|21:58:38|Trace:|CStorjConnectOpData::Send() in state 1|
|21:58:38|Command:|host us2.tardigrade.io:7777|
|21:58:38|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:58:38|Trace:|CControlSocket::SendNextCommand()|
|21:58:38|Trace:|CStorjConnectOpData::Send() in state 3|
|21:58:38|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:58:38|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:58:38|Trace:|CControlSocket::ResetOperation(0)|
|21:58:38|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:58:38|Trace:|CControlSocket::SendNextCommand()|
|21:58:38|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:58:38|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:58:38|Trace:|CStorjListOpData::Send() in state 0|
|21:58:38|Status:|Retrieving directory listing of /fireplace...|
|21:58:38|Trace:|CStorjListOpData::Send() in state 2|
|21:58:38|Command:|list /fireplace|
|21:58:48|Error:|Connection timed out after 10 seconds of inactivity|
|21:58:49|Trace:|CControlSocket::DoClose(2050)|
|21:58:49|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:49|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:58:49|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:49|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:58:49|Error:|File transfer failed|
|21:58:49|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:58:49|Trace:|CControlSocket::SendNextCommand()|
|21:58:49|Trace:|CStorjConnectOpData::Send() in state 0|
|21:58:49|Status:|Connecting to us2.tardigrade.io...|
|21:58:49|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:58:49|Response:|fzStorj started, protocol_version=3|
|21:58:49|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:58:49|Trace:|CControlSocket::SendNextCommand()|
|21:58:49|Trace:|CStorjConnectOpData::Send() in state 1|
|21:58:49|Command:|host us2.tardigrade.io:7777|
|21:58:49|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:58:49|Trace:|CControlSocket::SendNextCommand()|
|21:58:49|Trace:|CStorjConnectOpData::Send() in state 3|
|21:58:49|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:58:49|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:58:49|Trace:|CControlSocket::ResetOperation(0)|
|21:58:49|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:58:49|Trace:|CControlSocket::SendNextCommand()|
|21:58:49|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:58:49|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:58:49|Trace:|CStorjListOpData::Send() in state 0|
|21:58:49|Status:|Retrieving directory listing of /fireplace...|
|21:58:49|Trace:|CStorjListOpData::Send() in state 2|
|21:58:49|Command:|list /fireplace|
|21:58:59|Error:|Connection timed out after 10 seconds of inactivity|
|21:58:59|Trace:|CControlSocket::DoClose(2050)|
|21:58:59|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:59|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:58:59|Trace:|CControlSocket::ResetOperation(2114)|
|21:58:59|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:58:59|Error:|File transfer failed|
|21:58:59|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:59:00|Trace:|CControlSocket::SendNextCommand()|
|21:59:00|Trace:|CStorjConnectOpData::Send() in state 0|
|21:59:00|Status:|Connecting to us2.tardigrade.io...|
|21:59:00|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:59:00|Response:|fzStorj started, protocol_version=3|
|21:59:00|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:59:00|Trace:|CControlSocket::SendNextCommand()|
|21:59:00|Trace:|CStorjConnectOpData::Send() in state 1|
|21:59:00|Command:|host us2.tardigrade.io:7777|
|21:59:00|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:59:00|Trace:|CControlSocket::SendNextCommand()|
|21:59:00|Trace:|CStorjConnectOpData::Send() in state 3|
|21:59:00|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:59:00|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:59:00|Trace:|CControlSocket::ResetOperation(0)|
|21:59:00|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:59:00|Trace:|CControlSocket::SendNextCommand()|
|21:59:00|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:59:00|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:59:00|Trace:|CStorjListOpData::Send() in state 0|
|21:59:00|Status:|Retrieving directory listing of /fireplace...|
|21:59:00|Trace:|CStorjListOpData::Send() in state 2|
|21:59:00|Command:|list /fireplace|
|21:59:10|Error:|Connection timed out after 10 seconds of inactivity|
|21:59:10|Trace:|CControlSocket::DoClose(2050)|
|21:59:10|Trace:|CControlSocket::ResetOperation(2114)|
|21:59:10|Trace:|CStorjListOpData::Reset(2114) in state 2|
|21:59:10|Trace:|CControlSocket::ResetOperation(2114)|
|21:59:10|Trace:|CStorjFileTransferOpData::Reset(2114) in state 0|
|21:59:10|Error:|File transfer failed|
|21:59:10|Trace:|CFileZillaEnginePrivate::ResetOperation(2114)|
|21:59:10|Trace:|CControlSocket::SendNextCommand()|
|21:59:10|Trace:|CStorjConnectOpData::Send() in state 0|
|21:59:10|Status:|Connecting to us2.tardigrade.io...|
|21:59:10|Trace:|Going to execute PATHTO\FileZillaPortable\App\FileZilla\fzstorj.exe|
|21:59:10|Response:|fzStorj started, protocol_version=3|
|21:59:10|Trace:|CStorjConnectOpData::ParseResponse() in state 0|
|21:59:10|Trace:|CControlSocket::SendNextCommand()|
|21:59:10|Trace:|CStorjConnectOpData::Send() in state 1|
|21:59:10|Command:|host us2.tardigrade.io:7777|
|21:59:10|Trace:|CStorjConnectOpData::ParseResponse() in state 1|
|21:59:10|Trace:|CControlSocket::SendNextCommand()|
|21:59:10|Trace:|CStorjConnectOpData::Send() in state 3|
|21:59:10|Command:|pass *****************************************************************************************************************************************************************************************************************************************************************************************************************|
|21:59:10|Trace:|CStorjConnectOpData::ParseResponse() in state 3|
|21:59:10|Trace:|CControlSocket::ResetOperation(0)|
|21:59:10|Trace:|CStorjConnectOpData::Reset(0) in state 3|
|21:59:10|Trace:|CControlSocket::SendNextCommand()|
|21:59:10|Trace:|CStorjFileTransferOpData::Send() in state 0|
|21:59:10|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|21:59:10|Trace:|CStorjListOpData::Send() in state 0|
|21:59:10|Status:|Retrieving directory listing of /fireplace...|
|21:59:10|Trace:|CStorjListOpData::Send() in state 2|
|21:59:10|Command:|list /fireplace|
|21:59:18|Trace:|CStorjListOpData::ParseResponse() in state 2|
|21:59:18|Trace:|CControlSocket::ResetOperation(0)|
|21:59:18|Trace:|CStorjListOpData::Reset(0) in state 2|
|21:59:18|Trace:|CStorjFileTransferOpData::SubcommandResult(0) in state 0|
|21:59:18|Trace:|CControlSocket::SendNextCommand()|
|21:59:18|Trace:|CStorjFileTransferOpData::Send() in state 1|
|21:59:18|Trace:|CControlSocket::CheckOverwriteFile()|
|21:59:18|Trace:|CStorjFileTransferOpData::Send() in state 4|
continued in part 2 below
Part 2 (Too many characterss to post in one post):
|21:59:18|Command:|put PATHTO\Fireplace 10 hours full HD.mp4 /fireplace/Fireplace 10 hours full HD.mp4|
|00:13:59|Error:|upload failed: stream error: ecclient error: successful puts (77) less than success threshold (80)|
|00:13:59|Trace:|CStorjFileTransferOpData::ParseResponse() in state 4|
|00:13:59|Trace:|CControlSocket::ResetOperation(2)|
|00:13:59|Trace:|CStorjFileTransferOpData::Reset(2) in state 4|
|00:13:59|Error:|File transfer failed after transferring 2.550.136.832 bytes in 8081 seconds|
|00:13:59|Trace:|CFileZillaEnginePrivate::ResetOperation(2)|
|00:13:59|Trace:|storj.io/uplink/private/ecclient.(*ecClient).Put:173|
|00:13:59|Trace:|storj.io/uplink/private/ecclient.(*ecClient).PutSingleResult:180|
|00:13:59|Trace:|storj.io/uplink/private/storage/streams.(*Store).Put:246|
|00:13:59|Trace:|storj.io/uplink/private/stream.NewUpload.func1:47|
|00:13:59|Trace:|golang.org/x/sync/errgroup.(*Group).Go.func1:57|
|00:13:59|Trace:|CControlSocket::SendNextCommand()|
|00:13:59|Trace:|CStorjFileTransferOpData::Send() in state 0|
|00:13:59|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|00:13:59|Trace:|CStorjListOpData::Send() in state 0|
|00:13:59|Status:|Retrieving directory listing of /fireplace...|
|00:13:59|Trace:|CStorjListOpData::Send() in state 2|
|00:13:59|Command:|list /fireplace|
|00:14:00|Trace:|CStorjListOpData::ParseResponse() in state 2|
|00:14:00|Trace:|CControlSocket::ResetOperation(0)|
|00:14:00|Trace:|CStorjListOpData::Reset(0) in state 2|
|00:14:00|Trace:|CStorjFileTransferOpData::SubcommandResult(0) in state 0|
|00:14:00|Trace:|CControlSocket::SendNextCommand()|
|00:14:00|Trace:|CStorjFileTransferOpData::Send() in state 1|
|00:14:00|Trace:|CControlSocket::CheckOverwriteFile()|
|00:14:00|Trace:|CStorjFileTransferOpData::Send() in state 4|
|00:14:00|Command:|put PATHTO\Fireplace 10 hours full HD.mp4 /fireplace/Fireplace 10 hours full HD.mp4|
|04:06:17|Error:|upload failed: stream error: ecclient error: successful puts (2) less than or equal to repair threshold (35)|
|04:06:17|Trace:|CStorjFileTransferOpData::ParseResponse() in state 4|
|04:06:17|Trace:|CControlSocket::ResetOperation(2)|
|04:06:17|Trace:|CStorjFileTransferOpData::Reset(2) in state 4|
|04:06:17|Error:|File transfer failed after transferring 6.509.559.808 bytes in 13936 seconds|
|04:06:17|Trace:|CFileZillaEnginePrivate::ResetOperation(2)|
|04:06:17|Trace:|storj.io/uplink/private/ecclient.(*ecClient).Put:169|
|04:06:17|Trace:|storj.io/uplink/private/ecclient.(*ecClient).PutSingleResult:180|
|04:06:17|Trace:|storj.io/uplink/private/storage/streams.(*Store).Put:246|
|04:06:17|Trace:|storj.io/uplink/private/stream.NewUpload.func1:47|
|04:06:17|Trace:|golang.org/x/sync/errgroup.(*Group).Go.func1:57|
|04:06:17|Trace:|CControlSocket::SendNextCommand()|
|04:06:17|Trace:|CStorjFileTransferOpData::Send() in state 0|
|04:06:17|Status:|Starting upload of PATHTO\Fireplace 10 hours full HD.mp4|
|04:06:17|Trace:|CStorjListOpData::Send() in state 0|
|04:06:17|Status:|Retrieving directory listing of /fireplace...|
|04:06:17|Trace:|CStorjListOpData::Send() in state 2|
|04:06:17|Command:|list /fireplace|
|04:06:17|Trace:|CStorjListOpData::ParseResponse() in state 2|
|04:06:17|Trace:|CControlSocket::ResetOperation(0)|
|04:06:17|Trace:|CStorjListOpData::Reset(0) in state 2|
|04:06:17|Trace:|CStorjFileTransferOpData::SubcommandResult(0) in state 0|
|04:06:17|Trace:|CControlSocket::SendNextCommand()|
|04:06:17|Trace:|CStorjFileTransferOpData::Send() in state 1|
|04:06:17|Trace:|CControlSocket::CheckOverwriteFile()|
|04:06:17|Trace:|CStorjFileTransferOpData::Send() in state 4|
|04:06:17|Command:|put PATHTO\Fireplace 10 hours full HD.mp4 /fireplace/Fireplace 10 hours full HD.mp4|How is your device with FileZilla connected to the network?
Is it WiFi or wire?
It’s a wired connection.
This is interesting. I’ve just uploaded about 200GB in several files onto the US2 test server using the tardigrade integration on FileZilla Pro. Worked awesome (obviously a bit “start-stop” because of the encryption and redundant shards).
Downloads were also fine.
Bandwidth was saturated in both directions (1Gbps symmetrical) but only with multiple concurrent transfer.
From my experience, it worked absolutely as advertised 
I noted a few failed transfers but the multipart upload must be working, because none of the files had to restart from scratch.