So today I tested it and I must say I am not impressed.
I can connect succesfully to my bucket. But:
No files get shown
Uploads do not finish successfully
I keep getting errors like: finalizing upload failed: uplink: ecclient error: successful puts (70) less than success threshold (80)
failed uploads keep retrying
I still hope I am doing something wrong. However I can create remote folders , enter them and renaming them, uploads are starting, so I seem to be successfully connected.
Edit: Upload speed seems to be impressive. But it does not help when the file never finishes uploading.
Please, make sure that you do not block any outgoing port on your firewall. The storagenodes could have any port, all data transfer is happening directly between you and storagenodes.
Thanks. I turned off Firewall and it finishes uploads now.
Now I need to find out how to run this with Firewall turned on. I believe it should not require Firewall to be turned off.
Itās a client behind a server. Firewall is on the server. I need to check if it would require to allow all traffic from and to the client, which would make the firewall obsolete.
Does the error message āfinalizing upload failed: uplink: ecclient error: successful puts (70) less than success threshold (80)ā technically really mean what it says? 70 out of 80 parts have been uploaded with success? So only 10 parts receive a firewall blockage?
Yes. You just uploaded pieces only on nodes with allowed by your firewall ports. The satellite do not know that you want to have only nodes with a default port (I assume that you allowed only default port in your outbound rule).
Your uplink (libuplink in case of FileZilla) requests 110 nodes by default and starts uploading, however, only 70 from them have an allowed port. The minimum threshold is 80 by default
I see. I need to check. Does a satellite get feedback which parts get through and which donāt? And could the satellite determine from such a feedback that certain ports donāt work?
Letās say for example the satellite gets feedback 10 parts did not make it and all parts have ports other than the default ports and at the same time satellite gets feedback that all successful parts are on the default port, could the satellite make a selection of nodes for the remaining 10 pieces only to select only nodes with default ports?
This is p2p network. You must allow any outgoing ports to make it work.
The satellite do not control nodes or network. Itās an address book, metadata storage, payment, audit and repair processor. There is no control plane.
You can modify your uplink with such logic of node selection.
You should configure it to request only nodes with needed port.
However, you will skip 40/110 of the network because of that and reduce reliability for your data - because all nodes will be with known port. If someone wants to interfere, it could DDOS those nodes and your data could become not retrievable.
There are numerous methods of forwarding traffic to another host on the network. The method employed will probably be based on the network traffic restrictions and whether or not the one forwarding the traffic has full control of all networking configuration of the LAN.
One method which works in nearly every case is a reverse ssh tunnel, since most networks allow ssh outgoing connections.
Itās nearly impossible to post a solution that works for everyone. A specific scenario will be required before a solution can be laid out beyond the general idea. I typically use ssh for everything, because itās quick, easy, and can be implemented with public key authentication in a few simple steps.
āUnfortunatelyā it is a Windows server where the firewall sits on.
But isnāt it very common to have the firewall set to deny all outgoing and open only required ports specifically? At least a thread I found after quick googling is suggesting this: [SOLVED] Blocking all outgoing ports by default? - Networking
Also I would say with multiple clients a firewall running on a server is also common. In such a case an application aware firewall on the server does not help as the p2p application is running on the client only. There is no rule on the server firewall to tell it to allow this specific client application.
So at the end it means Tardigrade or whatever application it is built in that aims at security focused users with encryption and decentralization in place requires them to set their firewalls to allow unrestricted outgoing traffic from the network clients.
Client-server setups are very common in business environments, how would a business deal with such cases e.g. if they would like to use Filezilla on some clients to move some files onto Tardigrade instead of an FTP server?
Incoming traffic - it should block everything except services which should process requests from outside of your PC
Outgoing traffic - it is open by default. However, your administrator could decide to block any outgoing traffic too, itās not common, but possible. In such case you can connect only to services which is allowed by your administrator. This is dead end for any p2p application, which connects to random* ports to transfer data.
*By default any node in the p2p can use any port to listen requests, but tracker (the satellite in our case) do know what port each node uses and offer a list of nodes to the participant.
In your case you need to open an outgoing traffic, the incoming can still be blocked as it was to do not break your security.
The reasons for blocking outgoing traffic could be different - to reduce not related to your job traffic, to limit a volume of traffic generated by employee, to block p2p traffic, to protect the network from ban if one of the PC behind the firewall have a trojan which sends spam or malware.
The ftp service uses the same standard well known port 21, so it is easy to determine the ftp traffic. In case of p2p the outgoing port could be effectively random, there is no standard well-known port.
In case of Storj network there are nodes which uses the default port (itās not the same as a standard, since the operator can change it and this is normal), so, you could try to use only that default port and add a logic to the uplink to force it to request from the satellite only nodes with a default port. But it will reduce the available network - you will have only 70/110 (the ratio could change, since there is no rule to keep the default port) of the network available.
As a second solution - you can use a gateway service on your server, which will listen only some standard port, but operates with a Tardigrade on you behalf. This will almost eliminate any security, since your encryption phrase and API key would be on that gateway and the traffic between you and gateway will be unencrypted.
The third option is to use a ssh tunnel as suggested by @anon27637763. You can easy setup a ssh server and client on Windows:
I havenāt tested downloading yesterday with Filezilla. Opening incoming ports is not required for downloading from Tardigrade with Filezilla ? I will try later.
So the satellite could offer only nodes with a specific port open, like the default port? But I also see the downside: If parts move between nodes due to repair, or simply SNO changes port, files could āvanishā meaning they are available on the network, but client cannot access them as the open client port does not match the nodes open port, right? So even if I manage to upload a file via default port, there is no guarantee that I will be able to download again via default port only?
The network is working in such way (very simplified!): client establish a connection and uses it to communicate with servers (satellites and nodes), the answers from servers are going through that connection back to the client (FileZilla). So yes, it should work (except if your firewall sniff the traffic too and blocks it by own rules based on content, even if this is useless, because itās encrypted, but random encrypted sequence could have known sequence at random).
Exactly!
However, as I said before the satellite do not control which nodes you wants to accept. Your client must instruct satellite to do so. As a simplification: āhey satellite, give me 110 nodes!ā sat: āPlease, take themā, uplink: āGive me other nodes, not all have a default port and I need more!ā, sat: āhere they areā.
So, you canāt requests nodes with a specific characteristics right now, but you can request them over and over again, if they do not suitable for you.