I tested FileZilla and I must say I am not impressed

So today I tested it and I must say I am not impressed.

I can connect succesfully to my bucket. But:

  • No files get shown
  • Uploads do not finish successfully
  • I keep getting errors like: finalizing upload failed: uplink: ecclient error: successful puts (70) less than success threshold (80)
  • failed uploads keep retrying

I still hope I am doing something wrong. However I can create remote folders , enter them and renaming them, uploads are starting, so I seem to be successfully connected.

Edit: Upload speed seems to be impressive. But it does not help when the file never finishes uploading.

1 Like

Please, make sure that you do not block any outgoing port on your firewall. The storagenodes could have any port, all data transfer is happening directly between you and storagenodes.

1 Like

Thanks. I turned off Firewall and it finishes uploads now.
Now I need to find out how to run this with Firewall turned on. I believe it should not require Firewall to be turned off.

1 Like

I think you just need to add your app to exeption rule for outbound

3 Likes

It’s a client behind a server. Firewall is on the server. I need to check if it would require to allow all traffic from and to the client, which would make the firewall obsolete.

1 Like

If GNU/Linux and iptables… something like:

iptables -A OUTPUT -p tcp -m state --state ESTABLISHED,RELATED -j ACCEPT

might work…

iptables is an ordered list of rules, so if you add this rule first… the restrictions added later will still work.

However, I’m no expert on creating iptables rules… it’s confusing enough that I would be suspicious of anyone claiming to be an expert.

2 Likes

Only one way around. You should allow any outgoing traffic. You do not need to allow any incoming traffic, since you do not offer a service.

1 Like

Does the error message ‘finalizing upload failed: uplink: ecclient error: successful puts (70) less than success threshold (80)’ technically really mean what it says? 70 out of 80 parts have been uploaded with success? So only 10 parts receive a firewall blockage?

1 Like

Yes. You just uploaded pieces only on nodes with allowed by your firewall ports. The satellite do not know that you want to have only nodes with a default port (I assume that you allowed only default port in your outbound rule).

Your uplink (libuplink in case of FileZilla) requests 110 nodes by default and starts uploading, however, only 70 from them have an allowed port. The minimum threshold is 80 by default

1 Like

I see. I need to check. Does a satellite get feedback which parts get through and which don’t? And could the satellite determine from such a feedback that certain ports don’t work?
Let’s say for example the satellite gets feedback 10 parts did not make it and all parts have ports other than the default ports and at the same time satellite gets feedback that all successful parts are on the default port, could the satellite make a selection of nodes for the remaining 10 pieces only to select only nodes with default ports?

1 Like

This is p2p network. You must allow any outgoing ports to make it work.
The satellite do not control nodes or network. It’s an address book, metadata storage, payment, audit and repair processor. There is no control plane.
You can modify your uplink with such logic of node selection.
You should configure it to request only nodes with needed port.
However, you will skip 40/110 of the network because of that and reduce reliability for your data - because all nodes will be with known port. If someone wants to interfere, it could DDOS those nodes and your data could become not retrievable.

2 Likes

After experimenting a bit, there may be another way around…

  1. Create an uplink user on the firewall platform.
  2. Port forward the uplink client to the firewall uplink user.
  3. Create an outgoing rule on the firewall platform for the uplink user to open all outgoing ports.

iptables -A OUTPUT -p tcp -m owner --uid-owner 123 -j ACCEPT

1 Like

Yes, that should also work at the expense of speed.

1 Like

Could you elaborate om step 2 @beast ?
Not sure how that be configured …

1 Like

There are numerous methods of forwarding traffic to another host on the network. The method employed will probably be based on the network traffic restrictions and whether or not the one forwarding the traffic has full control of all networking configuration of the LAN.

One method which works in nearly every case is a reverse ssh tunnel, since most networks allow ssh outgoing connections.

It’s nearly impossible to post a solution that works for everyone. A specific scenario will be required before a solution can be laid out beyond the general idea. I typically use ssh for everything, because it’s quick, easy, and can be implemented with public key authentication in a few simple steps.

1 Like

Thanks. Now I get it. I was under the assumption the client was Linux based on the firewall. I see now that was premature on my part.

So ie. SSH tunnel with a socks and configure it in the client, could be an example of actual application?

1 Like

‘Unfortunately’ it is a Windows server where the firewall sits on.

But isn’t it very common to have the firewall set to deny all outgoing and open only required ports specifically? At least a thread I found after quick googling is suggesting this: https://community.spiceworks.com/topic/218737-blocking-all-outgoing-ports-by-default

Also I would say with multiple clients a firewall running on a server is also common. In such a case an application aware firewall on the server does not help as the p2p application is running on the client only. There is no rule on the server firewall to tell it to allow this specific client application.

So at the end it means Tardigrade or whatever application it is built in that aims at security focused users with encryption and decentralization in place requires them to set their firewalls to allow unrestricted outgoing traffic from the network clients.
Client-server setups are very common in business environments, how would a business deal with such cases e.g. if they would like to use Filezilla on some clients to move some files onto Tardigrade instead of an FTP server?

1 Like

There are two paths protected by any firewall:

  1. Incoming traffic - it should block everything except services which should process requests from outside of your PC
  2. Outgoing traffic - it is open by default. However, your administrator could decide to block any outgoing traffic too, it’s not common, but possible. In such case you can connect only to services which is allowed by your administrator. This is dead end for any p2p application, which connects to random* ports to transfer data.

*By default any node in the p2p can use any port to listen requests, but tracker (the satellite in our case) do know what port each node uses and offer a list of nodes to the participant.

In your case you need to open an outgoing traffic, the incoming can still be blocked as it was to do not break your security.

The reasons for blocking outgoing traffic could be different - to reduce not related to your job traffic, to limit a volume of traffic generated by employee, to block p2p traffic, to protect the network from ban if one of the PC behind the firewall have a trojan which sends spam or malware.

The ftp service uses the same standard well known port 21, so it is easy to determine the ftp traffic. In case of p2p the outgoing port could be effectively random, there is no standard well-known port.

In case of Storj network there are nodes which uses the default port (it’s not the same as a standard, since the operator can change it and this is normal), so, you could try to use only that default port and add a logic to the uplink to force it to request from the satellite only nodes with a default port. But it will reduce the available network - you will have only 70/110 (the ratio could change, since there is no rule to keep the default port) of the network available.

As a second solution - you can use a gateway service on your server, which will listen only some standard port, but operates with a Tardigrade on you behalf. This will almost eliminate any security, since your encryption phrase and API key would be on that gateway and the traffic between you and gateway will be unencrypted.

The third option is to use a ssh tunnel as suggested by @beast. You can easy setup a ssh server and client on Windows:

1 Like

I haven’t tested downloading yesterday with Filezilla. Opening incoming ports is not required for downloading from Tardigrade with Filezilla ? I will try later.

So the satellite could offer only nodes with a specific port open, like the default port? But I also see the downside: If parts move between nodes due to repair, or simply SNO changes port, files could ‘vanish’ meaning they are available on the network, but client cannot access them as the open client port does not match the nodes open port, right? So even if I manage to upload a file via default port, there is no guarantee that I will be able to download again via default port only?

1 Like

The network is working in such way (very simplified!): client establish a connection and uses it to communicate with servers (satellites and nodes), the answers from servers are going through that connection back to the client (FileZilla). So yes, it should work (except if your firewall sniff the traffic too and blocks it by own rules based on content, even if this is useless, because it’s encrypted, but random encrypted sequence could have known sequence at random).

Exactly!
However, as I said before the satellite do not control which nodes you wants to accept. Your client must instruct satellite to do so. As a simplification: “hey satellite, give me 110 nodes!” sat: “Please, take them”, uplink: “Give me other nodes, not all have a default port and I need more!”, sat: “here they are”.
So, you can’t requests nodes with a specific characteristics right now, but you can request them over and over again, if they do not suitable for you.

1 Like