Congestion Collapse?

Developer Discussions DCS
1

Hi, I am testing out tardigrade, trying to upload a 3GB zip file. However, it keeps going through waves of knocking out my wifi, and then continuing to upload, but this is very slow so I have cancelled it. Is there a bug, or some configuation issue on my end?

And as a side question… How can I delete a credit card?

1 Like
2

Regarding the credit card issue, you cannot delete it unless you add another one first. You cannot have an open account without a default payment method. If you were planning to switch to STORJ token instead, please first deposit $50 worth of STORJ and then make a support request at support.tardigrade.io to request your credit card to be removed.

3

The uplink creates a lot of connections to the nodes across the globe. Seems your router is unable to handle that via WiFi. Please, try to use a wired connection.

I can guess but do you use the Apple router?

4

No, this was using a TP-Link Archer A9. I’ll try a wired connection and different router when I get a chance, thanks!

5

There should be a way to reduce the number of connection etc, since some people have bad routers, As it is right now they are unable to use Storj (but most likely would be able to use S3 and similar).

1 Like
6

You can create or vote for the idea here: https://ideas.storj.io

However, reducing number of connections will reduce the speed. This is trade off for the decentralized cloud

1 Like
7

There should be a setting for it somewhere. Crashing a cheap router means the user is unable to use the service, which means that if I am using Storj in a way that results in regular users accessing it, some of my users would be unable to access the files, which would make using Storj for storage/cdn a bit pointless.
After all, Amazon most likely works with the same router.

I think I ran into a similar problem before and had to use other methods to transfer the file, because Storj was very fast, but also didn’t work because the network card on a router crashed.

3 Likes
8

Sorry for my late reply here!

So, I also have a tp-link router and it also kills my WiFi connection. The direct problem is that there are too many concurrent connections, but the reason that is a problem is because the default TCP congestion controller prioritizes all connections equally. If you have 2 connections they both get 1/2 of the bandwidth. If you have 102 connections, they each get 1/102 of the bandwidth.

We use lots of connections, but we don’t want to use a disproportionate share of the bandwidth. This is something BitTorrent went through a decade ago, and we need to resolve too, probably the same way, using uTP and LEDBAT. Switching to uTP and LEDBAT are in the whitepaper(!) and on our roadmap, but are significant chunks of work and it may still take us some time to get there (if you want to help with this effort, let me know!)

Until we are able to automatically switch the congestion controller for everyone by using uTP, there are still a few things you can do.

  1. Many people have had success by adjusting the QoS settings in their router. The router can almost certainly handle the number of connections just fine but you want to make sure Uplink traffic isn’t taking up the majority of your bandwidth. I was recently surprised to find out how many of our employees have already done this, when I was confused why more people weren’t having this problem.
  2. If you use Linux (and maybe OS X might also work), are comfortable with a small amount of development and compiling kernel modules, you can configure your sockets to use LEDBAT today! First, make sure your kernel has a LEDBAT TCP congestion controller module loaded. This PR brings this repo up to date for recent kernels: https://github.com/silviov/TCP-LEDBAT/pull/11. OS X might have LEDBAT built in (I think Apple Updater uses it). Then, cherry-pick this change (which will get merged soon once it makes it through review): https://review.dev.storj.io/c/storj/uplink/+/1813. This change allows you to set per-socket options for things using your Uplink. Once you have those two things in, you’ll need to set your socket options to use LEDBAT per socket, like this: https://github.com/jtolio/jam/blob/6052bfd5be86d6d07d0bea3fbc38f8bf7feeea4c/backends/storj/socket_linux.go. Then, if your binary does that, you’ll need to make sure it has cap_net_admin (you can run sudo setcap cap_net_admin=eip your-binary), and you’ll be off to the races.

Obviously #2 is… not very user friendly. We’re going to try and make this a bit more user friendly in advance of actually implementing uTP, but yeah, still a ways out.

So, I’d recommend #1 (I think the TP-Link Archer A9 might support OpenWRT, if the default interface doesn’t let you set QoS) for now, and then thank you for your patience with us while we make this better!

4 Likes
9

Oh oh oh, I just realized, if you have Windows 10 with WSL2, you probably also can use that LEDBAT Linux kernel module. Oh boy what a realization. I wonder if we can double down on that for the short term.

3 Likes