It has nothing common with an S3 implementation. ANY link will be a one thread download, because your browser is capable only one thread download any time, thus the reason why the downloaders are exists. Try it with GetRight (or any other normal downloader, which could download regions of the big file simultaneously, unlike any browser), you will be surprised.
1 Like
This thing still exists? Wow, I used it >20 years ago. Havenât heard the name for a decade at least.
1 Like
i liked FlashGet more.
saying that, i also got full 1Gbit/s download speed with that link with FireFox browser.
even now as we speak, must be something on your site
1 Like
Yeah, letâs be realistic, this is a pretty big limitation for data distribution in browsers. Download managers have gone out of fashion for over a decade now, barely anyone uses them anymore. The gateway could easily be tuned to support this by just doing the downloading of different segments simultaneously on that end.
That said, perhaps actual current customers havenât yet requested this feature enough to prioritize this. But a site like nexus mods would almost certainly not be satisfied with the current state of single thread performance.
1 Like
Actually not so easy. You may try it with a GatewayST with actual Storj network or GetwayMT in the GitHub - storj/up: Docker-compose files for running full Storj network locally.
You cannot tune the concurrency on the server side, because itâs basically unlimited. It should be tuned on a client side unfortunately.
Maybe only tune a linksharing service somehow, since itâs a client for the GatewayMT before the browser 
When I tried the last time I wasnât succeed, nothing helps on the server side, until one of the customers didnât find a working solution for their site - they wrote a JS code on their site to request a multipart downloads in several threads (but here the browser has a limit). So, I wonder is it possible to implement this on a linksharing service too?
1 Like
Perhaps I shouldnât have implied it was easy. But I donât think the single thread transfer from the gateway to the browser is the issue, as Iâm sure the Gateway MT is hosted on infra that can easily support 1gbit single transfers (and probably much more). Most likely the issue is that the gateway currently functions as just a transparent request translator, doing nothing more than replicating the number of threads requested by the client. But why wouldnât it be possible to adjust that behavior in such a way that the gateway uses multiple threads to pull data quicker from the storage nodes and then send it out over a single thread? It would probably require code changes and perhaps more resources to hold more segments (in memory most likely or fast storage) while reconstructing them into a single thread, but if as you say customers have already bumped into this limitation, it might be worth it.
Yeah, I wasnât saying that. I agreed with you that single threaded S3 should be able to saturate a 1gbit connection. However, I think itâs way more likely for the bottleneck to be single threaded fetching of segments by the gateway from the storj network, than the connection between the gateway and the client being the bottleneck.
1 Like
Itâs not the gateway > client speed, or the gateway > nodes speed. Itâs the nodes > gateway speed.
A clientâs download speed = aggregate upload speed of nodes. The vast majority of nodes are being run on residential connections (which are asymmetric, ie more down than up). Unless I missed something in math class, it should take 100 nodes of 10Mbps up to saturate a clientâs 1Gbps download.
or 3 like me with 300Mbps,
srsly, who got 10Mbp Up those days. give them at least a credit of a 50Mbps! lol
Most of cable internet customers in the US. Over a hundred million people. DOCSIS is still developing and now shifting to higher upstreams, but most people have 20Mbps upstream, and if you really pay a lot â 30. Comcast is deploying next gen networking in waves, but even then you will only get 200Mbps upstream on the most expensive plan.
I get 1200Mbps/25Mbps connection for $60/month (this includes unlimited traffic, on a promo deal for 2 years. I cancel every 2 years and create a new account to get another 2 promo years. Itâs ridiculously dumb). Yes, some places are lucky to get google or AT&T fiber, but not me. Only one of my nodes is on a gigabit symmetric fiber connection. The rest have minimum required upstream.
1 Like
yea but USA is the oldest internet infrastructure, they invented it!
Fiber from AT&T not widely accessible?
Or Verizon - they got as cheap fiber with huuuge upload, as mine here in Europe.
Itâs spotty. There are plenty of providers, they focus on new markets, and I live is a relatively old neighborhood, and I guess they donât ancitipate many customers or itâs really hard to get all the permits for infrastructure. I have a choice of symmetric 75Mbps DSL or 1200/25 asymmetric cable⌠Maybe I shall get both 
1 Like
Gateway in turn downloads from nodes. If the gateway gets the data slow, it canât send it out fast either.
That said, there is long tail cancellation and many faster nodes than 10-25mbps. Iâm pretty sure most downloads are served by 29 nodes with 50+mbps connections. But there is also communication with the satellites causing time to first byte for each segment to have a significant impact. If the gateway fetched multiple segments simultaneously, it would easily be able to saturate a 1gbit connection.
3 Likes
I wasnât under any wrong impression. My comment said exactly what @BrightSilence said: âtâs not the gateway > client speed, or the gateway > nodes speed. Itâs the nodes > gateway speed.â
To be clear: A client downloading from the gateway depends on the nodes supplying data to the gateway, which is then passed on to the client. I doubt the gateways are hosted on DSL connections, so there isnât any point in arguing about the gatewayâs upload.
3 Likes
You and @BrightSilence are correct, the download speed of the one thread limited by the speed of nodes (their upstream), however, these pieces downloaded in parallel, so even if each of US nodes has
the total speed will be higher than 20Mbps, but of course itâs hard to reach 1Gbps if use only one thread from the Gateway in a such case. However, if you would downloads several segments of that file in parallel, the total speed can easily saturate 1Gbps, because each segment is stored on their own nodes, which likely not intersect with other nodes stores other segments of the same file.
1 Like
Is there any tuning that can be done on the gateway to pull pieces faster than the client can request them and cache them (assuming that the client is actually trying to download a full file ofcourse)? That should (in theory) saturate the clientâs download even if the client doesnât use parallelism on his/her/its side.
If the nodes are fast enough to saturate nodes > gateway, how can the bottleneck be (and I quote) âsomewhere between gateway and nodeâ?
The bottleneck is the client not asking for parallel downloads when downloading over the gateway. If the gateway anticipates that the client is trying to download the entire file, it can pull that file from the nodes fast (in parallel) and stream it to the client slower (since itâs the client asking for one piece of the file at a time). The nodes send data to the gateway as fast as they can, the gateway sends data to the client as fast as the client is requesting it.
1 Like
We do what user @Mitsos said above (I wonder who he is?): Pull the entire file from the nodes, cache it on the gateway, stream it to the client as fast as the client is requesting it.
No idea, Iâm just an SNO.