I tested the uplink recently with a 3.4GB file on europe.
In the WebUI overview it shows that I have used 52GB egress but I definitely don’t remember downloading the file over 15 times! I only downloaded it a few times to test the speed. I am now downloading the file again to see how the value shown changes and will report back.
EDIT: I downloaded it again and now my egress shows 62GB! So downloading my 3.4GB file did costs me 3 times the egress of the file’s size. How is that possible? I understand that for uploading a file you have to upload it more than once so it is stored securely but for downloading you only need everything 1 time, not 3 times.
The other question about egress is the billing: I have used 52GB of my available 25GB. How is such a case handled? In production I would probably get an invoice since I didn’t add my credit card yet? How is it handled now during the testing phase? Can I just use as much egress as I like?
The last point is a bug in the ui: In the bucket overview the used storage is shown correctly but the Egress used by the bucket stays at 0GB-
For downloading use of more egress traffic compared to file volume is expected too as you start downloading more pieces than you actually need to restore file - just to get better download speed. And cancel “excess” transfers after you get enough pieces from fastest nodes.
But it should be in ~10-20% range for such overhead (traffic is ~110-120% range compared to file volume).
As uplink should ask 35 pieces of each file and stops after getting first 29 pcs.
But not ~3 times more.
I guess it can be a bug in uplink software!
It should initiate download of 35 pieces (from up to 80 total stored on the network). But it looks like it star downloading all available pieses (up 80). So you generate up to 80/29 = 2.76x more egress traffic compared to file volume.
And it can easily explain why i see SO MANY download canceled errors on my storage nodes. If uplink is ask and start downloading ALL data pieces while actually need only need <40% of them - then it will be cancelling >60% of all downloads it starts producing abnormal high cancel ratio for storage nodes.
As i have thought.
I see worst download success ratio for europe-west-1.tardigrade.io (almost all downloads from my nodes for this satellite are canceled before completed, but still generate a lof of egress traffic)
And best results with us-central-1.tardigrade.io (almost all download are finished successfully)
Despite fact that europe-west is closest/fastest satellite for my location and us-central-1 is relatively slow.
@jocelyn sorry for pinging. I know I put a lot of questions into this thread and they might get answered eventually.
The only questions (which are not tech related) I’d like to have answered sooner are:
@kevink,
You have not actually used 52GB of egress. @Mad_Max is correct in that we generate orders for ~80 nodes although you will most likely not use all of them to download the file, and we use that number to determine a maximum possible usage for egress per download. Since that number is actually about 3 times the object, we actually multiply your BW limit by three when determining whether you have exceeded the limit or not. This is understandably quite confusing and we have some people working on it as we speak
I downloaded even more and now I see what happens if you run out of bandwith, the gateway shows lots of “Exceeded Usage Limit” errors and I can’t download anything anymore.