Bandwidth utilization comparison thread

Node Operators FAQ
, , ,
946

you forget some important factors… like say huge data segments that are stored on the network, which will at times reach their repair triggers which might cause such swings, i would equate it to an avalanche… doesn’t take much to trigger one, and the build up can be very slow and incremental… or simply storj labs testing stuff, poking holes in test data to see what happens under certain conditions.

or something similar, i’m sure there are many variables we as SNO’s can’t really see, even tho we can see their effects.

1 Like
947

yep, I saw it too in yesterday’s data.

Node 1:

image
image343×649 33.1 KB

Node 2:

image
image339×652 30.4 KB

948

Heh, that’s why I kinda like the the breakdown by SAT to see if maybe it is Euro-N or Salt-lake being chatty:

Repair asia-east-1.tardigrade.io:7777 (121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6)
Y-axis from 0 to 662 MiB; cell = 66.2 MiB; 1 nodes
│                  i
│                  i
│                  i
│                  i
│                  i
│                  i  i     i
│                  i  ie i  i
│i     i  i        ie ie i  i
│i  i  i  i  i     ie ie ie ie
│i  i  ie i  i  i  ie ie ie ie
└──────────────────────────────
 01 02 03 04 05 06 07 08 09 10

Repair us-central-1.tardigrade.io:7777 (12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S)
Y-axis from 0 to 422 MiB; cell = 42.2 MiB; 1 nodes
│                        i
│                  i     i  i
│                  i  i  i  i
│                  i  i  ie i
│i                 i  i  ie i
│i     i  i        i  ie ie i
│i  i  i  i  i     i  ie ie i
│i  i  i  i  i     i  ie ie ie
│i  i  ie i  i  i  i  ie ie ie
│ie ie ie ie ie i  ie ie ie ie
└──────────────────────────────
 01 02 03 04 05 06 07 08 09 10

Repair europe-west-1.tardigrade.io:7777 (12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs)
Y-axis from 0 to 998 MiB; cell = 99.8 MiB; 1 nodes
│                  i
│                  i
│                  i
│                  i
│                  i
│                  i
│                  i  i     i
│i                 i  i     i
│i  i  i  i        i  ie i  i
│i  i  i  i  i  i  ie ie ie i
└──────────────────────────────
 01 02 03 04 05 06 07 08 09 10

Repair europe-north-1.tardigrade.io:7777 (12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB)
Y-axis from 0 to 18.8 GiB; cell = 1.88 GiB; 1 nodes
│                         e
│                         e
│                         e
│                         e
│                        ie
│                        ie
│                        ie
│                      e ie  e
│                  i  ie ie ie
│ie ie ie ie ie    ie ie ie ie
└──────────────────────────────
 01 02 03 04 05 06 07 08 09 10

Repair saltlake.tardigrade.io:7777 (1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE)
Y-axis from 0 to 3.62 GiB; cell = 371 MiB; 1 nodes
│                        i
│                        i
│                  i  i  i  i
│                  i  i  i  i
│i     i           i  i  i  i
│i  i  i           i  i  i  i
│i  i  i  i  i     i  i  i  i
│i  i  i  i  i  i  i  i  i  i
└────────────────────────────── * 2
 01 02 03 04 05 06 07 08 09 10

And so based on those, I’d kinda agree that they’re poking at the test data on Euro-N and Salt-Lake, but US-C has also been a tad up. I’d be some heavy conjecture to make an association its related to the election, but I’d be lying if I wouldn’t be giddy that something cool like scanned images of each ballot was uploaded to Tardigrade- that’d be some serious amount of data and use!

I am loving this massive influx of ingress this month though- the 9TB fiber node is nearing full finally.

949

i don’t really plot all that stuff… but yeah it’s nice with a little bit of ingress… sadly i’m still only getting 2/3 because of having a foreign node on my subnet… but i should have time to call my new ISP tomorrow, see if i can actually give me the kind of setup i want…
i do fear that i maybe have to get an enterprise connection tho… which usually is just more costs… so hopefully i can get away with a consumer one while still getting multiple “baremetal” or vlan ip’s
so thats going to be interesting… tho i suspect they will say … you want what!!!

okay next question… how many separate ip/24 subnets do you have access to…

ISP: Who is this…?!

1 Like
950

Admittedly my two nodes are spread over two different geographic points and with different access media (fiber vs DOCSIS/COAX) so IPs isn’t much of a question for me and technically I have access to another two /29 nets, yet, but that would start to get crowded on them.
As it is right now, the fiber node is on an IP with a few other low utilization servers, which is part of a /29 block- the Coax node is residential and has one Dynamic IP that’s paired with no-ip for Storj use.

I do have access to a 20-20 /30 fiber IP and another 50-50 /29 Fiber IP set. Once the 9TB fiber node is full, I might spin up another on that IP and also one on the 50-50 /29 Fiber. Getting IP’s on a business fiber line is pretty easy to do, until you ask for a block of more than 15 host addresses (ICANN requires justification is the excuse).

From a consumer internet perspective though- I believe Spectrum services this area an for $50/m I can get a modem for free and get 400/20 asymmetrical COAX. I know the technical limitation on the number of times I can do that, but not sure how many they’d let me tie to the same address/billing account.

also, my nodes may be growing, but so is the trash right now- the Fiber node has 102GB in trash due to all the recent deletes.

1 Like
951

trash will have peaks, happens for everyone, but if your setup have difficult keeping up it can also create trash or that’s a going theory of mine… i think xD
can’t remember if anybody else actually verified that or not… alas pretty sure if the storage solution have difficulty keeping up both memory and trash will start to go up and if its really bad then so will memory, but the memory thing seems to be loosely related to either latency (i got a drive that’s acting up from time to time) or simply because of my node size, almost at 15TB so it seems to use anywhere between 100mb maybe even 65mb at times, and up to 3gb when it peaks… tho not all monitoring shows it… so it may also just be allocated memory for caching that isn’t used… but it sure does allocate it.

currently i’m on 500-500 sync fiber and getting 1gbit -1gbit soon or i’m not sure what i’m going to get, it sort of depends a bit on what isp will provide me with the ability to register multiple ip’s across multiple subnets… and such… who has the hardware and what kind of cost barriers i run into…

the 1gbit / 1gbit is also 50$ here… but i’m located in a nice spot for internet, when living close to an internet freeway… but going enterprise internet service instead just makes the price explode and the bandwidth drop…

but i guess that’s because they expect it to actually be used and to deliver support and near 100% uptime.

well the bandwidth and the ip’s are seperate units and not really related aside from that if one uses a router it need to support multiple ip’s via something like vlan or whatever they use for that.

but really in theory if they got some fairly knowledgeable people i doubt the router should be a real limitation either… because most modern networking gear supposely suppose vlan tagging and then it would just be a configuration issue… ofc they may want to limit access / segment lines for individual security reasons, so they don’t interact over their wan if one exists, depending on individual setups.

what i really want to do is to hook my server up directly to the internet connection, and then connect to multiple subnets with multiple ip’s either by leasing them from someone… or from the isp directly

i suppose if i’m basically directly connected through the isp, then i might be able to fully disregard their ip related stuff and simply static configure my global ip’s on the server.

that might be a pretty viable option for me… but i can see why you would want multiple connections… i’m basically on a high end connection, and still here it’s not far above base consumer grade connections, i bet without even any installation aside from a new router i could get 10gbit sync
ofc it depends a bit on what fiber they put down… but it’s really wasn’t that long ago

if you do it directly yeah i’m sure, but if the isp is renting them to you for profit, then i doubt it’s really required… but not 100% sure…

any net thats not below ip/24 is only partial of a class C subnet
but yeah usually it’s only ip/32 for consumers, but doesn’t really matter since ip/24 and all that way to ip/32 is just less and less ip’s in a ip/24 / class c subnet which is what storj sorts by…

only advantage to be closer to ip/24 is that there is less room for compertition, but then on the other hand… it only takes one to half the bandwidth. or worse if it’s somebody with many nodes.

maybe i should look into renting ip global ip addresses… might be relevant for when i need to talk to the isp… because if i can run it all myself just need them to jack me into the brainstem of the internetz.
xD
having multiple internet connections tho it would be nice and might be an option would get old and real expensive in a hurry… but i might get some i got a pair… at one point depending on how the reliability turns out.

i suppose renting the global ip’s and static configuring them on the server would also solve the whole internet net connection failover if they are basically just a network connection…

then it wouldn’t matter and it would maybe even load balance so easily and without changing the number of ip’s with 1 or multiple connections running.

that’s what i need :smiley:

952

That’s not exactly how it works, large files are split up in 64mb segments prior to being split up with RS encoding and each segment is spread out over a different set of nodes. As a result the repair will not happen at the same time for those large files. It’s naturally spread out.

953

then we can say it might be because the unfilled storagenodes that got the data is in a sense a group and when enough of those fail it would have about the same result large scale repairs.
tho if no nodes vanished from the network i guess that wasn’t it… but i would suppose it could be prone to the same avalanche type effect like i mentioned earlier…

ofc there maybe mitigations in an attempt to avoid that

i duno, just saying lots of possible ways it could happen and give the result of large repair data without the network having to see many nodes vanish at the same time.

954

It could simply be a few large nodes being offline or getting disqualified or suspended. That’s the most logical explanation.

2 Likes
955

i hear that the simplest is usually the right explanation. :smiley:

956

On that same note it could be an indicator that the 1.16 patch may have broken a few nodes. Although that’s purely speculation.

2 Likes
957

i’m sure not going to update to this version when i keep seeing people complain about corrupt orders.db

so next update it is then… :smiley: i know we should auto update, but i still don’t regret running it manually… but i’m sure that’s only until when i forget… maybe ill make a dead man trigger to install / activate watchtower if i don’t update in time lol

and if people don’t think the new version might be sketchy at best…
well this was basically the next post i jumped into… seems like there without a doubt is an issue with the new update…

1 Like
958

HI all,
My egress suddenly dropped to almost zero overnight, just want to ask if you all seeing the same usage pattern.
image

My 4 nodes usually got total 6-7mbps egress traffic and has different IP addresses in separate /24 subnet.

1 Like
959

Is there anything in the logs? Do you still have ingress? Possibly a firewall or port forwarding issue.

960

Same herenode

961

I just brought one of my nodes back online after migrating it and was wondering the same thing. almost zero egress.

962

All logs look good.
I have 3 nodes already full and 1 node still have space. Logs show very sporadic egress. Ingress for that 1 node still proceed normally.
i thought there might they might stop some testing activities that why I asked here if anyone see same pattern as me with almost 0 egress suddenly.

963

i’ve seen people report zero ingress lately, but haven’t seen it myself…
maybe they like you didn’t notice their nodes where full :smiley: ingress will do that

964

zero ingress was mostly outdated software. we were talking about no egress.

965

oh right i missed that…just checked… and yeah my new node is basically zero egress less than 1 GB… the 9 month old node tho is still doing repair egress.

big drop in egress tho… maybe the others where talking about egress also… wasn’t really paying that much attention just scanned through the post quickly, didn’t seem interesting.

also makes better sense because ingress should be the same :smiley:

oh but to be fair the new node never got much egress to begin with… but still main node is at less than 1GB egress in counting only egress and not repair egress