I believe you but my egress jumped from 300-600 MB per day to 6GB right when I upgraded my HDD and started getting ingress again. Stayed right at 6GB since then but it’s only been a few days so not enough to even be significant.
But as you say the node selection is random so there’s no way that egress and ingress can be linked.
Yes you’re right, just edited my post.
Well everything has nuance. It’s not the node selection that’s causing this, but customer behavior. We’ve seen this before, recently uploaded files get downloaded more often than older files. If your node is full, after a while it doesn’t have any recently uploaded files. Egress is a bit different from ingress, because for a download you can’t randomly select the nodes. You have to select the nodes from those who actually have pieces for the segment. So node selection for download is less random. Hence why I said there would be more variation in those numbers. This is exaggerated a bit by some of the storj test patterns that also download recently uploaded pieces. But there are valid customer scenarios where this would be the case, like when it’s used to verify a backup after upload or something.
well i will assume what i saw in regard to my ingress was a proxmox avg bandwidth calculation that seems to deviate… like say if there is many spikes over a day… and then it goes to zero for an hour or 8… and then starts back up… it seems the proxmox avg will show offset numbers even tho the avg really is the same in both places… maybe because its avg becomes more accurate over time and then in the beginning looks higher because the storagenode goes from a couple of mbits and all the way to like 110 when busy…
thats my best guess currently… so it’s still there… it’s just not the node causing it… even tho the node is the only thing creating traffic on that graph…so yeah sorry my mistake…
ofc this explanation is also a guess… at some point i should be right xD
even a broken clock is right twice a day… lol
6th july
striker43 - ingress 112,83 - (missing egress)
dragonhogan - ingress 108,38 - egress 34,33
TheMightyGeek - ingress 113,09 - egress 6,87 - Egress % of stored 0.56%
SCG - ingress 108,71 - egress 32,64 - Egress % of stored 0,37%
(had 1hr and 40min approx downtime was testing a shutdown script tsk tsk)
it worked lol
if we should try to look at egress compared to data stored, you will need to include that in the screenshot in the future…
might need to do that in another way than % tho… the numbers get so tiny…might just try fraction tomorrow
Well there’s no need to be that hard on yourself. I admire how you dive into things and keep going at it. I just think you can do a little better when shining a light on something before taking a shot in the dark. Keeping this kind of context in mind can help with that.
Trust me, I have the opposite problem. I hate being wrong so I try my hardest to never be and miss a lot of things I could have guessed. Oh… And I may try, but I still get things wrong anyway and it sucks.
We all have our strong points and weak points, so it’s nice when people can complement each other. So I thank you for this conversation and your efforts to collect information.
2 Likes
it’s good with some healthy critic and objective views sometimes… way to few people ever really learn to think for themselves, i generally don’t let stuff bother me to much… everybody has their own reasons and misunderstandings happen in almost every sentence…
so i try to regard everything as a story unless proven otherwise 
and assume that everybody else is just about as confused about everything as i am lol…
it’s a big world and lots to learn… for those with eyes to see.
you also do a great job, and yeah it does show you like to be right… but i suppose most people like to be right… i do throw around ideas a lot… sometimes they will be wrong, some times they will be right…
it’s not really about being right or wrong… its about making progress in understanding…
and yeah sometimes i may trust my gut a bit to much… hadn’t considered that my graph could be wrong… which was actually an obvious place to look…
going to be implementing your one line audit fail node shutdown script in the near future…
tho it seems the process isn’t closed upon shutting down the node…
i did try running it in a emulated terminal using the screen command, not to comfy in messing around in all the linux processes yet… so was an easy way for me to isolate it and check on it later from multiple machines…
after the command line had shut down the node because of an error… (i ran the command to trigger on simply all errors) and i went into the emulated terminal, then it was still going… didn’t check if it still would catch an error and shutdown the node tho… something which i really should have… but for the next few days i will be trying to see if i cannot beat the others in ingress and egress xD
after all i do have, maybe not the best setup… but i should be pretty close to being one of the top tier nodes with my setup… i mean i got 3x raidz1 in one pool, with two ssd’s for accelerating the pool
thats like raw iops from hell and latency that is never higher than a 1/10 of a blink of an eye (25-30ms) on basically any data stored…
so the race is on
1 Like
Both ingress and egress dropped a little bit yesterday, might be due to the fact that I downloaded a movie and watched a few videos during the day, I’ll try to monitor my bandwidth to see if I saturate the connection.
7th july
TheMightyGeek - ingress 109,30 - egress 6,06 = 5.0 ‰ of stored - 1,3 TB
SGC - ingress 109,96 - egress 29,61 = 2.6 ‰ of stored - 11,21 TB
kevink - ingress 109,60 - egress 15,69 = 4.6 ‰ of stored - 3.2 TB approx
dragonhogan - ingress 104,21 - egress 29,11 = 2,8 ‰ of stored - 10,31TB
striker43 - ingress 109,76 - egress 25,58 = 2.5 ‰ of stored - 10,2TB
1 Like
from what i am seeing which should be without downtime this time and plenty of bandwidth… then i cannot see it… also you don’t really lack download… you lack upload so might be difficult to tell… ofc as your node fills it might also become more and more of a problem…
but your numbers are basically equal or better than mine, egress compared to data stored…
Yes ingress only dropped 3GB so nothing dramatic, I agree that upload bandwidth might be a problem as my node gets full but hopefully I’ll have the fiber by then. They are rolling it out to my neighborhood as we speak.
Interesting to see that my egress is almost double yours. I’ll see if it drops once my node is full.
thats from the 6th… .and today you got 0.50%
thats less than a 1/10th drop since yesterday
and my egress + 100minutes downtime was 0,36% while today it was 0,26, so thats like a drop of 1/3 or 1/4 since yesterday
so no, i don’t see it… doesn’t mean its not there… its just not reflected compared to my numbers.
Same here so maybe not the movies
impressive egress !
What’s your nodes capacity ?
only 3.4TB used and 10TB available. SGC has more egress with 11TB used. yours seems a bit low
oh wait, read your storage wrong… with 1.3TB its fine
yeah actually it’s my egress in relation to stored data that is kinda low…
yours is nearly at the same level as TheMightyGeek with 5.0 while you got 4.6 and i got 2.6 ‰ of stored.
ofc brightsilence mentioned somewhere that the older the data gets usually the less egress it has… but thats most likely a test data thing… to reward people for holding data for long periods and then after like say 6months or a year download it again…
but my ingress does seem a bit low… but insufficient data to really get good numbers… seems like writing it in permille works well… also very easy to do that math for it…
because one simply says divide the tb stored number into the gb egress… and then you got the permille
I guess that will change once customers start really using tardigrade rather than testing. It’s probably because people running tests upload data and then download it right after.
yeah we are moving into exciting times now… my initial guess was that the really migration onto tardigrade would really start some 6 months after the initial launch date… so that sysadmins would have 6 months to tinker and test, before even considering moving enterprise stuff to it…
and it was what end of march or beginning of april that tardigrade launched
so 2-3 months left until i expect we should start to see some major changed on the network…
ofc that is a very unqualified guess… not really my wheel house…
two quarters seems about right tho… ofc upgrades or might data migrations of enterprise i suppose is almost seasonal…
i’m excited to see what happens with this… even tho i’m still loosing money on hosting lol but still kinda new myself… just got a bit lucky arriving on the end of a huge test data ingress cycle.
the needle really starts to move when one can get 5mb/s for days at a time…