if you are so sure, that’s all.
Good luck!
anyways, i was curious and i checked iostat and it does not look bad to me
autodeleted after 10 seconds 
One year ago there was a big bottleneck with the sqlite database and i got several nodes with corrupted database. Mirrored logs solved it and i had no problem (zero issues) after that.
Storj improved very much sqlite databases… and i guess it is not needed anymore.
yeah those are nvme / pcie ssd numbers, my aging ssd’s cannot keep up with that…
if you want me to do the math for your nodes and list it with the others in the summery. then i will need screenshots of the dashboard…
or
total ingress, total egress and data stored…
after the day is done obviously, at GMT day shift… also keep in mind that GMT is not the same all year round… or everybody elses time shifts… GMT stays the same, so the offset changes…
but just make sure the node is recording the old day… then i’m sure it’s fine…
well mirrors are much safer storage… i guess if the ssd’s make a lot of errors… or if cables for them had poor connections or something in that regard… a mirror would make it much more reliable… ofc in that case scrubs should still reveal data errors on the drive in question…
i cannot imagine why it would help tho… but like i said… mirrors increase latencies… but with modern ssd’s well latency are basically none existing…
my ssd’s can still give me 20ms latency when or my log ssd can… and it only gets worse if i and a second one in a mirror or a span… so i decided to go with a ssd for each…
very nice numbers on yours tho… you should enable autotrim…
it will give you even better performance… but its not like its going to change anything for you…
hehe I was surprised about my egress too. Seems to saturate my 12Mbit/s upload quite a bit.
No idea why I have so much more egress/data ratio than you and the others… I have 36GB egress from europe-north and saltlake, the rest must be real customers. So almost 10GB egress from real customers isn’t bad at all. Maybe my node age makes a different for that part.
i had a spike today of 60mbit egress… not sure which satellite tho… and it was fairly brief
just noticed the huge spike in my hourly and then went to check it in netdata to make sure it was from the storagenode…
proxmox daily graph doesn’t even show it… not even when say to day max… it still seems to use maybe an hourly avg to build the day graph, or something like that… usually the spikes of like say 120mbit egress isn’t long enough to even show up in the hourly proxmox graph… but this one was… which is interesting…
maybe what we are seeing is the start of enterprise customers moving into tardigrade, a bit sooner than expected… but ofc the sysadmins will have to come first… .scope out the place, if they are getting ready to move in their workloads.
would be amazing if ingress is higher than test data… ofc way to early to tell anything of that sort… will most likely be an unknown until a year from now atleast…
i have a problem with this…
you cannot have these numbers… on your storagenode pool atleast … search time on a hdd is 2ms i think and thats like the minimum on a 7200rpm and 3ms or 4ms on a 5400rpm and the 7200rpm has a full sweep of the head time of 7ms… i think… anyways… you cannot have these numbers on your storagenode… unless if your storagenode is on SSD’s
so yeah… how did you get these numbers this isn’t your os drive or what not… maybe if there is no work you might see those numbers… but when the storagenode is running you should see more realistic numbers in the milisecond range…
those numbers are purely nvme ssd numbers.
what are we looking at your l2arc ?
zpool iostat -l 3600
i’m still seeing the deviation tho only 700ms this time… wonder what that is…
ofc today i’m doing a scrub… so maybe it ran into the 1mil files folder where i got like 3 folders which really gives the system some work… but looks very very odd…
maybe 1900 reads per sec is my pools max directly from the hdd’s but think i have seen more like 4k maybe or i got a failing drive somewhere…
opool is a sas hdd mirror pool
rpool is my OS ssd pool / boot
tank is my 3x3drive raidz1 + slog ssd(slog is 5gb and the rest of the drive is used for the os /boot drive + l2arc ssd (storagenode /vm / various stuff pool)
have thought about mirroring the boot drive, as it would make my system so much more robust… but haven’t got around to it… but really should…
Only egress traffic at the moment. hdd is 1 mechanical disk and logs+caches are 4 ssd’s. Completed ROI in January. I really dont care, i have different setups and ingress/egress is approx the same +/- 5%. It mostly depend on the size and age of ur node.
i guess egress could be so low… l2arc so big that your hdd is basically idling most of the time, which will put the latency numbers towards zero… until they just becomes a line if there is no real activity…
and then the ssd reads on the l2arc is most of the calls, which makes your pool latency look like pure ssd…
same here…nope actually his total egress was more like 70 because of the two nodes on one subnet.
2nd place in egress… ofc thats only until we calculate the permille i bet
still winning the ingress… ever so slightly… doesn’t seem like my scrub did much… for my ingress atleast… egress is way to erratic to be useful apperetly… wonder if the test data egress is over two day periods… i mean it would ofc have some sort of resolution… might as well be two days as 10sec… i mean the more you have to control the slower you switch the less confusing it gets 
but with out a doubt the best day for me on this months graph…almost 40gb maybe 25 above
the graph shows total ingress and egress numbers unless if one sets it for one of the others
ingress is fairly stable… avg has been around 2tb pr month (im told by older SNO’s)… maybe a bit more maybe a bit less… some months have been near zero and others 4+ even… 100gb pr day is quite nice… but don’t expect it to last…
i’m 4 months in and i’m at near 12tb… so thus far i’ve kept an avg of almost 3tb pr month…
so yeah from what i’ve seen this is the avg…
and ofc now it’s like 3 months or so since they opened up for enterprise customers… so within the next 3 months i would expect enterprise starting to move in… ofc will most likely take a year before we got any good idea about what that will mean…
but it will ofc give us some numbers early on… deviations from the normal node test traffic.
so digging into the topic of surge protection or such… i know most will have some sort of opinion, but i’m looking for real answer… like what do i need exactly … and why… i mean will a sophisticated fuse do the job, or do i need like thousands of $ worth of gear to truly protect my server…
anyways… ran across this and thought i would share because it was just so hilarious…
fat/oil isn’t really that conductive… water is basically electricity… so yeah… no surprise there…
still not much wiser, but can’t stop smiling lol
why not create a new thread?
maybe i will tomorrow when i’ve dug a little deeper, just wanted to share a good laugh…
with a privileged few
not much wiser indeed, I mean people do build computers and submerge them in mineral oil to cool them. But in theory distilled (purified) water shouldn’t affect the motherboard since it isn’t conductive at low voltages.
and indeed it doesn’t, but there is also little denying that water does seem to be very different from other substances, but water is in the most fundamental way two out of three parts hydrogren and that being the closest physical matter to the electron…
also water is what creates thunder… there is some kinda of meaningful close relation between then… and yes i am familiar with the current accepted theories about how it’s supposedly works… but alas all science is usually replaced by simpler and more encompassing explanations…
sure maybe it’s just the gravimetric pressure of our star that makes water special here… also funny how almost all electrical creation of the planet has to utilize water in it’s production… even solar is greatly affected if there is a high amount of water vapor in the atmosphere…
also water can be controlled by electrostatic fields… sure one might says its due to it having a highly different electrostatic pressure… but isn’t that just passing the problem on to a new particle lol… and then how is that explained … well we got these quarks… and then how are they explained… well we got these strings… and suddenly one is so far away from the real question that it’s drowned in other questions…
water seems to have properties relating to electricity which is still poorly understood… hell it might be the energy source of the future… what advanced civilization wouldn’t use water as a battery on a planet such as this…
and if hydrogen is what the universe is made off… and water is the closest thing we have to hydrogen… then it would be no surprise if the universe is infact an incompressible liquid, something which the greatest living electrician was a proponent for…
rant over
almost… really everything is conductive, its just a matter of how much voltage you got… or how little you are measuring… i had this experience i ran once… where i had used some old speaker terminals for connections… took a while to figure out that even at voltage around 1 or 2 then voltage would start to leak out through the wood, because it was old speakers… before they started using plastics around the wire terminals to reduce voltage/current leaks.
after that i ran into trouble with collecting power from the mains power wires 3-4 meters away… because i had made the setup so sensitive that it could pickup and store the changes in the magnetic fields… not really useful for much… but took a lot of work to figure out that was what was happening lol…
104.32GB per day = 3.17TB per month. Impressive. I wonder how long this rate will continue? At that rate your 24TB node might be full in just 4 months if we don’t factor in deletions.
I cannot contribute ingress stats right now because my node is “full”, (storage limit capped by me at 1TB), but I enjoy seeing what’s going on with the rest of the network.
