i’m on my 4th scrub… and then might do another one if i manage to remove one of the top level vdevs from my pool, so making good use of the “downtime” and trying to migrate from 8k block size to 512byte because my hardware is mostly 512byte based… and making sure my data integrity is flawless
scrubs does take the peak of what my system can perform, especially my read latency suffer… gets up to 100ms O.O so i rather run those when the network is mostly asleep
Any news so far regarding the node selection and continuation of the tests?
I sure do miss the steady influx of data.
There was a weekend…
3 Likes
System seems to be running pretty smooth…
got a bit more IO delay than i care for, but i’m apparently running on a bad zfs ashift, if they aren’t all in the same blocksize it causes additional latency / IO delay…
$ zpool iostat -l
capacity operations bandwidth total_wait disk_wait syncq_wait asyncq_wait scrub trim
pool alloc free read write read write read write read write read write read write wait wait
---------- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
rpool 1.98G 137G 0 19 4.78K 250K 817us 976us 549us 230us 29us 297us 1ms 776us 372us -
testpool 4.78M 4.50G 0 5 5 30.3K 491us 653us 491us 170us 2us 1us - 485us - -
zPool 13.6T 24.6T 11 129 225K 1021K 18ms 6ms 11ms 1ms 2ms 7us 16ms 5ms 10ms 132us
---------- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
but system seems to run pretty nice while on low usage… storagenode is running tho…
i like how the total write latency is 6ms xD
weirdly enough my read is slightly high at the moment… maybe it’s because the system was rebooted only last night so the l2arc hasn’t had time to wake up, and most likely won’t at this speed… only 3 weeks left until its filled lol
I think it’s pretty good for me right now so I don’t miss anything. I was able to move my containers to my other Raspi4, modified my network and Pi’s a bit, move my backups to my server without slowing down anything else and synchronize the blockchain for Raspiblitz at full speed. As far as I’m concerned, it can stay that way until tomorrow, when I can calmly upgrade my first node from 4- to 8TB (SMR to PMR) and push the backups up further. 320GB with 1MB upload is pretty annoying ^^
1 Like
yeah i was looking at buying a few new hdd’s but the used prices at the moment seems almost unreasonable, i suppose it’s a corona thing… production was shut down for a good while so that may have upset the supply and demand balance…
got two 6tb that’s 4kn drives and that have offset my entire pool during creation, so its running ashift 12 give a IO amplification of x16 or x8 on my drives that is 512 based drives.
but i guess it’s a sign that i should really be buying new drives anyways and then sell them when warranty is about to expire, seems to be the trick of the trade in datacenters.
right now the risk vs reward on used hdd’s seems to favor new drives anyways…
not like my node has been shoveling in cash during the first 2 months either… but hopefully over the next few months it will start to actually offset my base electricity costs atleast…
Oh yes, that’s the reason for everything right now. Covid. I’m just happy when the topic is finally history … So I got a Western Digital Elements again relatively cheap for 120 euros, because I couldn’t say no ^^ I would never buy a used one, simply because I don’t knows how long it has been running. I’m very satisfied with Western Digital, my NAS is now 12 years old and still runs and runs on the same hard drive and according to the interface the hard disk is still in good condition. Sure the storage space is a bit small today but it is not the only HDD I have and as long as it runs … 
1 Like
i’ve been buying a bit of a mix of old enterprise drives and a few new.
got them cheap so i evaluated that it was worth the risk… it seems they will have just about 4 years and 7 months or so on them… which basically makes it so the datacenter can sell them to a 2nd hand dealer, that dealer can offer 3 months returns on bad drives… and they will still fall within the manufactures warranty.
and thus the datacenter can ensure they always have a supply of the same kinds of drives, in the same number they initially ordered, making dealing with array sizes easy.
and they can sell them at not much lower or so it seems than new drives… rarely able to get 50% savings on an old hdd.
thus one could if able to sell them replace most of the fleet of drives and upgrade them fairly cheaply and thus remain within warranty, might be pretty smart way to do it actually… so i think ill try that approach.
i wouldn’t have noticed without buying their used drives tho… xD
1 Like
Ah okay I misunderstood, that means you buy used hard drives from companies? Okay, the idea is not bad, you definitely know where they come from. In my previous post I meant that I would never buy a used hard drive from a private person. However … who knows what data could be restored if it was not properly formatted and overwritten ^^ If there is still a guarantee and you are even entitled to return it then I understand that. Where can you get such sources? xD
well i’m not sure it’s worthwhile tho… but at the right times they sure can be cheap… but right now not so much… lots of companies around that sell second hand datacenter stuff.
else i just go on ebay and buy from well rated long running accounts… never really had an issue with that yet… so from time to time one finds a shop that has just the right size to sell good stuff cheaply.
that trend usually trails off as peoples businesses grow and they realize just how much of a hassle customers become in the long term and then they figure out thats what the other companies they thought they could beat in price, take their premium for…
but i don’t mind helping them to arrive at that conclusion lol
buying the stuff directly from the datacenters might be a pretty good deal tho… but i don’t really need hundreds of the same drives at once yet… xD
goes past my monitor with the storagenode log running…
docker logs --tail 20 storagenode --follow
and i stop for a moment because it seems to be dead… i check the time… look back… still seems dead the minutes fit but the hours doesn’t… (time zone adjustment) i guess
oh good it’s not dead, its just running so slow it looks dead… lol
1 Like
Having is better than needing 
Yes, I currently even activated standby for my hard drives because there is so little traffic. Then they can sleep too. 
1 Like
We deployed a fix on europe-north and are starting uploads again. I hope this time with “normal” traffic for everyone.
@570RJ please let us know if you see 200mbps again.
10 Likes
Yeah. Traffic seems to flow in again. 
Bur real traffic seems to be terribly low.
1 Like
hi, with which program do you see those graphics?
This is Zabbix 
20 chars…
1 Like
Traffic back for me too but a bit lower volume than before
Seems normal now. Thanks @littleskunk
1 Like