Hi,
Can you use defrag on the storj HDD? Win GUI install.
Thanks
You can, but I doubt it’s very useful. The only useful thing to defrag would be the db’s.
I’m now using again Diskeeper 18 Pro, but this is a commercial option. The goal is not only defrag but reduct the I/O activity with several optimizations, and put the hot reads in RAM cache by software (it works fine because luckily, when I accidentally power off my hard drives a few seconds, by moving a cable, the node was still downloading pieces successfully haha), but I like this also before it defrags only files who are detected as degrading the performance (they estimate it saves ~85%) and they minimize the obstruction with performances. It also do space reclamation automatically, so basically, the defragmention is no more necessary (but it can be done manually for the remaining fragmented files). Its important to say its, with PerfectDisk, one of the only software specilizaing in i/o reductions for entreprise grade (including SQL Servers) for dozens of years. That’s a serious company, and that’s why I talk about them (also, I’m not affiliated to them, I’m only user). You can watch their live demos on youtube (look for the engineer Spencer Allingham and Condusiv) explaining their technologies.
Its still difficult for me to say the difference at this time since the nodes are more peaceful at this time, but I already saw differences when I watch animes videos on the drive or by moving files after a few uses, because Diskeeper is analyzing the frequentely used files generating iops (so it includes the .sj files), then it manages to reduce the i/o with their Intelliwrite (their main oldest feature on several generations, which prevents fragmentation before it occurs by putting files in contigous space, it works closely in low level with the Windows write driver) and Intellimemory (wich is basically a smart Hot Read RAM Cache, not write). You can hear when it happens on IOMeter. The disk is spinning, then after a few times, it reduces spinning and stop because of course the RAM Cache for the read part. Its difficult though to say if its putting the db in ram, I assume not, because it needs to do write operation. I’ll move probably my db again on my solid idle spare SSD to save a few iops, even if its a bit more risky (but Samsung gives a good estimation of the wearing level to react at time, and I think 3D Nand technology on Pro SSDs are a good longevity).
When I made comparison with I/O activites on the benchmark freeware IOMeter, which simulates a lot of I/O activities, the gain with Diskeeper was real. To compare with an already solid WD Gold 12tb, I had :
- 185 iops without additional software/cache.
- 1500 iops with Diskeeper (with all their features enabled)
Otherwise, with Windows 10, be careful with the built-in if you’ve a big drive, because the defragsvc is really defragmenting the whole drive weekly. Personally, I would recommand to lower the frequency monthly if your allocated node is large like 8tb, because at my opinion, the operation is so intense it’s wearing/heating the drive for a useless gain in that case.
But to conclude, the I/O performance is a lot more important than the defragmenting itself about nodes performance, that’s why some operator use hardware like in NAS with SSD Cache, also ZFS (but I don’t know well this one, just read topics on the forum, and I don’t want invest into dedicated hardware). And Diskeeper/Perfectdisk are the solution by software, so probably less performant but also probably cheaper, also at this time, they don’t have option for SSD Cache (which could be an interesting option for the write/larger read cache side). Finally, I only test Diskeeper again because I had already a licence, so I didn’t have to invest, and I wouldn’t recommand it because I’m not totally sure its really worth it for the storage nodes (somebody with a I/O monitoring could help to see, I didn’t watched very well the i/o operations with the windows ressources observer). But if you want try, they have a free evaluation. I just wanted tell you this kind of software for i/o reduction exists.
Diskeeper reports himself the I/O eliminated, fragments prevented, and so on in their interface, but a comparison with a dedicated i/o monitoring on Windows by another S.N.O. could give a better idea over the performance improvements (also it mixes all the i/o reductions, including my system, games and SSDs which doesn’t enter in consideration with the node alone, so a dedicated Windows device for the nodes should be more interesting). But, because I follow the logs, I think the download successrate is way higher than before, because the frequently asked pieces are in ram cache.
Also, only for the cache, you can use software like Primocache, but it will not defragment or preventing it. But it will save i/o too for the data in cache of course.
summary for the last 24h. I streamed, launched two games, watched some animes video and two storage nodes running in the background (on two single WD Gold 12tb). It makes sense more read is reduced since it uses the RAM, but the Write side is only using optimisation/prevent fragmentation/instant defrag. Storage I/O Time Saved seems to be a fancy estimation though. But its only a visual for your info, the software is made to “set and forget” in the backgrounds services, and it does the thing right fine for this. There are much more details in the others sections (hourly: response time, data cached, iops, throughput, fragments prevented & eliminated, …)
And here is the generated pdf report: http://floxit.power-heberg.be/temp/diskeeper.pdf
About the other “traditional” defragmenters, at my opinion, they are totally useless since the build-in Windows already does it well. Except if you wish see a nice mapping overview like in the freeware Defraggler.
1 Like
Thanks, ok I won’t bother was just a little worried that the drive is 20% fragmented.
2 Likes
So yeah, no worries, you have normally nothing to do, Windows will do it fine. Normally by default, it will do it when the machine is idle (no mouse/user activity) for the next scheduled default weekly defrag, but you can just click on properties of the drive and launch it manually yourself. It will generate intense activity, but, normally, it shouldn’t disturb your node. Preferentially, try to do this when you see your node quite idle and avoid when there are a lot activities due to tests (at this time, it seems peaceful).