Hello dear Storj community,
At the start of the node or after a storagenode update or randomly scan pieces occur and make our HDD stressed at 99% and more. (100%)
I would like to know what is the answer from storj on that two points
Does Storj will pay this HDD activity (IOPS) to storagenodes operators ?
Is there in 2021 network v3 a real solution to achieve the scan pieces less intensively for our hardware ?
Since disk activity is 100% during scan pieces Storj should pay for HDD usage. Unless using bcache (a storagenode operator solution) with SSD is there another and official solution ? Achieving scan pieces by block or something else ?
Iam today frustrated to ear my HDD spinning at 100% for several minutes/hours, as i know that is trafic (internally called IOPS) and we are not paid for this even HDD are used for this scan pieces.
bellow the chart for scan_pieces from 04:00 to 11:00 am (in my timezone)
Hello dear Krystof,
Yes you can also include CPU activity in scan_pieces but guy we speak on storage not on IT as Service.
My disks are SATA 7200RPM. I dont think that they are SMR disks. HDD activity that i show on the attached chart is not only piece deleter it’s scan_pieces which permit each time to storj to identify if the pieces are still on the medium through docker container (lsstat).
I have 12 nodes on CMR disks and one SMR (Seagate Barracuda 5400rpm)
SMR is very slow and noisy and stressed as you write.
Full SMR node and i see the same as you… to be fair I have synology OS on this disk but i have this synology allocated only for storj.
This could probably be configured to run periodically, instead of at startup. Every 5 days for example, of course I don’t know what it does exactly, so I might be wrong. Does it simply enumerate all the files on the drive?
After restarting node or even a node update storagenode may scan files to be sure that files are still accessible accordingly to the db (files list). This method may prevent data to be loss on the network by repairing on a online node in case of the file isn’t here for many reasons as possible, HDD crash, removed by operator or any technical problem that can cause to a file to don’t be accessible on the volume. I can’t say if this pieces scan occur on Windows because i don’t have Windows node, but on linux raspberry this process consume 100% on HDD for a time depending on your volume capacity.
yes that is something we were hoping for for a long time now.
There already is an option to move the DBs to a different volume, just search the forum.
A cache is completely worthless for the piece filewalker because it scans all files. You can’t have all that (or all their metadata) in a cache. Even I with 30GB of zfs cache don’t have a visible improvement. (except for a little less load for writing files and reading from the dbs).
of course not… you get paid for storing files and uploading them. nothing more and nothing less.
No, audits are done by downloading the file from your node which the satellite chooses.
I’m sure you can find information in the source code and documentation. It’s basically for book keeping on the node.
Everyone on every system has the same problem because that’s how the storagenode software works. Some systems might perform better because they have better HDDs, bandwidth or whatever. But still everyone will have the problem of high IOwait after start.
--device-read-bps= Limit read rate (bytes per second) from a device
--device-read-iops= Limit read rate (IO per second) from a device
--device-write-bps= Limit write rate (bytes per second) to a device
Guess it could be worth a try. However, it will definitely make the filewalker take longer. And I’m not sure what effect it would have on the remaining operations of the node like egress, since both the filewalker and egress are read-iops. So maybe by halving the read-iops you might shoot yourself in the foot with twice the time for filewalker with problems in egress?