100% Disk usage for many many days

[HGPlays]

Hello - i know file walker takes a long time, and i have let it run to get everything thats going on lately with bloomfilters, trash and such…

I got DB’s on an nvme, logs also.

my many HDD’s have ALL been hammered for 10 days straight!

Can it really be that a node with 5-8TB needs to run for SOOOO long for filewalker at 100% usage?
i have these settings in configs:

Use the lazy filewalker, it’s true by default

#pieces.enable-lazy-filewalker: false

Dissable filewalker:

#storage2.piece-scan-on-startup: false

The disks are CMR. Nodes still runs great.

[Alexey]

You likely also have a lot of uploads, right?
See

Updates on Test Data

Hello Storj Community, We are excited to announce some changes that we have begun to implement regarding how we are using test data on the network! What is changing We are currently seeing exponential growth in our paid customer data and expect that trend to continue based on our sales pipeline. Because of this we want to ensure that we are sending timely demand signals to the node community so that the capacity and other resources we need for our growing customer base is available when they …

[HGPlays]

Sure - i do - about 400GB out since the month started - but split to 8 different drives tho.
So i really dont know if this can be the only reason for 100% usage on all disks for 24/7? filewalkers should have finished days ago.

[Alexey]

I guess so, because right now the lazy mode is enabled, so they will pause if there are IOPS from the customers. And resume, when there is no usage.
The disk maybe loaded on 100%, but actually can proceed requests with a high speed. The load is meaning that the drive is busy, even if there a few KiB of w/r operations.

However, if your node experiences also FATAL errors due to a readability or writeability checks timeouts, then you likely have an issue.

[HGPlays]

Okay - i hear what you are saying - it just seems VERY extreme that this has been going on for 10 days straight - on 8 disks. day and night. But thanks.

[Alexey]

Yes, just check the post about stress test above.

[snorkel]

If those lines are from your config, they are commented out and not used. You must remove the # and restart the node, in order to use those settings.
You can keep lazzy on false (disabled) because it’s useless and creates more problems.
You can run the used space FW once (piece scan on startup - true) to bring everything up to date, than put it on false. If you don’t touch the pieces in the storage dir, you don’t need to run it every time you restart the node. Mine is off for months and all looks good.

[Alexey]

@snorkel

You can keep lazzy on false (disabled) because it’s useless and creates more problems.

I would disagree. Usually it helps to do not perform a housekeeping while customers want to download/upload anything from/to your node.