Depending on the speed of your drive, but it might take days before the filewalker finished.
It’s a 7200rpm 16TB HDD.
Any prediction a bit more accurate?
Rather days than hours, but not a week.
It all depends on whether PC isn’t being restarted in time between, no storagenode updates, the real amount of files on your node, …
What file system are you using BTW?
@naxbc
fsutil fsinfo ntfsinfo E:
copy this in the admin shell. press enter. post the screenshot(output). Please.
And welcome back.
1 Like
It’s NTFS on a Ryzen 7 with 32GB RAM.
That looks fine… So only seems to be an unfinished file walker.
2 Likes
I’d love to hear the average times for other peoples start-time filewalkers. Does 4h/used-TB sound reasonable? So if OP has around 9TB filled that would mean ~36 hours?
I posted my fw run times. See Tuning the filewalker.
2 Likes
Maybe we wait for the bigger Bloom-filters, fix is on the way.
Plus improved solution for the future is planned.
Even after forgetting untrusted satelites i still have a hugh issue with disc discrepancy:
windows reported use:
dashboard:
1 Like
Hi, I do see a bigger difference. My node is full for at least 2 months at 12 TB +/- 0.5G free and trash.
But yet, the average disk space used is staying at 10TB. So that is a 2TB difference.
I have a xfs file system. Not sure if that is causing this. But I can only think of small files being stored. Where the average is calculated by the satellites, I guess based on file size.
So when there are a lot of small files, it can eat up my storage sectors on disk and not filling all sectors due to the disk block size. So I am loosing space on my disks. Which would be a shame, but nothing Storj can do about it.
But at least I would like to know if that could explain the difference between the two values. Sounds placeble?
2024-01-24T22:28:33Z INFO lazyfilewalker.gc-filewalker.subprocess gc-filewalker started {"process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "process": "storagenode", "createdBefore": "2024-01-17T17:59:59Z", "bloomFilterSize": 2097155}
2024-01-25T03:20:07Z INFO lazyfilewalker.gc-filewalker subprocess finished successfully {"process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2024-01-25T19:42:34Z INFO lazyfilewalker.gc-filewalker.subprocess gc-filewalker started {"process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "createdBefore": "2024-01-21T15:26:55Z", "bloomFilterSize": 364960, "process": "storagenode"}
2024-01-25T19:49:13Z INFO lazyfilewalker.gc-filewalker subprocess finished successfully {"process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2024-01-26T14:58:04Z INFO lazyfilewalker.gc-filewalker.subprocess gc-filewalker started {"process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "process": "storagenode", "createdBefore": "2024-01-22T17:59:58Z", "bloomFilterSize": 574785}
2024-01-26T15:31:29Z INFO lazyfilewalker.gc-filewalker subprocess finished successfully {"process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2024-01-28T08:29:37Z INFO lazyfilewalker.gc-filewalker.subprocess gc-filewalker started {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "process": "storagenode", "createdBefore": "2024-01-23T17:59:59Z", "bloomFilterSize": 1877458}
2024-01-28T12:25:18Z INFO lazyfilewalker.gc-filewalker subprocess finished successfully {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
| Satellite ID | Used space | Time |
|---|---|---|
12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S |
2.18TB | 4h 52m |
1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE |
1.24TB | 0h 7m |
121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 |
645.58GB | 0h 33m |
12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs |
2.77TB | 3h 56m |
So, for me it sounds like 1h23m/TB
Did all filewalkers finish their jobs for all satellites? Or do you still have errors related to them or FATAL errors?
Is it exFAT? Because 4.57TB cannot use almost 9TB
Also, do you have errors related to filewalkers?
In this thread we discussing disk discrepancies on nodes, where disk is not able to keep up and lazy filewalkers are failing to finish their job.
So, please check your logs for errors related to filewalkers (search for "walk|retain" and "failed|error"), also search for FATAL errors.
My bad. Will search further
No problem at all. Just add what you have done so far.
