I have a node. The real available disk space is 3.5TB. Dashboard showing 3.02TB used with a trash folder of 563GB. So zero space left which makes the engress stopped. But “df” command shows 455GB available and only 3.1TB used. What should I do now?
Have you tried restarting the node ?
It’s a free space in the allocation, not on the disk.
Please show result of the command
df --si -T
I would also want to suggest to check your disk for errors.
result of “df --si -T”
also checked disk for errors, nothing wrong with disk or xfs.
It’s the same node from this old topic Wrong used & remaining disk space - Node Operators / troubleshooting - Storj Community Forum (official)
Lol. I may just do a GE.
Now the mystery with the free space is disclosed: you have 3.8TB disk, you allocated 3.5TB to the node, and node used 3.02TB + 563.19GB = 3.58TB, so the free space in the allocation is 0, but the disk has a free space of 489GB.
I suspect some round errors here, but well. Please note the SI measure items, i.e. decimal units.
From the other hand 489GB is much more than 3.8TB - 3.58TB = 220GB. So may be either filewalker did not finish its job, or the filesystem is corrupted.
I hope you checked the filesystem in a special way for XFS:
By the way, what numbers for (replace /storagenode/storage
to your actual path)
du --si -s /storagenode/storage
and
du --si -s --apparent-size /storagenode/storage
Now you need to wait until trash would be automatically emptied and your node would have a free space in the allocation.
And remained problem with a discrepancy between reported used space (Average Disk Space Used This Month) and the actual used space.
I suspect the same issue as there: