Hi guys. I have a node of 3.5TB with a strange situation. The free disk space in dashboard is 0.83TB (image) but in fact I have like 1.45GB of free disk space (by checking with df -h). I just updated the software to 1.73.4. Is that the problem?
Thank you,
No, it is not. I think you just used your root partition which is and was not empty. It probably just holds your system and other files, too.
Edit: I rather wonder why it is even running if you updated recently. If the disk space left is less than the node expects to be free it shouldn’t even start.
There’s no other files at all bro. The system files is about 2GB, not 0.83TB. And the machine is dedicated to be a node so there’s no other storage. I’m still trying to figure out why the stats is wrong.
How did it look before? I don’t think it is wrong. I think it’s something different.
You should check you path within the config and the mounting points for your storage.
Nevertheless 3.5T allocated on 3.5T partition can cause problems.
I haven’t checked the node for like 2 weeks. But I remember there was nothing wrong. Then after the last software update the problem happens.
I’m running ncdu to be sure. It’ll take 1 day to complete I think.
Update: ncdu can’t complete. I opened a bug report on Github. Hope somebody can figure it out.
Please show the whole output of the command df -T --si and your mount option in the docker run command for the data location.
I would recommend to check the filesystem and fix errors as well.
Hi Alexey (you’re Alexey on Github?) I don’t use docker. Just linux binary I downloaded from github. Here’s the current result of df -T --si and web dashboard.
Will the node stop ingress automatically when it detects that there’s no free space available? I have another node of 2TB and it seems fine. Just this one with strange stats. This is not the first time, last year it happened once and the problem solved itself after a month. Maybe some task is scheduled to run monthly?
Yes, AlexeyALeonov (Alexey A. Leonov) · GitHub
Yes, it should, unless we introduce a bug (this is why we recommend to leave 10% free on your disk and never use 100%).
There is a Garbage Collector, it’s running periodically (collector.interval option/argument), filewalker on start and cache updater running periodically (storage2.cache-sync-interval option/argument).
Usually such a discrepancy could be result of disabled filewalker or corrupted filesystem.
Since you unable to calculate used space - the last is likely a reason.
So, please, force check and repair a filesystem.
I would also recommend to check an LVM metainformation for errors as well.
I am having something similar on one node. I am still trying to figure out what the problem is.
This is what multinode is showing:
And this is the node dashboard:
According to df the used space is 8.1 TB. Assigned space is correct with 8.5 TB as shown in the nodes dashboard. So multinode shows the correct remaining space from the OS while node dashboard shows the correct total. It seems that there is a problem with the used space display which would explain the figures. Correct display would bring both charts back to correct showing.
i have same? problem, after 1.73.4
I have a 8TB HDD
until now, the allocated space in storj, was 7.6TB and always i have ~350gb free on HDD (many times the node was full - the hdd has always 350gb free)
after update to 1.73.4, i see the remain space on HDD to shrinking, so go in storj config and i change the allocated space from 7.6TB to 7.2TB
my problem still persist. I tried to restart storagenode many times but no help. 
Maybe I must wait for 1 month for the problem to solve itself?
Does the node free disk space after you changed the config to lower allocated space? Maybe I should try to lower the allocated space to 2.5TB.
I think sth is wrong with storagenode. What version are you using? I’m using 1.73.4.
That specific node is on v1.73.4.
Okay after monitoring I see that my node will stop ingress new data when the free disk space <=512MB. Then the trash will be deleted a bit. But the stats doesn’t change at all. I’m currently being paid for 2.5TB of disk space while in reality it’s 3.5TB. It’s a $1.5 loss per month. It’s nothing but I still want to correct it.
about multinode dashboard bug with free space:
Did you check the filesystem?
Will do it and update when I’m home.
