What happens when my disk is physically full? Am I penalized for failed uploads?
I have 6TB disks. I set storj to 5.5TB allocation to allow the 10% wiggle room as suggested in the docs. I’ve found my nodes to go over the allocation by nearly 0.5 TB and almost fill the drives completely.
i have also had problems with disks filling up in the past.
And currently an issue where storj is reporting more free space on the drive than really exists (I’m setting the storj limit like a TB lower than the real capacity to keep it in check). The size mismatch seems to get better if I keep them up long enough to finish the used space filewalkers, but still off. Oh and one node must have had a crash sometime in the past and the “blobs/temp” folder had many thousands of files that totaled hundreds of gigs that I had to delete.
anyway, to answer your question, you’re not particularly penalized for failed uploads. it’s just like if your node was slow and lost a race.
What could be a problem is that your node might have a hard stop if you completely run out of disk space and can’t free up a tiny bit just to even let trash cleanup work.
Ahh yes, I forgot I set my logs to store on the disks themselves too. Going to clean those up right now.
The node will send a signal to the satellites if it has less than 5GB of free space in the allocation/disk to stop the ingress.
The discrepancy is likely related either to errors with databases in your logs or failed used-space-filewalker.
So, if you would fix both issues (they are usually related to a slow disk subsystem), then the discrepancy should go.
i think it would be nice to have control over this 5GB limit, i would say most of the bigger SNOs have dedicated disks for storj and using remaining space to limit how much space should be used (or atleast a hardcap if filewalker has problem) is much more accurate without a heavy load on the disks
I do not think the band aid is a solution, you need to fix issues with a databases/failing filewalkers instead.
Which ones do you have?
Not one of them, simply it takes >4 days to complete and with the current load there could be 4TB of mismatch with 1 restart in a “bad time”
Ah, I see. Yes, the feature to resume a used-space-filewalker is not deployed yet