No space left on device in logs

I am seeing this error multiple times after the file system involuntarily ran out of space. After freeing space the node keeps showing that error every now and then on puts. But most puts gets succesfully uploaded to the server without error.

Could you post that error to get a better idea ?

I have seen errors like this "pieces error: write /var/local/storj01/storage/temp/blob-219879470.partial: no space left on device", "errorVerbose": "pieces error: write /var/local/storj01/storage/temp/blob-219879470.partial: no space left on device\n\tstorj.io/storj/storagenode/pieces.(*Writer)

And "error": "order: proto: can't skip unknown wire type 4", "errorVerbose": "order: proto: can't skip unknown wire type 4\n\tstorj.io/storj/storagenode/orders.readLimit:553\n\tstorj.io/storj/storagenode/orders.(*FileStore).ListUnsentBySatellite.func1:245\n\tpath/filepath.walk:360\n\tpath/filepath.walk:384\n\tpath/filepath.Walk:406\n\tstorj.io/storj/storagenode/orders.(*FileStore)

I don’t know if they are related.

However for the last couple of hours no more such error messages showed up. So maybe it kind of fixed itself after a while.
And I remember I ran a fsck but it did not show any results. Maybe that fixed something too. I don’t know.

How much space are you using here ?

The path is in the data storage. So it is always the same amount that the overall data storage can use.

Maybe just to be clear again. The error message was correct. The OS reported wrong free space which lead to no more space left on the device. Unless the underlying issue is solved, I have freed some space. But still for some uploads the node has reported that there is not enough space. Despite the fact that I had freed many GBs. The weird thing was some put went through without error and some showed this error message.

fsck must have fixed that issue.

Let me rephrase it. How much space is that folder occupying ?

All those partials take up quite a bite of space. Currently there is a ticket to investigate and implement a method to delete it. The most I have seen is 1.2TB.

Yes, that is my guess atm too.

4.0K