Node full but keeps getting data

A better place reporting bugs might be here https://github.com/storj/storj/issues

I have same issue. My node was -4MB yesterday. Today it is -2GB. Something wrong here. Node is purely dedicated to storj and has not been touched in last couple days.

Looking at the log, it seems that deleted files are not accounted for when checking if there is available storage for the upload.

I created a bug report: https://github.com/storj/storj/issues/3751

2 Likes

Was this issue really fixed? It seems the fix is not working for me. The data exceeded its limit and I have no physical space on my HDD now. What should I do to prevent the node from being disqualified? I am on v0.29.3

When the problem arose, people like @littleskunk (see UPDATE: the SNO Garbage Collection & Disqualification bug has been fixed above) suggested to reduce the size allocated to your node to the minimum: 500GB.

You might try that, but Iā€™m not sure itā€™s the same issue in your case.

Hi all,

Same Issue at my Nodeā€¦
I have some free space on my Serverā€¦ But i think something is wrong here?
Iā€™m on Version v0.29.3

Thanks for help
Markus

Hello
I have 3 nodes that reached storage limit but still receive new data. It is lasting for a 3 days. How to stop receive data above the limit? I have a little free space on that nodes and soon it will be dq.
Nodes is 0.29.3

Restart your node and it will stop receiving data.
It might be the same bug I reported a few days back: https://github.com/storj/storj/issues/3751

thanks.
hm, looks like one of my nodes stop to receive new data without restarting it. Right at that moment ( a couple of hours ago) when I write this topic.
I have look at logs and there are only download events for now.
I cant check it on another nodes because i have increased a storage for few GB. It is need to wait when it fills up again.

In my case, it was rather severe, because it ended up using close to 12% more space than the allocated amount.

node still seems alive (before and after container restart). 0 bytes available on its storagenode partition before container restart. reduced -e STORAGE with 500MB. paste showing stats after container restart. i somehow immediately got 3.4GB of free space after container restart.
https://pastebin.com/DxahUJXu

so all in all, all of this actually seems to tick very, very well on its own, because:

  1. Im cutting corners and not saving 10% of disk space as a ā€œbufferā€
  2. Node hadnā€™t crashed
  3. It must have been piled up garbage collection, which got correctly dumped after container restart.

I am also not seeing the fix work on my node. Previously, I had 750Gb allocated to my node, and storage available was negative. I reduced stoage to 500Gb, followed the instructions here, and restarted my node. The dashboard has been reporting -314 GB for 3 days now. Iā€™m running version 0.29.3.

Same here. My node say -7.9 GB available and is 7.9GB over allocated space.

it start to get new data again,
so I restarted it. only download messages in logs for now
letā€™s see what will happen next

I am having same issue. After investigating more and monitoring the node. It looks like there might be a bug in how Remaining space is reported on SNO dashboard. My free space was 22GB when Remaining space reached 0 (a week ago). No it is reporting -31GB for remaining space while my free space is 40GB. So files actually were removed but my remaining space reported did not reflect it correctly. Because of that I did not change space allocation on my node and I never ran out of space, even though Remaining space on the SNO dashboard is telling me it is -31GB now.

Is there a fix planned? Since my last power 12 days ago, my free Disk Space was going from -17.14GB via -16.50GB to now -17.52GB. Or is there anything i can do? Iā€™m on Version 0.31.12.
Thanks Markus

The initial problem of getting data is fixed now.

The correct displaying of available space seems to be a problem but is mainly discussed in another thread.

My ā€œout of spaceā€ error condition has not stopped the satellites from trying to send me more data. As a result I have received over 2000 upload attempts that have failed in the last 24 hours. The success rate is just 1.119%. wasting bandwidth. screenshot below. For now I have lowered my storage limit down to 500G so that my dashboard now shows negative storage space available. Hopefully the satellites will stop sending me stuff that my node rejects.
outofspace2

If they donā€™t stop although your dashboard shows negative space, then create a new bug report because this should be fixed in 0.31.12