Same issue here, but I think “garbage colector” will be implemented soon, that will clean all the test data and free up some space 
Just because storage is full doesnt mean it goes offline mode is still has to pull data from it.
The garbage collector will not delete all test data. That is not its purpose. It will identify and remove inaccessible segments from storage nodes, after the Zombie Segment Cleaner has done so on the Satellite side. You can read more in the design docs about Zombie Segment Cleaner and Garbage Collector. Some nodes which hold a lot of inaccessible segments will see a lot of space freed up once we activate the Garbage Collector, while others may not see much change.
6 Likes
Thank you Helene for the feedback. When that activation will take place? Is there a date set?
devs are still reviewing some aspects of the code, so we were unable to activate it yet in this upcoming release. Hopefully it will be ready for the release after. It is important that we do extensive testing to make sure this works properly before activating it on the SNO side.
1 Like
FYI it will not delete the data, uploaded to your node neither customers nor testers.
2 Likes
Hi, I noticed something interesting. When the allocated disk space becomes 100% full (sometimes it is even exceeded by several MB), the amount of traffic performed by the node drops drastically to almost zero. After some time, when some space is freed, the node still transmits very little data. It just helps restart the container. Immediately after this traffic returns to an average of 10-15 Mbps.
Your node isnt freezing its because if your node is full your no longer going to be getting data, Unless there gets and not puts.
2 Likes
Yes but if situation changes node should start getting data again. But it didn’t happen. Node needs to restart.
Its because the satellites need to relearn that the node has space again it wont know right away.
Exactly, but this process should be automatic.
It is just takes time. Your node just doesnt get free space again instantly. I know because my nodes have filled up many times and after garbage collection I get over 500gigs free and it starts getting data again without me having to restart my node.
However now we also have user data but it depends on what data your node actually has stored yet and if those users are downloading their data.
It is. It is also recommended, covered in the docs and in the install that you should have at least 10% capacity free. Following the guides will decrease the likeness of your node being disqualified.
1 Like