My nodes are full too, but the memory usage is small
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
3d20fef76e67 storagenode5 0.03% 136.2MiB / 24.81GiB 0.54% 26.3GB / 5.33GB 0B / 0B 83
9fb28e5cf48a storagenode2 19.64% 302.6MiB / 24.81GiB 1.19% 167MB / 7.22GB 0B / 0B 106
storagenode2 is currently working on gc, used-space-filewalker and collector (which is full of the same WARN messages, however it didn’t try to delete the same pieces, all unique).
more details here:
Looks like the database got corrupted?
Please check all databases and fix them:
It cannot recreate just one missing database and has never been able to do so. You need to use this guide: