Honestly, I don’t care about those peanuts, burning money with storj anyway.
What really scares me is how they are unable to keep this running with less than 30 PB of customer data and no growth. What would happen if we had the exponential growth storj needs to survive?
Eh, better have this kind of problem than what competition deals with.
I would rather be hetzner then… 
Is there a crisis of big bloom filter resolved or it on working stage jet? Because i have like 330TB of data but payments so fare like 300TB only
As I understand it, they are looking into a related issue that has them pausing the filters while they investigate it. There are a lot of people working it right now, and details about specifics are not available for me to tell you more than that at the moment. Once it is resolved we’ll hopefully get one of the engineers to explain what happened.
I’m going to pretend I don’t know trash isn’t being taken out. So it looks like HDDs are filling.
If a massive BF eventually comes along and smashes us all in the face… we’ll deal with it then.
Just for stats, last BF for US1 was this:
2025-02-10T20:30:19Z INFO retain Moved pieces to trash during retain {"Process": "storagenode", "cachePath": "config/retain", "Deleted pieces": 135843, "Failed to delete": 0, "Pieces failed to read": 0, "Pieces count": 12964293, "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Duration": "7m25.789395217s", "Retain Status": "enabled"}
And the restore from trash for the same node:
2025-02-11T16:29:40Z INFO pieces:trash restore trash started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-02-11T16:58:16Z INFO pieces:trash restore trash started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-02-11T17:05:36Z INFO pieces:trash restore trash started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-02-11T17:07:26Z INFO pieces:trash restore trash finished {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-02-11T17:13:10Z INFO pieces:trash restore trash finished {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-02-11T17:25:39Z INFO pieces:trash restore trash finished {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-02-11T22:03:04Z INFO pieces:trash restore trash started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-02-11T22:03:05Z INFO pieces:trash restore trash finished {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-02-11T22:03:22Z INFO pieces:trash restore trash started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-02-11T22:03:30Z INFO pieces:trash restore trash finished {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-02-11T22:11:01Z INFO pieces:trash restore trash started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-02-11T22:11:04Z INFO pieces:trash restore trash finished {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-02-13T17:37:55Z INFO pieces:trash restore trash started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-02-13T17:37:58Z INFO pieces:trash restore trash finished {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-02-13T17:52:02Z INFO pieces:trash restore trash started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-02-13T17:52:04Z INFO pieces:trash restore trash finished {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-02-13T17:57:58Z INFO pieces:trash restore trash started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-02-13T17:58:43Z INFO pieces:trash restore trash finished {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-02-23T05:50:30Z INFO piecestore New bloomfilter is received {"Process": "storagenode", "satellite": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "creation": "2025-02-20T15:08:27Z"}
Has the error in generating filters been fixed yet? Even after processing, I still have ~20% unaccounted garbage instead of ~23%
I do not know yet. The investigation is ongoing and the fixes are applied. We will see. If these changes are helpful - I believe no more restore events would be needed.
As far as I understand, there are combination of several issues in different places. Engineers are tracking it.
Most of my nodes now have ~10% in trash. I’ve disabled all logging that’s not warn or above, so I can’t investigate if that’s actually the case, but it does correspond with the amount of deleted material advertised above.
Cheers StorJ teams 
How old is the node in the picture?
There are three:
I also see around 10% in trash from 108 nodes, so it looks like long waited cleanup.
How do you both have free space and overused? 
I believe it’s a multinode dashboard.
Yes it’s the Multi node dashboard. I have free space and overused too.
Also is see in my nodes about 10% Trash
@snorkel , @MarviBiene thanks lads, that makes sense
I lost about ~15%. I do tend to win a lot of races though.
