No bloom filters from US1

Ottetal · February 26, 2025, 8:35am

15% is on the higher side, but on top of regular bloom filters, it does not sound unreasonable.

snorkel · February 28, 2025, 2:26pm

Almost 2TB/IP trashed on my nodes.
Sweet!

Mark · February 28, 2025, 2:35pm

I guess the EU1 purge is here. Over 30% of my data for that satellite got trashed.

alpharabbit · February 28, 2025, 3:53pm

I wonder where we are in the process of purging all the zombie data? Is it over now?

Mark · February 28, 2025, 4:24pm

If my most recent daily satellite report is correct for Eu1 which shows just 316GB (and we all know how accurate the daily reports are haha) It looks like my node might trash more data soon (assuming there might be a lag in bloom filter generation). If my actual used space drops to the number on the report, I will have lost about 50% of the data. Time will tell.

nerdatwork · February 28, 2025, 4:45pm

Why not turn on the debug port and look at the metrics ?

Vadim · February 28, 2025, 4:47pm

I have 58TB in trash, at least 300 tb left

Mark · February 28, 2025, 4:51pm

I’m not even sure what I would be looking for inside the debug port metrics. Could you be more specific?

Mark · February 28, 2025, 4:53pm

Have you ever calculated what percent of the entire network you are storing?

nerdatwork · February 28, 2025, 5:02pm

Mark · February 28, 2025, 5:30pm

For me, the “blobs_usage” metric in the debug port looks like the same number that I’m seeing in my dashboards used space near the pie chart.

Dashboard=452.99GB
debug port=4.52993728768e+11
/api/sno/ = 452993728768

So I’m thinking the number in the debug port is based on my nodes local data and not what the satellite is saying about my node.

Vadim · February 28, 2025, 5:32pm

Why I need this, it is useless info.

Mark · February 28, 2025, 5:42pm

I just think it’s interesting. I once estimated that you store 0.5% of all the data on the network based on the “storage_total_bytes_after_expansion” for each of the satellites from https://stats.storjshare.io/data.json

alpharabbit · February 28, 2025, 5:45pm

You knowing this guy?
http://www.th3van.dk

Roxor · February 28, 2025, 5:59pm

Are we being untrashed… again? Too much growth in the last hour for it to be real uploads.

Mark · February 28, 2025, 6:11pm

Yeah, looks like it. All my nodes except for my one hashstore node is showing zero trash now.

alpharabbit · February 28, 2025, 6:56pm

snorkel · February 28, 2025, 7:06pm

Untrashed again?
Why this yoyo dance with data? Is it testing? Or again some bug?

Roxor · February 28, 2025, 7:11pm

I don’t mind: they need to be super careful with customer data. And if anything looks fishy… they only have that one-week window to ask for it back.

(And although you never want to test-in-Prod… it must be satisfying to see the trash-recovery system working as designed on 20000+ nodes! )

alpharabbit · February 28, 2025, 7:14pm

But it also proofs the delete process is not working as designed…