Guys, what happened? test data removed or big customers deleted their data?
1 Like
The Uncollected Garbage bug fixed (previously your node have data but have not been pay for it). Now it is just delete those data, you could see some comment around here When will "Uncollected Garbage" be deleted? - #337 by Climbingkid
4 Likes
With what version is this fixed? 1.110?
I think it not related to any version (bloom filter work on StorJ side), so the next bloom filter go to your node will remove a lot of data.
2 Likes
My node dropped from roughly 9tb to 3.3tb
With 4.73tb of trash.
What the?!
Is there some huge saltlake cleanup going on?
3 Likes
You got paid for 3.3tb this months. 4.73 was unpaid data , don’t worry
3 Likes
Sorry what? That’s what I’m worried about.
I had 9tb of data and then it just flagged to delete a bunch of it.
Why would I not be worried that I went from being a 9tb storage node to 3.x tb?
its discussed in more detail in that long thread linked.
TL;DR, there was a problem with storj setting up the garbage collection so even though data was deleted by the customer, it wasn’t being caught on our nodes as trash but still marked as “used”. So it fell into a weird unpaid category called nicknamed uncollected garbage.
the last day or two some big garbage collection bloom filters have been rolling out and many people have been having lots of this garbage now sucessfully marked as trash.
So the bad news is you were never getting paid for this by storj.
The good news is that at least now it’s marked and trash and then will be deleted to make room for real data.
2 Likes
TL;DR you never had 9TB PAID data.
it was not previously flagged as garbage. it was perfectly fine data per my dashboard until this week where now half the perfectly good data is now trash.
I thought it’s just cleaning up data that was previously flagged as trash?
nope, it should have been flagged as trash a long time ago but wasn’t because (massive hand waving) the storj trash collection process was overloaded.
That was the problem. Data that should have been gone or at least marked as trash wasn’t handled correctly.
That’s why your node showed incorrect data. You cannot simply rely on the node dashboard as there are a lot of issues with the data it is displaying. It is often incorrect or has discrepancies between different data sources.
So unfortunately:
I’m kind of surprised many have not notice the different between the actual pay amount and the storage store amount…
2 Likes
It is not that easy due to average calculation taking into account node online/offline times and also the held back system. And with the average data always wrong and behind the real usage, how could you rely on it?
1 Like
The only way is to compare the latest fully reported usage from the satellites (the highest point on the Average graph) with the usage reported by the OS (you may also take the paid usage from the Payout information for the past month as a baseline). Or with usage reported on the dashboard if it’s match the OS reported usage.
looks like previous period I had “5.01TBm” average in the paid usage.
But now I dropped to 3.3tb used with 4.74 trash
so my average space paid has for sure dropped. but maybe they trashed a bunch of production files within the month too?
The test data on Saltlake was meant to be self-deleting. But this did not work.
So it was paid data for 30 days. And after that it was garbage but did not get deleted as it should have been.
There is no production data on the Saltlake satellite.
I think @jammerdan is right, and this is removed TTL data. It can drop like this. You can check that if you would select the Saltlake satellite in the drop down menu.