We are still investigating this issue. Current theory is that different processes that collects and inserts tallies and rollups data on satellite side had some unusual intervals between each other and that caused unexpected storage values. Time to have correct at_rest values might be delayed up to 48h as this is how long tallies can be processed. Data looks to be cleaning up on satellite side, on storage node side it my require ~12h to get latest at_rest data (its cached). So please keep us posted if problem still exists on your nodes or it was resolved.
5 Likes
Will do. Thanks for the update @michaln.
I take it that means there is no correct way to interpret the data as its actually incorrect at the node end at the moment? In that case I wonât bother trying to make sense of it in the earning calculator and just hope it wonât happen again. 
I might add a warning if the last disk report deviates significantly from the average that estimates might not be accurate, to prevent the tool from being used to report high âuncollected garbageâ when its actually a stats issue.
We are trying to make it as accurate as possible, just we cannot guarantee that there wonât be some glitches from time to time. In future we will try to improve that but we have also challenges in other places too 
2 Likes
I understand that local node stats canât always be a priority. But I feel a responsibility when building a tool that exposes those stats and derives conclusions from them, to make sure people donât draw incorrect conclusions from it.
I saw several screenshots posted of the earnings calculator and people complaining about high amounts of uncollected garbage, even though itâs actually just a stats issue.
Iâll figure something out to add a disclaimer when there are large deviations from the average or when itâs the first few days of the month. I just want to prevent unnecessary reports of issues on the forums.
2 Likes
Previously mentioned ones have been resolved on my end. Just Saltlake is acting weird now.
Iâm sure that will fix itself too.
In the mean time, I have been working on adjusting the Earnings Calculator for the new bandwidth_usage table and Iâm adding a warning when the last reported disk usage by the satellite deviates more than 10% from the average and the difference is at least 100GB. What do you think @michaln ?
Iâve lowered both thresholds to force it to show the message on my test node for this screenshot
The first 2 days of the month, it will show this smaller note. As things may be missing when satellites havenât sent a single storage usage report yet.
I again forced the display of the message with fake data for this screenshot
2 Likes
Sounds good. This should add additional clarity. Thank you!
2 Likes
Seems itâs resolved by now.
No, you need to wait until tomorrow to be sure.
1 Like
Just summarizing the conclusion of this topic, with thanks to @michaln for providing information. Storj is trying to prevent these glitches from happening, but canât guarantee it wonât happen again. Both a sudden drop in data stored in the graph as well as incomplete reports in the dbâs will usually resolve itself within 2-3 days. Payouts are not effected. Donât worry or report issues unless it remains unfixed for at least 4-5 days. Iâve added a note when such situations occur in my earnings calculator to prevent unnecessary worry and reports on the forums, though most people go by the dashboard.
2 Likes