this still not fixed yet? I thought it should’ve been fixed at some latest versions?
Hi,
Does anyone else have this dip… I seem to have it for 3 days much lower on 2 nodes , wondering if it is a ‘me’ issue 
1 Like
It’s a known problem. All my nodes looking like that.
The issue is somewhat similar to Dashboard reporting disk is almost full when in reality its half full? but seems not quite the same.
If I read this screenshot correctly then the satellites think data available on my node is 4.79 GB, the node itself thinks it stores 496.03 GB and the file system (NTFS) reports 585.181GB / 545GiB.
The HDD is connected via USB 3. I have set the following parameters:
storage2.piece-scan-on-startup: true
pieces.enable-lazy-filewalker: false
The following error gets logged on service start:
2024-06-11T19:55:35+04:00 ERROR piecestore:cache error getting current used space: {"error": "filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled", "errorVerbose": "group:\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:74\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:79\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:713\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:74\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:79\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:713\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:74\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:79\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:713\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:74\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:79\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:713\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
Waiting for days or weeks for everything to get in order by itself seems risky given that the file system is running out of space. What are other options to make the node in sync with the satellites?
I can see that the us1 and saltlake.tardigrade report 0 GB usage but most data is in their blob folders in the file system. What if a empty those two blob folders? Will it break the node? What are other options?
Is it possible to drop all data of the node and resync it?
This is a known problem, probably only some missing statistics from satellites. Only storj can fix it.
2 Likes
Are payments being affected as it’s measuring the average disk GBm ?
So does mine. It is a well-known issue. Many people encounter that.
Any idea of what to do in the mean time? Shut the node down or let it run out of the disk space?
I would assume so, our TB avg is lower and since less income. I dont understand why storj officials is not saying any public about this. More then “bug” with statistics.
My payment in grafana is barely moving anymore so… for me it seems we are not getting payed accordingly at the moment. @Alexey sorry for pinging you but what is going on? why is my avg stored data not correct? And will this get fixed?
1 Like
Depends on wheter the missing data is paid or not… 
Nah, i got graph for 12TBs of egress, on just one node, it was not real, just a glitch, all from local node datas. They are updating so much lastly, that its safe to say we are in some kind of metamorphosis, but payment was fair tho.
Remember that old mascot?
Looked like caterpillar, it just may be changing into a butterfly!
It’s related to the satellites reporting, not a new version.
These satellites didn’t report the usage because the tally took more time than it should.
Our engineers would recalculate it before the payout anyway. Since it’s not fixed automatically, that’s mean that it requires a manual intervention to the process, so we need to wait.
3 Likes
I just set up the node on June 6 right when the issue started for everyone so I’m not concerned about the payment just need a clean way to reset everything before it runs out of disk.
It would be good if we reach a state again where we can rely on the numbers we see. Currently every day something else seems broken. And if you think it has been fixed, it is broken again soon after.
2 Likes
It wouldn’t. We increased the minimum available to 5GB recently:
Yes, the product is evolving and customers data coming. Everything is moving right now.
This one would be fixed eventually. However, in that exact case the Community cannot help with this issue.
I would think so too but my case doesn’t look affected by just the topic issue. There is another issue on top of that – see:
Looks like it causes the system to forget the current used space value and at some point think it used less. Not sure when it happens, suspect that the service terminates abruptly at some points, gets restarted by windows service rule I configured. And when it restarts it thinks it used less space than when it crushed. So disk space is leaking on top of the satellites statistics issue.