Wrong used & remaining disk space

Update after 1h of running

Dashboard stats
Used: 0B (no change)
Free: 0B (no change)
Trash: 40.1MB (increasing)
Overused: 40.1MB (increasing)

Current configs

storage.allocated-disk-space: 0 B (as @Alexey advise to stop ingress)
storage2.monitor.minimum-disk-space: 0 B (otherwise the node won’t start, disk not meet min requirement)
storage2.piece-scan-on-startup: true ( I don’t think this working because I see no CPU usage at all)

:frowning: my node is f*cked

It need to finish a filewalker process. Some SNO reported that it could take a few hours (in some cases even days).

I don’t think the filewalker is running, no CPU usage bro. Any way to check and make sure it’s running? Because the used space remains 0B, no increasing by time.

Update:

  • ingress stopped, only “downloaded”, no more “uploaded”
  • filewalker still not running, no CPU usage and disk stats is still 0B for used space.
  • these error messages keep showing in logs

Nothing change. Somebody please tell me what should I do now? @Alexey :frowning:

Did you simply delete the piece_spaced_used.db or did you re-create it?
What I mean is, if the database is present or not?

I deleted it then restart the node. :frowning: Is that the wrong way?

Yes I think so. From my experience the software does not recreate single database but only whole set.
So what you want to do is to shutdown the node, then move all databases into a new directory that you create. Then start the node. It should create all new databases now, which you can delete except piece_spaced_used.db. This is the one you want to keep. So you shutdown the node again, delete the other databases and move the old databases from the folder you had created back to their original place.
Sounds complicated but is quite easy.

1 Like

I have deleted piece_spaced_used.db, what should I do now? :)) Can I just delete all databases? =.=

@nyancodex do what @jammerdan said…

  • you want to shutdown the node
  • move all existing databases into a new directory
  • Then start the node. It should create all new blank databases now
  • shutdown the node again
  • delete the blank ones except piece_spaced_used.db. This is the one you want to keep.
  • move the old databases, from the new folder you created, back to their original place.
  • start the node
2 Likes

I just did it. Same situation :frowning: I don’t know why the filewalker not running.

  • ingress stopped, only “downloaded”, no more “uploaded”
  • filewalker still not running, no CPU usage and disk stats is still 0B for used space.
  • these error messages keep showing in logs

Just a general advice: It is always good to rename before final deletion or move to different folder so it can be undone in case something goes wrong.

That would erase your node stats and stats history completely beyond recovery.

That’s sad. Have you double checked that the piece_spaced_used.db is present now?

But then I have no idea what could be the reason for your issue.

yeah it’s here, I think something wrong with the filewalker. In your situation the filewalker works so it’ll recalculate the used space from the beginning, but in my case for some reasons it’s not working. :frowning:

image

Yes its very strange.
There is a way to connect to the node via debug port and check if the filewalker is running.

Please give me a link. =.=

And some more in this post and following ones: How does 'storage2.max-concurrent-requests' really work? - #12 by jammerdan

1 Like

Please show more of this log. There are lines before these 4 ERROR ones which may be helpful.

I did it the debug and got this, so by this we can conclude that filewalker is running?

[3072520588062033830,4584779262334866261] storj.io/storj/storagenode/pieces.(*CacheService).Run() (elapsed: 2.767654614s, orphaned)
[5364910219229466961,4584779262334866261] storj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite() (elapsed: 2.76763983s)
[3018707075876990546,4584779262334866261] storj.io/storj/storagenode/pieces.(*Store).WalkSatellitePieces() (elapsed: 2.767516401s)
[5311096707044423677,4584779262334866261] storj.io/storj/storage/filestore.(*Dir).WalkNamespace() (elapsed: 2.767510333s)
[6457291522628140242,4584779262334866261] storj.io/storj/storage/filestore.(*Dir).walkNamespaceInPath() (elapsed: 2.767507848s)
[3157767700770289893,4584779262334866261] storj.io/storj/storagenode/pieces.storedPieceAccess.Size() (elapsed: 48.427269ms)

nothing bro, just normal “downloaded”, “download started”, “download canceled” :frowning: gotta go now, will capture screenshot later if you need it.