Logs indicating the file walker process

Are there any specific log entries I could observe that would signify when the file walker process starts and ends? I don’t see any at the INFO level, and as the file walker process consumes a lot of resources, I suspect it would be good to see it there.

2 Likes

This has been suggested before in a thread, but I don’t think it made it to a poll or github issue.

2 Likes

Voted. This is important information for SNOs in my opinion.

1 Like

Was bored, peeked into some source code. Though, I don’t speak golang, so I might be wrong in some parts.

As far as I understand, there are two uses for what the community here calls the file walker process:

  • estimate the amount of disk space used by blobs (of a specific satellite or of all satellites, functions SpaceUsedBySatellite and SpaceUsedTotalAndBySatellite in pieces/store.go).
  • run the bloom filter (function retainPieces in retain/retain.go).

Both essentially call stat() on each blob for good reasons. The latter already has a DEBUG-level log item. So all it would take to implement this request would be to annotate the first two functions with log items before and after the call to WalkSatellitePieces, and maybe bumping the level of the existing log entries in the third one to INFO.

4 Likes

That sounds right to me.

2 Likes