ERROR blobscache satPiecesTotal < 0

Node Operators troubleshooting
1

Never seen such errors before. What does it mean?

2024-04-22T18:48:37+02:00       ERROR   blobscache      satPiecesTotal < 0      {"satPiecesTotal": -3072}
2024-04-22T18:48:37+02:00       ERROR   blobscache      satPiecesContentSize < 0        {"satPiecesContentSize": -2560}
2024-04-22T18:48:37+02:00       ERROR   blobscache      satPiecesTotal < 0      {"satPiecesTotal": -1280}
2024-04-22T18:48:37+02:00       ERROR   blobscache      satPiecesContentSize < 0        {"satPiecesContentSize": -768}
2

This is only mean that your used-space-filewalker didn’t finish its job or you disabled it. In the latter case I would suggest to enable it back, save the config and restart the node.

3

Yes , used-space-filewalker is running.

If this is expected behavior, why calling it error? Should be info message then.

4

Yes, it’s expected, because normally it should not occur, if used-space-filewalker wasn’t disabled. But when the used-space-filewalker will finish its work and update databases, this error should gone.

6

Same here with 200k ERROR entries per hour on one of my nodes.


2024-08-27T19:32:26Z	ERROR	blobscache	satPiecesTotal < 0	{"Process": "storagenode", "satPiecesTotal": -250368}
2024-08-27T19:32:26Z	ERROR	blobscache	satPiecesContentSize < 0	{"Process": "storagenode", "satPiecesContentSize": -249856}
2024-08-27T19:32:26Z	ERROR	blobscache	satPiecesTotal < 0	{"Process": "storagenode", "satPiecesTotal": -3840}
2024-08-27T19:32:26Z	ERROR	blobscache	satPiecesContentSize < 0	{"Process": "storagenode", "satPiecesContentSize": -3328}
2024-08-27T19:32:26Z	ERROR	blobscache	satPiecesTotal < 0	{"Process": "storagenode", "satPiecesTotal": -250368}
2024-08-27T19:32:26Z	ERROR	blobscache	satPiecesContentSize < 0	{"Process": "storagenode", "satPiecesContentSize": -249856}
2024-08-27T19:32:26Z	ERROR	blobscache	satPiecesTotal < 0	{"Process": "storagenode", "satPiecesTotal": -197888}
2024-08-27T19:32:26Z	ERROR	blobscache	satPiecesContentSize < 0	{"Process": "storagenode", "satPiecesContentSize": -197376}
2024-08-27T19:32:27Z	ERROR	blobscache	satPiecesTotal < 0	{"Process": "storagenode", "satPiecesTotal": -197888}
2024-08-27T19:32:27Z	ERROR	blobscache	satPiecesContentSize < 0	{"Process": "storagenode", "satPiecesContentSize": -197376}

Shouldn’t that be a warning instead of an error? It’s simply wasting resources, if no FATAL error occurs and the fw finishes sooner or later.

@Alexey

7

The answer has not changed. While databases are not updated, you may see this error.

Enable scan on startup, if you disabled it (it’s enabled by default), save the config and restart the node.
Then monitor that the filewalker wouldn’t crash (search for error and used-space in your logs).
To track its progress:

8

both nodes have --pieces.enable-lazy-filewalker="true"

9

Then please make sure, that it’s not failed during the way.