Error getting current used space:

Node Operators troubleshooting
, ,
1

Hi team,

I keep getting this error in the logs, and it restart

zError: readdirent config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/6g: no such file or directory

Any idea what causes this ?

also… seeing this

image
image829×770 42.1 KB

worrying thing… the node and system has not changed at all… other than the node updating to the latest one.

been running fine for 9 months

2

then goes to this… 1/2 seconds after that initail screenshot

image
image1168×784 52.1 KB

3

image
image1031×586 27.5 KB

4 
2023-02-03T13:38:40.230Z        INFO    piecestore      downloaded      {"Process": "storagenode", "Piece ID": "I2IEM4GCUFNI2TXYCOG7MMDUX4FTC6CXWECXDNGIOJMQVET7USLQ", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Action": "GET"}
2023-02-03T13:38:47.704Z        INFO    piecestore      download started        {"Process": "storagenode", "Piece ID": "DRKZQO2B6Y5CQX5T2OQZXH4AO3LPR6XCTR2A5LXTRJCUFAHBM2SA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2023-02-03T13:38:48.280Z        INFO    piecestore      download canceled       {"Process": "storagenode", "Piece ID": "DRKZQO2B6Y5CQX5T2OQZXH4AO3LPR6XCTR2A5LXTRJCUFAHBM2SA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2023-02-03T13:38:48.857Z        ERROR   piecestore:cache        error getting current used space:       {"Process": "storagenode", "error": "readdirent config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/6g: no such file or directory"}
2023-02-03T13:38:48.857Z        ERROR   services        unexpected shutdown of a runner {"Process": "storagenode", "name": "piecestore:cache", "error": "readdirent config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/6g: no such file or directory"}
2023-02-03T13:38:48.863Z        INFO    piecestore      downloaded      {"Process": "storagenode", "Piece ID": "NX27TOQWI3OF4POAGTC6AA7W7BLVZ4NWRPSPXHHSOZRW4QKBYTRQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
Error: readdirent config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/6g: no such file or directory
2023-02-03 13:38:48,892 INFO exited: storagenode (exit status 1; not expected)
2023-02-03 13:38:49,936 INFO spawned: 'storagenode' with pid 93
2023-02-03 13:38:49,938 WARN received SIGQUIT indicating exit request
2023-02-03 13:38:49,945 INFO waiting for storagenode, processes-exit-eventlistener, storagenode-updater to die
2023-02-03T13:38:49.953Z        INFO    Got a signal from the OS: "terminated"  {"Process": "storagenode-updater"}
2023-02-03 13:38:49,959 INFO stopped: storagenode-updater (exit status 0)
2023-02-03T13:38:50.070Z        INFO    Configuration loaded    {"Process": "storagenode", "Location": "/app/config/config.yaml"}
2023-02-03T13:38:50.072Z        INFO    Anonymized tracing enabled      {"Process": "storagenode"}
5

UPDATE ::

seems like a checkDisk has now fixed the issue