Node refuse to start

Node Operators troubleshooting
,
1

Hello,
I have a big 11 TB node NO more starting due to hashtable.
I saw the fix in a thread but something is missing for me

pi@mcanto:/tmp $ git clone git@github.com:storj/storj.git && cd stor
Clone in ‘storj’ in corso…
git@github.com: Permission denied (publickey).
fatal: Impossibile leggere dal repository remoto.

Assicurati di disporre dei privilegi d’accesso corretti
e che il repository esista.

How can I fix in linux the below error ?

Best regards
026-08-17T23:29:22Z FATAL Unrecoverable error {“process”: “storagenode”, “error”: “Failed to create storage node peer: hashstore: potential misconfiguration: missing hashtbl "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/meta/hashtbl" when log files exist in "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0"\n\tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[…].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[…].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290”, “errorVerbose”: “Failed to create storage node peer: hashstore: potential misconfiguration: missing hashtbl "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/meta/hashtbl" when log files exist in "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0"\n\tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[…].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[…].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290\n\tmain.cmdRun:109\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290”}

2

Use https clone instead of git@github.com:storj/storj.git:

This should work: git clone https://github.com/storj/storj.git --depth 1

3

You can install write-hashtbl without cloning the repo:

go install storj.io/storj/cmd/write-hashtbl@latest

It will be compiled to ~/go/bin

4

Hello,
I made it working with the help of kocoten1992.
I ran this command as root :
./write-hashtbl /NODE/NFS/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0

it ran for more than 2 hours scanning all folders until “ff”. No errors
I ran the node again and I have the same issue :frowning:

Any suggestion ?
I made some error ?

Best regards

2026-08-18T07:54:10Z INFO hashstore hashstore opened successfully {“process”: “storagenode”, “satellite”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “open_time”: “6.304019538s”, “logs_skipped”: 231, “logs_matched”: 89, “logs_mismatched”: 0}
2026-08-18T07:54:10Z INFO hashstore loaded hint file {“process”: “storagenode”, “satellite”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “store”: “s0”, “path”: “config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/meta/hint-0000000000000009”, “writable”: , “largest”: 0, “skip”: false}
2026-08-18T07:54:10Z ERROR failure during run {“process”: “storagenode”, “error”: “Failed to create storage node peer: hashstore: potential misconfiguration: missing hashtbl "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/meta/hashtbl" when log files exist in "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0"\n[tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[…].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[…].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290](http://tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[...].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[...].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290)”, “errorVerbose”: “Failed to create storage node peer: hashstore: potential misconfiguration: missing hashtbl "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/meta/hashtbl" when log files exist in "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0"\n[tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[…].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[…].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290\n\tmain.cmdRun:109\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290](http://tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[...].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[...].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290\n\tmain.cmdRun:109\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290)”}
2026-08-18T07:54:10Z FATAL Unrecoverable error {“process”: “storagenode”, “error”: “Failed to create storage node peer: hashstore: potential misconfiguration: missing hashtbl "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/meta/hashtbl" when log files exist in "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0"\n[tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[…].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[…].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290](http://tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[...].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[...].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290)”, “errorVerbose”: “Failed to create storage node peer: hashstore: potential misconfiguration: missing hashtbl "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0/meta/hashtbl" when log files exist in "config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s0"\n[tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[…].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[…].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290\n\tmain.cmdRun:109\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290](http://tstorj.io/storj/storagenode/hashstore.NewStore-range3:240\n\tstorj.io/storj/storagenode/hashstore.NewStore.NewStore.parseFiles[...].func13.func14:41\n\tpath/filepath.walkDir:311\n\tpath/filepath.walkDir:333\n\tpath/filepath.walkDir:333\n\tpath/filepath.WalkDir:401\n\tstorj.io/storj/storagenode/hashstore.NewStore.parseFiles[...].func13:27\n\tstorj.io/storj/storagenode/hashstore.NewStore:232\n\tstorj.io/storj/storagenode/hashstore.New:105\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:277\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:122\n\tstorj.io/storj/storagenode.New:607\n\tmain.cmdRun:107\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290\n\tmain.cmdRun:109\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.InitBeforeExecute.func1.2:389\n\tstorj.io/common/process.InitBeforeExecute.func1:407\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:115\n\tmain.main:34\n\truntime.main:290)”}

5

Hi, hope this can help Claude

Looking at the thread — his run actually worked. He just left the output in the wrong place.

What went wrong: write-hashtbl writes the new table as a file named hashtbl into the current working directory, not into the folder you pass as the argument. He ran ./write-hashtbl /NODE/NFS/…/s0 from wherever the binary was, so the table landed next to the binary. s0/meta/ still has no hashtbl, hence the identical error.

6

Hello,
you are right,
it seems that a file has been created in /tmp from where I ran the command :
-rw------- 1 root root 268439552 18 ago 09.44 /tmp/hashtbl

what shoud I do ? where should I copy it ? in s0/meta ?
root@mcanto:/NODE/storage/hashstore/121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6/s0# ls -l meta
totale 131092
-rw------- 1 root root 67112960 18 ago 01.27 hashtbl-0000000000000272
-rw------- 1 root root 1295 18 ago 09.54 hint-0000000000000517
-rw-r–r-- 1 root root 0 11 gen 2025 lock

thanks

7

My friend, my job here is done, I just show you the blue print, go and talk to AI :slight_smile:.

8

Hello,
sorry I didn’t get that in your link there were the procedure.
I created a new meta directory and moved the hashtable generated by the tool.
Now the node is starting with a very long list of mismatch log tail and also some " ERROR contact:amnesty failed".
Below some logs.

Let’s wait hoping that will start.

Best regards

2026-08-18T09:53:29Z WARN hashstore mismatched log tail {“process”: “storagenode”, “satellite”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “store”: “s0”, “id”: 801, “path”: “config/storage/hashstore/121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6/s0/21/log-0000000000000321-00000000”, “table”: null, “log”: [{“Key”:“Y2D2JIFWXHSFLUISY2LUQKXCE2ZS4J7PRMQEWKENVDW3RKQSTRKA”,“Offset”:1072009792,“Log”:801,“Length”:2319872,“Created”:20680,“Expires”:0},{“Key”:“JG7ANTKP2AG6EF3VSSDFDIIPEE5B4J3EIGUEK4JXBR4ADRV2PJBA”,“Offset”:1072008192,“Log”:801,“Length”:1536,“Created”:20680,“Expires”:0},{“Key”:“2FFSHEYU5NSULQDDHWFMTNMJIS2TZRFHTVE7SBYTOHT3SHLLF77A”,“Offset”:1072005312,“Log”:801,“Length”:2816,“Created”:20680,“Expires”:0},{“Key”:“IJMXFKESORN74OGHPSIVQGWLC65EITYC4M6MQB7VANINT5UU4BUQ”,“Offset”:1071742336,“Log”:801,“Length”:262912,“Created”:20680,“Expires”:0},{“Key”:“UDVJS2VFR4OF7ZT4WOUB325PDYSDCMDHJZCFF4VIYTFF6SP2FW6A”,“Offset”:1071698496,“Log”:801,“Length”:43776,“Created”:20680,“Expires”:0}]}
2026-08-18T09:53:33Z INFO contact:amnesty sending amnesty batch to satellite {“process”: “storagenode”, “satellite”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “piece_count”: 8}
2026-08-18T09:53:33Z ERROR contact:amnesty failed to get satellite URL for amnesty report {“process”: “storagenode”, “satellite”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “piece_count”: 8, “error”: “trust: satellite is untrusted”, “errorVerbose”: “trust: satellite is untrusted\n[tstorj.io/storj/storagenode/trust.init:29\n\truntime.doInit1:8103\n\truntime.doInit:8070\n\truntime.main:258](http://tstorj.io/storj/storagenode/trust.init:29\n\truntime.doInit1:8103\n\truntime.doInit:8070\n\truntime.main:258)”}
2026-08-18T09:54:02Z WARN hashstore mismatched log tail {“process”: “storagenode”, “satellite”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “store”: “s0”, “id”: 803, “path”: “config/storage/hashstore/121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6/s0/23/log-0000000000000323-00000000”, “table”: null, “log”: [{“Key”:“7QPSYNPTLGDM5JOV34ZMVXCYRHIX42UAOIQK2ZXUY3VDQBIYVVDA”,“Offset”:1072425728,“Log”:803,“Length”:1345280,“Created”:20600,“Expires”:0},{“Key”:“7QKTRHBM73NOKZGDA7F3KI4M2OTCSBABINHEJT73GHT7365HJKOQ”,“Offset”:1072423616,“Log”:803,“Length”:2048,“Created”:20600,“Expires”:0},{“Key”:“7QAUCFEDLSPDXPF5SRI56O6QESE5SIENNGYFTUPTSPPHCAUEBTSA”,“Offset”:1072422272,“Log”:803,“Length”:1280,“Created”:20600,“Expires”:0},{“Key”:“7QWOSGMHSXCYKNKSNX7QLB3JAVNFTMICJZ5PQ63ADAA2DISSVQGQ”,“Offset”:1072415808,“Log”:803,“Length”:6400,“Created”:20600,“Expires”:0},{“Key”:“7QU5D3GHRO2UHWFRKDWG5THCQJS475MSZVOZLOSTMXA2DAGWLESQ”,“Offset”:1072413440,“Log”:803,“Length”:2304,“Created”:20600,“Expires”:0}]}
2026-08-18T09:54:38Z WARN hashstore mismatched log tail {“process”: “storagenode”, “satellite”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “store”: “s0”, “id”: 548, “path”: “config/storage/hashstore/121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6/s0/24/log-0000000000000224-00000000”, “table”: null, “log”: [{“Key”:“TD43ZWBFNM3NSKCOXEJEKIQX4HQP3CWCKWMHAMM6ZVRJEBBL6P2Q”,“Offset”:1073578368,“Log”:548,“Length”:2319872,“Created”:20616,“Expires”:0},{“Key”:“SQEHVFR5JWVYWKPUDMIQBDNEJVUAEYZ2XDITIWQSDCTGO4R6TSGA”,“Offset”:1073576512,“Log”:548,“Length”:1792,“Created”:20616,“Expires”:0},{“Key”:“ZR3DTWGDKSPWZZWYN5SYVMKQ7Q6LYTLQDBQL3XXPXEGGD4CMWBOA”,“Offset”:1073574400,“Log”:548,“Length”:2048,“Created”:20616,“Expires”:0},{“Key”:“4TP3YIQQUVTSECM6ENK6GG36HLFHRKO3QK3QRCMK547T6FYVZBPA”,“Offset”:1073572544,“Log”:548,“Length”:1792,“Created”:20616,“Expires”:0},{“Key”:“D6RNSYRV4GK3KICFIVL43TM6KEU7KYQ6JOK5NZUKNYCS64JKK5DA”,“Offset”:1073569664,“Log”:548,“Length”:2816,“Created”:20616,“Expires”:0}]}

9

Warns should be fixed automatically. If node is started, then please keep it running.

10

yes, the node started during the night but there were a lot of issue with

2026-08-19T03:19:56Z ERROR blobscache piecesTotal < 0 {process: storagenode, piecesTotal: -2319360}
2026-08-19T03:19:56Z ERROR blobscache piecesContentSize < 0 {process: storagenode, piecesContentSize: -2318848}
2026-08-19T03:19:56Z ERROR blobscache satPiecesTotal < 0 {process: storagenode, satPiecesTotal: -2319360}
2026-08-19T03:19:56Z ERROR blobscache satPiecesContentSize < 0 {process: storagenode, satPiecesContentSize: -2318848}
2026-08-19T03:19:56Z ERROR blobscache piecesTotal < 0 {process: storagenode, piecesTotal: -98048}

I also find blobscache file empty and I deleted.
Before the restart I’m running a fsck because linux was complaining.

Best regards

11

You likely need also to remove the prefixes database and restart the node.

12

Hello,
I deleted (renamed) the used_space_per_prefix.db* but the issue is still there, see below logs.
Apart this the node seems to be recovered (upload & download are coming).
PS even if I stop the node with docker stop -t 300 storagenode, the db are not fully closed :
pi@mcanto:/STORJ_LOCAL $ ls -l
totale 6440
-rw-r–r-- 1 root root 36864 20 ago 10.44 bandwidth.db
-rw-r–r-- 1 root root 32768 20 ago 10.46 bandwidth.db-shm
-rw-r–r-- 1 root root 41232 20 ago 10.44 bandwidth.db-wal
-rw-r–r-- 1 pi pi 24576 20 ago 10.44 garbage_collection_filewalker_progress.db
-rw-r–r-- 1 pi pi 32768 20 ago 10.46 garbage_collection_filewalker_progress.db-shm
-rw-r–r-- 1 pi pi 32992 20 ago 10.44 garbage_collection_filewalker_progress.db-wal
-rw-r–r-- 1 pi pi 126976 20 ago 10.46 heldamount.db
-rw-r–r-- 1 pi pi 32768 20 ago 10.46 heldamount.db-shm
-rw-r–r-- 1 pi pi 4124152 20 ago 10.46 heldamount.db-wal
-rw-r–r-- 1 pi pi 16384 20 ago 10.44 info.db
-rw-r–r-- 1 pi pi 32768 20 ago 10.46 info.db-shm
-rw-r–r-- 1 pi pi 32992 20 ago 10.44 info.db-wal
drwxr-xr-x 2 pi pi 4096 18 ago 00.55 LOG
drwx------ 2 pi pi 4096 30 mar 11.25 lost+found
-rw-r–r-- 1 pi pi 24576 20 ago 10.44 notifications.db
-rw-r–r-- 1 pi pi 32768 20 ago 10.46 notifications.db-shm
-rw-r–r-- 1 pi pi 32992 20 ago 10.44 notifications.db-wal
-rw-r–r-- 1 root root 32768 20 ago 10.44 orders.db
-rw-r–r-- 1 root root 32768 20 ago 10.46 orders.db-shm
-rw-r–r-- 1 root root 32992 20 ago 10.44 orders.db-wal
-rw-r–r-- 1 root root 28672 20 ago 10.44 piece_expiration.db
-rw-r–r-- 1 root root 32768 20 ago 10.46 piece_expiration.db-shm
-rw-r–r-- 1 root root 41232 20 ago 10.44 piece_expiration.db-wal
drwxr-xr-x 6 pi pi 4096 14 feb 2026 piece_expirations
-rw-r–r-- 1 root root 24576 20 ago 10.44 pieceinfo.db
-rw-r–r-- 1 root root 32768 20 ago 10.46 pieceinfo.db-shm
-rw-r–r-- 1 root root 32992 20 ago 10.44 pieceinfo.db-wal
-rw-r–r-- 1 root root 24576 20 ago 10.44 piece_spaced_used.db
-rw-r–r-- 1 root root 32768 20 ago 10.46 piece_spaced_used.db-shm
-rw-r–r-- 1 root root 32992 20 ago 10.44 piece_spaced_used.db-wal
-rw-r–r-- 1 pi pi 24576 20 ago 10.44 pricing.db
-rw-r–r-- 1 pi pi 32768 20 ago 10.46 pricing.db-shm
-rw-r–r-- 1 pi pi 65952 20 ago 10.46 pricing.db-wal
-rw-r–r-- 1 root root 32768 20 ago 10.44 reputation.db
-rw-r–r-- 1 root root 32768 20 ago 10.46 reputation.db-shm
-rw-r–r-- 1 root root 32992 20 ago 10.44 reputation.db-wal
-rw-r–r-- 1 root root 32768 20 ago 10.44 satellites.db
-rw-r–r-- 1 root root 32768 20 ago 10.46 satellites.db-shm
-rw-r–r-- 1 root root 41232 20 ago 10.44 satellites.db-wal
-rw-r–r-- 1 pi pi 24576 20 ago 10.44 secret.db
-rw-r–r-- 1 pi pi 32768 20 ago 10.46 secret.db-shm
-rw-r–r-- 1 pi pi 32992 20 ago 10.44 secret.db-wal
-rw-r–r-- 1 root root 45056 20 ago 10.44 storage_usage.db
-rw-r–r-- 1 root root 32768 20 ago 10.46 storage_usage.db-shm
-rw-r–r-- 1 root root 32992 20 ago 10.44 storage_usage.db-wal
-rw-r–r-- 1 root root 20480 20 ago 10.44 used_serial.db
-rw-r–r-- 1 root root 32768 20 ago 10.46 used_serial.db-shm
-rw-r–r-- 1 root root 41232 20 ago 10.44 used_serial.db-wal
-rw-r–r-- 1 root root 4096 20 ago 10.44 used_space_per_prefix.db
-rw-r–r-- 1 pi pi 651264 20 ago 06.23 used_space_per_prefix.db.old
-rw-r–r-- 1 root root 32768 20 ago 10.46 used_space_per_prefix.db-shm
-rw-r–r-- 1 pi pi 32768 20 ago 10.30 used_space_per_prefix.db-shm.old
-rw-r–r-- 1 root root 74192 20 ago 10.46 used_space_per_prefix.db-wal
-rw-r–r-- 1 pi pi 53592 20 ago 10.30 used_space_per_prefix.db-wal.old

This behavior happens also in other nodes, before I remember that shm and wal was automatically removed / merged to the .db

Best regards

2026-08-20T08:46:41Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “6624DJYOH6KVCUV3SV5LDIZ62GW7HZETPHY2TTPFULGCS6TJ2YNA”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 7424, “remote_address”: “109.61.92.72:45406”}
2026-08-20T08:46:41Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -428288}
2026-08-20T08:46:41Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -427776}
2026-08-20T08:46:41Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -428288}
2026-08-20T08:46:41Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -427776}
2026-08-20T08:46:41Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -284416}
2026-08-20T08:46:41Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -283904}
2026-08-20T08:46:41Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -284416}
2026-08-20T08:46:41Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -283904}
2026-08-20T08:46:41Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -182016}
2026-08-20T08:46:41Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -181504}
2026-08-20T08:46:41Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -182016}
2026-08-20T08:46:41Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -181504}
2026-08-20T08:46:41Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -7424}
2026-08-20T08:46:41Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -6912}
2026-08-20T08:46:41Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -7424}
2026-08-20T08:46:41Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -6912}
2026-08-20T08:46:42Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -37376}
2026-08-20T08:46:42Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -36864}
2026-08-20T08:46:42Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -37376}
2026-08-20T08:46:42Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -36864}
2026-08-20T08:46:42Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “B4ARNN2ZLM4ZS3HMBASTUVQO2VIJWU5YU4DPTQDN2DCHAAW5ZP6Q”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 5888, “remote_address”: “79.127.205.226:56726”}
2026-08-20T08:46:42Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -182016}
2026-08-20T08:46:42Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -181504}
2026-08-20T08:46:42Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -182016}
2026-08-20T08:46:42Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -181504}
2026-08-20T08:46:42Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “S6ZHRKF5V46A7U2MGGNHX4DHBAYAE46TMUYFYMBUOODP7RJM7KKQ”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET_AUDIT”, “offset”: 11264, “size”: 256, “remote_address”: “79.127.205.250:35382”}
2026-08-20T08:46:42Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -10240}
2026-08-20T08:46:42Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -9728}
2026-08-20T08:46:42Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -10240}
2026-08-20T08:46:42Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -9728}
2026-08-20T08:46:42Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -182016}
2026-08-20T08:46:42Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -181504}
2026-08-20T08:46:42Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -182016}
2026-08-20T08:46:42Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -181504}
2026-08-20T08:46:42Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -37376}
2026-08-20T08:46:42Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -36864}
2026-08-20T08:46:42Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -37376}
2026-08-20T08:46:42Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -36864}

13

Perhaps it was cached. Please stop and remove the container (or stop the service) and delete this DB.

Please make sure that the scan on startup is enabled (it’s enabled by default) and run the node back.
If errors would remain after the filewalker is finished, then you would need to re-create the database piece_spaced_used.db, only deleting it will prevent the node from starting unlike deleting the prefixes database.

14

Hello,
these are my standard settings :
pi@mcanto:~/storj-docker $ cat docker-compose.yml | egrep “FILEWALKER|STARTUP”
- STORJ_PIECES_ENABLE_LAZY_FILEWALKER=true
- STORJ_STORAGE2_PIECE_SCAN_ON_STARTUP=false

I recreated piece_spaced_used.db using your link and deleted (again) used_space_per_prefix.db

The node starts and I can receive upload download .
Check the status of 10 minutes of run
pi@mcanto:~ $ grep 2026-08-21T08:2 /STORJ_LOCAL/LOG/node.log|grep “blobscache”|grep ERROR|wc -l
31084
pi@mcanto:~ $ grep 2026-08-21T08:2 /STORJ_LOCAL/LOG/node.log|grep “uploaded”|wc -l
717
pi@mcanto:~ $ grep 2026-08-21T08:2 /STORJ_LOCAL/LOG/node.log|grep “downloaded”|wc -l
2233

Any suggestion on how I can fix ?
Below a full log of few seconds

Best regards

2026-08-21T08:33:21Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “WASWWZ5BREK5YQXZHLXAGTJGW6Q34WAXHCI3NPGZVZETW36QRD2Q”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 3328, “remote_address”: “109.61.92.70:48316”}
2026-08-21T08:33:21Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -238848}
2026-08-21T08:33:21Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -238336}
2026-08-21T08:33:21Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -238848}
2026-08-21T08:33:21Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -238336}
2026-08-21T08:33:21Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -5376}
2026-08-21T08:33:21Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -4864}
2026-08-21T08:33:21Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -5376}
2026-08-21T08:33:21Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -4864}
2026-08-21T08:33:21Z INFO piecestore uploaded {“process”: “storagenode”, “piece_id”: “KJDJBMTLNLKXEP5SUWB7BMZ6EMNQ3JWPKEFPT4HWOC2QWWDQYCEQ”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “PUT”, “remote_address”: “95.173.206.163:46190”, “size”: 2319360}
2026-08-21T08:33:22Z INFO piecestore uploaded {“process”: “storagenode”, “piece_id”: “Q37KCFYGUBCOMSUBAZSDNTXKEL4HXTGV3HXVQKZYXZX5SVOMTY4A”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “PUT”, “remote_address”: “79.127.219.43:55834”, “size”: 2174464}
2026-08-21T08:33:22Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -1024}
2026-08-21T08:33:22Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -512}
2026-08-21T08:33:22Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -1024}
2026-08-21T08:33:22Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -512}
2026-08-21T08:33:22Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -7424}
2026-08-21T08:33:22Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -6912}
2026-08-21T08:33:22Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -7424}
2026-08-21T08:33:22Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -6912}
2026-08-21T08:33:22Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -1024}
2026-08-21T08:33:22Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -512}
2026-08-21T08:33:22Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -1024}
2026-08-21T08:33:22Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -512}

15

This need to be changed to true to fix a discrepancy.

16

Hi Alexey,
it’s running since two days but I don’t see any improvement.
Below my configuration and also the dashboard showing wrong info

pi@mcanto:~ $ df -h /NODE
File system Dim. Usati Dispon. Uso% Montato su
/dev/mapper/vgSTORJ-lvSTORJ 11T 9,0T 1,2T 89% /NODE

pi@mcanto:~ $ docker exec storagenode /app/dashboard.sh

Storage Node Dashboard ( Node Version: v1.155.9 )

======================

ID 12mgrK7bsPX5EJwjBqKkf7yR3R5GhDnJ5bgj3qy2Cjjay1KadAa
Status ONLINE
Uptime 49h53m39s

               Available          Used        Egress      Ingress
 Bandwidth           N/A     202.91 GB     105.85 GB     97.06 GB (since Aug 1)
      Disk           0 B       4.27 TB

Internal 127.0.0.1:7778
External mcanto.ddns.net:28967

pi@mcanto:~ $ cd storj-docker
pi@mcanto:~/storj-docker $ cat docker-compose.yml | egrep “false|true”
- STORJ_PIECES_ENABLE_LAZY_FILEWALKER=false
- STORJ_STORAGE2_PIECE_SCAN_ON_STARTUP=true
- STORJ_STORAGE2_HASHSTORE_WRITE_TO_NEW=false
- STORJ_STORAGE2_HASHSTORE_TTL_TO_NEW=false
- STORJ_STORAGE2_HASHSTORE_PASSIVE_MIGRATE=false

below also last logs

How can I be sure that it’s really fixing ?
Any suggestion ?
Thank you very much in advance

Best regards

2026-08-24T13:12:41Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -182016}
2026-08-24T13:12:41Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -181504}
2026-08-24T13:12:41Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -182016}
2026-08-24T13:12:41Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -181504}
2026-08-24T13:12:41Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “PWQVK6LVY5M7X3AE2OJ6CLAHXZ36HDJTC3XPDRLBETZFHQ2BJUMA”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 6656, “remote_address”: “109.61.92.77:54780”}
2026-08-24T13:12:41Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -182016}
2026-08-24T13:12:41Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -181504}
2026-08-24T13:12:41Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -182016}
2026-08-24T13:12:41Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -181504}
2026-08-24T13:12:41Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “HDK7J5Q5PS7BSEDFVMPBKHNP5KP2LVGDN4JGFENP7ET4G3BWYKFA”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 6400, “remote_address”: “109.61.92.86:59204”}
2026-08-24T13:12:41Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -182016}
2026-08-24T13:12:41Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -181504}
2026-08-24T13:12:41Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -182016}
2026-08-24T13:12:41Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -181504}
2026-08-24T13:12:41Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “RYDOIO2I4WHTGJOOGS3HY6EHS7JXHUGKS4VB3VRJWHXARUL6KBWA”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 6400, “remote_address”: “109.61.92.73:51750”}
2026-08-24T13:12:41Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -14336}
2026-08-24T13:12:41Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -13824}
2026-08-24T13:12:41Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -14336}
2026-08-24T13:12:41Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -13824}
2026-08-24T13:12:41Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “4FY7XJCBPBKN7WWBA3SHD7SUI2YLAN2WKMKXUCZPMR3NNKLULIFA”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 6144, “remote_address”: “109.61.92.70:59518”}
2026-08-24T13:12:41Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “5ZV6NV5PTT2S5QM5SQLU2T37I63YSRELHKKPRQSAUELUTU3DUL6A”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 3584, “remote_address”: “109.61.92.71:44458”}
2026-08-24T13:12:41Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “T2FKLHIRMOMO3YOE7SVNY5EPFLRSVXEDT3THUQUHV4JGJSXRM6EQ”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 3328, “remote_address”: “109.61.92.86:59216”}
2026-08-24T13:12:41Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “LNH7OIBJNK2RXWBXJI4OXVP7AWM37D4VKRW6DADTP6IGYL5YHD6A”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 0, “size”: 181248, “remote_address”: “109.61.92.73:51154”}

17

You need to stop and remove the container, re-create the database piece_spaced_used.db, stop and remove the container again, delete prefixes database, make sure that STORJ_STORAGE2_PIECE_SCAN_ON_STARTUP=true, and start the node. When the scan would be completed, these errors should gone.

As alternative, you can enable migration to hashstore for all satellites and restart the node to do not have old pains of piecestore.

18

Hello,
it seems fixed,
no more errors.
Just recreating the piece_spaced_used.db fixed. I already did it and didn’t worked , maybe I did something wrong.

Now I’m running the full scan because the space reported by the dashboard is almost half of the used.

Storage Node Dashboard ( Node Version: v1.155.9 )

======================

ID 12mgrK7bsPX5EJwjBqKkf7yR3R5GhDnJ5bgj3qy2Cjjay1KadAa
Status ONLINE
Uptime 12m46s

               Available          Used        Egress      Ingress
 Bandwidth           N/A     223.60 GB     126.32 GB     97.28 GB (since Aug 1)
      Disk       1.20 TB       4.72 TB

Internal 127.0.0.1:7778
External mcanto.ddns.net:28967
^Cpi@mcanto:~ $ df -h /NODE
File system Dim. Usati Dispon. Uso% Montato su
/dev/mapper/vgSTORJ-lvSTORJ 11T 9,0T 1,2T 89% /NODE

thank you very much

19

I appear to be getting hashstore mismatched log tail

like this and like you have mentioned @mcanto73 … - everytime if i restart the node , it does this mismatched log tail … even after i leave for a long time and it comes back online with dashboard… even after that if restrat - it does this mismatched log tail all over it seems …

did you experience this?

20

@Alexey

unfortunately after few time the errors come back :frowning:
Pieces scan at startup is running and the node is up since 26 hours.
Also storage used seems to increase

see logs below

Best regards

2026-08-26T10:32:50Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “BO7KN76LEWQYTAUC6DUNMJL4BHYRT2K2T5KW5NJ73L3CDUEW377Q”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 5376, “size”: 256, “remote_address”: “79.127.219.33:42358”}
2026-08-26T10:32:50Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -2174976}
2026-08-26T10:32:50Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -2174464}
2026-08-26T10:32:50Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -2174976}
2026-08-26T10:32:50Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -2174464}
2026-08-26T10:32:50Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “O26TKDTP3KSUWVETYGTL3464BYLPACVBJJJCY36XTPSW7XW23KNQ”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET_REPAIR”, “offset”: 0, “size”: 181504, “remote_address”: “79.127.205.242:53219”}
2026-08-26T10:32:50Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “5J6NRVC2MOJ62KFUDQ2KGFIBVM3VGFMMV25KEUOVV4DZ5QKW7XRQ”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 503040, “size”: 256, “remote_address”: “79.127.219.42:33872”}
2026-08-26T10:32:50Z ERROR blobscache piecesTotal < 0 {“process”: “storagenode”, “piecesTotal”: -182016}
2026-08-26T10:32:50Z ERROR blobscache piecesContentSize < 0 {“process”: “storagenode”, “piecesContentSize”: -181504}
2026-08-26T10:32:50Z ERROR blobscache satPiecesTotal < 0 {“process”: “storagenode”, “satPiecesTotal”: -182016}
2026-08-26T10:32:50Z ERROR blobscache satPiecesContentSize < 0 {“process”: “storagenode”, “satPiecesContentSize”: -181504}
2026-08-26T10:32:50Z INFO piecestore downloaded {“process”: “storagenode”, “piece_id”: “BO7KN76LEWQYTAUC6DUNMJL4BHYRT2K2T5KW5NJ73L3CDUEW377Q”, “satellite_id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “action”: “GET”, “offset”: 127232, “size”: 512, “remote_address”: “79.127.219.34:52908”}
^C
pi@mcanto:~ $ grep 2026-08-26T /STORJ_LOCAL/LOG/node.log|grep blobscache|wc -l
3941290
pi@mcanto:~ $ docker exec storagenode /app/dashboard.sh

Storage Node Dashboard ( Node Version: v1.155.9 )

======================

ID 12mgrK7bsPX5EJwjBqKkf7yR3R5GhDnJ5bgj3qy2Cjjay1KadAa
Status ONLINE
Uptime 26h35m40s

               Available          Used        Egress       Ingress
 Bandwidth           N/A     290.10 GB     163.18 GB     126.92 GB (since Aug 1)
      Disk       0.72 TB       5.30 TB

Internal 127.0.0.1:7778
External mcanto.ddns.net:28967

pi@mcanto:~ $ df -h /NODE
File system Dim. Usati Dispon. Uso% Montato su
/dev/mapper/vgSTORJ-lvSTORJ 11T 9,0T 1,2T 90% /NODE