Hello.
Could someone pass some ideas on how to i deal with this error?
Node does not start. No other errors visible. Tried to restart all machine, did not helped.
Thank you
2025-01-31T19:22:34Z INFO server kernel support for server-side tcp fast open remains disabled. {“Process”: “storagenode”}
2025-01-31T19:22:34Z INFO server enable with: sysctl -w net.ipv4.tcp_fastopen=3 {“Process”: “storagenode”}
2025-01-31T19:22:34Z ERROR failure during run {“Process”: “storagenode”, “error”: “Failed to create storage node peer: hashstore: unable to flock: hashstore: resource temporarily unavailable\n\tstorj.io/storj/storagenode/hashstore.NewStore:105\n\tstorj.io/storj/storagenode/hashstore.New:85\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:271\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:105\n\tstorj.io/storj/storagenode.New:618\n\tmain.cmdRun:84\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:392\n\tstorj.io/common/process.cleanup.func1:410\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tmain.main:34\n\truntime.main:272”, “errorVerbose”: “Failed to create storage node peer: hashstore: unable to flock: hashstore: resource temporarily unavailable\n\tstorj.io/storj/storagenode/hashstore.NewStore:105\n\tstorj.io/storj/storagenode/hashstore.New:85\n\tstorj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:271\n\tstorj.io/storj/storagenode/piecestore.NewHashStoreBackend:105\n\tstorj.io/storj/storagenode.New:618\n\tmain.cmdRun:84\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:392\n\tstorj.io/common/process.cleanup.func1:410\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tmain.main:34\n\truntime.main:272\n\tmain.cmdRun:86\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:392\n\tstorj.io/common/process.cleanup.func1:410\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tmain.main:34\n\truntime.main:272”}
Error: Failed to create storage node peer: hashstore: unable to flock: hashstore: resource temporarily unavailable storj.io/storj/storagenode/hashstore.NewStore:105 storj.io/storj/storagenode/hashstore.New:85 storj.io/storj/storagenode/piecestore.(*HashStoreBackend).getDB:271 storj.io/storj/storagenode/piecestore.NewHashStoreBackend:105 storj.io/storj/storagenode.New:618
main.cmdRun:84
main.newRunCmd.func1:33 storj.io/common/process.cleanup.func1.4:392 storj.io/common/process.cleanup.func1:410 github.com/spf13/cobra.(*Command).execute:983 github.com/spf13/cobra.(*Command).ExecuteC:1115 github.com/spf13/cobra.(*Command).Execute:1039 storj.io/common/process.ExecWithCustomOptions:112
main.main:34
runtime.main:272
2025-01-31 19:22:34,343 WARN stopped: storagenode (exit status 1)
2025-01-31 19:22:34,344 WARN stopped: processes-exit-eventlistener (terminated by SIGTERM)
That error indicates some other process already has a lock on the same hashstore. Is it possible you have multiple nodes configured to use the same storage directory?
It also happened to one of my nodes last night, which was working fine, after updating to 1.121.2, it gave the hashstore flock error. I haven’t activated the migration, I’m still on piecestore. I threw away the bin, piecestore, filestatcache folders, removed and recreated the node and it finally started
Unfortunately I didn’t save the error, I was busy restarting the node, but I think that was it. In my opinion it has to do with the badger cache, but I’m not an expert, I’ll throw it out there
2025-04-12T11:37:09Z ERROR failure during run {"Process": "storagenode", "error": "Error opening database on storagenode: database: orders couldn't be read (\"config/storage/orders.db\"): stat config/storage/orders.db: input/output error\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:359\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:337\n\tstorj.io/storj/storagenode/storagenodedb.OpenExisting:302\n\tmain.cmdRun:67\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:392\n\tstorj.io/common/process.cleanup.func1:410\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tmain.main:34\n\truntime.main:272", "errorVerbose": "Error opening database on storagenode: database: orders couldn't be read (\"config/storage/orders.db\"): stat config/storage/orders.db: input/output error\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:359\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:337\n\tstorj.io/storj/storagenode/storagenodedb.OpenExisting:302\n\tmain.cmdRun:67\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:392\n\tstorj.io/common/process.cleanup.func1:410\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tmain.main:34\n\truntime.main:272\n\tmain.cmdRun:69\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:392\n\tstorj.io/common/process.cleanup.func1:410\n\tgithub.com/spf13/cobra.(*Command).execute:985\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\n\tgithub.com/spf13/cobra.(*Command).Execute:1041\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tmain.main:34\n\truntime.main:272"}
I threw away the bin, piecestore, filestatcache folders, removed and recreated the node but don’t work
the database are ok
