Node crash after power failure

Node Operators troubleshooting
,
1

Node unable to start after power failure.

{"log":"2025-05-27 06:35:42,417 INFO spawned: 'storagenode' with pid 49\n","stream":"stdout","time":"2025-05-27T06:35:42.418024165Z"} {"log":"2025-05-27 06:35:42,418 INFO spawned: 'storagenode-updater' with pid 50\n","stream":"stdout","time":"2025-05-27T06:35:42.419117477Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Configuration loaded\u0009{\"Process\": \"storagenode-updater\", \"Location\": \"/app/config/config.yaml\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428822907Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"server.private-address\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428839137Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"console.address\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428842211Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"server.address\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428844748Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"healthcheck.enabled\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428847173Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"operator.wallet-features\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428849663Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"contact.external-address\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428852046Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"storage.allocated-disk-space\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428854414Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"operator.email\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428856766Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"healthcheck.details\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428859085Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Invalid configuration file key\u0009{\"Process\": \"storagenode-updater\", \"Key\": \"operator.wallet\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428864429Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Anonymized tracing enabled\u0009{\"Process\": \"storagenode-updater\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.428872859Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Running on version\u0009{\"Process\": \"storagenode-updater\", \"Service\": \"storagenode-updater\", \"Version\": \"v1.127.1\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.429532763Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Downloading versions.\u0009{\"Process\": \"storagenode-updater\", \"Server Address\": \"https://version.storj.io\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.42989658Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Configuration loaded\u0009{\"Process\": \"storagenode\", \"Location\": \"/app/config/config.yaml\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.446915177Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Anonymized tracing enabled\u0009{\"Process\": \"storagenode\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.446971845Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Operator email\u0009{\"Process\": \"storagenode\", \"Address\": \"mr.arfathkhalid@gmail.com\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.44730414Z"} {"log":"2025-05-27T06:35:42Z\u0009INFO\u0009Operator wallet\u0009{\"Process\": \"storagenode\", \"Address\": \"0xC83a4020afe013e08ef26ffeEE277814005664Ad\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.447311002Z"} {"log":"2025-05-27T06:35:42Z\u0009ERROR\u0009failure during run\u0009{\"Process\": \"storagenode\", \"error\": \"Error opening revocation database: revocation database: boltdb: read-only file system\\n\\tstorj.io/storj/private/kvstore/boltdb.New:46\\n\\tstorj.io/storj/private/revocation.openDBBolt:52\\n\\tstorj.io/storj/private/revocation.OpenDB:35\\n\\tstorj.io/storj/private/revocation.OpenDBFromCfg:23\\n\\tmain.cmdRun:76\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\", \"errorVerbose\": \"Error opening revocation database: revocation database: boltdb: read-only file system\\n\\tstorj.io/storj/private/kvstore/boltdb.New:46\\n\\tstorj.io/storj/private/revocation.openDBBolt:52\\n\\tstorj.io/storj/private/revocation.OpenDB:35\\n\\tstorj.io/storj/private/revocation.OpenDBFromCfg:23\\n\\tmain.cmdRun:76\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\\n\\tmain.cmdRun:78\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\"}\n","stream":"stdout","time":"2025-05-27T06:35:42.490537389Z"} {"log":"Error: Error opening revocation database: revocation database: boltdb: read-only file system\n","stream":"stdout","time":"2025-05-27T06:35:42.490554389Z"} {"log":"\u0009storj.io/storj/private/kvstore/boltdb.New:46\n","stream":"stdout","time":"2025-05-27T06:35:42.49055721Z"} {"log":"\u0009storj.io/storj/private/revocation.openDBBolt:52\n","stream":"stdout","time":"2025-05-27T06:35:42.49055951Z"} {"log":"\u0009storj.io/storj/private/revocation.OpenDB:35\n","stream":"stdout","time":"2025-05-27T06:35:42.490561892Z"} {"log":"\u0009storj.io/storj/private/revocation.OpenDBFromCfg:23\n","stream":"stdout","time":"2025-05-27T06:35:42.49056518Z"} {"log":"\u0009main.cmdRun:76\n","stream":"stdout","time":"2025-05-27T06:35:42.490567402Z"} {"log":"\u0009main.newRunCmd.func1:33\n","stream":"stdout","time":"2025-05-27T06:35:42.49056958Z"} {"log":"\u0009storj.io/common/process.cleanup.func1.4:392\n","stream":"stdout","time":"2025-05-27T06:35:42.49057172Z"} {"log":"\u0009storj.io/common/process.cleanup.func1:410\n","stream":"stdout","time":"2025-05-27T06:35:42.49057386Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).execute:985\n","stream":"stdout","time":"2025-05-27T06:35:42.490575975Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).ExecuteC:1117\n","stream":"stdout","time":"2025-05-27T06:35:42.490584098Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).Execute:1041\n","stream":"stdout","time":"2025-05-27T06:35:42.490586512Z"} {"log":"\u0009storj.io/common/process.ExecWithCustomOptions:112\n","stream":"stdout","time":"2025-05-27T06:35:42.490588657Z"} {"log":"\u0009main.main:34\n","stream":"stdout","time":"2025-05-27T06:35:42.490590809Z"} {"log":"\u0009runtime.main:283\n","stream":"stdout","time":"2025-05-27T06:35:42.49059302Z"} {"log":"2025-05-27 06:35:42,493 WARN exited: storagenode (exit status 1; not expected)\n","stream":"stdout","time":"2025-05-27T06:35:42.493636005Z"} {"log":"2025-05-27T06:35:43Z\u0009INFO\u0009Current binary version\u0009{\"Process\": \"storagenode-updater\", \"Service\": \"storagenode\", \"Version\": \"v1.127.1\"}\n","stream":"stdout","time":"2025-05-27T06:35:43.264322799Z"} {"log":"2025-05-27T06:35:43Z\u0009INFO\u0009New version is being rolled out but hasn't made it to this node yet\u0009{\"Process\": \"storagenode-updater\", \"Service\": \"storagenode\"}\n","stream":"stdout","time":"2025-05-27T06:35:43.264339706Z"} {"log":"2025-05-27T06:35:43Z\u0009INFO\u0009Current binary version\u0009{\"Process\": \"storagenode-updater\", \"Service\": \"storagenode-updater\", \"Version\": \"v1.127.1\"}\n","stream":"stdout","time":"2025-05-27T06:35:43.27191359Z"} {"log":"2025-05-27T06:35:43Z\u0009INFO\u0009New version is being rolled out but hasn't made it to this node yet\u0009{\"Process\": \"storagenode-updater\", \"Service\": \"storagenode-updater\"}\n","stream":"stdout","time":"2025-05-27T06:35:43.271928803Z"} {"log":"2025-05-27 06:35:44,272 INFO success: processes-exit-eventlistener entered RUNNING state, process has stayed up for \u003e than 1 seconds (startsecs)\n","stream":"stdout","time":"2025-05-27T06:35:44.272436709Z"} {"log":"2025-05-27 06:35:44,273 INFO spawned: 'storagenode' with pid 82\n","stream":"stdout","time":"2025-05-27T06:35:44.273398486Z"} {"log":"2025-05-27 06:35:44,273 INFO success: storagenode-updater entered RUNNING state, process has stayed up for \u003e than 1 seconds (startsecs)\n","stream":"stdout","time":"2025-05-27T06:35:44.273546209Z"} {"log":"2025-05-27T06:35:44Z\u0009INFO\u0009Configuration loaded\u0009{\"Process\": \"storagenode\", \"Location\": \"/app/config/config.yaml\"}\n","stream":"stdout","time":"2025-05-27T06:35:44.300122007Z"} {"log":"2025-05-27T06:35:44Z\u0009INFO\u0009Anonymized tracing enabled\u0009{\"Process\": \"storagenode\"}\n","stream":"stdout","time":"2025-05-27T06:35:44.30014512Z"} {"log":"2025-05-27T06:35:44Z\u0009INFO\u0009Operator email\u0009{\"Process\": \"storagenode\", \"Address\": \"mr.arfathkhalid@gmail.com\"}\n","stream":"stdout","time":"2025-05-27T06:35:44.300434088Z"} {"log":"2025-05-27T06:35:44Z\u0009INFO\u0009Operator wallet\u0009{\"Process\": \"storagenode\", \"Address\": \"0xC83a4020afe013e08ef26ffeEE277814005664Ad\"}\n","stream":"stdout","time":"2025-05-27T06:35:44.300508254Z"} {"log":"2025-05-27T06:35:44Z\u0009ERROR\u0009failure during run\u0009{\"Process\": \"storagenode\", \"error\": \"Error opening revocation database: revocation database: boltdb: read-only file system\\n\\tstorj.io/storj/private/kvstore/boltdb.New:46\\n\\tstorj.io/storj/private/revocation.openDBBolt:52\\n\\tstorj.io/storj/private/revocation.OpenDB:35\\n\\tstorj.io/storj/private/revocation.OpenDBFromCfg:23\\n\\tmain.cmdRun:76\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\", \"errorVerbose\": \"Error opening revocation database: revocation database: boltdb: read-only file system\\n\\tstorj.io/storj/private/kvstore/boltdb.New:46\\n\\tstorj.io/storj/private/revocation.openDBBolt:52\\n\\tstorj.io/storj/private/revocation.OpenDB:35\\n\\tstorj.io/storj/private/revocation.OpenDBFromCfg:23\\n\\tmain.cmdRun:76\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\\n\\tmain.cmdRun:78\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\"}\n","stream":"stdout","time":"2025-05-27T06:35:44.333890887Z"} {"log":"Error: Error opening revocation database: revocation database: boltdb: read-only file system\n","stream":"stdout","time":"2025-05-27T06:35:44.333914344Z"} {"log":"\u0009storj.io/storj/private/kvstore/boltdb.New:46\n","stream":"stdout","time":"2025-05-27T06:35:44.333917367Z"} {"log":"\u0009storj.io/storj/private/revocation.openDBBolt:52\n","stream":"stdout","time":"2025-05-27T06:35:44.333919666Z"} {"log":"\u0009storj.io/storj/private/revocation.OpenDB:35\n","stream":"stdout","time":"2025-05-27T06:35:44.333921825Z"} {"log":"\u0009storj.io/storj/private/revocation.OpenDBFromCfg:23\n","stream":"stdout","time":"2025-05-27T06:35:44.333924537Z"} {"log":"\u0009main.cmdRun:76\n","stream":"stdout","time":"2025-05-27T06:35:44.333926727Z"} {"log":"\u0009main.newRunCmd.func1:33\n","stream":"stdout","time":"2025-05-27T06:35:44.333928888Z"} {"log":"\u0009storj.io/common/process.cleanup.func1.4:392\n","stream":"stdout","time":"2025-05-27T06:35:44.33393103Z"} {"log":"\u0009storj.io/common/process.cleanup.func1:410\n","stream":"stdout","time":"2025-05-27T06:35:44.333933249Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).execute:985\n","stream":"stdout","time":"2025-05-27T06:35:44.333935689Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).ExecuteC:1117\n","stream":"stdout","time":"2025-05-27T06:35:44.33393781Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).Execute:1041\n","stream":"stdout","time":"2025-05-27T06:35:44.333939941Z"} {"log":"\u0009storj.io/common/process.ExecWithCustomOptions:112\n","stream":"stdout","time":"2025-05-27T06:35:44.333942076Z"} {"log":"\u0009main.main:34\n","stream":"stdout","time":"2025-05-27T06:35:44.333944228Z"} {"log":"\u0009runtime.main:283\n","stream":"stdout","time":"2025-05-27T06:35:44.333946368Z"} {"log":"2025-05-27 06:35:44,335 WARN exited: storagenode (exit status 1; not expected)\n","stream":"stdout","time":"2025-05-27T06:35:44.335726688Z"} {"log":"2025-05-27 06:35:46,338 INFO spawned: 'storagenode' with pid 91\n","stream":"stdout","time":"2025-05-27T06:35:46.338492768Z"} {"log":"2025-05-27T06:35:46Z\u0009INFO\u0009Configuration loaded\u0009{\"Process\": \"storagenode\", \"Location\": \"/app/config/config.yaml\"}\n","stream":"stdout","time":"2025-05-27T06:35:46.365170921Z"} {"log":"2025-05-27T06:35:46Z\u0009INFO\u0009Anonymized tracing enabled\u0009{\"Process\": \"storagenode\"}\n","stream":"stdout","time":"2025-05-27T06:35:46.365233742Z"} {"log":"2025-05-27T06:35:46Z\u0009INFO\u0009Operator email\u0009{\"Process\": \"storagenode\", \"Address\": \"mr.arfathkhalid@gmail.com\"}\n","stream":"stdout","time":"2025-05-27T06:35:46.365576648Z"} {"log":"2025-05-27T06:35:46Z\u0009INFO\u0009Operator wallet\u0009{\"Process\": \"storagenode\", \"Address\": \"0xC83a4020afe013e08ef26ffeEE277814005664Ad\"}\n","stream":"stdout","time":"2025-05-27T06:35:46.36558471Z"} {"log":"2025-05-27T06:35:46Z\u0009ERROR\u0009failure during run\u0009{\"Process\": \"storagenode\", \"error\": \"Error opening revocation database: revocation database: boltdb: read-only file system\\n\\tstorj.io/storj/private/kvstore/boltdb.New:46\\n\\tstorj.io/storj/private/revocation.openDBBolt:52\\n\\tstorj.io/storj/private/revocation.OpenDB:35\\n\\tstorj.io/storj/private/revocation.OpenDBFromCfg:23\\n\\tmain.cmdRun:76\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\", \"errorVerbose\": \"Error opening revocation database: revocation database: boltdb: read-only file system\\n\\tstorj.io/storj/private/kvstore/boltdb.New:46\\n\\tstorj.io/storj/private/revocation.openDBBolt:52\\n\\tstorj.io/storj/private/revocation.OpenDB:35\\n\\tstorj.io/storj/private/revocation.OpenDBFromCfg:23\\n\\tmain.cmdRun:76\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\\n\\tmain.cmdRun:78\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\"}\n","stream":"stdout","time":"2025-05-27T06:35:46.408639743Z"} {"log":"Error: Error opening revocation database: revocation database: boltdb: read-only file system\n","stream":"stdout","time":"2025-05-27T06:35:46.408662115Z"} {"log":"\u0009storj.io/storj/private/kvstore/boltdb.New:46\n","stream":"stdout","time":"2025-05-27T06:35:46.408665128Z"} {"log":"\u0009storj.io/storj/private/revocation.openDBBolt:52\n","stream":"stdout","time":"2025-05-27T06:35:46.408667388Z"} {"log":"\u0009storj.io/storj/private/revocation.OpenDB:35\n","stream":"stdout","time":"2025-05-27T06:35:46.408669547Z"} {"log":"\u0009storj.io/storj/private/revocation.OpenDBFromCfg:23\n","stream":"stdout","time":"2025-05-27T06:35:46.408672349Z"} {"log":"\u0009main.cmdRun:76\n","stream":"stdout","time":"2025-05-27T06:35:46.408674569Z"} {"log":"\u0009main.newRunCmd.func1:33\n","stream":"stdout","time":"2025-05-27T06:35:46.408676738Z"} {"log":"\u0009storj.io/common/process.cleanup.func1.4:392\n","stream":"stdout","time":"2025-05-27T06:35:46.408678873Z"} {"log":"\u0009storj.io/common/process.cleanup.func1:410\n","stream":"stdout","time":"2025-05-27T06:35:46.40868103Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).execute:985\n","stream":"stdout","time":"2025-05-27T06:35:46.408683193Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).ExecuteC:1117\n","stream":"stdout","time":"2025-05-27T06:35:46.408685328Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).Execute:1041\n","stream":"stdout","time":"2025-05-27T06:35:46.408687459Z"} {"log":"\u0009storj.io/common/process.ExecWithCustomOptions:112\n","stream":"stdout","time":"2025-05-27T06:35:46.408689583Z"} {"log":"\u0009main.main:34\n","stream":"stdout","time":"2025-05-27T06:35:46.408691769Z"} {"log":"\u0009runtime.main:283\n","stream":"stdout","time":"2025-05-27T06:35:46.408693918Z"} {"log":"2025-05-27 06:35:46,410 WARN exited: storagenode (exit status 1; not expected)\n","stream":"stdout","time":"2025-05-27T06:35:46.410427946Z"} {"log":"2025-05-27 06:35:49,413 INFO spawned: 'storagenode' with pid 100\n","stream":"stdout","time":"2025-05-27T06:35:49.41377884Z"} {"log":"2025-05-27T06:35:49Z\u0009INFO\u0009Configuration loaded\u0009{\"Process\": \"storagenode\", \"Location\": \"/app/config/config.yaml\"}\n","stream":"stdout","time":"2025-05-27T06:35:49.440951746Z"} {"log":"2025-05-27T06:35:49Z\u0009INFO\u0009Anonymized tracing enabled\u0009{\"Process\": \"storagenode\"}\n","stream":"stdout","time":"2025-05-27T06:35:49.440995189Z"} {"log":"2025-05-27T06:35:49Z\u0009INFO\u0009Operator email\u0009{\"Process\": \"storagenode\", \"Address\": \"mr.arfathkhalid@gmail.com\"}\n","stream":"stdout","time":"2025-05-27T06:35:49.441299125Z"} {"log":"2025-05-27T06:35:49Z\u0009INFO\u0009Operator wallet\u0009{\"Process\": \"storagenode\", \"Address\": \"0xC83a4020afe013e08ef26ffeEE277814005664Ad\"}\n","stream":"stdout","time":"2025-05-27T06:35:49.441392342Z"} {"log":"2025-05-27T06:35:49Z\u0009ERROR\u0009failure during run\u0009{\"Process\": \"storagenode\", \"error\": \"Error opening revocation database: revocation database: boltdb: read-only file system\\n\\tstorj.io/storj/private/kvstore/boltdb.New:46\\n\\tstorj.io/storj/private/revocation.openDBBolt:52\\n\\tstorj.io/storj/private/revocation.OpenDB:35\\n\\tstorj.io/storj/private/revocation.OpenDBFromCfg:23\\n\\tmain.cmdRun:76\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\", \"errorVerbose\": \"Error opening revocation database: revocation database: boltdb: read-only file system\\n\\tstorj.io/storj/private/kvstore/boltdb.New:46\\n\\tstorj.io/storj/private/revocation.openDBBolt:52\\n\\tstorj.io/storj/private/revocation.OpenDB:35\\n\\tstorj.io/storj/private/revocation.OpenDBFromCfg:23\\n\\tmain.cmdRun:76\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\\n\\tmain.cmdRun:78\\n\\tmain.newRunCmd.func1:33\\n\\tstorj.io/common/process.cleanup.func1.4:392\\n\\tstorj.io/common/process.cleanup.func1:410\\n\\tgithub.com/spf13/cobra.(*Command).execute:985\\n\\tgithub.com/spf13/cobra.(*Command).ExecuteC:1117\\n\\tgithub.com/spf13/cobra.(*Command).Execute:1041\\n\\tstorj.io/common/process.ExecWithCustomOptions:112\\n\\tmain.main:34\\n\\truntime.main:283\"}\n","stream":"stdout","time":"2025-05-27T06:35:49.483163127Z"} {"log":"Error: Error opening revocation database: revocation database: boltdb: read-only file system\n","stream":"stdout","time":"2025-05-27T06:35:49.483188004Z"} {"log":"\u0009storj.io/storj/private/kvstore/boltdb.New:46\n","stream":"stdout","time":"2025-05-27T06:35:49.48319498Z"} {"log":"\u0009storj.io/storj/private/revocation.openDBBolt:52\n","stream":"stdout","time":"2025-05-27T06:35:49.483197738Z"} {"log":"\u0009storj.io/storj/private/revocation.OpenDB:35\n","stream":"stdout","time":"2025-05-27T06:35:49.483200117Z"} {"log":"\u0009storj.io/storj/private/revocation.OpenDBFromCfg:23\n","stream":"stdout","time":"2025-05-27T06:35:49.483203361Z"} {"log":"\u0009main.cmdRun:76\n","stream":"stdout","time":"2025-05-27T06:35:49.483205576Z"} {"log":"\u0009main.newRunCmd.func1:33\n","stream":"stdout","time":"2025-05-27T06:35:49.483207744Z"} {"log":"\u0009storj.io/common/process.cleanup.func1.4:392\n","stream":"stdout","time":"2025-05-27T06:35:49.483209894Z"} {"log":"\u0009storj.io/common/process.cleanup.func1:410\n","stream":"stdout","time":"2025-05-27T06:35:49.483212024Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).execute:985\n","stream":"stdout","time":"2025-05-27T06:35:49.483214138Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).ExecuteC:1117\n","stream":"stdout","time":"2025-05-27T06:35:49.483216258Z"} {"log":"\u0009github.com/spf13/cobra.(*Command).Execute:1041\n","stream":"stdout","time":"2025-05-27T06:35:49.483218384Z"} {"log":"\u0009storj.io/common/process.ExecWithCustomOptions:112\n","stream":"stdout","time":"2025-05-27T06:35:49.483220494Z"} {"log":"\u0009main.main:34\n","stream":"stdout","time":"2025-05-27T06:35:49.48322264Z"} {"log":"\u0009runtime.main:283\n","stream":"stdout","time":"2025-05-27T06:35:49.483230364Z"} {"log":"2025-05-27 06:35:49,484 WARN exited: storagenode (exit status 1; not expected)\n","stream":"stdout","time":"2025-05-27T06:35:49.484976132Z"} {"log":"2025-05-27 06:35:50,485 INFO gave up: storagenode entered FATAL state, too many start retries too quickly\n","stream":"stdout","time":"2025-05-27T06:35:50.48561044Z"} {"log":"2025-05-27 06:35:51,487 WARN received SIGTERM indicating exit request\n","stream":"stdout","time":"2025-05-27T06:35:51.487173832Z"} {"log":"2025-05-27 06:35:51,487 INFO waiting for processes-exit-eventlistener, storagenode-updater to die\n","stream":"stdout","time":"2025-05-27T06:35:51.487254957Z"} {"log":"2025-05-27T06:35:51Z\u0009INFO\u0009Got a signal from the OS: \"terminated\"\u0009{\"Process\": \"storagenode-updater\"}\n","stream":"stdout","time":"2025-05-27T06:35:51.487327027Z"} {"log":"2025-05-27 06:35:51,487 WARN received SIGQUIT indicating exit request\n","stream":"stdout","time":"2025-05-27T06:35:51.487359634Z"} {"log":"2025-05-27 06:35:51,488 INFO stopped: storagenode-updater (exit status 0)\n","stream":"stdout","time":"2025-05-27T06:35:51.488354484Z"} {"log":"2025-05-27 06:35:52,489 WARN stopped: processes-exit-eventlistener (terminated by SIGTERM)\n","stream":"stdout","time":"2025-05-27T06:35:52.48985055Z"}

Help would be appreciated, Thanks!

2

Should I delete “revocation.db” in data folder and start the node?

3

Did you check your file system? It might be mounted as read only due to corruption or at least that specific file is corrupted. Check your filesystem:

1 Like
4

Can I just delete the “revocation.db” file and start the node? Do I lose anything deleting it?

5

Yes I think it gets recreated if it is not present. You can rename it first instead of deletion to be safe though.

6

Through WinSCP root user, it doesn’t allow me to rename, delete or even download the file to my PC.

7

That’s probably due to this:

Your file system seems to be mounted read-only due to corruption.

As already suggested:

1 Like
8

When I run sudo fsck -ycjp /dev/sdf I get,

fsck from util-linux 2.38.1
e2fsck 1.47.0 (5-Feb-2023)
fsck.ext2: Invalid argument while trying to open /dev/sdf

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>
9

You need to run it against the partition you want to check. So verify if you have multiple partitions on that device and run it against the correct partition then, e.g. fsck -ycjp /dev/sdf1 or fsck -ycjp /dev/sdf2 or whatever partition number your node is running.

You can see the partitions on your device by running lsblk /dev/sdf.

1 Like
10

Something is seriously wrong with that drive.. It auto disconnected, so I rebooted and now system hangs/Linux doesn’t boot.

11

This sounds really bad. Your drive seems to have serious issues.

If it really doesn’t boot and isn’t just slow on booting due to all the checks it is performing, I’d suggest to run complete e2fsck check on boot. Maybe starting a live image and run fsck from there on that disk is also an option.

1 Like