Node database corruption - Help requested

Node Operators troubleshooting
1

Hoping to get some guidance on a database corruption event - looking at the forums I only see posts around 2020 - nothing recent.

  • How can I tell which database was corrupted?
  • Do we have new steps to resolve corruption other than drop table and recreate?
2024-07-06T11:43:26Z    INFO    db.migration.58 Remove unused trash column      {"Process": "storagenode"}
2024-07-06T11:43:26Z    ERROR   failure during run      {"Process": "storagenode", "error": "Error migrating tables for database on storagenode: migrate: v58: database disk image is malformed\n\tstorj.io/storj/private/migrate.(*Migration).Run:212\n\tstorj.io/storj/storageprocess.cleanup.func1.4:393\n\tstorj.io/common/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:: migrate: v58: database disk image is malformed\n\tstorj.io/storj/private/migrate.(*Migration).Run:212\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:387\n\tmain.cmdRun:98\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:393\n\tstoobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tmain.main:34\n\truntime.main:271\n\tmain.cmdRun:100\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:393\n\tstorj.i.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tmain.main:34\n\truntime.main:271"}
Error: Error migrating tables for database on storagenode: migrate: v58: database disk image is malformed
        storj.io/storj/private/migrate.(*Migration).Run:212
        storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:387
        main.cmdRun:98
        main.newRunCmd.func1:33
        storj.io/common/process.cleanup.func1.4:393
        storj.io/common/process.cleanup.func1:411
        github.com/spf13/cobra.(*Command).execute:983
        github.com/spf13/cobra.(*Command).ExecuteC:1115
        github.com/spf13/cobra.(*Command).Execute:1039
        storj.io/common/process.ExecWithCustomOptions:112
        main.main:34
        runtime.main:271
2024-07-06 11:43:26,610 INFO exited: storagenode (exit status 1; not expected)
2
3

This allowed me to identify and resolve the problem. If the document maintainer could reach out there were a few items that could be improved / corrected

4

You could mention the improvements in a reply in this thread.

1 Like
5

Or even suggest a pull request to the GitHub - storj/docs: Source for Storj docs repo.
We would be happy to review and merge (and I would mention this new link in the Support Portal KB instead of the current article).