Node crashed after power loss. database disk image is malformed

Node Operators troubleshooting
1

Hi,
first I have restored my orders.db after a crash, than I have deleted and the tables new created with the SQL syntax, orders.db is now empty and ok, but after I start the node , the last line of the log is this:

PS C:\Users\foximulder> cat “$env:ProgramFiles\Storj\Storage Node\storagenode.log” -tail 20
2020-06-14T08:58:26.136+0200 INFO Configuration loaded {“Location”: “C:\Program Files\Storj\Storage Node\config.yaml”}
2020-06-14T08:58:26.162+0200 DEBUG Anonymized tracing disabled
2020-06-14T08:58:26.163+0200 DEBUG debug server listening on 127.0.0.1:49386
2020-06-14T08:58:26.172+0200 INFO Operator email {“Address”: “foximulder@hotmail.com”}
2020-06-14T08:58:26.172+0200 INFO Operator wallet {“Address”: “0x1B6B5C7DC3bE0A390f434f01870d1228913Ec3Cd”}
2020-06-14T08:58:26.186+0200 DEBUG Version info {“Version”: “1.6.3”, “Commit Hash”: “d86941d14ec34b5d9368e356d32cfbc858fb7721”, “Build Timestamp”: “2020-06-04 12:08:23 +0200 CEST”, “Release Build”: true}
2020-06-14T08:58:26.846+0200 DEBUG version Allowed minimum version from control server. {“Minimum Version”: “1.3.0”}
2020-06-14T08:58:26.846+0200 DEBUG version Running on allowed version. {“Version”: “1.6.3”}
2020-06-14T08:58:26.847+0200 INFO Telemetry enabled
2020-06-14T08:58:26.847+0200 DEBUG Initialized batcher {“ID”: “1uiBijz7HDGa8j71mwD5yPehWG6cjLYZsY71ywQhMiZfA7zN4T”}
2020-06-14T08:58:26.850+0200 INFO db.migration.42 Drop used serials table
2020-06-14T08:58:28.219+0200 FATAL Unrecoverable error {“error”: “Error creating tables for master database on storagenode: migrate: database disk image is malformed\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).Migration.func10:1455\n\tstorj.io/storj/private/migrate.Func.Run:263\n\tstorj.io/storj/private/migrate.(*Migration).Run.func1:171\n\tstorj.io/storj/private/dbutil/txutil.withTxOnce:75\n\tstorj.io/storj/private/dbutil/txutil.WithTx:36\n\tstorj.io/storj/private/migrate.(*Migration).Run:170\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:293\n\tmain.cmdRun:180\n\tstorj.io/private/process.cleanup.func1.4:359\n\tstorj.io/private/process.cleanup.func1:377\n\tgithub.com/spf13/cobra.(*Command).execute:840\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:945\n\tgithub.com/spf13/cobra.(*Command).Execute:885\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:66\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”, “errorVerbose”: “Error creating tables for master database on storagenode: migrate: database disk image is malformed\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).Migration.func10:1455\n\tstorj.io/storj/private/migrate.Func.Run:263\n\tstorj.io/storj/private/migrate.(*Migration).Run.func1:171\n\tstorj.io/storj/private/dbutil/txutil.withTxOnce:75\n\tstorj.io/storj/private/dbutil/txutil.WithTx:36\n\tstorj.io/storj/private/migrate.(*Migration).Run:170\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:293\n\tmain.cmdRun:180\n\tstorj.io/private/process.cleanup.func1.4:359\n\tstorj.io/private/process.cleanup.func1:377\n\tgithub.com/spf13/cobra.(*Command).execute:840\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:945\n\tgithub.com/spf13/cobra.(*Command).Execute:885\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:66\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n\tmain.cmdRun:182\n\tstorj.io/private/process.cleanup.func1.4:359\n\tstorj.io/private/process.cleanup.func1:377\n\tgithub.com/spf13/cobra.(*Command).execute:840\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:945\n\tgithub.com/spf13/cobra.(*Command).Execute:885\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:66\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}

Can I also clear the used_Serials.db?

2

Please check your other databases with this instruction.

3

D:\storj\storage>d:\sqlite\sqlite3.exe bandwidth.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe heldamount.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe info.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe notifications.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe orders.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe piece_expiration.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe piece_spaced_used.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe pieceinfo.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe piecestore.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe pricing.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe piece_expiration.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe reputation.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe satellites.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe storage_usage.db “PRAGMA integrity_check;”
ok
D:\storj\storage>d:\sqlite\sqlite3.exe used_serial.db “PRAGMA integrity_check;”
Error: database disk image is malformed

ok, will dump/read the db.
…

4

I have stopped the read from I have only 10% after 2 Days. My orig used_Serial.db had 2GB.
I have created a new one with the syntax from here:

Now the node is running and UP. :heart_eyes:

@BrightSilence Is it not possible in the preFlight mode to create the db if it not exists?