Node Failing to Start

Node Operators troubleshooting
1

This is a Windows GUI install.
I can no longer start my Node. I checked for malformed databases and had 3, heldamount.db, piece_expiration.db, and storage_usage.db, but after repairing them still nothing. I doubled checked the databases and then to test I removed all databases and tried letting the Node rebuild them, then after that failed I restored the databases. Thinking it might be a disk issue I did a checkdisk and that came back clean. Trying to manually start the service gives a “Error 1067: The process terminated unexpectedly” I am out of ideas.

This is what I am seeing my log file, but I am at a lost as to what it means.

2023-10-11T12:18:32-05:00 FATAL Unrecoverable error {“error”: “Error creating tables for master database on storagenode: migrate: v32: table paystubs already exists\n\tstorj.io/storj/private/migrate.(*Migration).Run:209\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:370\n\tmain.cmdRun:95\n\tmain.newRunCmd.func1:32\n\tstorj.io/private/process.cleanup.func1.4:398\n\tstorj.io/private/process.cleanup.func1:416\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomOptions:112\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:78\n\tstorj.io/private/process.ExecWithCustomConfig:73\n\tstorj.io/private/process.Exec:63\n\tmain.(*service).Execute.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75”, “errorVerbose”: “Error creating tables for master database on storagenode: migrate: v32: table paystubs already exists\n\tstorj.io/storj/private/migrate.(*Migration).Run:209\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:370\n\tmain.cmdRun:95\n\tmain.newRunCmd.func1:32\n\tstorj.io/private/process.cleanup.func1.4:398\n\tstorj.io/private/process.cleanup.func1:416\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomOptions:112\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:78\n\tstorj.io/private/process.ExecWithCustomConfig:73\n\tstorj.io/private/process.Exec:63\n\tmain.(*service).Execute.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n\tmain.cmdRun:97\n\tmain.newRunCmd.func1:32\n\tstorj.io/private/process.cleanup.func1.4:398\n\tstorj.io/private/process.cleanup.func1:416\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomOptions:112\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:78\n\tstorj.io/private/process.ExecWithCustomConfig:73\n\tstorj.io/private/process.Exec:63\n\tmain.(*service).Execute.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75”}

2

Wellcome to the Community d1wGDqj,

how long is the node offline?

my guess, try this (discarded).

or wait for awsome @Alexey for further help tomorrow, i think:

1 Like
3

Likely the suggestion Node Failing to Start - #2 by daki82 will not work, because it throws an error that this table already exist, so you actually need to insert a row to the versions table, but please give me result of this sqlite3 command for the heldamount.db database:

select * from versions;

The correct output (date may vary):

sqlite> select * from versions;
32|2020-03-19 22:59:23.3953808 +0300 MSK m=+0.929216701
33|2020-03-19 22:59:23.3953808 +0300 MSK m=+0.929216701
37|2020-05-14 00:57:16.1967959 +0300 MSK m=+0.902128601
43|2020-07-28 21:30:20.8443607 +0300 MSK m=+0.969467501
49|2021-02-17 01:11:14.5355229 +0300 MSK m=+0.762999601
50|2021-02-17 01:11:14.5765211 +0300 MSK m=+0.803997801
51|2021-02-26 17:11:13.7571647 +0300 MSK m=+0.786998101

You may also re-create this database, following this instruction (you must not skip any step):

1 Like
4

Was able to get the Node up and running with the your link “How to fix database: file is not a database error”. It is odd as that is what I tried before, but whatever it is working now.
Thank You for your help.

1 Like