Error: Error creating tables for master database on storagenode: migrate: no such table: main.order_archive_

Node keeps restarting. storagenode logs says:

2020-03-14T12:40:43.194Z INFO version running on version v0.34.6
2020-03-14T12:40:43.247Z INFO db.migration.27 Add index archived_at to ordersDB
Error: Error creating tables for master database on storagenode: migrate: no such table: main.order_archive_
storj.io/storj/private/dbutil/txutil.WithTx:47
storj.io/storj/private/migrate.(*Migration).Run:170
storj.io/storj/storagenode/storagenodedb.(*DB).CreateTables:275
main.cmdRun:188
storj.io/storj/pkg/process.cleanup.func1.2:312
storj.io/storj/pkg/process.cleanup.func1:330
github.com/spf13/cobra.(*Command).execute:826
github.com/spf13/cobra.(*Command).ExecuteC:914
github.com/spf13/cobra.(*Command).Execute:864
storj.io/storj/pkg/process.ExecWithCustomConfig:84
storj.io/storj/pkg/process.ExecCustomDebug:66
main.main:328
runtime.main:203

Welcome to the forum @istie!

How is your HDD connected ? Is it a Windows or Linux node ?

Thx for your reply!

It’s a linux node (raspberry buster) with an external hdd of 2TB (external powered) mounted via fstab. Node is online since sept 2019

Can you show your docker run command and remove any personal details from it ?

docker run -d --restart unless-stopped -p 28967:28967 -p 14002:14002 -e WALLET=“0x****************************************” -e EMAIL="myemail@mail.com" -e ADDRESS=“my.address:28967” -e BANDWIDTH=“5TB” -e STORAGE=“1TB” --mount type=bind,source="/home/pi/.local/share/storj/identity/storagenode",destination=/app/identity --mount type=bind,source="/mnt/STORJ/_STORJ-DATA",destination=/app/config --name storagenode storjlabs/storagenode:beta

Try to fix the database:

  1. Stop the storagenode
  2. Make a backup of orders.db database (for example by copying to orders.db.bak).
  3. Execute either with a local installed sqlite3 or with a docker container with sqlite3:
sqlite3 orders.db
CREATE TABLE unsent_order (
                                                satellite_id  BLOB NOT NULL,
                                                serial_number BLOB NOT NULL,

                                                order_limit_serialized BLOB      NOT NULL, -- serialized pb.OrderLimit
                                                order_serialized       BLOB      NOT NULL, -- serialized pb.Order
                                                order_limit_expiration TIMESTAMP NOT NULL, -- when is the deadline for sending it

                                                uplink_cert_id INTEGER NOT NULL,

                                                FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
                                        );
CREATE TABLE order_archive_ (
                                                satellite_id  BLOB NOT NULL,
                                                serial_number BLOB NOT NULL,

                                                order_limit_serialized BLOB NOT NULL,
                                                order_serialized       BLOB NOT NULL,

                                                uplink_cert_id INTEGER NOT NULL,

                                                status      INTEGER   NOT NULL,
                                                archived_at TIMESTAMP NOT NULL,

                                                FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
                                        );
CREATE UNIQUE INDEX idx_orders ON unsent_order(satellite_id, serial_number);
CREATE TABLE versions (version int, commited_at text);
CREATE INDEX idx_order_archived_at ON order_archive_(archived_at);
.exit

Executed commands with sqlite3 but still node keeps restarting
logs say:

2020-03-14T16:43:45.884Z INFO version running on version v0.34.6
2020-03-14T16:43:46.377Z INFO db.migration.27 Add index archived_at to ordersDB
Error: Error creating tables for master database on storagenode: migrate: index idx_order_archived_at already exists
storj.io/storj/private/dbutil/txutil.WithTx:47
storj.io/storj/private/migrate.(*Migration).Run:170
storj.io/storj/storagenode/storagenodedb.(*DB).CreateTables:275
main.cmdRun:188
storj.io/storj/pkg/process.cleanup.func1.2:312
storj.io/storj/pkg/process.cleanup.func1:330
github.com/spf13/cobra.(*Command).execute:826
github.com/spf13/cobra.(*Command).ExecuteC:914
github.com/spf13/cobra.(*Command).Execute:864
storj.io/storj/pkg/process.ExecWithCustomConfig:84
storj.io/storj/pkg/process.ExecCustomDebug:66
main.main:328
runtime.main:203

Is there no tool or script which can repair a broken database, Or build it correct again?
Or am I asking for something really stupid?

  1. Stop the storagenode
  2. execute the script:
sqlite3 orders.db
drop index idx_order_archived_at;
.exit
  1. Start the storagenode

Yes, you can try to remove the database and lose the statistic and some unsent orders (they will not be paid), the storagenode should recreate a database. But it may not work. So, better to use a provided fix.

Thx for your answers, tried your suggestion but still not ok.

2020-03-14T19:22:58.405Z INFO version running on version v0.34.6
2020-03-14T19:22:58.474Z INFO db.migration Database Version {“version”: 31}
Error: Error during preflight check for storagenode databases: storage node preflight database error: piece_expiration: expected schema does not match actual: &dbschema.Schema{
Tables: *dbschema.Table{
&{Name: “piece_expirations”, Columns: *dbschema.Column{&{Name: “deletion_failed_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “piece_expiration”, Type: “TIMESTAMP”}, &{Name: “piece_id”, Type: “BLOB”}, &{Name: “satellite_id”, Type: “BLOB”}, &{Name: “trash”, Type: “INTEGER”}}, PrimaryKey: string{“piece_id”, “satellite_id”}},

Looks like more databases are corrupt isn’t it.
You say: “better use a provided fix” so deleting database is not an option…?
But If I delete a database, which ones should be saved and which ones I can safely delete?

Please, stop the node and execute:

sqlite3 piece_expiration.db
drop test_table;
.exit

And try to start the node

Sorry it’s still restarting.
And with the command: “drop test_table;”
I get:
sqlite> drop test_table;
Error: near “test_table”: syntax error

logs:
2020-03-16T23:15:39.097Z INFO version running on version v0.34.6
2020-03-16T23:15:39.176Z INFO db.migration Database Version {“version”: 31}
2020-03-16T23:15:40.761Z INFO Got a signal from the OS: “terminated”
Error: Error during preflight check for storagenode databases: storage node preflight database error: piece_spaced_used: expected schema does not match actual: &dbschema.Schema{
Tables: *dbschema.Table{
&{Name: “piece_space_used”, Columns: *dbschema.Column{&{Name: “content_size”, Type: “INTEGER”}, &{Name: “satellite_id”, Type: “BLOB”, IsNullable: true}, &{Name: “total”, Type: “INTEGER”}}},

  1. Stop the storagenode
  2. execute
sqlite3 piece_space_used.db
drop table test_table;
.exit

Thanks for all your replies but is still not working :frowning:

command:
sqlite> drop table test_table;
Error: no such table: test_table
sqlite> .exit

logs: (much longer this time…??):
2020-03-17T20:14:51.479Z INFO Configuration loaded from: /app/config/config.yaml
2020-03-17T20:14:51.484Z INFO Operator email: ***************
2020-03-17T20:14:51.485Z INFO operator wallet: **************************************
2020-03-17T20:14:52.658Z INFO version running on version v0.34.6
2020-03-17T20:14:52.766Z INFO db.migration Database Version {“version”: 31}
Error: Error during preflight check for storagenode databases: storage node preflight database error: piece_spaced_used: expected schema does not match actual: &dbschema.Schema{
Tables: *dbschema.Table{
&{Name: “piece_space_used”, Columns: *dbschema.Column{&{Name: “content_size”, Type: “INTEGER”}, &{Name: “satellite_id”, Type: “BLOB”, IsNullable: true}, &{Name: “total”, Type: “INTEGER”}}},

  • Tables: *dbschema.Table{
  •   &{
    
  •   	Name: "used_serial_",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "expiration", Type: "TIMESTAMP"},
    
  •   		&{Name: "satellite_id", Type: "BLOB"},
    
  •   		&{Name: "serial_number", Type: "BLOB"},
    
  •   	},
    
  •   },
    
  • },
  • Tables: nil,
  • Indexes: *dbschema.Index{
  •   &{Name: "idx_used_serial_", Table: "used_serial_", Columns: []string{"expiration"}},
    
  •   &{
    
  •   	Name:    "pk_used_serial_",
    
  •   	Table:   "used_serial_",
    
  •   	Columns: []string{"satellite_id", "serial_number"},
    
  •   },
    
  • },
  • Tables: *dbschema.Table{
  •   &{
    
  •   	Name: "used_serial_",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "expiration", Type: "TIMESTAMP"},
    
  •   		&{Name: "satellite_id", Type: "BLOB"},
    
  •   		&{Name: "serial_number", Type: "BLOB"},
    
  •   	},
    
  •   },
    
  • },
  • Tables: nil,
  • Indexes: *dbschema.Index{
  •   &{Name: "idx_used_serial_", Table: "used_serial_", Columns: []string{"expiration"}},
    
  •   &{
    
  •   	Name:    "pk_used_serial_",
    
  •   	Table:   "used_serial_",
    
  •   	Columns: []string{"satellite_id", "serial_number"},
    
  •   },
    
  • },
  • Tables: nil,
  • Tables: *dbschema.Table{

  •   &{
    
  •   	Name: "test_table",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "id", Type: "int"},
    
  •   		&{Name: "name", Type: "varchar(30)", IsNullable: true},
    
  •   	},
    
  •   	PrimaryKey: []string{"id"},
    
  •   },
    
  • },
    Indexes: nil,
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203
    2020-03-17T20:15:41.619Z INFO Configuration loaded from: /app/config/config.yaml
    2020-03-17T20:15:41.624Z INFO Operator email: *************
    2020-03-17T20:15:41.626Z INFO operator wallet: **************************************
    2020-03-17T20:15:43.884Z INFO version running on version v0.34.6
    2020-03-17T20:15:44.009Z INFO db.migration Database Version {“version”: 31}
    Error: Error during preflight check for storagenode databases: storage node preflight database error: reputation: expected schema does not match actual: &dbschema.Schema{
    Tables: *dbschema.Table{
    &{Name: “reputation”, Columns: *dbschema.Column{&{Name: “audit_reputation_alpha”, Type: “REAL”}, &{Name: “audit_reputation_beta”, Type: “REAL”}, &{Name: “audit_reputation_score”, Type: “REAL”}, &{Name: “audit_success_count”, Type: “INTEGER”}, &{Name: “audit_total_count”, Type: “INTEGER”}, &{Name: “disqualified”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “satellite_id”, Type: “BLOB”}, &{Name: “updated_at”, Type: “TIMESTAMP”}, &{Name: “uptime_reputation_alpha”, Type: “REAL”}, &{Name: “uptime_reputation_beta”, Type: “REAL”}, &{Name: “uptime_reputation_score”, Type: “REAL”}, &{Name: “uptime_success_count”, Type: “INTEGER”}, &{Name: “uptime_total_count”, Type: “INTEGER”}}, PrimaryKey: string{“satellite_id”}},

  •   &{
    
  •   	Name: "test_table",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "id", Type: "int"},
    
  •   		&{Name: "name", Type: "varchar(30)", IsNullable: true},
    
  •   	},
    
  •   	PrimaryKey: []string{"id"},
    
  •   },
    

    },
    Indexes: nil,
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203
    2020-03-17T20:16:01.646Z INFO Configuration loaded from: /app/config/config.yaml
    2020-03-17T20:16:01.650Z INFO Operator email: **************
    2020-03-17T20:16:01.651Z INFO operator wallet: **************************************
    2020-03-17T20:16:02.251Z INFO version running on version v0.34.6
    2020-03-17T20:16:02.306Z INFO db.migration Database Version {“version”: 31}
    Error: Error during preflight check for storagenode databases: storage node preflight database error: piece_expiration: expected schema does not match actual: &dbschema.Schema{
    Tables: *dbschema.Table{
    &{Name: “piece_expirations”, Columns: *dbschema.Column{&{Name: “deletion_failed_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “piece_expiration”, Type: “TIMESTAMP”}, &{Name: “piece_id”, Type: “BLOB”}, &{Name: “satellite_id”, Type: “BLOB”}, &{Name: “trash”, Type: “INTEGER”}}, PrimaryKey: string{“piece_id”, “satellite_id”}},

  •   &{
    
  •   	Name: "test_table",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "id", Type: "int"},
    
  •   		&{Name: "name", Type: "varchar(30)", IsNullable: true},
    
  •   	},
    
  •   	PrimaryKey: []string{"id"},
    
  •   },
    

    },
    Indexes: *dbschema.Index{&{Name: “idx_piece_expirations_deletion_failed_at”, Table: “piece_expirations”, Columns: string{“deletion_failed_at”}}, &{Name: “idx_piece_expirations_piece_expiration”, Table: “piece_expirations”, Columns: string{“piece_expiration”}}, &{Name: “idx_piece_expirations_trashed”, Table: “piece_expirations”, Columns: string{“satellite_id”, “trash”}, Partial: “trash = 1”}},
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203
    2020-03-17T20:16:37.454Z INFO Configuration loaded from: /app/config/config.yaml
    2020-03-17T20:16:37.461Z INFO Operator email: *************
    2020-03-17T20:16:37.462Z INFO operator wallet: **************************************
    2020-03-17T20:16:38.628Z INFO version running on version v0.34.6
    2020-03-17T20:16:38.768Z INFO db.migration Database Version {“version”: 31}
    Error: Error during preflight check for storagenode databases: storage node preflight database error: piece_spaced_used: expected schema does not match actual: &dbschema.Schema{
    Tables: *dbschema.Table{
    &{Name: “piece_space_used”, Columns: *dbschema.Column{&{Name: “content_size”, Type: “INTEGER”}, &{Name: “satellite_id”, Type: “BLOB”, IsNullable: true}, &{Name: “total”, Type: “INTEGER”}}},

  •   &{
    
  •   	Name: "test_table",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "id", Type: "int"},
    
  •   		&{Name: "name", Type: "varchar(30)", IsNullable: true},
    
  •   	},
    
  •   	PrimaryKey: []string{"id"},
    
  •   },
    

    },
    Indexes: *dbschema.Index{&{Name: “idx_piece_space_used_satellite_id”, Table: “piece_space_used”, Columns: string{“satellite_id”}}},
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203

Unfortunately this db file is misspelled in the installation. So the command should be.

sqlite3 piece_spaced_used.db
drop table test_table;
.exit

And actually a few more have the same issue.

sqlite3 storage_usage.db
drop table test_table;
.exit
sqlite3 piece_expiration.db
drop table test_table;
.exit
sqlite3 reputation.db
drop table test_table;
.exit

There is an issue with used_serial as well, but I’m not entirely sure what.

I really wonder what happened to this node to have so many different database issues…

Again Thanks for all the help. The node had no problems from sept. 2019
I’ve been on vacation when the node went offline.
Mabybe a power outtage which corrupted my databases? I really do not know…
Is it easier to startup a new node?
Can I use the same identity (email and authorisation code)?
Can I take my reputaion with me?
Thx again!

Commands I tried this time on your request:

pi@gekkojoris:/mnt/STORJ/_STORJ-DATA/storage sqlite3 piece_spaced_used.db SQLite version 3.27.2 2019-02-25 16:06:06 Enter ".help" for usage hints. sqlite> drop table test_table; sqlite> .exit pi@gekkojoris:/mnt/STORJ/_STORJ-DATA/storage sqlite3 storage_usage.db
SQLite version 3.27.2 2019-02-25 16:06:06
Enter “.help” for usage hints.
sqlite> drop table test_table;
sqlite> .exit
pi@gekkojoris:/mnt/STORJ/_STORJ-DATA/storage sqlite3 piece_expiration.db SQLite version 3.27.2 2019-02-25 16:06:06 Enter ".help" for usage hints. sqlite> drop table test_table; sqlite> .exit pi@gekkojoris:/mnt/STORJ/_STORJ-DATA/storage sqlite3 reputation.db
SQLite version 3.27.2 2019-02-25 16:06:06
Enter “.help” for usage hints.
sqlite> drop table test_table;
sqlite> .exit
pi@gekkojoris:/mnt/STORJ/_STORJ-DATA/storage $

Status:
pi@gekkojoris:/mnt/STORJ/_STORJ-DATA/storage sudo docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 500c4f0a40c3 storjlabs/storagenode:beta "/entrypoint" 41 seconds ago Up 6 seconds 0.0.0.0:14002->14002/tcp, 0.0.0.0:28967->28967/tcp storagenode pi@gekkojoris:/mnt/STORJ/_STORJ-DATA/storage sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b0b2cc04e5e storjlabs/storagenode:beta “/entrypoint” 2 minutes ago Restarting (1) 42 seconds ago storagenode
pi@gekkojoris:/mnt/STORJ/_STORJ-DATA/storage $

Again lots news in logs:
2020-03-17T22:56:01.326Z INFO Configuration loaded from: /app/config/config.yaml
2020-03-17T22:56:01.331Z INFO Operator email: **************
2020-03-17T22:56:01.331Z INFO operator wallet: ***************************************
2020-03-17T22:56:01.896Z INFO version running on version v0.34.6
2020-03-17T22:56:01.962Z INFO db.migration Database Version {“version”: 31}
Error: Error during preflight check for storagenode databases: storage node preflight database error: info: expected schema does not match actual: &dbschema.Schema{

  • Tables: nil,
  • Tables: *dbschema.Table{
  •   &{
    
  •   	Name: "used_serial_",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "expiration", Type: "TIMESTAMP"},
    
  •   		&{Name: "satellite_id", Type: "BLOB"},
    
  •   		&{Name: "serial_number", Type: "BLOB"},
    
  •   	},
    
  •   },
    
  • },
  • Tables: nil,
  • Indexes: *dbschema.Index{
  •   &{Name: "idx_used_serial_", Table: "used_serial_", Columns: []string{"expiration"}},
    
  •   &{
    
  •   	Name:    "pk_used_serial_",
    
  •   	Table:   "used_serial_",
    
  •   	Columns: []string{"satellite_id", "serial_number"},
    
  •   },
    
  • },
  • Indexes: nil,
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203
    2020-03-17T22:56:20.321Z INFO Configuration loaded from: /app/config/config.yaml
    2020-03-17T22:56:20.325Z INFO Operator email: **************
    2020-03-17T22:56:20.326Z INFO operator wallet: ***************************************
    2020-03-17T22:56:21.812Z INFO version running on version v0.34.6
    2020-03-17T22:56:21.915Z INFO db.migration Database Version {“version”: 31}
    Error: Error during preflight check for storagenode databases: storage node preflight database error: satellites: expected schema does not match actual: &dbschema.Schema{
    Tables: *dbschema.Table{
    &{Name: “satellite_exit_progress”, Columns: *dbschema.Column{&{Name: “bytes_deleted”, Type: “INTEGER”}, &{Name: “completion_receipt”, Type: “BLOB”, IsNullable: true}, &{Name: “finished_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “initiated_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “satellite_id”, Type: “BLOB”}, &{Name: “starting_disk_usage”, Type: “INTEGER”}}, PrimaryKey: string{“satellite_id”}},
    &{Name: “satellites”, Columns: *dbschema.Column{&{Name: “added_at”, Type: “TIMESTAMP”}, &{Name: “node_id”, Type: “BLOB”}, &{Name: “status”, Type: “INTEGER”}}, PrimaryKey: string{“node_id”}},

  •   &{
    
  •   	Name: "test_table",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "id", Type: "int"},
    
  •   		&{Name: "name", Type: "varchar(30)", IsNullable: true},
    
  •   	},
    
  •   	PrimaryKey: []string{"id"},
    
  •   },
    

    },
    Indexes: nil,
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203
    2020-03-17T22:56:25.334Z INFO Configuration loaded from: /app/config/config.yaml
    2020-03-17T22:56:25.341Z INFO Operator email: **************
    2020-03-17T22:56:25.342Z INFO operator wallet: ***************************************
    2020-03-17T22:56:27.062Z INFO version running on version v0.34.6
    2020-03-17T22:56:27.165Z INFO db.migration Database Version {“version”: 31}
    Error: Error during preflight check for storagenode databases: storage node preflight database error: satellites: expected schema does not match actual: &dbschema.Schema{
    Tables: *dbschema.Table{
    &{Name: “satellite_exit_progress”, Columns: *dbschema.Column{&{Name: “bytes_deleted”, Type: “INTEGER”}, &{Name: “completion_receipt”, Type: “BLOB”, IsNullable: true}, &{Name: “finished_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “initiated_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “satellite_id”, Type: “BLOB”}, &{Name: “starting_disk_usage”, Type: “INTEGER”}}, PrimaryKey: string{“satellite_id”}},
    &{Name: “satellites”, Columns: *dbschema.Column{&{Name: “added_at”, Type: “TIMESTAMP”}, &{Name: “node_id”, Type: “BLOB”}, &{Name: “status”, Type: “INTEGER”}}, PrimaryKey: string{“node_id”}},

  •   &{
    
  •   	Name: "test_table",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "id", Type: "int"},
    
  •   		&{Name: "name", Type: "varchar(30)", IsNullable: true},
    
  •   	},
    
  •   	PrimaryKey: []string{"id"},
    
  •   },
    

    },
    Indexes: nil,
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203
    2020-03-17T22:56:32.437Z INFO Configuration loaded from: /app/config/config.yaml
    2020-03-17T22:56:32.442Z INFO Operator email: **************
    2020-03-17T22:56:32.442Z INFO operator wallet: ***************************************
    2020-03-17T22:56:33.214Z INFO version running on version v0.34.6
    2020-03-17T22:56:33.269Z INFO db.migration Database Version {“version”: 31}
    Error: Error during preflight check for storagenode databases: storage node preflight database error: used_serial: expected schema does not match actual: &dbschema.Schema{

  • Tables: *dbschema.Table{
  •   &{
    
  •   	Name: "used_serial_",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "expiration", Type: "TIMESTAMP"},
    
  •   		&{Name: "satellite_id", Type: "BLOB"},
    
  •   		&{Name: "serial_number", Type: "BLOB"},
    
  •   	},
    
  •   },
    
  • },
  • Tables: nil,
  • Indexes: *dbschema.Index{
  •   &{Name: "idx_used_serial_", Table: "used_serial_", Columns: []string{"expiration"}},
    
  •   &{
    
  •   	Name:    "pk_used_serial_",
    
  •   	Table:   "used_serial_",
    
  •   	Columns: []string{"satellite_id", "serial_number"},
    
  •   },
    
  • },
  • Tables: *dbschema.Table{
  •   &{
    
  •   	Name: "used_serial_",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "expiration", Type: "TIMESTAMP"},
    
  •   		&{Name: "satellite_id", Type: "BLOB"},
    
  •   		&{Name: "serial_number", Type: "BLOB"},
    
  •   	},
    
  •   },
    
  • },
  • Tables: nil,
  • Indexes: *dbschema.Index{
  •   &{Name: "idx_used_serial_", Table: "used_serial_", Columns: []string{"expiration"}},
    
  •   &{
    
  •   	Name:    "pk_used_serial_",
    
  •   	Table:   "used_serial_",
    
  •   	Columns: []string{"satellite_id", "serial_number"},
    
  •   },
    
  • },
  • Indexes: nil,
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203
    2020-03-17T22:56:49.503Z INFO Configuration loaded from: /app/config/config.yaml
    2020-03-17T22:56:49.507Z INFO Operator email: **************
    2020-03-17T22:56:49.507Z INFO operator wallet: ***************************************
    2020-03-17T22:56:51.275Z INFO version running on version v0.34.6
    2020-03-17T22:56:51.377Z INFO db.migration Database Version {“version”: 31}
    Error: Error during preflight check for storagenode databases: storage node preflight database error: satellites: expected schema does not match actual: &dbschema.Schema{
    Tables: *dbschema.Table{
    &{Name: “satellite_exit_progress”, Columns: *dbschema.Column{&{Name: “bytes_deleted”, Type: “INTEGER”}, &{Name: “completion_receipt”, Type: “BLOB”, IsNullable: true}, &{Name: “finished_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “initiated_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “satellite_id”, Type: “BLOB”}, &{Name: “starting_disk_usage”, Type: “INTEGER”}}, PrimaryKey: string{“satellite_id”}},
    &{Name: “satellites”, Columns: *dbschema.Column{&{Name: “added_at”, Type: “TIMESTAMP”}, &{Name: “node_id”, Type: “BLOB”}, &{Name: “status”, Type: “INTEGER”}}, PrimaryKey: string{“node_id”}},

  •   &{
    
  •   	Name: "test_table",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "id", Type: "int"},
    
  •   		&{Name: "name", Type: "varchar(30)", IsNullable: true},
    
  •   	},
    
  •   	PrimaryKey: []string{"id"},
    
  •   },
    

    },
    Indexes: nil,
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203
    2020-03-17T22:57:06.884Z INFO Configuration loaded from: /app/config/config.yaml
    2020-03-17T22:57:06.889Z INFO Operator email: **************
    2020-03-17T22:57:06.892Z INFO operator wallet: ***************************************
    2020-03-17T22:57:07.495Z INFO version running on version v0.34.6
    2020-03-17T22:57:07.555Z INFO db.migration Database Version {“version”: 31}
    Error: Error during preflight check for storagenode databases: storage node preflight database error: satellites: expected schema does not match actual: &dbschema.Schema{
    Tables: *dbschema.Table{
    &{Name: “satellite_exit_progress”, Columns: *dbschema.Column{&{Name: “bytes_deleted”, Type: “INTEGER”}, &{Name: “completion_receipt”, Type: “BLOB”, IsNullable: true}, &{Name: “finished_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “initiated_at”, Type: “TIMESTAMP”, IsNullable: true}, &{Name: “satellite_id”, Type: “BLOB”}, &{Name: “starting_disk_usage”, Type: “INTEGER”}}, PrimaryKey: string{“satellite_id”}},
    &{Name: “satellites”, Columns: *dbschema.Column{&{Name: “added_at”, Type: “TIMESTAMP”}, &{Name: “node_id”, Type: “BLOB”}, &{Name: “status”, Type: “INTEGER”}}, PrimaryKey: string{“node_id”}},

  •   &{
    
  •   	Name: "test_table",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "id", Type: "int"},
    
  •   		&{Name: "name", Type: "varchar(30)", IsNullable: true},
    
  •   	},
    
  •   	PrimaryKey: []string{"id"},
    
  •   },
    

    },
    Indexes: nil,
    }

    storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:301
    main.cmdRun:198
    storj.io/storj/pkg/process.cleanup.func1.2:312
    storj.io/storj/pkg/process.cleanup.func1:330
    github.com/spf13/cobra.(*Command).execute:826
    github.com/spf13/cobra.(*Command).ExecuteC:914
    github.com/spf13/cobra.(*Command).Execute:864
    storj.io/storj/pkg/process.ExecWithCustomConfig:84
    storj.io/storj/pkg/process.ExecCustomDebug:66
    main.main:328
    runtime.main:203
    2020-03-17T22:57:28.461Z INFO Configuration loaded from: /app/config/config.yaml
    2020-03-17T22:57:28.466Z INFO Operator email: **************
    2020-03-17T22:57:28.467Z INFO operator wallet: ***************************************
    2020-03-17T22:57:29.045Z INFO version running on version v0.34.6
    2020-03-17T22:57:29.149Z INFO db.migration Database Version {“version”: 31}
    Error: Error during preflight check for storagenode databases: storage node preflight database error: info: expected schema does not match actual: &dbschema.Schema{

  • Tables: nil,
  • Tables: nil,
  • Tables: *dbschema.Table{
  •   &{
    
  •   	Name: "used_serial_",
    
  •   	Columns: []*dbschema.Column{
    
  •   		&{Name: "expiration", Type: "TIMESTAMP"},
    
  •   		&{Name: "satellite_id", Type: "BLOB"},
    
  •   		&{Name: "serial_number", Type: "BLOB"},
    
  •   	},
    
  •   },
    
  • },
  • Tables: nil,
  • Indexes: *dbschema.Index{
  •   &{Name: "idx_used_serial_", Table: "used_serial_", Columns: []string{"expiration"}},
    
  •   &{
    
  •   	Name:    "pk_used_serial_",
    
  •   	Table:   "used_serial_",
    
  •   	Columns: []string{"satellite_id", "serial_number"},
    
  •   },
    
  • },
sqlite3 used_serial.db
CREATE TABLE used_serial_ (
                                                satellite_id  BLOB NOT NULL,
                                                serial_number BLOB NOT NULL,
                                                expiration    TIMESTAMP NOT NULL
                                        );
CREATE UNIQUE INDEX pk_used_serial_ ON used_serial_(satellite_id, serial_number);
CREATE INDEX idx_used_serial_ ON used_serial_(expiration);
.exit

Looks like a huge damage to the databases.
Have you had a power outage?

See my previous answer and some questions…

With a new node you will lose the gained reputation and held amount, so better to fix it.

If you want to start over, then you need to generate a new identity, request a new authorization token and start with clean data.

Also satellites.db seems to have this test table as well.

sqlite3 satellites.db
drop table test_table;
.exit