Getting "Error: Error creating tables for master database on storagenode: migrate: sql: Scan error on column index 0, name "MAX(version)":"

Error: Error creating tables for master database on storagenode: migrate: sql: Scan error on column index 0, name "MAX(version)": converting driver.Value type []uint8 ("\xa2\x8bO\x04\xe1\v\xae\x85\xd6\u007fLl\xb8+\xf8\xd4\xc0\xf0\xf4z\x8e\xa7&'RM\xebn\xc0\x00\x00\x00") to a int64: invalid syntax
	storj.io/storj/private/dbutil/txutil.WithTx:53
	storj.io/storj/private/migrate.(*Migration).getLatestVersion:214
	storj.io/storj/private/migrate.(*Migration).Run:153
	storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:293
	main.cmdRun:180
	storj.io/private/process.cleanup.func1.4:359
	storj.io/private/process.cleanup.func1:377
	github.com/spf13/cobra.(*Command).execute:840
	github.com/spf13/cobra.(*Command).ExecuteC:945
	github.com/spf13/cobra.(*Command).Execute:885
	storj.io/private/process.ExecWithCustomConfig:88
	storj.io/private/process.ExecCustomDebug:70
	main.main:320
	runtime.main:203

I had a malformed DB before that… because it’s morning I may have done something stupid while recovering my orders.db… what can I do to check if indeed that is the issue?

As I’m trying to check integrity of orders.db, I am seeing this:

# sqlite3 orders.db  'PRAGMA INTEGRITY_CHECK;'
*** in database main ***
Main freelist: size is 130 but should be 12333
On tree page 743 cell 7: Rowid 4296 out of order
Page 14507: btreeInitPage() returns error code 11
Page 14508: btreeInitPage() returns error code 11
Page 14509: btreeInitPage() returns error code 11
On tree page 667 cell 7: Rowid 3704 out of order
Page 14491: btreeInitPage() returns error code 11
Page 14510: btreeInitPage() returns error code 11
Page 14512: btreeInitPage() returns error code 11
Page 14513: btreeInitPage() returns error code 11
Page 14514: btreeInitPage() returns error code 11
On tree page 576 cell 7: Rowid 2976 out of order
On tree page 798 cell 7: Rowid 4736 out of order
Page 979: btreeInitPage() returns error code 11
On tree page 564 cell 7: Rowid 2880 out of order
Page 14481: btreeInitPage() returns error code 11
Page 11345: btreeInitPage() returns error code 11
On tree page 549 cell 7: Rowid 2760 out of order
Page 894: btreeInitPage() returns error code 11
On tree page 617 cell 7: Rowid 3304 out of order
Page 14483: btreeInitPage() returns error code 11
Page 1199: btreeInitPage() returns error code 11
Page 14469: btreeInitPage() returns error code 11
Page 1427: btreeInitPage() returns error code 11
On tree page 620 cell 7: Rowid 3328 out of order
On tree page 571 cell 7: Rowid 2936 out of order
On tree page 601 cell 7: Rowid 3176 out of order
Page 896: btreeInitPage() returns error code 11
Page 1215: btreeInitPage() returns error code 11
On tree page 778 cell 7: Rowid 4576 out of order
On tree page 705 cell 7: Rowid 4008 out of order
Page 1263: btreeInitPage() returns error code 11
On tree page 686 cell 7: Rowid 3856 out of order
On tree page 750 cell 7: Rowid 4352 out of order
On tree page 575 cell 7: Rowid 2968 out of order
On tree page 737 cell 7: Rowid 4248 out of order
On tree page 561 cell 7: Rowid 2856 out of order
On tree page 651 cell 7: Rowid 3576 out of order
On tree page 746 cell 7: Rowid 4320 out of order
On tree page 621 cell 7: Rowid 3336 out of order
On tree page 614 cell 7: Rowid 3280 out of order
On tree page 610 cell 7: Rowid 3248 out of order
On tree page 638 cell 7: Rowid 3472 out of order
On tree page 557 cell 7: Rowid 2824 out of order
On tree page 747 cell 7: Rowid 4328 out of order
On tree page 483 cell 7: Rowid 2232 out of order
On tree page 478 cell 7: Rowid 2192 out of order
Page 14482: btreeInitPage() returns error code 11
Page 14487: btreeInitPage() returns error code 11
Page 14489: btreeInitPage() returns error code 11
Page 14490: btreeInitPage() returns error code 11
On tree page 567 cell 7: Rowid 2904 out of order
On tree page 786 cell 7: Rowid 4640 out of order
On tree page 2 cell 146: Rowid 1144 out of order
On tree page 634 cell 7: Rowid 3440 out of order
On tree page 680 cell 7: Rowid 3808 out of order
On tree page 728 cell 7: Rowid 4176 out of order
Page 969: btreeInitPage() returns error code 11
On tree page 462 cell 7: Rowid 2064 out of order
On tree page 2 cell 142: Child page depth differs
On tree page 2 cell 141: Rowid 1104 out of order
On tree page 2 cell 141: Child page depth differs
Page 1155: btreeInitPage() returns error code 11
On tree page 637 cell 7: Rowid 3464 out of order
On tree page 797 cell 7: Rowid 4728 out of order
On tree page 785 cell 7: Rowid 4632 out of order
On tree page 781 cell 7: Rowid 4600 out of order
Page 11327: btreeInitPage() returns error code 11
Page 11325: btreeInitPage() returns error code 11
On tree page 741 cell 7: Rowid 4280 out of order
Page 1482: btreeInitPage() returns error code 11
Page 1175: btreeInitPage() returns error code 11
Page 14560: btreeInitPage() returns error code 11
Page 1082: btreeInitPage() returns error code 11
On tree page 758 cell 7: Rowid 4416 out of order
On tree page 546 cell 7: Rowid 2736 out of order
Page 1102: btreeInitPage() returns error code 11
On tree page 765 cell 7: Rowid 4472 out of order
On tree page 796 cell 7: Rowid 4720 out of order
Page 14485: btreeInitPage() returns error code 11
Page 11312: btreeInitPage() returns error code 11
Page 1458: btreeInitPage() returns error code 11
On tree page 783 cell 7: Rowid 4616 out of order
On tree page 768 cell 7: Rowid 4496 out of order
On tree page 760 cell 7: Rowid 4432 out of order
On tree page 803 cell 7: Rowid 4776 out of order
Page 14511: btreeInitPage() returns error code 11
On tree page 539 cell 7: Rowid 2680 out of order
Page 1265: btreeInitPage() returns error code 11
Page 14517: btreeInitPage() returns error code 11
On tree page 725 cell 7: Rowid 4152 out of order
On tree page 624 cell 7: Rowid 3360 out of order
On tree page 699 cell 7: Rowid 3960 out of order
Page 935: btreeInitPage() returns error code 11
On tree page 807 cell 1: Rowid 4802 out of order
On tree page 720 cell 338: 2nd reference to page 803
On tree page 720 cell 333: 2nd reference to page 798
On tree page 720 cell 332: 2nd reference to page 797
On tree page 720 cell 331: 2nd reference to page 796
On tree page 720 cell 322: 2nd reference to page 787

I’m still trying to figure what I did and see what I can do to fix it.
any help is appreciated

It’s still corrupted. You can use this article to recover what could be recovered:

Please, take a look on sqlite3 version. It should be not older than v3.25.2

And you can start with an empty orders.db database as well (you will lose all unsent orders and will not be paid for them, there should not be a lot of such orders anyway):

  1. Stop the storagenode
  2. Rename the orders.db
  3. Execute either with a local sqlite3 or with a docker version
sqlite3 /path/to/orders.db
CREATE TABLE unsent_order (
                                                satellite_id  BLOB NOT NULL,
                                                serial_number BLOB NOT NULL,

                                                order_limit_serialized BLOB      NOT NULL, -- serialized pb.OrderLimit
                                                order_serialized       BLOB      NOT NULL, -- serialized pb.Order
                                                order_limit_expiration TIMESTAMP NOT NULL, -- when is the deadline for sending it

                                                uplink_cert_id INTEGER NOT NULL,

                                                FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
                                        );
CREATE TABLE order_archive_ (
                                                satellite_id  BLOB NOT NULL,
                                                serial_number BLOB NOT NULL,

                                                order_limit_serialized BLOB NOT NULL,
                                                order_serialized       BLOB NOT NULL,

                                                uplink_cert_id INTEGER NOT NULL,

                                                status      INTEGER   NOT NULL,
                                                archived_at TIMESTAMP NOT NULL,

                                                FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
                                        );
CREATE UNIQUE INDEX idx_orders ON unsent_order(satellite_id, serial_number);
CREATE TABLE versions (version int, commited_at text);
CREATE INDEX idx_order_archived_at ON order_archive_(archived_at);
.exit
  1. Start the storagenode
  2. Check your logs

Thanks Alexey
I’m ok with losing a few orders.

just as a note, I ran into Error creating tables for master database on storagenode: migrate: index idx_order_archived_at already exists after recreating orders.db, but I remember from some other post to simply delete the index and start the node again

Yes, it’s just depends on migration version stored in the database.