Please, do not forget to update your docker run command for the storagenode and watchtower.
This 2 throw error, all others are OK
C:\Users\vuli1>sqlite3 O:/StrojV4/storage/piece_spaced_used.db “PRAGMA integrity_check;”
Error: file is not a database
C:\Users\vuli1>sqlite3 O:/StrojV4/storage/pricing.db “PRAGMA integrity_check;”
Error: file is not a database
File is not a database errors can’t be repaired, so we’re going to have to recreate them.
Rename both files first to back up the old versions just to be sure.
Then do this:
sqlite3 O:/StrojV4/storage/piece_spaced_used.db
CREATE TABLE versions (version int, commited_at text);
CREATE TABLE piece_space_used (
total INTEGER NOT NULL DEFAULT 0,
content_size INTEGER NOT NULL,
satellite_id BLOB
);
CREATE UNIQUE INDEX idx_piece_space_used_satellite_id ON piece_space_used(satellite_id);
.quit
And for the other file:
sqlite3 O:/StrojV4/storage/pricing.db
CREATE TABLE versions (version int, commited_at text);
CREATE TABLE pricing (
satellite_id BLOB NOT NULL,
egress_bandwidth_price bigint NOT NULL,
repair_bandwidth_price bigint NOT NULL,
audit_bandwidth_price bigint NOT NULL,
disk_space_price bigint NOT NULL,
PRIMARY KEY ( satellite_id )
);
.quit
hmm, now that interesting. Thx for your help.
I did why you wrote, made a new storagenode and start it, guess what. It’s restarting.
C:\Users\vuli1>docker logs --tail 20 storagenode
2020-08-05T13:01:16.687Z INFO Operator wallet {“Address”: "0x000000000000000
2020-08-05T13:01:17.654Z INFO Telemetry enabled
2020-08-05T13:01:17.659Z INFO db.migration.35 Create pricing table
Error: Error creating tables for master database on storagenode: migrate: table pricing already exists
storj.io/storj/private/migrate.SQL.Run:274
storj.io/storj/private/migrate.(*Migration).Run.func1:179
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:178
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:295
main.cmdRun:183
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203
Than I deleted this file "pricing " and it started without any problem.
EDIT: One more problem, now I have all my space as a free space instead full. I had 4TB and now it is shouwing like FREE space.
EDIT: No worries, it has been corrected.
I guess you were previously running an older version. This file was added in a database migration step. What you did was the correct way to fix this.
This is because we had to recreate the piece_spaced_used.db. There is no loss of data for that since the node walks the pieces to see how much space is used. That just takes some time after boot, hence why it showed as empty until that was done.
I use windows GUI in windows 10 pro. I got same error on my full 7.6 TB node. Please help me. I’m low tech user.
2020-08-16T23:59:51.389+0700 FATAL Unrecoverable error {“error”: “Error starting master database on storagenode: storage node database error: file is not a database\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:363\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:349\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:313\n\tstorj.io/storj/storagenode/storagenodedb.Open:245\n\tmain.cmdRun:151\n\tstorj.io/private/process.cleanup.func1.4:353\n\tstorj.io/private/process.cleanup.func1:371\n\tgithub.com/spf13/cobra.(*Command).execute:840\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:945\n\tgithub.com/spf13/cobra.(*Command).Execute:885\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:66\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”, “errorVerbose”: “Error starting master database on storagenode: storage node database error: file is not a database\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:363\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:349\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:313\n\tstorj.io/storj/storagenode/storagenodedb.Open:245\n\tmain.cmdRun:151\n\tstorj.io/private/process.cleanup.func1.4:353\n\tstorj.io/private/process.cleanup.func1:371\n\tgithub.com/spf13/cobra.(*Command).execute:840\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:945\n\tgithub.com/spf13/cobra.(*Command).Execute:885\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:66\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n\tmain.cmdRun:153\n\tstorj.io/private/process.cleanup.func1.4:353\n\tstorj.io/private/process.cleanup.func1:371\n\tgithub.com/spf13/cobra.(*Command).execute:840\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:945\n\tgithub.com/spf13/cobra.(*Command).Execute:885\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:66\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}
PS C:\Users\khuat> ls K:\StorjDATA
Directory: K:\StorjDATA
Mode LastWriteTime Length Name
d----- 04/19/2020 00:17 blobs
d----- 08/16/2020 23:07 garbage
d----- 08/16/2020 23:08 temp
d----- 04/22/2020 01:33 trash
-a---- 08/16/2020 23:08 11390976 bandwidth.db
-a---- 08/12/2020 00:06 49152 heldamount.db
-a---- 08/12/2020 09:19 32768 heldamount.db-shm
-a---- 08/14/2020 09:22 1919952 heldamount.db-wal
-a---- 08/16/2020 23:08 16384 info.db
-a---- 08/12/2020 00:06 65536 notifications.db
-a---- 08/12/2020 09:19 32768 notifications.db-shm
-a---- 08/12/2020 09:19 65952 notifications.db-wal
-a---- 08/14/2020 20:40 597999616 orders.db
-a---- 08/12/2020 00:06 24576 pieceinfo.db
-a---- 08/12/2020 09:19 32768 pieceinfo.db-shm
-a---- 08/12/2020 09:19 65952 pieceinfo.db-wal
-a---- 08/12/2020 00:06 229376 piece_expiration.db
-a---- 08/12/2020 09:19 32768 piece_expiration.db-shm
-a---- 08/14/2020 09:57 1462632 piece_expiration.db-wal
-a---- 08/12/2020 00:06 24576 piece_spaced_used.db
-a---- 08/12/2020 09:19 32768 piece_spaced_used.db-shm
-a---- 08/14/2020 20:18 1672752 piece_spaced_used.db-wal
-a---- 08/12/2020 00:06 24576 pricing.db
-a---- 08/12/2020 09:19 32768 pricing.db-shm
-a---- 08/12/2020 09:19 164832 pricing.db-wal
-a---- 08/12/2020 00:06 24576 reputation.db
-a---- 08/12/2020 09:19 32768 reputation.db-shm
-a---- 08/14/2020 17:20 856992 reputation.db-wal
-a---- 08/12/2020 00:06 32768 satellites.db
-a---- 08/12/2020 09:19 32768 satellites.db-shm
-a---- 08/12/2020 09:19 82432 satellites.db-wal
-a---- 08/12/2020 00:06 167936 storage_usage.db
-a---- 08/12/2020 09:19 32768 storage_usage.db-shm
-a---- 08/14/2020 09:22 984712 storage_usage.db-wal
-a---- 08/12/2020 00:06 53084160 used_serial.db
-a---- 08/12/2020 09:19 32768 used_serial.db-shm
-a---- 08/12/2020 09:19 82432 used_serial.db-wal
Please, check all your databases and find out which are unrecoverable corrupted:
FYI - “file is not a database” can’t be fixed by this guide
PS C:\Users\khuat> ls K:\StorjDATA
Directory: K:\StorjDATA
Mode LastWriteTime Length Name
d----- 04/19/2020 00:17 blobs
d----- 08/16/2020 23:07 garbage
d----- 08/16/2020 23:08 temp
d----- 04/22/2020 01:33 trash
-a---- 08/16/2020 23:08 11390976 bandwidth.db
-a---- 08/12/2020 00:06 49152 heldamount.db
-a---- 08/12/2020 09:19 32768 heldamount.db-shm
-a---- 08/14/2020 09:22 1919952 heldamount.db-wal
-a---- 08/16/2020 23:08 16384 info.db
-a---- 08/12/2020 00:06 65536 notifications.db
-a---- 08/12/2020 09:19 32768 notifications.db-shm
-a---- 08/12/2020 09:19 65952 notifications.db-wal
-a---- 08/14/2020 20:40 597999616 orders.db
-a---- 08/12/2020 00:06 24576 pieceinfo.db
-a---- 08/12/2020 09:19 32768 pieceinfo.db-shm
-a---- 08/12/2020 09:19 65952 pieceinfo.db-wal
-a---- 08/12/2020 00:06 229376 piece_expiration.db
-a---- 08/12/2020 09:19 32768 piece_expiration.db-shm
-a---- 08/14/2020 09:57 1462632 piece_expiration.db-wal
-a---- 08/12/2020 00:06 24576 piece_spaced_used.db
-a---- 08/12/2020 09:19 32768 piece_spaced_used.db-shm
-a---- 08/14/2020 20:18 1672752 piece_spaced_used.db-wal
-a---- 08/12/2020 00:06 24576 pricing.db
-a---- 08/12/2020 09:19 32768 pricing.db-shm
-a---- 08/12/2020 09:19 164832 pricing.db-wal
-a---- 08/12/2020 00:06 24576 reputation.db
-a---- 08/12/2020 09:19 32768 reputation.db-shm
-a---- 08/14/2020 17:20 856992 reputation.db-wal
-a---- 08/12/2020 00:06 32768 satellites.db
-a---- 08/12/2020 09:19 32768 satellites.db-shm
-a---- 08/12/2020 09:19 82432 satellites.db-wal
-a---- 08/12/2020 00:06 167936 storage_usage.db
-a---- 08/12/2020 09:19 32768 storage_usage.db-shm
-a---- 08/14/2020 09:22 984712 storage_usage.db-wal
-a---- 08/12/2020 00:06 53084160 used_serial.db
-a---- 08/12/2020 09:19 32768 used_serial.db-shm
-a---- 08/12/2020 09:19 82432 used_serial.db-wal
PS C:\Users\khuat> ls “C:\Program Files\Storj\Storage Node”
Directory: C:\Program Files\Storj\Storage Node
Mode LastWriteTime Length Name
-a---- 08/16/2020 23:34 7525 config.yaml
-a---- 05/16/2020 09:18 7503 config.yaml.bak
-a---- 08/12/2020 09:19 32768 revocations.db
-a---- 05/16/2020 09:16 385 storagenode-updater-recovery.log
-a---- 08/16/2020 23:59 21960688 storagenode-updater.exe
-a---- 08/16/2020 23:59 3851483 storagenode-updater.log
-a---- 08/03/2020 05:12 21952496 storagenode-updater.old.exe
-a---- 08/16/2020 23:59 26130416 storagenode.exe
-a---- 03/04/2020 15:44 24471536 storagenode.old.v0.34.6.exe
-a---- 03/21/2020 01:54 24788464 storagenode.old.v0.35.3.exe
-a---- 03/31/2020 04:08 24788464 storagenode.old.v1.0.1.exe
-a---- 04/07/2020 22:01 24700400 storagenode.old.v1.1.1.exe
-a---- 04/30/2020 23:41 25377776 storagenode.old.v1.3.3.exe
-a---- 05/19/2020 21:03 25094640 storagenode.old.v1.4.2.exe
-a---- 05/30/2020 21:17 24809456 storagenode.old.v1.5.2.exe
-a---- 06/08/2020 20:37 24891888 storagenode.old.v1.6.3.exe
-a---- 06/27/2020 15:38 24892400 storagenode.old.v1.6.4.exe
-a---- 07/21/2020 02:08 24892400 storagenode.old.v1.6.5.exe
-a---- 08/03/2020 05:12 26109936 storagenode.old.v1.9.5.exe
-a---- 06/27/2020 15:38 24892400 storagenode1.exe
-a---- 08/16/2020 23:59 2437 storagenode1.log
-a---- 08/16/2020 23:31 1921391854 storagenode1.log.bak
-a---- 08/16/2020 22:14 1430 trust-cache.json
Please, check database files:
Please help me
PS K:\StorjDATA> Get-ChildItem K:\StorjDATA*.db -File | %{_.Name + " " + (./sqlite3.exe $_.FullName “PRAGMA integrity_check;”)}
bandwidth.db ok
heldamount.db ok
info.db ok
notifications.db ok
Error: file is not a database
orders.db
pieceinfo.db ok
piece_expiration.db ok
piece_spaced_used.db ok
pricing.db ok
reputation.db ok
satellites.db ok
storage_usage.db ok
used_serial.db ok
Please:
- Stop the storagenode (elevated PowerShell):
Stop-Service storagenode
- Remove the
K:\StorjDATA\orders.db - Execute
K:\StorjDATA\sqlite3.exe K:\StorjDATA\orders.db
- When you see the
sqlite>prompt, execute this script:
CREATE TABLE unsent_order (
satellite_id BLOB NOT NULL,
serial_number BLOB NOT NULL,
order_limit_serialized BLOB NOT NULL, -- serialized pb.OrderLimit
order_serialized BLOB NOT NULL, -- serialized pb.Order
order_limit_expiration TIMESTAMP NOT NULL, -- when is the deadline for sending it
uplink_cert_id INTEGER NOT NULL,
FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
);
CREATE TABLE order_archive_ (
satellite_id BLOB NOT NULL,
serial_number BLOB NOT NULL,
order_limit_serialized BLOB NOT NULL,
order_serialized BLOB NOT NULL,
uplink_cert_id INTEGER NOT NULL,
status INTEGER NOT NULL,
archived_at TIMESTAMP NOT NULL,
FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
);
CREATE UNIQUE INDEX idx_orders ON unsent_order(satellite_id, serial_number);
CREATE TABLE versions (version int, commited_at text);
.exit
- Start the storagenode
Start-Service storagenode
- Check your logs
1 Like
It’s online. Thank you!
1 Like
What is the command to run on Windows 10 to see which DB is coming back as not a database? I have sqlite3 installed.The log file still does not show what file it is talking about (this should be fixed). Also, where is a listing of all the DB creation scripts to recreate any that go bad (this is something that should happen if the DB is missing anyways).
Thank you
Yes, have done that before. Now, how do I recreate the corrupted DB? I delete it and it will not start. Maybe a posting with how to recreate each DB? Or have it do it automatically if it is not found?
What is name of the corrupted DB?
I can give you a SQL script to re-create a schema, but I need to know which one.
The DBs are recreating only if there is no DBs at all. This is not an option for your situation.
@Alexey I got new database error. Please help me. Thank you!
PS C:\WINDOWS\system32> cd ‘K:\StorjDATA’
PS K:\StorjDATA>
PS K:\StorjDATA> Get-ChildItem K:\StorjDATA*.db -File | %{_.Name + " " + (./sqlite3.exe $_.FullName “PRAGMA integrity_check;”)}
Error: file is not a database
bandwidth.db
heldamount.db ok
info.db ok
notifications.db ok
orders.db ok
pieceinfo.db ok
piece_expiration.db ok
piece_spaced_used.db ok
pricing.db ok
reputation.db ok
satellites.db ok
storage_usage.db ok
used_serial.db ok
- Stop the storagenode
- Remove the
bandwidth.db - Execute either with a local installed
sqlite3or with a docker version (described in the above guide), replace the/path/to/bandwidth.dbto your actual path:
sqlite3 /path/to/bandwidth.db
- When you see a
sqlite>prompt, execute this script:
CREATE TABLE bandwidth_usage (
satellite_id BLOB NOT NULL,
action INTEGER NOT NULL,
amount BIGINT NOT NULL,
created_at TIMESTAMP NOT NULL
);
CREATE INDEX idx_bandwidth_usage_satellite ON bandwidth_usage(satellite_id);
CREATE INDEX idx_bandwidth_usage_created ON bandwidth_usage(created_at);
CREATE TABLE bandwidth_usage_rollups (
interval_start TIMESTAMP NOT NULL,
satellite_id BLOB NOT NULL,
action INTEGER NOT NULL,
amount BIGINT NOT NULL,
PRIMARY KEY ( interval_start, satellite_id, action )
);
.exit
- Start the storagenode
- Check your logs
2 Likes
It’s working. Thanks you!
1 Like