Error starting master database on storagenode: stoage node database error: file is not a database

PS G:\Storj\storage> sls “storage.path:” ‘C:\Program Files\Storj\Storage Node\config.yaml’

C:\Program Files\Storj\Storage Node\config.yaml:158:storage.path: G:\Storj\storage\

docker run -d --restart unless-stopped -p 28967:28967 -p 127.0.0.1:14002:14002 -e WALLET=“censor” -e EMAIL=“censor” -e ADDRESS=“censor:28967” -e BANDWIDTH=“2TB” -e STORAGE=“500GB” --mount type=bind,source=“C:\Users\censor\AppData\Roaming\Storj\Identity\storagenode”,destination=/app/identity --mount type=bind,source=“g:\storj”,destination=/app/config --name storagenode storjlabs/storagenode:beta

The bandwidth and storage have since increased since I last used the docker version (around June 2019 I think is when I transitioned to Windows GUI).

Also, sorry for the delay, was asleep.

Ok, but from where is G:\Storj\blobs and database files in the G:\Storj\?

To fix the orders.db:

  1. Stop the storagenode
  2. Remove the G:\Storj\storage\orders.db
  3. Execute
sqlite3 G:\Storj\storage\orders.db
CREATE TABLE unsent_order (
                                                satellite_id  BLOB NOT NULL,
                                                serial_number BLOB NOT NULL,
                                                order_limit_serialized BLOB      NOT NULL, -- serialized pb.OrderLimit
                                                order_serialized       BLOB      NOT NULL, -- serialized pb.Order
                                                order_limit_expiration TIMESTAMP NOT NULL, -- when is the deadline for sending it
                                                uplink_cert_id INTEGER NOT NULL,
                                                FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
                                        );
CREATE TABLE order_archive_ (
                                                satellite_id  BLOB NOT NULL,
                                                serial_number BLOB NOT NULL,
                                                order_limit_serialized BLOB NOT NULL,
                                                order_serialized       BLOB NOT NULL,
                                                uplink_cert_id INTEGER NOT NULL,
                                                status      INTEGER   NOT NULL,
                                                archived_at TIMESTAMP NOT NULL,
                                                FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
                                        );
CREATE UNIQUE INDEX idx_orders ON unsent_order(satellite_id, serial_number);
CREATE TABLE versions (version int, commited_at text);
.exit
  1. Start the storagenode
  2. Check your logs

I checked Storj/blobs and it looks like that folder is empty currently while storj/storage/blobs has 6 folders.
I ran what you typed in and it encountered errors at order_serialized BLOB NOT NULL, – serialized pb.Order during unsent_order:

At line:4 char:91
+ ...                    order_limit_serialized BLOB      NOT NULL, -- seri ...
+                                                                  ~
Missing expression after ',' in pipeline element.
At line:4 char:119
+ ... rder_limit_serialized BLOB      NOT NULL, -- serialized pb.OrderLimit
+                                                                          ~
Missing closing ')' in expression.
At line:5 char:91
+ ...                    order_serialized       BLOB      NOT NULL, -- seri ...
+                                                                  ~
Missing expression after ',' in pipeline element.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : MissingExpression

Should I start the node anyway?

Looks like a copy-paste issue.
Please, run the

sqlite3 G:\Storj\storage\orders.db

first.
When you see a sqlite> prompt, copy the remained part (started from CREATE)

Seems to be running fine now after doing that, here’s the output from the log.

2020-08-01T13:22:57.379-0500	INFO	Telemetry enabled
2020-08-01T13:22:57.386-0500	INFO	db.migration.27	Add index archived_at to ordersDB
2020-08-01T13:22:57.424-0500	INFO	db.migration	Database Version	{"version": 43}
2020-08-01T13:22:58.386-0500	INFO	preflight:localtime	start checking local system clock with trusted satellites' system clock.
2020-08-01T13:22:59.255-0500	INFO	preflight:localtime	local system clock is in sync with trusted satellites' system clock.
2020-08-01T13:22:59.256-0500	INFO	bandwidth	Performing bandwidth usage rollups
2020-08-01T13:22:59.257-0500	INFO	Node 12CMjYzRsLfx6GAjMTPg9JQ8sJWwpTuDNmVUEMR1Kavd77pgMpv started
2020-08-01T13:22:59.257-0500	INFO	Public server started on [::]:28967
2020-08-01T13:22:59.257-0500	INFO	Private server started on 127.0.0.1:7778
2020-08-01T13:22:59.259-0500	INFO	trust	Scheduling next refresh	{"after": "3h40m56.956315756s"}

Based on the total disk space module on the dashboard, it seems to be successfully grabbing everything correctly.

1 Like

5 posts were split to a new topic: Error starting master database on storagenode: storage node database error: unable to open database file: operation not permitted

same here. just happen. I am on docker too. Windows 10

hm, i tried but still do not start.
I removed storagenode
add it back with docker run -d --privileged --restart unless-stopped…

storagenode is reastaring

Please, show 20 last lines from the log

storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203
2020-08-05T08:24:42.300Z INFO Configuration loaded {“Location”: “/app/config/config.yaml”}
2020-08-05T08:24:42.314Z INFO Operator email {“Address”: “0000@hotmail.com”}
2020-08-05T08:24:42.314Z INFO Operator wallet {“Address”: “0x000000000000000000000000”}
Error: Error starting master database on storagenode: storage node database error: file is not a database
storj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:272
storj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:212
storj.io/storj/storagenode/storagenodedb.New:174
main.cmdRun:150
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203

This is completely different issue. You have a write cache enabled on your disk and storagenode was abruptly stopped without clear shutdown for some reason.
Perhaps you even did not update the docker run command to the latest one: https://documentation.storj.io/setup/cli/storage-node#running-the-storage-node
And the watchtower command: https://documentation.storj.io/setup/cli/software-updates#automatic-updates

Regarding how to fix this issue:

  1. Stop and remove the storagenode container
docker stop -t 300 storagenode
docker rm storagenode
  1. Revert back the Docker desktop to 2.1.0.5
  2. Disable write cache on the disk with data
  3. Check all databases to figure out which one is unrecoverable corrupted:

5. Post here, what database should be created from scratch

I am gonna try, but write cash is disabled on this drive. Could this with update?
2. docker is already 2.1.0.5
3. write cash already disabled.

  1. :face_with_monocle:

Please, do not forget to update your docker run command for the storagenode and watchtower.

This 2 throw error, all others are OK

C:\Users\vuli1>sqlite3 O:/StrojV4/storage/piece_spaced_used.db “PRAGMA integrity_check;”
Error: file is not a database

C:\Users\vuli1>sqlite3 O:/StrojV4/storage/pricing.db “PRAGMA integrity_check;”
Error: file is not a database

File is not a database errors can’t be repaired, so we’re going to have to recreate them.

Rename both files first to back up the old versions just to be sure.

Then do this:

sqlite3 O:/StrojV4/storage/piece_spaced_used.db

CREATE TABLE versions (version int, commited_at text);
CREATE TABLE piece_space_used (
                                                total INTEGER NOT NULL DEFAULT 0,
                                                content_size INTEGER NOT NULL,
                                                satellite_id BLOB
                                        );
CREATE UNIQUE INDEX idx_piece_space_used_satellite_id ON piece_space_used(satellite_id);

.quit

And for the other file:

sqlite3 O:/StrojV4/storage/pricing.db

CREATE TABLE versions (version int, commited_at text);
CREATE TABLE pricing (
                                                satellite_id BLOB NOT NULL,
                                                egress_bandwidth_price bigint NOT NULL,
                                                repair_bandwidth_price bigint NOT NULL,
                                                audit_bandwidth_price bigint NOT NULL,
                                                disk_space_price bigint NOT NULL,
                                                PRIMARY KEY ( satellite_id )
                                        );

.quit

hmm, now that interesting. Thx for your help.
I did why you wrote, made a new storagenode and start it, guess what. It’s restarting.

C:\Users\vuli1>docker logs --tail 20 storagenode
2020-08-05T13:01:16.687Z INFO Operator wallet {“Address”: "0x000000000000000
2020-08-05T13:01:17.654Z INFO Telemetry enabled
2020-08-05T13:01:17.659Z INFO db.migration.35 Create pricing table
Error: Error creating tables for master database on storagenode: migrate: table pricing already exists
storj.io/storj/private/migrate.SQL.Run:274
storj.io/storj/private/migrate.(*Migration).Run.func1:179
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:178
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:295
main.cmdRun:183
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203

Than I deleted this file "pricing " and it started without any problem.

EDIT: One more problem, now I have all my space as a free space instead full. I had 4TB and now it is shouwing like FREE space.

EDIT: No worries, it has been corrected.

I guess you were previously running an older version. This file was added in a database migration step. What you did was the correct way to fix this.

This is because we had to recreate the piece_spaced_used.db. There is no loss of data for that since the node walks the pieces to see how much space is used. That just takes some time after boot, hence why it showed as empty until that was done.

I use windows GUI in windows 10 pro. I got same error on my full 7.6 TB node. Please help me. I’m low tech user.

2020-08-16T23:59:51.389+0700 FATAL Unrecoverable error {“error”: “Error starting master database on storagenode: storage node database error: file is not a database\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:363\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:349\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:313\n\tstorj.io/storj/storagenode/storagenodedb.Open:245\n\tmain.cmdRun:151\n\tstorj.io/private/process.cleanup.func1.4:353\n\tstorj.io/private/process.cleanup.func1:371\n\tgithub.com/spf13/cobra.(*Command).execute:840\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:945\n\tgithub.com/spf13/cobra.(*Command).Execute:885\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:66\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”, “errorVerbose”: “Error starting master database on storagenode: storage node database error: file is not a database\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:363\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:349\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:313\n\tstorj.io/storj/storagenode/storagenodedb.Open:245\n\tmain.cmdRun:151\n\tstorj.io/private/process.cleanup.func1.4:353\n\tstorj.io/private/process.cleanup.func1:371\n\tgithub.com/spf13/cobra.(*Command).execute:840\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:945\n\tgithub.com/spf13/cobra.(*Command).Execute:885\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:66\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n\tmain.cmdRun:153\n\tstorj.io/private/process.cleanup.func1.4:353\n\tstorj.io/private/process.cleanup.func1:371\n\tgithub.com/spf13/cobra.(*Command).execute:840\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:945\n\tgithub.com/spf13/cobra.(*Command).Execute:885\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:66\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}

PS C:\Users\khuat> ls K:\StorjDATA

Directory: K:\StorjDATA

Mode LastWriteTime Length Name


d----- 04/19/2020 00:17 blobs
d----- 08/16/2020 23:07 garbage
d----- 08/16/2020 23:08 temp
d----- 04/22/2020 01:33 trash
-a---- 08/16/2020 23:08 11390976 bandwidth.db
-a---- 08/12/2020 00:06 49152 heldamount.db
-a---- 08/12/2020 09:19 32768 heldamount.db-shm
-a---- 08/14/2020 09:22 1919952 heldamount.db-wal
-a---- 08/16/2020 23:08 16384 info.db
-a---- 08/12/2020 00:06 65536 notifications.db
-a---- 08/12/2020 09:19 32768 notifications.db-shm
-a---- 08/12/2020 09:19 65952 notifications.db-wal
-a---- 08/14/2020 20:40 597999616 orders.db
-a---- 08/12/2020 00:06 24576 pieceinfo.db
-a---- 08/12/2020 09:19 32768 pieceinfo.db-shm
-a---- 08/12/2020 09:19 65952 pieceinfo.db-wal
-a---- 08/12/2020 00:06 229376 piece_expiration.db
-a---- 08/12/2020 09:19 32768 piece_expiration.db-shm
-a---- 08/14/2020 09:57 1462632 piece_expiration.db-wal
-a---- 08/12/2020 00:06 24576 piece_spaced_used.db
-a---- 08/12/2020 09:19 32768 piece_spaced_used.db-shm
-a---- 08/14/2020 20:18 1672752 piece_spaced_used.db-wal
-a---- 08/12/2020 00:06 24576 pricing.db
-a---- 08/12/2020 09:19 32768 pricing.db-shm
-a---- 08/12/2020 09:19 164832 pricing.db-wal
-a---- 08/12/2020 00:06 24576 reputation.db
-a---- 08/12/2020 09:19 32768 reputation.db-shm
-a---- 08/14/2020 17:20 856992 reputation.db-wal
-a---- 08/12/2020 00:06 32768 satellites.db
-a---- 08/12/2020 09:19 32768 satellites.db-shm
-a---- 08/12/2020 09:19 82432 satellites.db-wal
-a---- 08/12/2020 00:06 167936 storage_usage.db
-a---- 08/12/2020 09:19 32768 storage_usage.db-shm
-a---- 08/14/2020 09:22 984712 storage_usage.db-wal
-a---- 08/12/2020 00:06 53084160 used_serial.db
-a---- 08/12/2020 09:19 32768 used_serial.db-shm
-a---- 08/12/2020 09:19 82432 used_serial.db-wal

Please, check all your databases and find out which are unrecoverable corrupted:


FYI - “file is not a database” can’t be fixed by this guide