I checked Storj/blobs and it looks like that folder is empty currently while storj/storage/blobs has 6 folders.
I ran what you typed in and it encountered errors at order_serialized BLOB NOT NULL, – serialized pb.Order during unsent_order:
At line:4 char:91
+ ... order_limit_serialized BLOB NOT NULL, -- seri ...
+ ~
Missing expression after ',' in pipeline element.
At line:4 char:119
+ ... rder_limit_serialized BLOB NOT NULL, -- serialized pb.OrderLimit
+ ~
Missing closing ')' in expression.
At line:5 char:91
+ ... order_serialized BLOB NOT NULL, -- seri ...
+ ~
Missing expression after ',' in pipeline element.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : MissingExpression
Should I start the node anyway?
Alexey
August 1, 2020, 6:10pm
13
Looks like a copy-paste issue.
Please, run the
sqlite3 G:\Storj\storage\orders.db
first.
When you see a sqlite>
prompt, copy the remained part (started from CREATE
)
Seems to be running fine now after doing that, here’s the output from the log.
2020-08-01T13:22:57.379-0500 INFO Telemetry enabled
2020-08-01T13:22:57.386-0500 INFO db.migration.27 Add index archived_at to ordersDB
2020-08-01T13:22:57.424-0500 INFO db.migration Database Version {"version": 43}
2020-08-01T13:22:58.386-0500 INFO preflight:localtime start checking local system clock with trusted satellites' system clock.
2020-08-01T13:22:59.255-0500 INFO preflight:localtime local system clock is in sync with trusted satellites' system clock.
2020-08-01T13:22:59.256-0500 INFO bandwidth Performing bandwidth usage rollups
2020-08-01T13:22:59.257-0500 INFO Node 12CMjYzRsLfx6GAjMTPg9JQ8sJWwpTuDNmVUEMR1Kavd77pgMpv started
2020-08-01T13:22:59.257-0500 INFO Public server started on [::]:28967
2020-08-01T13:22:59.257-0500 INFO Private server started on 127.0.0.1:7778
2020-08-01T13:22:59.259-0500 INFO trust Scheduling next refresh {"after": "3h40m56.956315756s"}
Based on the total disk space module on the dashboard, it seems to be successfully grabbing everything correctly.
1 Like
Alexey
Split this topic
August 4, 2020, 8:23pm
15
bovcan
August 5, 2020, 7:05am
16
same here. just happen. I am on docker too. Windows 10
bovcan
August 5, 2020, 7:38am
17
hm, i tried but still do not start.
I removed storagenode
add it back with docker run -d --privileged --restart unless-stopped…
storagenode is reastaring
Alexey
August 5, 2020, 7:46am
18
Please, show 20 last lines from the log
bovcan
August 5, 2020, 8:25am
19
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203
2020-08-05T08:24:42.300Z INFO Configuration loaded {“Location”: “/app/config/config.yaml”}
2020-08-05T08:24:42.314Z INFO Operator email {“Address”: “0000@hotmail.com ”}
2020-08-05T08:24:42.314Z INFO Operator wallet {“Address”: “0x000000000000000000000000”}
Error: Error starting master database on storagenode: storage node database error: file is not a database
storj.io/storj/storagenode/storagenodedb.(*DB ).openDatabase:272
storj.io/storj/storagenode/storagenodedb.(*DB ).openDatabases:212
storj.io/storj/storagenode/storagenodedb.New:174
main.cmdRun:150
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command ).execute:840
github.com/spf13/cobra.(*Command ).ExecuteC:945
github.com/spf13/cobra.(*Command ).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203
Alexey
August 5, 2020, 8:49am
20
This is completely different issue. You have a write cache enabled on your disk and storagenode was abruptly stopped without clear shutdown for some reason.
Perhaps you even did not update the docker run
command to the latest one: https://documentation.storj.io/setup/cli/storage-node#running-the-storage-node
And the watchtower command: https://documentation.storj.io/setup/cli/software-updates#automatic-updates
Regarding how to fix this issue:
Stop and remove the storagenode container
docker stop -t 300 storagenode
docker rm storagenode
Revert back the Docker desktop to 2.1.0.5
Disable write cache on the disk with data
Check all databases to figure out which one is unrecoverable corrupted:
5. Post here, what database should be created from scratch
bovcan
August 5, 2020, 8:57am
21
I am gonna try, but write cash is disabled on this drive. Could this with update?
2. docker is already 2.1.0.5
3. write cash already disabled.
Alexey
August 5, 2020, 9:02am
22
Please, do not forget to update your docker run
command for the storagenode and watchtower.
bovcan
August 5, 2020, 9:25am
23
This 2 throw error, all others are OK
C:\Users\vuli1>sqlite3 O:/StrojV4/storage/piece_spaced_used.db “PRAGMA integrity_check;”
Error: file is not a database
C:\Users\vuli1>sqlite3 O:/StrojV4/storage/pricing.db “PRAGMA integrity_check;”
Error: file is not a database
File is not a database errors can’t be repaired, so we’re going to have to recreate them.
Rename both files first to back up the old versions just to be sure.
Then do this:
sqlite3 O:/StrojV4/storage/piece_spaced_used.db
CREATE TABLE versions (version int, commited_at text);
CREATE TABLE piece_space_used (
total INTEGER NOT NULL DEFAULT 0,
content_size INTEGER NOT NULL,
satellite_id BLOB
);
CREATE UNIQUE INDEX idx_piece_space_used_satellite_id ON piece_space_used(satellite_id);
.quit
And for the other file:
sqlite3 O:/StrojV4/storage/pricing.db
CREATE TABLE versions (version int, commited_at text);
CREATE TABLE pricing (
satellite_id BLOB NOT NULL,
egress_bandwidth_price bigint NOT NULL,
repair_bandwidth_price bigint NOT NULL,
audit_bandwidth_price bigint NOT NULL,
disk_space_price bigint NOT NULL,
PRIMARY KEY ( satellite_id )
);
.quit
bovcan
August 5, 2020, 1:05pm
25
hmm, now that interesting. Thx for your help.
I did why you wrote, made a new storagenode and start it, guess what. It’s restarting.
C:\Users\vuli1>docker logs --tail 20 storagenode
2020-08-05T13:01:16.687Z INFO Operator wallet {“Address”: "0x000000000000000
2020-08-05T13:01:17.654Z INFO Telemetry enabled
2020-08-05T13:01:17.659Z INFO db.migration.35 Create pricing table
Error: Error creating tables for master database on storagenode: migrate: table pricing already exists
storj.io/storj/private/migrate.SQL.Run:274
storj.io/storj/private/migrate.(*Migration ).Run.func1:179
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration ).Run:178
storj.io/storj/storagenode/storagenodedb.(*DB ).MigrateToLatest:295
main.cmdRun:183
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command ).execute:840
github.com/spf13/cobra.(*Command ).ExecuteC:945
github.com/spf13/cobra.(*Command ).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203
Than I deleted this file "pricing " and it started without any problem.
EDIT: One more problem, now I have all my space as a free space instead full. I had 4TB and now it is shouwing like FREE space.
EDIT: No worries, it has been corrected.
I guess you were previously running an older version. This file was added in a database migration step. What you did was the correct way to fix this.
bovcan:
EDIT: One more problem, now I have all my space as a free space instead full. I had 4TB and now it is shouwing like FREE space.
EDIT: No worries, it has been corrected.
This is because we had to recreate the piece_spaced_used.db. There is no loss of data for that since the node walks the pieces to see how much space is used. That just takes some time after boot, hence why it showed as empty until that was done.
PS C:\Users\khuat> ls K:\StorjDATA
Directory: K:\StorjDATA
Mode LastWriteTime Length Name
d----- 04/19/2020 00:17 blobs
d----- 08/16/2020 23:07 garbage
d----- 08/16/2020 23:08 temp
d----- 04/22/2020 01:33 trash
-a---- 08/16/2020 23:08 11390976 bandwidth.db
-a---- 08/12/2020 00:06 49152 heldamount.db
-a---- 08/12/2020 09:19 32768 heldamount.db-shm
-a---- 08/14/2020 09:22 1919952 heldamount.db-wal
-a---- 08/16/2020 23:08 16384 info.db
-a---- 08/12/2020 00:06 65536 notifications.db
-a---- 08/12/2020 09:19 32768 notifications.db-shm
-a---- 08/12/2020 09:19 65952 notifications.db-wal
-a---- 08/14/2020 20:40 597999616 orders.db
-a---- 08/12/2020 00:06 24576 pieceinfo.db
-a---- 08/12/2020 09:19 32768 pieceinfo.db-shm
-a---- 08/12/2020 09:19 65952 pieceinfo.db-wal
-a---- 08/12/2020 00:06 229376 piece_expiration.db
-a---- 08/12/2020 09:19 32768 piece_expiration.db-shm
-a---- 08/14/2020 09:57 1462632 piece_expiration.db-wal
-a---- 08/12/2020 00:06 24576 piece_spaced_used.db
-a---- 08/12/2020 09:19 32768 piece_spaced_used.db-shm
-a---- 08/14/2020 20:18 1672752 piece_spaced_used.db-wal
-a---- 08/12/2020 00:06 24576 pricing.db
-a---- 08/12/2020 09:19 32768 pricing.db-shm
-a---- 08/12/2020 09:19 164832 pricing.db-wal
-a---- 08/12/2020 00:06 24576 reputation.db
-a---- 08/12/2020 09:19 32768 reputation.db-shm
-a---- 08/14/2020 17:20 856992 reputation.db-wal
-a---- 08/12/2020 00:06 32768 satellites.db
-a---- 08/12/2020 09:19 32768 satellites.db-shm
-a---- 08/12/2020 09:19 82432 satellites.db-wal
-a---- 08/12/2020 00:06 167936 storage_usage.db
-a---- 08/12/2020 09:19 32768 storage_usage.db-shm
-a---- 08/14/2020 09:22 984712 storage_usage.db-wal
-a---- 08/12/2020 00:06 53084160 used_serial.db
-a---- 08/12/2020 09:19 32768 used_serial.db-shm
-a---- 08/12/2020 09:19 82432 used_serial.db-wal
Alexey
August 16, 2020, 5:07pm
29
Please, check all your databases and find out which are unrecoverable corrupted:
FYI - “file is not a database” can’t be fixed by this guide
PS C:\Users\khuat> ls K:\StorjDATA
Directory: K:\StorjDATA
Mode LastWriteTime Length Name
d----- 04/19/2020 00:17 blobs
d----- 08/16/2020 23:07 garbage
d----- 08/16/2020 23:08 temp
d----- 04/22/2020 01:33 trash
-a---- 08/16/2020 23:08 11390976 bandwidth.db
-a---- 08/12/2020 00:06 49152 heldamount.db
-a---- 08/12/2020 09:19 32768 heldamount.db-shm
-a---- 08/14/2020 09:22 1919952 heldamount.db-wal
-a---- 08/16/2020 23:08 16384 info.db
-a---- 08/12/2020 00:06 65536 notifications.db
-a---- 08/12/2020 09:19 32768 notifications.db-shm
-a---- 08/12/2020 09:19 65952 notifications.db-wal
-a---- 08/14/2020 20:40 597999616 orders.db
-a---- 08/12/2020 00:06 24576 pieceinfo.db
-a---- 08/12/2020 09:19 32768 pieceinfo.db-shm
-a---- 08/12/2020 09:19 65952 pieceinfo.db-wal
-a---- 08/12/2020 00:06 229376 piece_expiration.db
-a---- 08/12/2020 09:19 32768 piece_expiration.db-shm
-a---- 08/14/2020 09:57 1462632 piece_expiration.db-wal
-a---- 08/12/2020 00:06 24576 piece_spaced_used.db
-a---- 08/12/2020 09:19 32768 piece_spaced_used.db-shm
-a---- 08/14/2020 20:18 1672752 piece_spaced_used.db-wal
-a---- 08/12/2020 00:06 24576 pricing.db
-a---- 08/12/2020 09:19 32768 pricing.db-shm
-a---- 08/12/2020 09:19 164832 pricing.db-wal
-a---- 08/12/2020 00:06 24576 reputation.db
-a---- 08/12/2020 09:19 32768 reputation.db-shm
-a---- 08/14/2020 17:20 856992 reputation.db-wal
-a---- 08/12/2020 00:06 32768 satellites.db
-a---- 08/12/2020 09:19 32768 satellites.db-shm
-a---- 08/12/2020 09:19 82432 satellites.db-wal
-a---- 08/12/2020 00:06 167936 storage_usage.db
-a---- 08/12/2020 09:19 32768 storage_usage.db-shm
-a---- 08/14/2020 09:22 984712 storage_usage.db-wal
-a---- 08/12/2020 00:06 53084160 used_serial.db
-a---- 08/12/2020 09:19 32768 used_serial.db-shm
-a---- 08/12/2020 09:19 82432 used_serial.db-wal
PS C:\Users\khuat> ls “C:\Program Files\Storj\Storage Node”
Directory: C:\Program Files\Storj\Storage Node
Mode LastWriteTime Length Name
-a---- 08/16/2020 23:34 7525 config.yaml
-a---- 05/16/2020 09:18 7503 config.yaml.bak
-a---- 08/12/2020 09:19 32768 revocations.db
-a---- 05/16/2020 09:16 385 storagenode-updater-recovery.log
-a---- 08/16/2020 23:59 21960688 storagenode-updater.exe
-a---- 08/16/2020 23:59 3851483 storagenode-updater.log
-a---- 08/03/2020 05:12 21952496 storagenode-updater.old.exe
-a---- 08/16/2020 23:59 26130416 storagenode.exe
-a---- 03/04/2020 15:44 24471536 storagenode.old.v0.34.6.exe
-a---- 03/21/2020 01:54 24788464 storagenode.old.v0.35.3.exe
-a---- 03/31/2020 04:08 24788464 storagenode.old.v1.0.1.exe
-a---- 04/07/2020 22:01 24700400 storagenode.old.v1.1.1.exe
-a---- 04/30/2020 23:41 25377776 storagenode.old.v1.3.3.exe
-a---- 05/19/2020 21:03 25094640 storagenode.old.v1.4.2.exe
-a---- 05/30/2020 21:17 24809456 storagenode.old.v1.5.2.exe
-a---- 06/08/2020 20:37 24891888 storagenode.old.v1.6.3.exe
-a---- 06/27/2020 15:38 24892400 storagenode.old.v1.6.4.exe
-a---- 07/21/2020 02:08 24892400 storagenode.old.v1.6.5.exe
-a---- 08/03/2020 05:12 26109936 storagenode.old.v1.9.5.exe
-a---- 06/27/2020 15:38 24892400 storagenode1.exe
-a---- 08/16/2020 23:59 2437 storagenode1.log
-a---- 08/16/2020 23:31 1921391854 storagenode1.log.bak
-a---- 08/16/2020 22:14 1430 trust-cache.json
Alexey
August 16, 2020, 5:09pm
31
Please, check database files:
Please, check all your databases and find out which are unrecoverable corrupted:
FYI - “file is not a database” can’t be fixed by this guide