Storj node continue restarting

Good day! Just manually updated storj node from [v1.10.1] and received following logs:

Error: Error starting master database on storagenode: database: database storage_usage does not exist stat config/storage/storage_usage.db: no such file or directory storj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:344 storj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:313 storj.io/storj/storagenode/storagenodedb.Open:245 main.cmdRun:151 storj.io/private/process.cleanup.func1.4:353 storj.io/private/process.cleanup.func1:371 github.com/spf13/cobra.(*Command).execute:840 github.com/spf13/cobra.(*Command).ExecuteC:945 github.com/spf13/cobra.(*Command).Execute:885 storj.io/private/process.ExecWithCustomConfig:88 storj.io/private/process.ExecCustomDebug:70 main.main:330 runtime.main:203

DB file is actually missing …

ls /mnt/st4000/storage bandwidth.db heldamount.db orders.db piece_spaced_used.db satellites.db used_serial.db blobs info.db piece_expiration.db pricing.db temp garbage notifications.db pieceinfo.db reputation.db trash

All other DBs are fine …

./bandwidth.dbok
./orders.dbok
./notifications.dbok
./pricing.dbok
./piece_expiration.dbok
./info.dbok
./satellites.dbok
./used_serial.dbok
./piece_spaced_used.dbok
./pieceinfo.dbok
./heldamount.dbok
./reputation.dbok

Kindly ask your assistance!

Hello @Jeffry,
Welcome to the forum!

  1. Stop the storagenode
  2. Execute either with a local installed sqlite3 or with a docker version:
sqlite3 /mnt/st4000/storage/storage_usage.db
  1. When you see sqlite> prompt execute this script:
CREATE TABLE storage_usage (
                                                satellite_id BLOB NOT NULL,
                                                at_rest_total REAL NOT NUll,
                                                interval_start TIMESTAMP NOT NULL,
                                                PRIMARY KEY (satellite_id, interval_start)
                                        );
.exit
  1. Start the storagenode
  2. Check your logs.
2 Likes

Thank you! Node is back online!
Good lesson for me to backup DBs before manual updating…
Still no idea why whatchtower didn’t upgrade node automatically …

1 Like

Please, stop and remove the watchtower container and update it: https://documentation.storj.io/setup/cli/software-updates#automatic-updates

Already 2 days passed since new version released, yet node wasn’t upgraded by watchtower …
docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6eb3af8c3067 storjlabs/watchtower "/watchtower storage…" 3 days ago Up 3 days watchtower 2bd6d04acd32 storjlabs/storagenode:latest "/entrypoint" 6 days ago Up 6 days 127.0.0.1:14002->14002/tcp, 0.0.0.0:28967->28967/tcp storagenode

The only difference that in manual the containes should have name ““storjlabs/watchtower:latest””

Storj gradually rolls out updates, updating windows GUI nodes first. Don’t expect the docker images to be updated for at least a week after the changelog has been published.

Understood, yet still running v1.11.1…

I wouldn’t be worried. It can take up to 10 days after the changelog is published. The minimum allowed version is currently v1.9.0, and you will see a notification in your web dashboard if your node is running an old version past the expected date. You can force an update if you want, but then you won’t know if watchtower is working properly. If your node isn’t updated by Sept 19th, then I would start looking for problems.

I was looking this post earlier and finally found it:

Great timeline & explanation, thx, waiting then :grinning: