Master database corrupt after watchtower upgrades storagenode

My Storj node won’t start. Is anything wrong with my run command? (I ran the setup instructions many months ago)

docker run -d --restart unless-stopped --stop-timeout 300 \ -p 28967:28967 \ -p \ -e WALLET="<wallet>" \ -e EMAIL="" \ -e ADDRESS="" \ -e STORAGE="9.5TB" \ --mount type=bind,source="/home/jb/.local/share/storj/identity/storagenode",destination=/app/identity \ --mount type=bind,source="/mnt/hangten/storjFarm/node1",destination=/app/config \ --name storagenode storjlabs/storagenode


Error: Error starting master database on storagenode: database: file is not a database*DB).openDatabase:323*DB).openExistingDatabase:305*DB).openDatabases:281 main.cmdRun:160*Command).execute:842*Command).ExecuteC:950*Command).Execute:887 main.main:385 runtime.main:204

Nothing has changed on this machine. It’s been running fine for months. Watchtower logs show

time="2021-02-07T04:35:30Z" level=info msg="Found new storjlabs/storagenode:latest image (sha256:426ced3a1c89ed400d1db44437295f38d85c448221710bf50f15a76f93b8bcf5)" time="2021-02-07T04:35:31Z" level=info msg="Stopping /storagenode (d9ff0e4b1a27f5b2af88312c0361c162bfd00ec2a0b62b815429ee0c43ec3d66) with SIGTERM" time="2021-02-07T04:35:36Z" level=info msg="Creating /storagenode" time="2021-02-07T04:35:43Z" level=info msg="Removing image sha256:7d88b04aa4ed02967871ae4f5d378c87ce6db59ea22d33b839f89f90becb62cc" time="2021-02-10T01:44:30Z" level=info msg="Waiting for running update to be finished..."

Try steps from here and update the post.

I’ve completed the steps for fixing malformed dbs, but several resulted in empty db files (size=0). Are there any other recovery steps, or must I wipe the node data and start over?

What databases was corrupted?
If you give me a name, I can give you a script to recover them. You can remove all zeroed databases - they are unrecoverable.
The other easy way to restore databases would be:

  1. Stop the storagenode
  2. Move all healthy databases to temporary folder
  3. Start your node and check logs when databases got created:
  4. Stop the node when databases are created
  5. Move backed up databases to the original place and replace created ones
  6. Start storagenode again - now it should not complain anymore.