Need help. Error starting master database on storagenode

Hello - i recently must have fucked something up on my 10TB storj node… Currently when i try to boot the node i get this error logs:

Here is the commands i run to start/setup:

docker run --rm -e SETUP=“true” --mount type=bind,source=“C:\Users\andre\AppData\Roaming\Storj\Identity2\Identity\storagenode”,destination=/app/identity --mount type=bind,source=“E:\StorjData”,destination=/app/config --name storagenode storjlabs/storagenode:latest

docker run -d --restart unless-stopped --stop-timeout 300 -p 28968:28967/tcp -p 28968:28967/udp -p 127.0.0.1:14003:14002 -e WALLET=“0xFBF22d386c10aeb329A8C245e99F9AB4Db92D16d” -e EMAIL="legoandreas@gmail.com" -e ADDRESS=“83.89.251.143:28968” -e STORAGE=“10TB” --mount type=bind,source=“C:\Users\andre\AppData\Roaming\Storj\Identity2\Identity\storagenode”,destination=/app/identity --mount type=bind,source=“E:\StorjData”,destination=/app/config --name storagenode storjlabs/storagenode:latest

Can anyone help me? i am affraid to loose my node.

After a reboot and deletion of the docker file i ran the setup commands again and the node seems to be runnning now… But i am afraid this will happend again so if ANYONE knows what went wrong or what i messed up - please explain. Also i will try to provice all the info i can.

I just saw that i have a 95% suspension - am i fuc**d? :frowning: That whould be over a year of storj going to the “trash” if i get suspended. Is there anything i can do?

So the node just went down again. There is 100% a big problem i cant seem to find. I will wait with booting it until someone can give me a direction of fixing it.

It’s only online that has gone down, stay calm and see if the percentages go back up.
If you stopped it you can do a chdsk

Well europe north has a 95% suspension score. And i can launch the node for only 10 minutes until it crashes again - so something must be wrong?

Try checking the disk

No problems on the disk. something else must be wrong then:

You’re running docker on Windows?

Post the most recent log entries just before last crash.

Yes on windows - can you tell me how to get the latest logs before crash?

It’s not possible unless you’ve redirected the logs - How do I redirect my logs to a file? - Storj Node Operator Docs

Its should be here then:

This is your error…

2023-02-23T13:50:46.201Z	INFO	Configuration loaded	{"Process": "storagenode", "Location": "/app/config/config.yaml"}
2023-02-23T13:50:46.204Z	INFO	Anonymized tracing enabled	{"Process": "storagenode"}
2023-02-23T13:50:46.212Z	INFO	Operator email	{"Process": "storagenode", "Address": "legoandreas@gmail.com"}
2023-02-23T13:50:46.213Z	INFO	Operator wallet	{"Process": "storagenode", "Address": "0xFBF22d386c10aeb329A8C245e99F9AB4Db92D16d"}
2023-02-23T13:50:46.234Z	FATAL	Unrecoverable error	{"Process": "storagenode", "error": "Error starting master database on storagenode: database: info opening file \"config/storage/info.db\" failed: unable to open database file: input/output error\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:331\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:308\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:283\n\tstorj.io/storj/storagenode/storagenodedb.OpenExisting:250\n\tmain.cmdRun:193\n\tstorj.io/private/process.cleanup.func1.4:377\n\tstorj.io/private/process.cleanup.func1:395\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:92\n\tmain.main:478\n\truntime.main:250", "errorVerbose": "Error starting master database on storagenode: database: info opening file \"config/storage/info.db\" failed: unable to open database file: input/output error\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:331\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:308\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:283\n\tstorj.io/storj/storagenode/storagenodedb.OpenExisting:250\n\tmain.cmdRun:193\n\tstorj.io/private/process.cleanup.func1.4:377\n\tstorj.io/private/process.cleanup.func1:395\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:92\n\tmain.main:478\n\truntime.main:250\n\tmain.cmdRun:195\n\tstorj.io/private/process.cleanup.func1.4:377\n\tstorj.io/private/process.cleanup.func1:395\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:92\n\tmain.main:478\n\truntime.main:250"}

Unfortunately it’s docker so I can’t help. At a guess I would assume you’re not looking in the correct location for the databases as it shows unable to open database file: input/output error.

1 Like

never run it for the working node. It’s a one-time command for a new identity only. If you would use it for your working node, you may destroy it. So, please remove it from your batch script.

you have 95% on the online score, not suspension score. To fully recover your node should be online for the next 30 days. Each downtime requires another 30 days to recover.

This error means that your filesystem is corrupted and needs repair. Please stop and remove the storagenode container, then run chkdsk /f e:\ from the elevated cmd/powershell, if everything is fine, but node still crash with this error, then you can recreate the info.db database:

So i ran the windows repair command - but it seems to have done nothing. so time to recreate info.db?

So i just ran the SQLite3 check on all database files - and it seems that it found no errors. Now i feel lost - what to do?

You could try moving the databases and then seeing if the node will start by recreating them.

I just launched the node again - so far it has been runnning for 40 minutes without any issues. and has downloaded approx 1.5GB

Should i just let it run and keep monitoring?

Either

  • you keep it running as is and accept you’ve lost the node statistics up to this point. Earnings are still fine as they’re managed by the satellite.
  • stop the node and try to copy back some of the old databases until you find the one(s) causing the problem.

I never delete any databases - i actually did not change anything… So i dont understand why its working right now - when it was not before…

Also - right now all my logs are written to a file. Should i change that back? that file whould become huge right?