Container bootloop: Structure needs cleaning

I assume an update got released and the watchtower applied it automatically, however I seem to be stuck in a bootloop. The container takes about a minute to start, logs failed to check for file existence: stat config/config.yaml: structure needs cleaning and then reboots again.
Any idea whats going on?

Thanks!

Try to remove the config file and restart the node

the config file didnt exist, I tried creating it but it wouldn’t let me (same reason structure needs cleaning) so I googled it and tried e2fsck to ‘fix’ the disk. It deleted everything inside my storagenode data folder.

I can restart the storagenode just fine now. But I assume my reputation will have taken a big hit for this as it deleted a good 200GB worth of files. (Though interestingly enough it only deleted storj files, nothing outside of that folder)

I’m a total linux noob and this was probably an OTT course of action to take. Not sure why it happened in the first place, the power hadn’t been disrupted, afaik everything was fine until watchtower tried updating it.

If this happens again in future, is there a less destructive path I can take?

If it deleted a whole data folder, then your identity will be disqualified sooner or later.
You should download a latest identity binary, subscribe to the wait list again for a new authorization token, generate a new identity and sign it with a new authorization token and start a new node.

Please, update your watchtower:

docker stop watchtower
docker rm watchtower
docker pull storjlabs/watchtower

And follow this guide: https://documentation.storj.io/setup/cli/software-updates#automatic-updates

It deleted the data folder (the app/config mount path in the launch command) , not the one with the identity stuff in it (app/identity mount), so I should be alright here? I restarted and I’ve got bandwidth usage and 20GB stored locally (i assume this wouldnt happen if I did need to re-auth)

I’ve updated my watchtower now. Thanks!

It’ll only work for a bit most likely. If your data is gone, your node will fail audits until it’s disqualified. Take a look at your log to see if you see missing file errors. You’ll likely see a whole bunch of those if the data is indeed gone. I suggest getting a new identity ready for when it eventually is disqualified.

Thanks I’ll try and get a new identity. It seems the sign up form isnt working at the moment https://storj.io/sign-up-node-operator/ can’t click the continue button after checking all the boxes. I’ll keep trying though

Use the Chrome or Firefox and disable all ad blockers extensions.
The Brave doesn’t work

awesome that worked, running with a new identity now. thanks everyone!

1 Like