Node content seems unvisible to STORJ after 10 days offline

Hi everybody ! I host a 7TB node with a Rock64 under debian stretch.
I happen to have a special problem :

  • everything was working so fine, the disk was filling, uploading, the dashboard confirming traffic and the Storj coins sent to my wallet
  • after an email from storj labs about the new beta version (10th of march) I didn’t react for 10 days
  • my node went offline after several days
  • I finally opened the link in the mail to open the “Install the auto updater link”
  • so did I install it. It didn’t worked to reload a new storagenode version
  • so I did it manually
  • and the stuff was always restarting (even without the auto updater)
  • so I started stopping, removing, pulling and re running the node on and on
  • finally I removed the “-d” and “–restart unless-stopped” operands to get these messages :
    “2020-03-23T15:49:51.763Z WARN piecestore:monitor Disk space is less than requested. Allocating space {“bytes”: 8230543360}
    2020-03-23T15:49:51.764Z ERROR piecestore:monitor Total disk space less than required minimum {“bytes”: 500000000000}
    2020-03-23T15:49:51.766Z INFO bandwidth Performing bandwidth usage rollups
    2020-03-23T15:49:51.770Z ERROR version Failed to do periodic version check: version control client error: Get https://version.storj.io: context canceled
    Error: piecestore monitor: disk space requirement not met”

It seems the 4,3GB of used storage from my node are unvisible to docker/storj. Thus in a 8TB disk with 4,3TB used my 7TB node just can’t start
What can I do to fix it ?
Should I format my disk and restart for scratch ?

Thank you by advance

ps : docker run -p 28967:28967 -e WALLET=“my wallet” -e EMAIL="st3fc10@yahoo.fr" -e ADDRESS=“my ip:28967” -e BANDWIDTH=“70TB” -e STORAGE=“7TB” --mount type=bind,source="/home/smart/.local/share/storj/identity/storagenode/",destination=/app/identity --mount type=bind,source="/media/smart/Sg_NA8X8LP6/",destination=/app/config --name storagenode storjlabs/storagenode:beta

It seems you are using an automatic mount point in /media for your hdd. Those aren’t reliable and could change name. Please mount your HDD in /mnt using /etc/fstab and adjust the paths in the run command accordingly.

2 Likes

I suspect BrightSilence is right too. This is a common error, here are some tips to avoid it in the future: https://documentation.storj.io/before-you-begin/prerequisites#system-requirements (see the Linux tab)
Good news is your node is not disqualified and your audit score is still extremely high! Let us know how this goes, and thanks for being such a great member of the SNO community!

1 Like

Here is some further explanation about why you should not be using automount with /media:

You have used an GUI automount option (your disk is mounted inside the /media), which works only after login. Moreover, it can mount to a different folder every time.
After reboot, the container will be started before the GUI automount has happened and your node will start to store the data inside the mountpoint instead of the disk. Your node will fail too many audits until you log in. After the login, the data in the mountpoint will be hidden. And you will fail audits again but this time, because the hidden data is inaccessible.
Please, use only static mount for the storage. You can read more here

1 Like

Thank you so much BrightSilence, jennifer and heunland : It worked ! And I even learned the manual mount I delayed for so long.

I just used the “mount” command instead of rewriting “/etc/fstab”

After a short inquiry on my automount /media name I found that, for no reason my mount name had a “1” added at the end of it… Causing all this trouble.

I’ll soon rewire my stuff and send photos of my node on the topic made for that.

Thx

That will only work until you reboot or reattaach your device. Then that mountpoint will be gone again… you need to put it into fstab

1 Like

The effect may be worse than needing to manually mount the drive.

Since the drive will not be mounted upon reboot, but the docker container will probably restart… the storagenode may be running and online without access to the prior data. If this happens, the node will become disqualified rather quickly.

2 Likes

Previously there was a chance the name would be different after a reboot, now it will by definition be gone after a reboot. We pointed you to fstab for a reason. It’s the only way to reliably mount on boot.

Please follow the instructions here:
https://documentation.storj.io/resources/faq/linux-static-mount

1 Like