Trouble when releasing new Version

Node Operators troubleshooting
1

Hi all
At the last few Updates I always got troubles on one Node. The docker contrainer was pulled, the old was deleted but the new could not automatically started. I need to start the docker then once manually. After I started id manually everything works well. My Node is now about 11 months old and I got these troubles only since about the last 3 updates. Is there something I can do? The Problem is that when my node is updating at the night and I only see it in the morning. The offline time would be very close or are larger then allowed. I don’t want to be disqualifed in the future for this.

Thanks for help
Markus

2

Are you using watchtower ?

3

Yes, installed and is working. Should i delete watchtower and reinstall it?

1 Like
4

Hey, for now we are working on auto-update logic for docker distribution, including ability to restart docker container after update. We hope these changes will help to avoid this situation.

2 Likes
5

And what should I do at the moment? So i would not be disqualified because of this?

6

I would not worry about DQ right now as DQ for downtime is still disabled. But this is certainly a problem you’ll want to get to the bottom of. I would suggest removing and reinstalling watchtower. And just to be sure we should remove old images.

docker stop -t 300 watchtower
docker rm watchtower
docker ps -a

Make sure there aren’t any instances of watchtower running, possibly under a different name. Then:

docker system prune -a

This will remove old images from the cache. Then reinstall watchtower as per the official docs:
https://documentation.storj.io/setup/cli/software-updates
I think you can add the --debug flag to the watchtower run command to get a bit more in the watchtower logs.

1 Like
7

Do I have to stop and remove the storagenode container before this? Or can I let the storagenode container running?

8

You can leave the storagenode container running. docker system prune only removes old images and current images for non-running containers. So really, it is best to leave the storagenode containers running. More info:

9

Okay, found another watchtower docker is running… Should I stop and delete this?

    CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                                                NAMES
166b8a8fc759        storjlabs/storagenode:latest   "/entrypoint"            34 hours ago        Up 34 hours         0.0.0.0:14002->14002/tcp, 0.0.0.0:28967->28967/tcp   storagenode
5a20d45c9076        84803293c0e3                   "/watchtower storage…"   11 months ago       Up 35 hours                                                              eXrNzqZkfIQnkbTDzmgxbtgezXkcGNmk

Thanks

10

Yes. You should be able to stop and rm it by referencing the container ID shown there.

11

Okay, stopped and removed. Now installing the watchtower as documented. And I will see it at the next updates…

Thanks

2 Likes