I use docker-compose for everything on my server and I added Storj and Watchtower to it, my node is still on v1.1.1, is there something i have configured incorrectly?
Storj is pushing rolling updates so docker nodes will be updated slowly. If your watchtower is running smoothly then it will update automatically. Be patient!
If you are using this server for anything else, I would strongly advise against using watchtower as the watchtower container effectively has root access to the entire host system. If you subscribe to the principle of least privilege, watchtower should only be used on systems that do nothing but run Storj nodes.
The storjlabs/storagenode:beta image is published on Docker Hub. Storj publishes many interim containers likely due to an automated CI/CD process, but ultimately only attaches the beta tag to the release containers that SNOs are to pull to their machines. So only when the image is pushed to Docker Hub can a SNO actually get the latest beta tagged release.
Storj also publishes their own fork of the containrrr/watchtower image that updates Docker containers on a host. Watchtower functions to automatically check Docker Hub for the latest images for running containers on a set interval. The upstream image results in a watchtower container that defaults to checking every 300 seconds, or 5 minutes. This can be modified by setting the --interval switch in the watchtower command.
The storjlabs/watchtower fork of the upstream image makes the key modification to set the interval internally to a random number between 12 and 72 hours. Therefore, depending on what the random interval time is generated for a given SNO’s system, it may take up to 72 hours for watchtower to check Docker Hub for an updated beta image after it’s been pushed there by the Storj team.
What is the mechanism for pushing the rolling updates if not via watchtower? They can only leverage the software that is already deployed for Docker-based SNOs.
The 72h starts after the docker image is published, but it may take longer for the docker image to be published. Updates roll out roughly like this.
Though some times the phased rollouts of binary nodes takes a little longer or shorter depending on the needs. The docker image isn’t always published on day 8. But usually around that time.
Thanks for the calendar @BrightSilence. My main message is that people worried about their watchtower installations should rest easy as long as the Docker image is not yet updated on Docker Hub. If we’re in the blue zone on the calendar and people are not getting updated within 72 hours, then they should start questioning whether watchtower isn’t working correctly. Checking the beta tag on Docker Hub allows one to be sure whether the image is available or not.
The difference is the same as -v (your version) and --mount (the must have version):
If you use -v or --volume to bind-mount a file or directory that does not yet exist on the Docker host, -v creates the endpoint for you. It is always created as a directory.
If you use --mount to bind-mount a file or directory that does not yet exist on the Docker host, Docker does not automatically create it for you, but generates an error.
For the storagenode the first behavior is the way to disqualification if your drive doesn’t mounted for any reason, because the data will be stored inside the container and will be removed with docker-compose down
All of my docker containers remain on the same drive, if it’s unmounted nothing would work anyway, but I will update it next time I get around to updating my compose. Does the watchtower setup look correct, and will it touch any container that isn’t the storagenode?
Do the same for watchtower. Otherwise the containers will get a fun but inconsistent names every time to run docker-compose up -d and your watchtower container won’t be able to update them.
Good call, you’re right. I’ve gotten into the habit of naming my containers, so I was mixing this up with running docker run without the --name argument.
Did anybody managed to get proper update interval either via WATCHTOWER_SCHEDULE (–schedule) or WATCHTOWER_POLL_INTERVAL (–interval)?
Doesn’t matter which one i define, after starting storjlabs/watchtower, the log shows start time which is doesn’t align with any of that. “Starting Watchtower and scheduling first run: 2021-05-13 07:12:42 -0400 EDT”.
Restarts of container is another random time value. ENV var or command, “schedule” or “interval”, no difference.