The problem with the watchtower

@littleskunk

I just installed a watchtower:

watch_local

On docker hub I can see:

Is this normal behavior ???

My GUI system that supports i.a. docker claims I have a version from 11 months ago …

omv_created

Here is a workaround :wink:

Summary

1 Like

Please, select a 64 bit version, not 32 bit and compare again


I would like to suggest you to remove the image
docker stop watchtower
docker rm watchtower
docker rmi storjlabs/watchtower:latest

and download it again

docker pull storjlabs/watchtower

Then run it: https://documentation.storj.io/setup/cli/software-updates#automatic-updates

This version doesn’t support rolling updates, which is required function.

So, please update storjlabs/watchtower to the current version and include to the regular update process, or contribute this function (update on randomizing time frame) to the original watchtower like a standard option.
I personally have a workaround with the same function (start watchtower at a random time on each server)

Deleting the image (from rmi) and downloading it again doesn’t change anything, still Digest: sha256: 31c0c558ec8adad5ce … and the version like 11 months ago …

and
docker logs watchtower
shows nothing, logs nothing …

It will be replaced with the storagenode-updater. For the docker version too.

1 Like

What is your distro and arch? uname -a

Debian:
Linux omvzde 4.15.18-30-pve #1 SMP PVE 4.15.18-58 (Fri, 12 Jun 2020 13:53:01 +0200) x86_64 GNU/Linux

and Synology:
Linux synol 4.4.59+ #25426 SMP PREEMPT Wed Jul 8 03:21:29 CST 2020 x86_64 GNU/Linux synology_apollolake_918+

download from some internet cache instead of directly from docker.hub?

Hm, seems it’s correct Digest

storjlabs/watchtower@sha256:31c0c558ec8adad5cec6cb86320c58018289ca22dafa019c31cd5d6f65f03655

So, it’s latest. But what the real problem?
The docker version should be on 1.13.3, it will be updated only after Windows versions, which are on 20% accordingly to

storjlabs / watchtower:
does nothing, log is empty … on Debian, on Synology

containrrr / watchtower:

time = "2020-10-10T12: 16: 59Z" level = debug msg = "Checking containers for updated images"
time = "2020-10-10T12: 16: 59Z" level = debug msg = "Retrieving running containers" ```

etc, etc...

It will do nothing, because there is nothing to do.
Please, wait when the new version of storagenode image will be published.

adding --debug to the command from the official documentation finally showed that the watchtower does something :wink:

@Alexey

can add --debug option to the documentation (link) so that SNO can see in the log what the watchtower is doing ???

better are (docker logs watchtower):

time="2020-10-11T08:35:41Z" level=debug msg="Sleeping for a seconds to ensure the docker api client has been properly initialized."
time="2020-10-11T08:35:42Z" level=debug msg="Retrieving running containers"
time="2020-10-11T08:35:42Z" level=debug msg="There are no additional watchtower containers"
time="2020-10-11T08:35:42Z" level=debug msg="Starting Watchtower and scheduling first run: 2020-10-13 01:44:27 +0000 UTC m=+148125.689440268" 

than an empty log …

Otherwise…

the documentation includes:
“Watchtower will look for new updates to the Docker container on Docker Hub every 6 hours”

and in fact it is:
“time =” 2020-10-11T08: 35: 42Z “level = debug msg =” Starting Watchtower and scheduling first run: 2020-10-13 01:44:27 "

and in my opinion it is not 6 hours …

time="2020-10-09T09:29:05Z" level=debug msg="Scheduled next run: 2020-10-09 22:01:43 +0000 UTC"

More like ~12.5 hours

Hm, it’s random between 12 and 72 hours


Yesterday (2020-10-13) I did a server update and had to restart it …
Here’s what I can see from the storjlabs/watchtower log:

time="2020-10-13T01:44:44Z" level=debug msg="Scheduled next run: 2020-10-14 18:53:12 +0000 UTC"
time="2020-10-13T06:29:49Z" level=info msg="Waiting for running update to be finished..."
time="2020-10-13T06:30:32Z" level=debug msg="Sleeping for a seconds to ensure the docker api client has been properly initialized."
time="2020-10-13T06:30:33Z" level=debug msg="Retrieving running containers"
time="2020-10-13T06:30:33Z" level=debug msg="There are no additional watchtower containers"
time="2020-10-13T06:30:33Z" level=debug msg="Starting Watchtower and scheduling first run: 2020-10-15 15:16:09 +0000 UTC m=+204336.408521214"
time="2020-10-13T07:05:34Z" level=info msg="Waiting for running update to be finished..."
time="2020-10-13T07:06:25Z" level=debug msg="Sleeping for a seconds to ensure the docker api client has been properly initialized."
time="2020-10-13T07:06:26Z" level=debug msg="Retrieving running containers"
time="2020-10-13T07:06:26Z" level=debug msg="There are no additional watchtower containers"
time="2020-10-13T07:06:26Z" level=debug msg="Starting Watchtower and scheduling first run: 2020-10-15 22:31:00 +0000 UTC m=+228274.729756129"

Every restart, the storjlabs/watchtower update time limit is moved … that’s how I got it.
Is this how it is supposed to work?

Yes, as I said before:

1 Like

So something that will restart the watchtower more often than every 12 hours can lead to it not updating anything? (these are theoretical considerations)

1 Like

Seems so. Why do you need to restart it?

1 Like