Following install instructions Software Updates - Storj Docs I had watchtower running stably for years, but as of a few months ago it started crashing with the error log Watchtower crashes reporting client version is too old and restart looping, similar to Storjlabs/watchtower stop after update docker?. I see some workarounds there, and the claim that watchtower isn’t needed.
Given the official docs saying:
We recommend that you install Watchtower in the event that the base image needs updating.
Is there an official position on the handling here? e.g. Watchtower updates to fix the issue, doc updates to recommend against watchtower, or something else?
You can use the mentioned workaround to make it work. You can also do not use watchtower and update the base image manually, it’s not updated too often, and the node inside will update itself automatically anyway.
We updating a base image with security updates as needed.
I hope you’re using a different watchtower. The base one has been basically abandoned and it’s not safe to use anymore. It’s been over a year since an update.
There are a couple forks out there. I just don’t have it handy to give you the name if you didn’t know that.
Why isn’t safe? If isn’t safe Storj should take care of it. The official docs point to the Storj’s fork of the Watchtower, so it’s their business to take measures. I just follow the docs.
The actual solution is to not use docker. Then you won’t need watchtower. Use container management that does not suck. Podman, for example, supports containers auto update out of the box and without broken crutches. Use it. (Watchtower is conceptually broken and cannot be fixed — container that controls its own orchestrator is a high grade bullshit and just abhorrent poor design, that docker forced on people by being bloated garbage and not supporting basic features)
For a Docker install, can’t I replace the Watchtower with a simple scheduled script that runs once a day, or once a week? That only updates the storagenode image?
docker pull storjlabs/storagenode:latest
It will keep the image up-to-date, without the need of container restart.
What do you say?
I appreciate the workarounds and alternatives, but I’d like to follow official docs and avoid workarounds that increase maintenance burden for the future.
Regarding watchtower, is there plans to update its docker API supported versions, or is it unmaintained and effectively deprecated?
I understand watchtower is just to update the container of storagenode which is only needed for OS base image, but then who watches the watchtower? Do I then need to manually periodically pull its container? If so why even have watchtower?
It updates itself if you mentioned it in the run command. If you don’t mention any image in the run command, it will update all the images, including watchtower.
And exactly that is what I need. To keep the image updated, ready for a container re-creation. I don’t need to recreate the container with each base image update, because the node updates itself through the dedicated binary downloaded in bin. So no need to use podman.
And as I understand, that base image dosen’t contain the storagenode binary, but only a downloader for it.
You do. That the whole point. When storj updates their image, for whatever reason, you want to update to it. I mean, you don’t have to, you can keep using the same container forever, but if you are going to be updating – then that’s when you do. With podman you don’t need to do anything explicitly – you label the container for auto-updating and … that’s it.
Though communicated poorly in the thread, there are apparently other container managers out there that handle this better. I’d be happy to learn but telling me its all bullshit just use something else isn’t extremely helpful.
Just pulling a new container in the world of docker doesn’t fix anything, unless you restart your container. Watchtower did those two things for you, albiet very crudely and abruptly.
Personally I run it under the TrueNAS Scale and the software inside it updates itself. There are updates as the underlying stuff is changed but in general you don’t need to update the container version of store node all that often comparatively speaking since it updates itself while running.
I disagree. You weren’t to told to just use something else. You were told to use somethng else with justification and explanation. Here is it here condensed to the bones:
The container orchestration solution that
Is bloated and buggy
does not support basic container lifecycle features
Requiring users to undermine the world order architecture by having to to use third party container to control said orchestrator to achieve that basic task
is the very definition of bullshit.
When the tool you accidentally picked turned out to be a dud — you change the tool, not continue suffering and building a castle of crutches around it in perpetuity.
Using alternative, non-fundamentally-broken orchestrator is therefore the only sane solution here.
There is a proverb in Russian that addresses what’s going on with docket here: Мыши давились, кололись, плакали, но продолжали жрать кактус. (Literal translation: The mice choked, were pricked, cried — but kept eating the cactus.).
Whether this is lack of awareness of alternatives, or sunk cost fallacy — does not change the validity of the solution of using a different container manager.
Except you don’t know shit about my setup, or what causes me pain, or anything other than the fact I’m using watchtower today.
It’s worked for me over three years, I’ve never had a problem with it except that the original maintainer stopped updating it, and it finally broke when some minimum version thing came out.
The pain you talk about? I’ve not seen any of it, my containers stay up to date. That’s it, that’s all I needed.
But you come in swinging screaming about its all bullshit, with no context. So, completely unhelpful.
Nor do I need or care to. You are missing the point. Re-read my posts. Knowing that you have to use watchtower is enough evidence that you chose an ill-fitting, albeit popular, commercial solution. You can dig deeper on why do people continue to chew on this cactus – be my guest. But reasons don’t matter, and I gave you a gist twice already. Thrice, including this post.
Right. A crutch, that users had to develop, finally fell off. The very users, who developed it, gave up. They should have given up earlier – but sometimes cactuses are hard to recognize, and sunk cost fallacy is a real thing. If you need any sign from a universe to abandon docker – here it is. > “This is the signal, Jerry! Thats is the signal!” (from Seinfeld, I’d love to insert a video snipper here but can’t find onefound it)
You are here in this thread discussing said crutch.
If it is still not clear: giving a container control over orchestrator is ridiculous and backwards and in all ways broken, even if it “works”. Solution that forces you to this bullshit to the point of being normalized amongs users has no business being anywhere in your or anybody else’s arsenal of tools. Period.
I provided context. Do you also want me to read my own posts to you: Here.
What else do you need? Why are you defending docker? Do you work for them? This is getting too close to Stockholm syndrome.
Switch to normal tools. The fact that it happens to work is a very low bar to clear. Do better. Be the change you want to see in the world. yada yada.
Thanks for the update! My existing watchtower containers did not immediately update automatically on start (I’m not sure if they’re expected to). I pulled with docker pull storjlabs/watchtower and then removed and re-ran a one of the containers using docker run -d --restart=always --name watchtower -v /var/run/docker.sock:/var/run/docker.sock storjlabs/watchtower storagenode watchtower --stop-timeout 300s from Software Updates - Storj Docs but that gives me exec /watchtower: no such file or directory. Should the run command be updated for the new container?
