I’ve been reporting this for some time now and it still does not work. I have reinstalled watchtower multiple times after being asked to do so. However, nothing I have done has ever fixed the problem. Here are the processes running in my environment:
storj@pine64:~$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cb06bc2768c1 storjlabs/storagenode:latest "/entrypoint" 10 days ago Up 10 days 0.0.0.0:14002->14002/tcp, 127.0.0.1:7777->7777/tcp, 0.0.0.0:28967->28967/tcp storagenode
c5628a1d2b83 storjlabs/watchtower:latest "/watchtower storage…" 2 weeks ago Up 2 weeks watchtower
So, as you can see watchtower has been running for more than 2 weeks (it was reinstalled at that time as well). I am currently running version 1.4.2 of the storagenode software. I received an email saying I should update to 1.5.2. So, I checked to see if watchtower had done the update and as I expected it didn’t. This is a snippit of the current watchtower log:
2020-06-02T23:47:01.702482875Z time="2020-06-02T23:47:01Z" level=debug msg="Scheduled next run: 2020-06-03 05:47:00 +0000 UTC"
2020-06-03T05:47:00.002548937Z time="2020-06-03T05:47:00Z" level=debug msg="Checking containers for updated images"
2020-06-03T05:47:00.007402754Z time="2020-06-03T05:47:00Z" level=debug msg="Retrieving running containers"
2020-06-03T05:47:00.053015714Z time="2020-06-03T05:47:00Z" level=debug msg="Pulling storjlabs/storagenode:latest for /storagenode"
2020-06-03T05:47:00.054041788Z time="2020-06-03T05:47:00Z" level=debug msg="No credentials for storjlabs in /config.json"
2020-06-03T05:47:00.054130749Z time="2020-06-03T05:47:00Z" level=debug msg="Got auth value: "
2020-06-03T05:47:00.054308838Z time="2020-06-03T05:47:00Z" level=debug msg="Got image name: storjlabs/storagenode:latest"
2020-06-03T05:47:00.054569471Z time="2020-06-03T05:47:00Z" level=debug msg="No authentication credentials found for storjlabs/storagenode:latest"
2020-06-03T05:47:01.393621683Z time="2020-06-03T05:47:01Z" level=debug msg="No new images found for /storagenode"
2020-06-03T05:47:01.393792605Z time="2020-06-03T05:47:01Z" level=debug msg="Pulling storjlabs/watchtower:latest for /watchtower"
2020-06-03T05:47:01.393901942Z time="2020-06-03T05:47:01Z" level=debug msg="No credentials for storjlabs in /config.json"
2020-06-03T05:47:01.393955818Z time="2020-06-03T05:47:01Z" level=debug msg="Got auth value: "
2020-06-03T05:47:01.394058030Z time="2020-06-03T05:47:01Z" level=debug msg="Got image name: storjlabs/watchtower:latest"
2020-06-03T05:47:01.394110490Z time="2020-06-03T05:47:01Z" level=debug msg="No authentication credentials found for storjlabs/watchtower:latest"
2020-06-03T05:47:02.123636503Z time="2020-06-03T05:47:02Z" level=debug msg="No new images found for /watchtower"
2020-06-03T05:47:02.123861260Z time="2020-06-03T05:47:02Z" level=debug msg="This is the watchtower container /watchtower"
2020-06-03T05:47:02.123940471Z time="2020-06-03T05:47:02Z" level=debug msg="Scheduled next run: 2020-06-03 11:47:00 +0000 UTC"
2020-06-03T11:47:00.002460387Z time="2020-06-03T11:47:00Z" level=debug msg="Checking containers for updated images"
2020-06-03T11:47:00.003829012Z time="2020-06-03T11:47:00Z" level=debug msg="Retrieving running containers"
2020-06-03T11:47:00.049967795Z time="2020-06-03T11:47:00Z" level=debug msg="Pulling storjlabs/storagenode:latest for /storagenode"
2020-06-03T11:47:00.050179843Z time="2020-06-03T11:47:00Z" level=debug msg="No credentials for storjlabs in /config.json"
2020-06-03T11:47:00.050241386Z time="2020-06-03T11:47:00Z" level=debug msg="Got auth value: "
2020-06-03T11:47:00.050297096Z time="2020-06-03T11:47:00Z" level=debug msg="Got image name: storjlabs/storagenode:latest"
2020-06-03T11:47:00.050353223Z time="2020-06-03T11:47:00Z" level=debug msg="No authentication credentials found for storjlabs/storagenode:latest"
2020-06-03T11:47:00.822919668Z time="2020-06-03T11:47:00Z" level=debug msg="No new images found for /storagenode"
2020-06-03T11:47:00.823418350Z time="2020-06-03T11:47:00Z" level=debug msg="Pulling storjlabs/watchtower:latest for /watchtower"
2020-06-03T11:47:00.823729235Z time="2020-06-03T11:47:00Z" level=debug msg="No credentials for storjlabs in /config.json"
2020-06-03T11:47:00.823971909Z time="2020-06-03T11:47:00Z" level=debug msg="Got auth value: "
2020-06-03T11:47:00.824265584Z time="2020-06-03T11:47:00Z" level=debug msg="Got image name: storjlabs/watchtower:latest"
2020-06-03T11:47:00.824332253Z time="2020-06-03T11:47:00Z" level=debug msg="No authentication credentials found for storjlabs/watchtower:latest"
2020-06-03T11:47:01.510298973Z time="2020-06-03T11:47:01Z" level=debug msg="No new images found for /watchtower"
2020-06-03T11:47:01.510589107Z time="2020-06-03T11:47:01Z" level=debug msg="This is the watchtower container /watchtower"
2020-06-03T11:47:01.510660776Z time="2020-06-03T11:47:01Z" level=debug msg="Scheduled next run: 2020-06-03 17:47:00 +0000 UTC"
Are those No authentication … messages the cause of the problem?
Can you show your docker run command ? Remove any personal info from it
@nerdatwork
This is my docker run command:
docker run -d --restart unless-stopped -p 28967:28967 -p 14002:14002 -e WALLET="0x<WALLET>" -e EMAIL="x@yaddatech.com" -e ADDRESS="x.yaddatech.com:28967" -e BANDWITH="20TB" -e STORAGE="3.6TB" --mount type=bind,source="/opt/storj/identity/storjnode",destination=/app/identity --mount type=bind,source="/storj4tb",destination=/app/config --name storagenode storjlabs/storagenode:latest
You should not be using the latest tag. It could lead to many issues, including corruption of databases. And since databases can’t be rolled back it’s really hard to go back to a stable release again. Though you might be in luck that watchtower didn’t update as 1.4.2 can be upgraded to the current actual release with the :beta tag. Please stick to the documentation and don’t go rogue on using other stuff.
Additionally the safe shutdown delay is missing from your command and the deprecated bandwidth option is still there. Please remove the container and start it with a new docker run command based on Storage Node - Storj Docs
Could you also please share the command you used to start watchtower?
2 Likes
@BrightSilence
This is the run command for watchtower:
docker run -d --restart=always --name watchtower -v /var/run/docker.sock:/var/run/docker.sock storjlabs/watchtower:latest storagenode watchtower --debug --stop-timeout 300s --interval 21600
Should the latest be removed here as well? And are you saying I should be specifying :beta instead of :latest for the storagenode run command?
https://documentation.storj.io/setup/cli/software-updates#automatic-updates
Refer this.
Also post your updated docker command to make sure its correctly updated.
Yes in both cases please refer to the documentation for the correct versions of the commands you should be using. I’d rather not post them here as they can change and the documentation always has the latest version.
1 Like
ok, so I am now confused. Are the pull requests also beta requests as well. My pull requests currently has this:
docker pull storjlabs/storagenode:latest --platform aarch64
should it be:
docker pull storjlabs/storagenode:beta --platform aarch64
I thought the beta period had ended.
So just to clarify before I make any changes. All my docker commands should now be the following:
docker run -d --restart unless-stopped --stop-timeout 300 -p 28967:28967 -p 14002:14002 -p 7777:7777 -e WALLET="x" -e EMAIL="galewis@yaddatech.com" -e ADDRESS="x.yaddatech.com:28967" -e STORAGE="3.6TB" --mount type=bind,source="/opt/storj/identity/storjnode",destination=/app/identity --mount type=bind,source="/storj4tb",destination=/app/config --name storagenode storjlabs/storagenode:beta
docker pull storjlabs/storagenode:beta --platform aarch64
docker run -d --restart=always --name watchtower -v /var/run/docker.sock:/var/run/docker.sock storjlabs/watchtower:beta storagenode watchtower --debug --stop-timeout 300s --interval 21600
What are you using to run your node ?
@nerdatwork
Pine64
storj@pine64 : ~$ uname -a
Linux pine64 3.10.105-bsp-1.2-ayufan-136 #1 SMP PREEMPT Sat Oct 27 21:18:35 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
One item to note on this comment…SNOs, like myself, that have had node(s) running for a while may have been unaware of changes in the guidance/documentation. My first node has been running since Aug 2019 and when I set that up, the template for docker run command was in alignment with what the OP posted, including the “storjlabs/storagenode:latest”. I don’t know when the docker run command template changed, nor was I aware of that until I spun up my second node in Apr 2020 and while following the instructions in the documentation and noticed that it changed.
If I hadn’t have been setting up the second node, I don’t know if even at this point I would have known about the docker run command change.
Perhaps this is a topic to submit to in the “ideas/suggestions” part of the forum for some sort of announcement and changelog/rev history from Storj when the documentation is updated.
As far as I remember, there was no latest tag. It was first alpha then beta.
I agree updates to important content in documentation should be notified via notifications in forum/email/dashboard. I too came across it when I was referring it to another SNO.
You should make a post under ideas/suggestions for the same.
Did you refer documentation for this ?
1 Like
Ah, yes my mistake. When I set up my first node (RPi) the documentation had the docker run command template with “…storjlabs/storagenode:arm”
I’m not sure why you’re still not referring to the documentation for this. Watchtower command is on the next page from the one we linked.
You don’t have to change anything from it, just copy paste.
As for the run command for your node, I think it looks right now. And yes, you should be using :beta for the pull command as well.
2 Likes
I was surprised by the email as well, as my nodes are working fine and watchtower updates when it so chooses. Mine was automatically updated last week to 1.4.2 so the urgency to update to 1.5.2 seems weird. When I followed the link for Linux deployments it just offers the watchtower utility and that seems to be working fine. I assumed the email was premature and part of a new notification system.
I would be worried about the authentication errors, but I am not in a position to offer a suggestion on that.
1 Like
I’m still having trouble getting watchtower to work. I am still seeing these types of errors:
2020-07-04T19:48:22.814084522Z time="2020-07-04T19:48:22Z" level=debug msg="Got auth value: "
2020-07-04T19:48:22.814136273Z time="2020-07-04T19:48:22Z" level=debug msg="Got image name: storjlabs/storagenode:latest"
2020-07-04T19:48:22.814164191Z time=“2020-07-04T19:48:22Z” level=debug msg=“No authentication credentials found for storjlabs/storagenode:latest”
2020-07-04T19:48:26.324413548Z time=“2020-07-04T19:48:26Z” level=debug msg=“No new images found for /storagenode”
2020-07-04T19:48:26.324625138Z time=“2020-07-04T19:48:26Z” level=debug msg=“Pulling storjlabs/watchtower:latest for /watchtower”
2020-07-04T19:48:26.325110861Z time=“2020-07-04T19:48:26Z” level=debug msg=“No credentials for storjlabs in /config.json”
2020-07-04T19:48:26.325292700Z time=“2020-07-04T19:48:26Z” level=debug msg="Got auth value: "
2020-07-04T19:48:26.325615376Z time=“2020-07-04T19:48:26Z” level=debug msg=“Got image name: storjlabs/watchtower:latest”
2020-07-04T19:48:26.325715837Z time=“2020-07-04T19:48:26Z” level=debug msg=“No authentication credentials found for storjlabs/watchtower:latest”
2020-07-04T19:48:27.184293332Z time=“2020-07-04T19:48:27Z” level=debug msg=“No new images found for /watchtower”
2020-07-04T19:48:27.184420752Z time=“2020-07-04T19:48:27Z” level=debug msg=“Scheduled next run: 2020-07-05 10:57:55 +0000 UTC”
Now, I’ve been given a misleading advice as to whether I should be using the beta or latest suffix for the docker start command. I have tried both with the same result. The documentation link below says it should be latest so if that is wrong then that documentation needs to be changed.
https://documentation.storj.io/setup/cli/storage-node?_ga=2.201327826.256223045.1593727866-1714981838.1576885869#running-the-storage-node
Could someone from the support team who can help me to get watchtower to work please respond to this posting. What does No authentication credentials found for storjlabs/storagenode:latest mean and is that the cause of the problem or is there something else wrong? Like I said I’ve tried both beta and latest with the same result.
Is your Storagenode working and only your Watchtower is not?
Were you having a problem pulling the watchtower image or just running it?
What does docker ps -a show?
The storagenode works regardless of whether I use beta or latest. It is only watchtower that does not update the storagenode. I have updated watchtower and storagenode manually many times but watchtower itself does not do the automatic update and reports that authentication message.
There is the ps output:
storj@pine64:~$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f0881c59d33f storjlabs/storagenode:latest "/entrypoint" 6 hours ago Up 6 hours 0.0.0.0:7777->7777/tcp, 0.0.0.0:14002->14002/tcp, 0.0.0.0:28967->28967/tcp storagenode
465bef0dbb9a storjlabs/watchtower "/watchtower storage…" 2 days ago Up 2 days watchtower
1 Like