Docker 1.50.4 restarts many times after "unable to resolve host address 'version.storj.io'"

Node Operators troubleshooting
,
1

Hello,
I have updated my stoagenodes to the latest tag. There sometimes seems to be a problem with resolving dns?

If

wget: unable to resolve host address 'version.storj.io'

fails: It stops the container immediately and restart it because of --restart unless-stopped. I had some storagenodes that restarted many times over until it could resolve the dns.

Environment:
Synology DS918+ with DSM 7.0.1-42218 Update 3.

uname -a:
Linux Sembeth 4.4.180+ #42218 SMP Mon Oct 18 19:17:56 CST 2021 x86_64 GNU/Linux synology_apollolake_918+

docker run -d --restart unless-stopped --stop-timeout 300 \
    --log-driver local --log-opt max-size=2m --log-opt max-file=2 \
    -p 28967:28967/tcp \
    -p 28967:28967/udp \
    -p 14002:14002 \
    -e WALLET="xxxxxx" \
    -e EMAIL="xxxxxx" \
    -e ADDRESS="xxxxxx:28967" \
    -e STORAGE="7TB" \
    --mount type=bind,source="/volumeUSB1/usbshare/identity",destination=/app/identity \
    --mount type=bind,source="/volumeUSB1/usbshare/config",destination=/app/config \
    --name storj1 storjlabs/storagenode:latest

Logs:

downloading storagenode-updater
--2022-03-25 17:54:38--  https://version.storj.io/processes/storagenode-updater/minimum/url?os=linux&arch=amd64
Resolving version.storj.io (version.storj.io)... 35.224.88.204
Connecting to version.storj.io (version.storj.io)|35.224.88.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 92 [text/plain]
Saving to: 'STDOUT'

     0K                                                       100% 51.1M=0s

2022-03-25 17:54:39 (51.1 MB/s) - written to stdout [92/92]

--2022-03-25 17:54:39--  https://github.com/storj/storj/releases/download/v1.50.4/storagenode-updater_linux_amd64.zip
Resolving github.com (github.com)... 140.82.121.3
Connecting to github.com (github.com)|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/128089774/dfa144f5-ef82-4c12-85ce-c92acb9f202d?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220325%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220325T175439Z&X-Amz-Expires=300&X-Amz-Signature=48b8132851ef6a52a6311f77774ac5659fbf512a5493b2ede9707aa336ef2759&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=128089774&response-content-disposition=attachment%3B%20filename%3Dstoragenode-updater_linux_amd64.zip&response-content-type=application%2Foctet-stream [following]
--2022-03-25 17:54:39--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/128089774/dfa144f5-ef82-4c12-85ce-c92acb9f202d?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220325%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220325T175439Z&X-Amz-Expires=300&X-Amz-Signature=48b8132851ef6a52a6311f77774ac5659fbf512a5493b2ede9707aa336ef2759&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=128089774&response-content-disposition=attachment%3B%20filename%3Dstoragenode-updater_linux_amd64.zip&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.111.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8107339 (7.7M) [application/octet-stream]
Saving to: '/tmp/storagenode-updater.zip'

     0K .......... .......... .......... .......... ..........  0% 2.34M 3s
    50K .......... .......... .......... .......... ..........  1% 2.23M 3s
   100K .......... .......... .......... .......... ..........  1% 42.3M 2s
   150K .......... .......... .......... .......... ..........  2% 62.8M 2s
   200K .......... .......... .......... .......... ..........  3% 4.73M 2s
   250K .......... .......... .......... .......... ..........  3% 10.2M 2s
   300K .......... .......... .......... .......... ..........  4% 13.2M 1s
   350K .......... .......... .......... .......... ..........  5% 9.98M 1s
   400K .......... .......... .......... .......... ..........  5% 14.3M 1s
   450K .......... .......... .......... .......... ..........  6% 8.12M 1s
   500K .......... .......... .......... .......... ..........  6% 9.94M 1s
   550K .......... .......... .......... .......... ..........  7% 10.4M 1s
   600K .......... .......... .......... .......... ..........  8% 20.6M 1s
   650K .......... .......... .......... .......... ..........  8% 14.1M 1s
   700K .......... .......... .......... .......... ..........  9% 13.0M 1s
   750K .......... .......... .......... .......... .......... 10% 10.1M 1s
   800K .......... .......... .......... .......... .......... 10% 14.1M 1s
   850K .......... .......... .......... .......... .......... 11% 12.8M 1s
   900K .......... .......... .......... .......... .......... 11% 3.64M 1s
   950K .......... .......... .......... .......... .......... 12% 5.50M 1s
  1000K .......... .......... .......... .......... .......... 13% 8.43M 1s
  1050K .......... .......... .......... .......... .......... 13% 8.52M 1s
  1100K .......... .......... .......... .......... .......... 14% 6.48M 1s
  1150K .......... .......... .......... .......... .......... 15% 5.08M 1s
  1200K .......... .......... .......... .......... .......... 15% 8.52M 1s
  1250K .......... .......... .......... .......... .......... 16% 5.84M 1s
  1300K .......... .......... .......... .......... .......... 17% 3.99M 1s
  1350K .......... .......... .......... .......... .......... 17%  901K 1s
  1400K .......... .......... .......... .......... .......... 18% 5.65M 1s
  1450K .......... .......... .......... .......... .......... 18% 2.43M 1s
  1500K .......... .......... .......... .......... .......... 19% 3.20M 1s
  1550K .......... .......... .......... .......... .......... 20% 9.10M 1s
  1600K .......... .......... .......... .......... .......... 20% 9.16M 1s
  1650K .......... .......... .......... .......... .......... 21% 9.48M 1s
  1700K .......... .......... .......... .......... .......... 22% 11.0M 1s
  1750K .......... .......... .......... .......... .......... 22% 8.46M 1s
  1800K .......... .......... .......... .......... .......... 23% 12.6M 1s
  1850K .......... .......... .......... .......... .......... 23% 10.0M 1s
  1900K .......... .......... .......... .......... .......... 24% 4.96M 1s
  1950K .......... .......... .......... .......... .......... 25% 9.65M 1s
  2000K .......... .......... .......... .......... .......... 25% 8.30M 1s
  2050K .......... .......... .......... .......... .......... 26% 9.55M 1s
  2100K .......... .......... .......... .......... .......... 27% 4.94M 1s
  2150K .......... .......... .......... .......... .......... 27% 5.72M 1s
  2200K .......... .......... .......... .......... .......... 28% 8.87M 1s
  2250K .......... .......... .......... .......... .......... 29% 9.94M 1s
  2300K .......... .......... .......... .......... .......... 29% 11.3M 1s
  2350K .......... .......... .......... .......... .......... 30% 10.1M 1s
  2400K .......... .......... .......... .......... .......... 30% 10.8M 1s
  2450K .......... .......... .......... .......... .......... 31% 13.0M 1s
  2500K .......... .......... .......... .......... .......... 32% 19.1M 1s
  2550K .......... .......... .......... .......... .......... 32% 24.5M 1s
  2600K .......... .......... .......... .......... .......... 33% 22.0M 1s
  2650K .......... .......... .......... .......... .......... 34% 12.7M 1s
  2700K .......... .......... .......... .......... .......... 34% 77.4M 1s
  2750K .......... .......... .......... .......... .......... 35%  138M 1s
  2800K .......... .......... .......... .......... .......... 35%  125M 1s
  2850K .......... .......... .......... .......... .......... 36%  168M 1s
  2900K .......... .......... .......... .......... .......... 37%  168M 1s
  2950K .......... .......... .......... .......... .......... 37%  130M 1s
  3000K .......... .......... .......... .......... .......... 38%  149M 1s
  3050K .......... .......... .......... .......... .......... 39%  164M 1s
  3100K .......... .......... .......... .......... .......... 39%  136M 1s
  3150K .......... .......... .......... .......... .......... 40% 14.1M 1s
  3200K .......... .......... .......... .......... .......... 41%  156M 1s
  3250K .......... .......... .......... .......... .......... 41%  174M 1s
  3300K .......... .......... .......... .......... .......... 42%  165M 1s
  3350K .......... .......... .......... .......... .......... 42%  155M 1s
  3400K .......... .......... .......... .......... .......... 43%  137M 1s
  3450K .......... .......... .......... .......... .......... 44%  166M 1s
  3500K .......... .......... .......... .......... .......... 44%  169M 1s
  3550K .......... .......... .......... .......... .......... 45% 75.4M 0s
  3600K .......... .......... .......... .......... .......... 46% 66.2K 1s
  3650K .......... .......... .......... .......... .......... 46% 8.55M 1s
  3700K .......... .......... .......... .......... .......... 47% 5.66M 1s
  3750K .......... .......... .......... .......... .......... 47% 27.9M 1s
  3800K .......... .......... .......... .......... .......... 48%  166M 1s
  3850K .......... .......... .......... .......... .......... 49% 41.3M 1s
  3900K .......... .......... .......... .......... .......... 49% 12.2M 1s
  3950K .......... .......... .......... .......... .......... 50% 11.3M 1s
  4000K .......... .......... .......... .......... .......... 51% 6.80M 1s
  4050K .......... .......... .......... .......... .......... 51% 10.5M 1s
  4100K .......... .......... .......... .......... .......... 52% 13.6M 1s
  4150K .......... .......... .......... .......... .......... 53% 13.7M 1s
  4200K .......... .......... .......... .......... .......... 53% 12.2M 1s
  4250K .......... .......... .......... .......... .......... 54% 15.4M 1s
  4300K .......... .......... .......... .......... .......... 54% 5.21M 1s
  4350K .......... .......... .......... .......... .......... 55% 22.5M 1s
  4400K .......... .......... .......... .......... .......... 56% 30.1M 1s
  4450K .......... .......... .......... .......... .......... 56% 11.0M 1s
  4500K .......... .......... .......... .......... .......... 57% 6.42M 1s
  4550K .......... .......... .......... .......... .......... 58% 16.8M 1s
  4600K .......... .......... .......... .......... .......... 58% 12.5M 1s
  4650K .......... .......... .......... .......... .......... 59% 10.3M 1s
  4700K .......... .......... .......... .......... .......... 59% 15.1M 1s
  4750K .......... .......... .......... .......... .......... 60% 8.32M 1s
  4800K .......... .......... .......... .......... .......... 61% 12.8M 1s
  4850K .......... .......... .......... .......... .......... 61% 10.2M 1s
  4900K .......... .......... .......... .......... .......... 62% 8.95M 1s
  4950K .......... .......... .......... .......... .......... 63% 14.0M 1s
  5000K .......... .......... .......... .......... .......... 63% 14.5M 1s
  5050K .......... .......... .......... .......... .......... 64% 9.92M 1s
  5100K .......... .......... .......... .......... .......... 65% 13.2M 1s
  5150K .......... .......... .......... .......... .......... 65% 9.69M 1s
  5200K .......... .......... .......... .......... .......... 66% 6.99M 1s
  5250K .......... .......... .......... .......... .......... 66% 8.16M 1s
  5300K .......... .......... .......... .......... .......... 67% 13.6M 1s
  5350K .......... .......... .......... .......... .......... 68% 8.23M 1s
  5400K .......... .......... .......... .......... .......... 68% 14.0M 1s
  5450K .......... .......... .......... .......... .......... 69% 9.30M 1s
  5500K .......... .......... .......... .......... .......... 70% 8.88M 1s
  5550K .......... .......... .......... .......... .......... 70% 5.96M 1s
  5600K .......... .......... .......... .......... .......... 71% 48.9M 1s
  5650K .......... .......... .......... .......... .......... 71% 14.5M 1s
  5700K .......... .......... .......... .......... .......... 72% 5.53M 1s
  5750K .......... .......... .......... .......... .......... 73% 23.4M 0s
  5800K .......... .......... .......... .......... .......... 73% 13.2M 0s
  5850K .......... .......... .......... .......... .......... 74% 7.99M 0s
  5900K .......... .......... .......... .......... .......... 75% 9.34M 0s
  5950K .......... .......... .......... .......... .......... 75% 8.25M 0s
  6000K .......... .......... .......... .......... .......... 76% 10.6M 0s
  6050K .......... .......... .......... .......... .......... 77% 4.28M 0s
  6100K .......... .......... .......... .......... .......... 77%  136M 0s
  6150K .......... .......... .......... .......... .......... 78% 78.4M 0s
  6200K .......... .......... .......... .......... .......... 78% 10.2M 0s
  6250K .......... .......... .......... .......... .......... 79% 12.7M 0s
  6300K .......... .......... .......... .......... .......... 80% 4.87M 0s
  6350K .......... .......... .......... .......... .......... 80% 9.94M 0s
  6400K .......... .......... .......... .......... .......... 81% 8.67M 0s
  6450K .......... .......... .......... .......... .......... 82% 12.1M 0s
  6500K .......... .......... .......... .......... .......... 82% 8.99M 0s
  6550K .......... .......... .......... .......... .......... 83% 3.07M 0s
  6600K .......... .......... .......... .......... .......... 83% 6.37M 0s
  6650K .......... .......... .......... .......... .......... 84% 9.65M 0s
  6700K .......... .......... .......... .......... .......... 85% 8.47M 0s
  6750K .......... .......... .......... .......... .......... 85% 50.1M 0s
  6800K .......... .......... .......... .......... .......... 86% 22.5M 0s
  6850K .......... .......... .......... .......... .......... 87% 14.8M 0s
  6900K .......... .......... .......... .......... .......... 87% 10.6M 0s
  6950K .......... .......... .......... .......... .......... 88% 10.6M 0s
  7000K .......... .......... .......... .......... .......... 89% 6.97M 0s
  7050K .......... .......... .......... .......... .......... 89% 3.24M 0s
  7100K .......... .......... .......... .......... .......... 90% 62.4M 0s
  7150K .......... .......... .......... .......... .......... 90% 19.9M 0s
  7200K .......... .......... .......... .......... .......... 91% 12.3M 0s
  7250K .......... .......... .......... .......... .......... 92% 9.43M 0s
  7300K .......... .......... .......... .......... .......... 92% 9.29M 0s
  7350K .......... .......... .......... .......... .......... 93% 9.65M 0s
  7400K .......... .......... .......... .......... .......... 94% 24.4M 0s
  7450K .......... .......... .......... .......... .......... 94% 14.0M 0s
  7500K .......... .......... .......... .......... .......... 95% 4.65M 0s
  7550K .......... .......... .......... .......... .......... 95% 8.17M 0s
  7600K .......... .......... .......... .......... .......... 96% 10.7M 0s
  7650K .......... .......... .......... .......... .......... 97% 13.2M 0s
  7700K .......... .......... .......... .......... .......... 97% 7.31M 0s
  7750K .......... .......... .......... .......... .......... 98% 8.33M 0s
  7800K .......... .......... .......... .......... .......... 99% 9.70M 0s
  7850K .......... .......... .......... .......... .......... 99% 6.70M 0s
  7900K .......... .......                                    100% 56.5M=1.6s

2022-03-25 17:54:41 (4.87 MB/s) - '/tmp/storagenode-updater.zip' saved [8107339/8107339]

downloading storagenode
--2022-03-25 17:54:41--  https://version.storj.io/processes/storagenode/minimum/url?os=linux&arch=amd64
Resolving version.storj.io (version.storj.io)... failed: Try again.
wget: unable to resolve host address 'version.storj.io'
http://: Invalid host name.
2

Seems you have problems with your DNS. Try to run your node with option --dns 8.8.8.8 (specify it somewhere before --name).

1 Like
3

Well, it seems to be my DNS. After a few restarts of the container, the dns was finally solved and downloaded. I just wanted to give some feedback.

This topic can be closed.

4

this is suddenly happening to one of my 4 nodes aswell vs 1.50.4. nodes are all the same configuration. Resolving on the host is fine though.

1 Like
5

Hello everyone,
I’m giving trueNas a go to see if it meets my needs for running nodes. However I’m running on the same issue described above, when I try to setup my node with

sudo docker run --rm -e SETUP="true" --user $(id -u):$(id -g) --mount type=bind,source=/mnt/Mirror/gab/vetting0/identity,destination=/app/identity --mount type=bind,source=/mnt/Mirror/gab/vetting0,destination=/app/config --name storagenode storjlabs/storagenode:latest

it fails with

downloading storagenode-updater
--2022-10-02 15:27:51--  https://version.storj.io/processes/storagenode-updater/minimum/url?os=linux&arch=amd64
Resolving version.storj.io (version.storj.io)... failed: Temporary failure in name resolution.
wget: unable to resolve host address 'version.storj.io'
http://: Invalid host name.

I tried changing the dns in the trueNas web interface, in the docker command with --dns 8.8.8.8 and adding --restart unless-stopped but it just keeps restarting with the same error.
The system is running on an Intel Core i7-4770 with a fresh install of TrueNas SCALE 22.02.4
I could use some help because I can’t seem to find the issue.
Thanks !

6

Please try to change the DNS setting in DHCP on your router, after that you need to request an IP update on TrueNAS (or just reboot it).

7

Hi Alexey,
thanks a lot for your reply ! I did what you said and at least I now can ping version.storj.io from the machine. The node however still gives the same error…
Do you have any other tricks up your sleeve that could help me ?

Also I don’t know if it’s relevant but I have another node running on a different machine (ubuntu baremetal) and it doesn’t have any issues.

8

I think you need to file a support ticket to IXsystems, because it seems a very popular issue with TrueNAS Scale. Unfortunately I did not find any public thread with a normal solution, except several hacks (like bypassing a k8s network stack completely, as described here: SOLVED - TrueNAS Scale Cluster DNS not working | TrueNAS Community, or modify the docker daemon configuration like here: https://www.reddit.com/r/truenas/comments/kyah2i/comment/goye851/?utm_source=share&utm_medium=web2x&context=3)

1 Like
9

So I moved the note to an ubuntu server VM and it started up no prolem
thanks for your help !!

1 Like