Multinode Could not fetch the version of the node

I just finished active migrating 1 out of 5 nodes to hashstore, it took 15/16h roughly (380GB data/1.8TB space) so I decided to migrate another 1.8TB node too.

However after I removed the container and re-ran the node with the following command:

sudo docker run -d --restart unless-stopped --stop-timeout 300     
-p 28971:28967/tcp     
-p 28971:28967/udp     
-p 0.0.0.0:14006:14002     
-e WALLET="0xAddress"     
-e EMAIL="mail@mail.com"     
-e ADDRESS="address.ddns.net:28971"     
-e STORAGE="1.8TB"     
--mount type=bind,source="/home/user/.local/share/storj/identity/node5",destination=/app/identity     
--mount type=bind,source="/mnt/node5",destination=/app/config     
--name node-5 storjlabs/storagenode:latest     
--hashstore.compaction.rewrite-multiple=10     
--hashstore.compaction.alive-fraction=0.70     
--hashstore.hashtbl.mmap     
--hashstore.table-default-kind=HashTbl

Multinode now is failing to run properly and returns the following errors:

2025-12-04T12:06:24Z    ERROR   nodes:service   Could not fetch the version of the node:        {"error": "rpc: tcp connector failed: rpc: dial tcp *Hidden_For_Privacy*:28967: operation was canceled", "errorVerbose": "rpc: tcp connector failed: rpc: dial tcp *Hidden_For_Privacy*:28967: operation was canceled\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}
2025-12-04T12:06:24Z    ERROR   nodes:service   Could not fetch the version of the node:        {"error": "rpc: tcp connector failed: rpc: dial tcp: lookup address.ddns.net: operation was canceled", "errorVerbose": "rpc: tcp connector failed: rpc: dial tcp: lookup address.ddns.net: operation was canceled\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}
2025-12-04T12:06:24Z    ERROR   nodes:service   Could not fetch the version of the node:        {"error": "rpc: tcp connector failed: rpc: dial tcp: lookup address.ddns.net: operation was canceled", "errorVerbose": "rpc: tcp connector failed: rpc: dial tcp: lookup address.ddns.net: operation was canceled\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}
2025-12-04T12:06:24Z    ERROR   nodes:service   Could not fetch the version of the node:        {"error": "rpc: tcp connector failed: rpc: dial tcp: lookup address.ddns.net: operation was canceled", "errorVerbose": "rpc: tcp connector failed: rpc: dial tcp: lookup address.ddns.net: operation was canceled\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}
2025-12-04T12:06:24Z    ERROR   nodes:service   Could not fetch the version of the node:        {"error": "rpc: tcp connector failed: rpc: dial tcp: lookup address.ddns.net: operation was canceled", "errorVerbose": "rpc: tcp connector failed: rpc: dial tcp: lookup address.ddns.net: operation was canceled\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}

So I can’t load the Multinode dashboard, despite it was working correctly even after I migrated the previous node.

The nodes are working just fine, as I can see from the logs.

node-4:

2025-12-04T12:28:39Z    INFO    piecestore      downloaded      {"Process": "storagenode", "Piece ID": "52M*Hidden_For_Privacy*6AGLQLI5X7ZZ5QQ2N3YJ5GWIX33AN5K5LPJDA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 0, "Size": 905984, "Remote Address": "*Hidden_For_Privacy*46272"}
2025-12-04T12:28:39Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "HGD*Hidden_For_Privacy*UCPSWBEEFGBX45LQWNXBQLAEYRMAORK4YAHRA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*.98:47652", "Size": 285440}
2025-12-04T12:28:39Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "BCH*Hidden_For_Privacy*HAGHZF6LJ4OJAVIY52QQC4V6LQ37JM42MRBVQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:55466", "Size": 9216}
2025-12-04T12:28:43Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "V5M*Hidden_For_Privacy*LSRQ6DOMVPJ4AMOGDNSBAQKVY4W7AEVMWIQMA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:51376", "Size": 2319360}
2025-12-04T12:28:44Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "RBF*Hidden_For_Privacy*5HAKDKGAU324T3AYJIY64SSUNQXR4DGYEPG6A", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:59204", "Size": 12032}
2025-12-04T12:28:51Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "GLU*Hidden_For_Privacy*TESIRQXILLNQ2552QIHEH52MTZM7CUQ7DOX7Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:44962", "Size": 9216}
2025-12-04T12:28:54Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "E5U*Hidden_For_Privacy*XM3WXC6OHJRVIQYEXHJFA74Z4OLBLU3GGSN4A", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:38662", "Size": 133632}
2025-12-04T12:28:54Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "QCT*Hidden_For_Privacy*TFDV2ZWG2MHOWI4V25TUYPCL37TO4WCKMJ5JQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:40198", "Size": 187648}

node-5:

2025-12-04T12:34:32Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "E2B*Hidden_For_Privacy*HMRCK6E6B7RHU3A5IP5DU2CRVSGE7VX3HVOSQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:45366", "Size": 291328}
2025-12-04T12:34:32Z    INFO    piecestore      downloaded      {"Process": "storagenode", "Piece ID": "ORA*Hidden_For_Privacy*IOSAVNIKRWJ2ZUITJKDUTJMH4J3BGRIKOK4TA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 0, "Size": 7936, "Remote Address": "*Hidden_For_Privacy*:55038"}
2025-12-04T12:34:34Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "FKB*Hidden_For_Privacy*UUQWICKRJPAM3KLUEFTQKE5E52VY7SFIPB47Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:45366", "Size": 276224}
2025-12-04T12:34:36Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "BWH*Hidden_For_Privacy*UR4NWXFHDMK7AJ4IIUOLHTPKSO7SZNRCTAFCA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:51980", "Size": 1024}
2025-12-04T12:34:38Z    INFO    piecestore      uploaded        {"Process": "storagenode", "Piece ID": "RQ6*Hidden_For_Privacy*XIKSSLZ7543TBOTBUI2O4QTIYAWAOMTFZCVXA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Remote Address": "*Hidden_For_Privacy*:34186", "Size": 7680}
2025-12-04T12:34:40Z    INFO    piecestore      downloaded      {"Process": "storagenode", "Piece ID": "PSC*Hidden_For_Privacy*5SXYRQMLV554JUJ7EI22DQ2YMI7ZUKLZI23IQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 0, "Size": 9472, "Remote Address": "*Hidden_For_Privacy*:36320"}

Any suggestions?

Please restart the multinode dashboard after all nodes are started. There is a some delay when the node with hashstore become online after start - it does small fsck of hashstore, it should be improved though in the next releases, but still.
So I think that the multinode dashboard is never retry if it was unable to connect on the first attempt. Also, please make sure that your node is online and not restarting.
Also I suspect that maybe your DNS is misbehaving and multinode cannot resolve the external address of that node:

3 Likes

As you said, it worked fine after few minutes, thanks @Alexey :folded_hands:

1 Like