Node is unreachable

Hello,

I’m posting here in the hope that someone can help me diagnose what’s wrong with my node. I’ve just migrated from Windows GUI to linux CLI (docker). Everything seems to be setup fine, but the node can’t be seen from the outside. I’ve opened and pointed all the required ports, but still get those messages in the logs :

2022-06-29T07:58:05.192Z ERROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “xxxxxxxxxxxxxxxxxx”, “attempts”: 7, “error”: “ping satellite: failed to dial storage node (ID: xxxxxxxxxxxxxx) at address xxxxxxxxxxxxx:28967: rpc: tcp connector failed: rpc: dial tcp xxxxxxxx:28967: connect: connection refused”, “errorVerbose”: “ping satellite: failed to dial storage node (ID: xxxxxxxxxxxx) at address xxxxxxxxxxxx:28967: rpc: tcp connector failed: rpc: dial tcp xxxxxxxxxxxx:28967: connect: connection refused\n\tstorjio/storj/storagenode/contact.(*Service).pingSatelliteOnce:139\n\tstorjio/storj/storagenode/contact.(*Service).pingSatellite:98\n\tstorjio/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorjio/common/sync2.(*Cycle).Run:92\n\tstorjio/common/sync2.(*Cycle).Start.func1:71\n\tgolangorg/x/sync/errgroup.(*Group).Go.func1:57”}

Is there anything I can check to diagnose the issue? I think it’s an issue with my new server, I’m going to try and run on a different server to double check. Any help appreciated!

When I run this command : ncat -l -p 28967

I get this :
image

So the problem seems to be with docker?

Is your domain correctly pointed to your local IP?

Hello, yes I’ve narrowed it down to docker. There seems to be a problem with the listener.

The weird thing is that I’m able to get the dashboard from another local machine.

you most likely got the docker run command wrong when launching…
the ip or port for the node could be wrong, the dashboard has its own ip and port setting in the docker run command…

for reference.

example

docker run -d --restart unless-stopped --stop-timeout 300 -p 192.168.1.100:28967:28967/tcp -p 192.168.1.100:28967:28967/udp \
#--log-opt max-size=1m \
-p 192.168.1.100:14002:14002 -e WALLET="0x111111111111111111111" \
-e EMAIL="your@email.com" -e ADDRESS="global.ip.inet:28967" \
-e STORAGE="4TB" --mount type=bind,source="/sn3/id-sn3",destination=/app/identity \
--mount type=bind,source="/sn3/storj",destination=/app/config --name sn3 storjlabs/storagenode:latest \
--filestore.write-buffer-size 4096kiB --pieces.write-prealloc-size 4096kiB

there are a few extras in this, but all you need to worry about is the network part…
as you can see the 14002 port is the dashboard and the others 28967 ports are the storagenode.

also keep in mind one end of the 28967 always has to remain as such because that connects to the storagenode which default in its config.yaml is configured to 28967

like if i wanted to change the lan port used it would look something like this.

docker run -d --restart unless-stopped --stop-timeout 300 -p 192.168.1.100:20001:28967/tcp -p 192.168.1.100:20001:28967/udp \
#--log-opt max-size=1m \
-p 192.168.1.100:14002:14002 -e WALLET="0x111111111111111111111" \
-e EMAIL="your@email.com" -e ADDRESS="global.ip.inet:28967" \

Hello @aurelius ,
Welcome to the forum!

Please check your WAN IP on your router and compare it with IP on Open Port Check Tool - Test Port Forwarding on Your Router, they should match.
If you use DDNS hostname, check that it’s resolved to the same IP as your WAN IP and public IP:

nslookup ddns.hostname.tld 8.8.8.8

Please also check your identity: Step 5. Create an Identity - Storj Docs
If you bind host port to the specific interface, as @SGC did - then make sure that you specified the correct local IP for bind or remove it completely to listen on all interfaces, i.e. -p 28967:28967/tcp -p 28967:28967/udp

1 Like

Yes both are configured correctly, exactly as stated by Alexey

All of this works well, the ddns resolves correctly and to the right ip, and the open port check tool works fine with the command type above (but not with docker)

That was migrated correctly

Then please post your full docker run command with all your parameters (you can mask private information) between two new lines with three backticks, like this:

```
docker run -d ...
```
sudo docker run -d --restart unless-stopped --stop-timeout 300     -p 28967:28967/tcp     -p 28967:28967/udp   -p 14002:14002     -e WALLET="xxx"     -e EMAIL="xxx"     -e ADDRESS="xxx:28967"     -e STORAGE="5TB"     --user $(id -u):$(id -g)     --mount type=bind,source="/mnt/F038B71538B6DA34/storj/identity",destination=/app/identity     --mount type=bind,source="/mnt/F038B71538B6DA34/storj",destination=/app/config     --name storagenode storjlabs/storagenode:latest

The command looks correct, then we only need to check your address in the -e ADDRESS option.
Please execute the command and compare resulted IP with the WAN IP and IP on yougetsignal:

nslookup here.your.external.address 8.8.8.8

As stated before, the ddns name resolves to the correct ip.

I can reach the dashboard from inside my network, and the port is open to the outside and correctly directed (it goes through with ncat -l -p 28967 on the same server - either with ddns or ip). I just can’t reach the docker container for a reason I can’t diagnose.

My node has been offline for a few days now, my hopes aren’t very high.

If you used ncat with the same port, you need to close it and re-create the storagenode container, it can bind to the interface only on creation. So, if port has been used, it cannot be re-used again, until the previous allocation would be freed up.
Or you can try to use a different external port, i.e. -p 28968:28967/tcp -p 28968:28967/udp -e ADDRESS=your.external.address:28968 and modify port forwarding rule to forward 28968 TCP+UDP to your host with docker.

I did close it and restart the server. I re-created the storagenode docker container many times since then.

Did you try to use a different port?

I just did it. It still doesn’t work.

I found recently removing the —user Command working for me better recently !

Give it a try !

Thanks for the suggestion. I just tried without the --user argument and it didn’t fix it unfortunately.

A tool I found useful to double check if the node is reachable is the following:

http://storjnet.info/ping_my_node

What does it tell if you try?

  • started
  • via: St.Petersburg, Russia
  • TCP: couldn’t connect to node: rpc: dial tcp xxx:28967: connect: connection refused
  • QUIC: couldn’t connect to node: rpc: quic: context deadline exceeded
  • via: France (proxy, no QUIC)
  • TCP: couldn’t connect to node: rpc: socks connect tcp 37.59.37.96:2080->xxx:28967: unknown error connection refused
  • done.

Is the node running when you checking?