Error : contact:chore pingSatellites failed follow-up

subwolf · October 28, 2019, 10:24am

This is getting bad, please help. I deliberately changed my IP to it’s own, so now my startup config looks like this:

docker run -d --restart unless-stopped -p 28967:28967 \
    -p 14002:14002 \
    -e WALLET="0xxxxxxxxxxxxxxxxxxxxxxxxxx" \
    -e EMAIL="subwolf+home@gmail.com" \
    -e ADDRESS="192.168.1.130:28967" \
    -e BANDWIDTH="100TB" \
    -e STORAGE="1.0TB" \
    -v "/media/subwolf/Games/storj/identity/storagenode":/app/identity \
    -v "/media/subwolf/Games/storj/store":/app/config \
    --name storagenode storjlabs/storagenode:alpha

The docker logs show this:

2019-10-28T09:19:06.950Z        INFO    version running on version v0.23.5
2019-10-28T09:29:07.554Z        ERROR   contact:chore   pingSatellites failed   {"error": "rpc error: code = Unavailable desc = transport is closing"}
2019-10-28T09:34:06.936Z        INFO    version running on version v0.23.5
2019-10-28T09:49:06.942Z        INFO    version running on version v0.23.5

If I check netstat and try to connect:

root@subwolf-tk:~# netstat -a | grep 28967
tcp6       0      0 [::]:28967              [::]:*                  LISTEN
root@subwolf-tk:~# telnet 192.168.1.130 28967
Trying 192.168.1.130...
telnet: Unable to connect to remote host: Connection refused

I have a server in the USA working fine. My home PC is up 24/7 with unlimited fibre from my ISP. There is no reason for this to happen now.

subwolf · October 28, 2019, 10:27am

Just changed the external address back to subwolf.ddns.net and now I get:

2019-10-28T10:26:33.537Z        ERROR   contact:chore   pingSatellites failed   {"error": "rpc error: code = Internal desc = contact: couldn't connect to client at addr: subwolf.ddns.net:28967 due to internal error."}

Verified the IP is correct and working.

KernelPanick · October 28, 2019, 4:31pm

This is a class C - internal address only. You need your WAN IP www.ipmonkey.com or like you set to ddns address.

heunland · October 28, 2019, 6:23pm

You have used the -v option instead of --mount, as described in the documentation.
The difference is very dangerous for your case: if docker will not map the disk to the container for any reason, with the -v option it will create an empty disk inside the container. The customers’ data will be stored inside this container. When you then remove the container, the data will be removed with it.
In case of --mount it will just fail. Please, replace the -v options to the --mount options ASAP. The --mount option has a different syntax. Please, read about it in the section Running the Storage Node of our Documentation.
You have used an GUI automount option (your disk is mounted inside the /media), which works only after login. Moreover, it can mount to a different folder every time.
After reboot, the container will be started before the GUI automount has happened and your node will start to store the data inside the mountpoint instead of the disk. Your node will fail too many audits until you log in. After the login, the data in the mountpoint will be hidden. And you will fail audits again but this time, because the hidden data is inaccessible.
Please, use only static mount for the storage. You can read more here

Please, fix both of these problems ASAP and keep your node online.

For more details, please check Please, update your `docker run` command!

heunland · October 28, 2019, 6:31pm

Also please note that we have been in beta for some time now, so please change --name storagenode storjlabs/storagenode:alpha to --name storagenode storjlabs/storagenode:beta and update to the latest release v0.24.5

heunland · October 28, 2019, 6:37pm

Also please note that if you need to file support tickets, the correct way to do it is to submit your request at https://support.storj.io/hc/en-us/requests/new thanks!

subwolf · October 28, 2019, 8:00pm

I did, not sure if there was a response or not.

subwolf · October 28, 2019, 8:20pm

I have changed the mount options and the mount paths were already in fstab defined manually and not subject to change.

Current run script:

docker run -d --restart unless-stopped -p 28967:28967 \
    -p 14002:14002 \
    -e WALLET="0xXXXXXXXXXXXXXXXXXXXXXX" \
    -e EMAIL="subwolf+home@gmail.com" \
    -e ADDRESS="subwolf.ddns.net:28967" \
    -e BANDWIDTH="100TB" \
    -e STORAGE="1.0TB" \
    --mount type=bind,source="/media/subwolf/Games/storj/identity/storagenode",destination=/app/identity \
    --mount type=bind,source="/media/subwolf/Games/storj/store",destination=/app/config \
    --name storagenode storjlabs/storagenode:beta

Still getting in the logs:

2019-10-28T20:15:30.567Z        INFO    piecestore:monitor      Remaining Bandwidth     {"bytes": 99451296638208}
2019-10-28T20:15:30.669Z        INFO    version running on version v0.24.5
2019-10-28T20:15:31.913Z        ERROR   contact:chore   pingSatellites failed   {"error": "rpc error: code = Internal desc = contact: couldn't connect to client at addr: subwolf.ddns.net:28967 due to internal error."}

Here is what iptables lists as the chain:

Chain DOCKER (1 references)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             172.17.0.3           tcp dpt:28967
ACCEPT     tcp  --  anywhere             172.17.0.3           tcp dpt:14002

172.17.0.3 is interface docker0.

heunland · October 28, 2019, 8:29pm

we received an email to hello@storj.io from you, this is not the correct email to file support tickets.

peem · October 28, 2019, 8:41pm

https://www.yougetsignal.com/tools/open-ports/
Port [28967] is closed on subwolf.ddns.net
your WAN IP: 2.98.183.99 ?

heunland · October 28, 2019, 8:47pm

Ok we now also received a support ticket in our ticketing system. You will receive an answer soon if the issue is not resolved here on the forum meanwhile.

subwolf · October 28, 2019, 8:48pm

IPGorilla says:

My IP Address: 2.98.183.99

peem · October 28, 2019, 8:49pm

what does your router say?
WAN IP is…?

subwolf · October 28, 2019, 8:51pm

peem · October 28, 2019, 8:55pm

port 28967 on the router you have redirected to what address? something like 192.168.x.x?

subwolf · October 28, 2019, 8:57pm

enp27s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.130  netmask 255.255.255.0  broadcast 192.168.1.255

peem · October 28, 2019, 9:01pm

try in storagenode docker run command:
-e ADDRESS=“2.98.183.99:28967”

subwolf · October 28, 2019, 9:02pm

2019-10-28T21:02:34.491Z        INFO    version running on version v0.24.5
2019-10-28T21:02:35.718Z        ERROR   contact:chore   pingSatellites failed   {"error": "rpc error: code = Internal desc = contact: couldn't connect to client at addr: 2.98.183.99:28967 due to internal error."}

peem · October 28, 2019, 9:10pm

is your connection any WiFi?

subwolf · October 28, 2019, 9:11pm

Direct to the router, gigabit ethernet. I’m stumped.