ERROR nodestats:cache Get stats query failed {“error”: “node stats service error: unable to connect to the satellite 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs: rpccompat: context deadline exceeded”, “errorVerbose”: “node stats service error: unable to connect to the satellite 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs: rpccompat: context deadline exceeded\n\tstorj.io/storj/storagenode/nodestats.(*Service).dial:129\n\tstorj. io/storj/storagenode/nodestats.(*Service).GetReputationStats:65\n\tstorj. io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:108\n\tstorj. io/storj/storagenode/nodestats.(*Cache).satelliteLoop:170\n\tstorj. io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:107\n\tstorj. io/storj/storagenode/nodestats.(*Cache).Run.func1:79\n\tstorj. io/common/sync2.(*Cycle).Run:147\n\tstorj. io/common/sync2.(*Cycle).Start.func1:68\n\tgolang. org/x/sync/errgroup.(*Group).Go.func1:57”}
port on the router and firewall are opened but appear to be the node have port closed
root@STORJ:~# docker port storagenode
14002/tcp -> 0.0.0.0:14002
28967/tcp -> 0.0.0.0:28967
DNAT red0 TCP remoteip 59116 192.168.20.59(firewall ip) 28967 (traffic in is ok)
node status is online, latest ping 9 min ago.
I see some connections established if run:
docker exec storagenode netstat -tap
Active Internet connections (servers and established)
tcp 0 0 82963062b317:60000 REMOTE-IP.bc.googleusercontent.com:https ESTABLISHED 1/storagenode
I think is a network problem of the nested virtualization of docker within proxmox container on a virtual network managed by ipfire firewall.
but if i go back assigning the container to the normal network behind normal router the log error is the same
ERROR nodestats:cache Get stats query failed {“error”: “node stats service error: unable to connect to the satellite 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs: rpccompat: context deadline exceeded”, “errorVerbose”: “node stats service error: unable to connect to the satellite 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs: rpccompat: context deadline exceeded\n\tstorj. io/storj/storagenode/nodestats.(*Service).dial:129\n\tstorj. io/storj/storagenode/nodestats.(*Service).GetReputationStats:65\n\tstorj. io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:108\n\tstorj. io/storj/storagenode/nodestats.(*Cache).satelliteLoop:170\n\tstorj. io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:107\n\tstorj. io/storj/storagenode/nodestats.(*Cache).Run.func1:79\n\tstorj. io/common/sync2.(*Cycle).Run:147\n\tstorj. io/common/sync2.(*Cycle).Start.func1:68\n\tgolang. org/x/sync/errgroup.(*Group).Go.func1:57”}
yes also in the normal network with a simple router the port result closed ,the node online, ping ok, the same log error, and no data exchange
I have removed and installed from 0 the container and now the port result opened…
UPDATE: no data in out exchange… but node result online.
Such error only mean the one thing: your node is offline because of limiting of outgoing traffic.
Please, allow any outgoing traffic to any destination.
Hi. On hardware and virtual network the output traffic is allowed.
I think is a problem with the kernel and the proxmox nested virtualization.
I need to create a new identity and setup for the prolungate downtime and rating downgrade?
It’s up on you. The disqualification for downtime is currently disabled: Design draft: New way to measure SN uptimes
I got this similar error a few days ago when tinkering with my new ISP’s router.
At some point, after changing several firewall configurations, I started to see that error and undoing the last firewall configurations that started to show me the error didn’t help. I restarted the container an the problem persisted.
Finally, I thought that after that much tinkering perhaps the router got crazy, so I rebooted it and stopped the container and start the container once the router was started and with internet connection again. That solved the problem.
It may not be your case, but ensure that your router isn’t blocking the traffic and if you haven’t already done it, give a chance to reboot it, stop the storage node container and start it again once the router is up again.
Hi.
I have the same error in a container running in another my server in a data center with proxmox.
I think is the nested virtualization of proxmox and docker.
There is no way to not use docker image?
Why you use the nested virtualization in the first place?
The docker is not virtualization. It’s kernel-level isolation. You can think about it as a chroot.
The nested virtualization when you run a hypervizor inside the hypervizor. Why you do it?
Yes it’s the same of the proxmox containers ,kernel isolation. Proxmox call this literally "nested virtualization"in the container privilege options.
Would be better to have a normal software installation process for all the people’s and professional that already have servers always on.
Anyone start a new container and with a little .sh start a new storj node 
Now there are virtualization hypervisor in any house of the world no?! 
Then try to do not use a nested virtualization in the VM with Docker.
Also, can you install the docker directly on proxmox host?
To run docker within a proxmox container is like to run a virtual machine inside a virtual machine… The result can be random sometimes…
No, the base usage of a system like proxmox… xenserver etc… Is to no modify the hypervisor in any way, to maintain the stability of the system.
The only software I install on an hypervisor host is OSSEC and rkhunter.
Run docker inside the host that at same time run others kind of containers it’s not a good idea.
The test that I can do can be to run a full virtualized Debian kvm64 with inside docker… But this will use more resources.
Finally, with proxmox containers and docker inside, I haved the same network problem on two different hosts where the proxmox containers run within an isolated virtual network behind a virtual firewall. If the container is outside the closed virtual network it works. Could not be a simple coincidence.
For home office host can be ok, but if you have some servers in a data center this couldn’t be an option, because you don’t have an hardware firewall and is important to not expose the host, using dedicated separated public IP managed by a dedicated virtual firewall… And all the other machine behind the firewall in the isolated virtual network.
For this approach, is really better to have simple installation script of storj, that will run easy the same on a container or fully virtualized os.
I do not suggest using Docker inside LXC. This is not a option. And you already have issues.
With Proxmox and docker there is only two ways:
- install the docker on host with Proxmox
- install the docker on Linux VM
Also, you can run the storagenode without docker, but such setup is not supported (it’s searchable on forum).
I will try the VM.
Docker inside the host absolutely not, expose the host , can create problems with the host and security breach conflicting with the host firewall that is managed by proxmox.