Storagenode does not start (network connectivity problem )

[itnok]

It looks like my current Internet provider (AT&T) is blocking NTP in both directions allegedly to prevent Reflective DDOS attacks.

Had to reboot my system (due to StorJ storagenode docker container leaking memory and ending up allocating over 32GB of RAM!), but now the node does not want to come up anymore, because it fails its pre-flight check (log image attached).

Does anybody have any idea how to solve this problem?

Thank you

[KernelPanick]

you could try over https:
http://www.vervest.org/htp/

[itnok]

Thank you for the pointer, BUT how should I convince the storagenode to do the same for its sanity check? It looks like the node is doing NTP requests, isn’t it?

Thanks

[Alexey]

This looks like a connectivity issue. Please, check that your firewall or provider doesn’t block the outgoing traffic to different ports, not only NTP.
Please, use this checklist to troubleshoot:

My Node is OFFLINE for 2 hours

It should connect in seconds. There are a few things you should check. Start with checking the port. On https://www.yougetsignal.com/tools/open-ports/?port=28967 you can fill in the domain or IP you are using in the run command. Then check the port. If it’s closed, it could still be several things. Make sure the WAN IP you see in your router settings matches the IP shown on the above page. Make sure your port forward is set up correctly. That means no filter on incoming IPs, forwarding to th…

[itnok]

Thanks Alexey for your reply, actually (of course) it was my bad.

The default route got switched when I had to restart the node and Docker was trying to use a NIC that is not facing (nor have access) outside.

My NTP problem with AT&T persist, but the problem was indeed connectivity related (the node could not get the list of trusted satellites). Switching the NIC priority solved the issue.

Please mind that the restart of the system was due to the node eating up to 32GB of RAM. Was impossible to kill or forcefully remove and I had to restart the whole machine to get back the system to sanity (currently running the storagenode v1.1.1 inside Docker).

Thanks for helping!

[Alexey]

Such huge usage of RAM indicates the slow storage. Are you using any kind of network storage?
The SMB and NFS are not compatible with the storagenode, the only supported network protocol is iSCSI. But even then it’s slower than a local connected drive.

[itnok]

The node is running inDocker on a Synology NAS. Access to the disk pool is direct as the dedicated directory is bind mounted. The underlying filesystem is Btrfs.

Had no problem whatsoever in the past months. The problem occurred only starting yesterday. Only realized only today when the NAS came on its knees and all other services started to be unresponsive.

The docker container was showing a usage of RAM (roughly 2.5GB) higher, but when I killed it (should say I tried!) it freed more than 20GB, hence my doubt about a memory leak. Usually the node does not consume more than 1.9GB.

After trying to kill the node I had to restart the whole NAS because Docker could not kill the container in any way, nor stop it (even though the processes inside where dead/zombied).

[direktorn]

how is this a NTP or storj problem? your clock is incorrect. fix that. how would “both directions” be a problem? are you saying storj wants YOU to be a NTPd?

[itnok]

The clock was correct (BUT the NTP server was not reachable), and the problem (if you read the previous answers/posts) was related to network connectivity rather than NTP. The node could not reach satellite nodes to confirm time correctness.

[Alexey]

Removed NTP from the title