Docker container network disconnecting

Node Operators troubleshooting
,
1

Hi there,

I needed to stop the storj node and when I try to restart it I basically can’t. It keeps restarting undefinitely. I try to run docker logs but is pointless as the container seems to not be starting at all.

I’m running storj on a RockPro64 and the command I use is:

docker run -d --restart unless-stopped -p 28967:28967 \
-p 127.0.0.1:14002:14002 \
-e WALLET="WALLET" \
-e EMAIL="EMAIL" \
-e ADDRESS="ADDRES.ddns.net:28967" \
-e BANDWIDTH="6TB" \
-e STORAGE="7TB" \
--mount type=bind,source="/home/user/storj/identity/storagenode",destination=/app/identity \
--mount type=bind,source="/mnt/data/Storj",destination=/app/config \
--name storagenode storjlabs/storagenode:arm

docker events& shows this:

2020-12-01T19:51:20.044496824Z container create 2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b (image=storjlabs/storagenode:arm, name=storagenode)
2020-12-01T19:51:20.256308081Z network connect 7c0f338d845e1eb5541c5f73b2078b8058abf1c337d7059ffc6fe651a7724aeb (container=2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b, name=bridge, type=bridge)
2020-12-01T19:51:20.982133709Z container start 2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b (image=storjlabs/storagenode:arm, name=storagenode)
2020-12-01T19:51:28.021954216Z container die 2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b (exitCode=1, image=storjlabs/storagenode:arm, name=storagenode)
2020-12-01T19:51:28.310610338Z network disconnect 7c0f338d845e1eb5541c5f73b2078b8058abf1c337d7059ffc6fe651a7724aeb (container=2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b, name=bridge, type=bridge)
2020-12-01T19:51:28.575381773Z network connect 7c0f338d845e1eb5541c5f73b2078b8058abf1c337d7059ffc6fe651a7724aeb (container=2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b, name=bridge, type=bridge)
2020-12-01T19:51:29.266834562Z container start 2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b (image=storjlabs/storagenode:arm, name=storagenode)
2020-12-01T19:51:36.406279535Z container die 2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b (exitCode=1, image=storjlabs/storagenode:arm, name=storagenode)
2020-12-01T19:51:36.823986991Z network disconnect 7c0f338d845e1eb5541c5f73b2078b8058abf1c337d7059ffc6fe651a7724aeb (container=2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b, name=bridge, type=bridge)
2020-12-01T19:51:37.065193615Z network connect 7c0f338d845e1eb5541c5f73b2078b8058abf1c337d7059ffc6fe651a7724aeb (container=2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b, name=bridge, type=bridge)
2020-12-01T19:51:37.777741667Z container start 2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b (image=storjlabs/storagenode:arm, name=storagenode)
2020-12-01T19:51:44.904634899Z container die 2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b (exitCode=1, image=storjlabs/storagenode:arm, name=storagenode)
2020-12-01T19:51:45.246357366Z network disconnect 7c0f338d845e1eb5541c5f73b2078b8058abf1c337d7059ffc6fe651a7724aeb (container=2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b, name=bridge, type=bridge)
2020-12-01T19:51:45.537643328Z network connect 7c0f338d845e1eb5541c5f73b2078b8058abf1c337d7059ffc6fe651a7724aeb (container=2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b, name=bridge, type=bridge)
2020-12-01T19:51:46.246895914Z container start 2bce0c6fa6850ef56e1c73140b262570f67399fb955d14d849f8c0c9db7e176b (image=storjlabs/storagenode:arm, name=storagenode)

Any help would be much appreciated as I don’t want my node to be disqualified.

Thanks in advance.

2 
docker logs storagenode 2>&1 | head -n 30

Convert personal data to xxxx

3

No output at all.

~$ docker logs storagenode 2>&1 | head -n 30
~$

Every few seconds the state changes

~$ docker container ls
CONTAINER ID        IMAGE                       COMMAND             CREATED             STATUS                          PORTS               NAMES
2bce0c6fa685        storjlabs/storagenode:arm   "/entrypoint"       13 minutes ago      Restarting (1) 55 seconds ago                       storagenode
~$ docker container ls
CONTAINER ID        IMAGE                       COMMAND             CREATED             STATUS              PORTS                                                  NAMES
2bce0c6fa685        storjlabs/storagenode:arm   "/entrypoint"       13 minutes ago      Up 2 seconds        127.0.0.1:14002->14002/tcp, 0.0.0.0:28967->28967/tcp   storagenode
~$ docker container ls
CONTAINER ID        IMAGE                       COMMAND             CREATED             STATUS                         PORTS               NAMES
2bce0c6fa685        storjlabs/storagenode:arm   "/entrypoint"       13 minutes ago      Restarting (1) 7 seconds ago                       storagenode

EDIT: My bad, I forgot last time I worked on the setup (months ago) I had redirected the logs to a file. Sorry about that. In any case, there was nothing useful in the logs…

4

https://documentation.storj.io/setup/cli/storage-node

there is a newer run command

Besides, I’m not sure if it should be “storjlabs / storagenode: arm” or just “storjlabs / storagenode: latest”

1 Like
5

I’ve already tried both images. None of them fix the issue. The only other difference in the new command is the BANDWIDTH argument and --stop-timeout flag.

6 
ls /mnt/data/Storj
ls /home/user/storj/identity/storagenode

shows your data?

7

Yes.

$ ls /mnt/data/Storj
config.yaml  identity  node.log  orders  revocations.db  storage  trust-cache.json
$ ls /home/ramon/storj/identity/storagenode
ca.1584199014.cert  ca.cert  ca.key  identity.1584199014.cert  identity.cert  identity.key

It looks like some issue initialising the container as even if I run the command without -d so that I’m attached to the output I get no logs at all.

8

Ok, I got it working.

I thought it was a problem with the default bridge network from docker because I’ve created another bridge network and then started the container using the flag --network new_bridge and it worked.

Then I’ve tried again with the default bridge network and it worked, so I’m clueless about what the issue was :thinking:

Anyway, hope this helps somebody who might face the same problem.

9

Cool :slight_smile:
It can be useful for “dockers”

1 Like