Node Error Forcing Shutdown

Node Operators troubleshooting
,
1

So yesterday my node randomly went offline, I wasn’t able to find the logs. Now today I ran the logs after it shutdown again. This is the only ERROR log I could find?

2023-10-02T13:19:39Z    ERROR   piecestore      upload failed   {"process": "storagenode", "Piece ID": "FEQPCSQHL7AUMM4SHOVXTDQY2A5N66ZRWWUKPVR3XCCAF3GH3AMA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "PUT", "error": "manager closed: unexpected EOF", "errorVerbose": "manager closed: unexpected EOF\n\tgithub.com/jtolio/noiseconn.(*Conn).readMsg:225\n\tgithub.com/jtolio/noiseconn.(*Conn).Read:171\n\tstorj.io/drpc/drpcwire.(*Reader).ReadPacketUsing:96\n\tstorj.io/drpc/drpcmanager.(*Manager).manageReader:226", "Size": 65536, "Remote Address": "184.104.224.98:44260"}

Here is a paste of the last 100 logs storjlog - Pastebin.com

I also ran docker container ls and noticed that watchtower and storagenode are not running. So I ran the start command for storage node and received this error:

Error response from daemon: failed to start shim: mkdir /run/snap.docker/containerd/daemon/io.containerd.runtime.v2.task/moby/11ed2645b1537b04971867b6e67c346f31017e0f0e264e71bad993977aca2edf: file exists: unknown
Error: failed to start containers: storagenode

I’ve since rebooted my server again and the node is back online. Any idea whats going on?

2

since there is no fatal error, my guess:

maybe check the ram if it happens more often…

1 Like
3

Not sure that’s the issue… :frowning:

4

Is this on a VPS? Or otherwise virtualized? Failure to start shim can sometimes be attributed to the virtual environment having an issue, or a drive space or RAM issue.

1 Like
5

Thanks for the reply, yes it is a VPS, drive space doesn’t seem to be an issue, I’ll take a look at the RAM since this is the second person alluding to a possible memory issue.

1 Like
6

Seems you using a snap version of docker. It’s better to uninstall it and use a normal docker instead:

To uninstall the snap version:

sudo snap remove docker
3 Likes
7

Ah, this is what it was… thank you @Alexey

2 Likes