Node repeatly stop every few hrs after did a update yesterday

[Miner]

@BrightSilence

Not really, just when updates happen. Which I agree is a problem. But at the moment the choice on your end is for it to restart or to stop. I think restart is the better option between those two. But I do feel like the container shouldn’t stop at all while updating. That would be something for Storj to fix. All you can do in the mean time is pick the best of 2 not so ideal choices.

Of course I meant only in case of an update; the container restarts, from my perspective as a node operator, at a random time.
I absolutely support the idea to not require a container restart on rolling updates.
Further I support the idea, to not require any rolling restart of resource intensive tasks, without providing a method to apply an individual schedule in any form.

@BrightSilence

I had the same fear initially, but this is actually not the case. The image doesn’t come with the binaries anymore. Instead on start it downloads the latest updater, then checks using the updater which version your node should be on according to the rollout and downloads the binary. This prevents downgrades.

Thank you for testing and clarifying this. I can confirm, that the updater downloads the recent version on startup.

Either this mechanism had failed on April 7th or maybe there were multiple updates?
Because when I found the container exited due to an update and my --restart no
setting, I recreated the container and X hours later, the updater requested another restart.

[oihjdoifglsdjklfk]

a quick update:

as I mention that I experienced two poweroff after update check

there’s no poweroff/restart after that for 30+hrs

[Alexey]

We are evolving. The new tech could have a bugs, this is expected.

[KernelPanick]

i realize this is unsupported, but a warning for those using iSCSI.

–restart unless-stopped is dangerous if you have lost storage. The node should shut down and stay off if storage is unavailable. with the setting --restart no. This results in an offline node after auto updating. I use a cronjob to detect node offline state, if offline - check storage, if storage online, start node.

–mount (bind source) should stop the container from starting, but i’d rather have an additional safety check and not bounce the node too frequently.

Update to 1.64.1 required two manual starts.

[Alexey]

If the storage is lost, the node should not start, unless you did also the setup step in the second time when the storage is missing.
So you can use --restart unless-stopped without problems, but your node could crash too often (it also checks the readability and writability and will crash otherwise).

[my_kenneth]

I tried both --restart unless-stopped or --restart always

the node is still terminated every hour as below.

[Alexey]

Does container still running after that?
And is your docker daemon running?

[my_kenneth]

Hi @Alexey ,

Docker daemon is still there(as I saw other container is running) ,
just the storj container getting exit only.

Ps

1. docker and storj node is latest
2. Pc has been rebooted/ restarted.

[Alexey]

Container should be restarted, if it dies. Does container remain as stopped?

docker ps -a

Do you have watchtower running? Does it has only one instance?

Please try also remove the image and pull it again, then run the node.
I would also recommend checking your system logs, it may have been killed due to OOM.