Started seeing this a couple days ago, figure it’s probably related to the announcement here: Updates on Test Data Host is running Debian linux 12 on an i7-5960X with 64GB ram, has an ssd as the OS drive and a pair of 12TB hdds connected to the sata3 interface for StorJ data. [additional note] The…

You dont think the earlier errors about the invalid configuration keys isn’t the issue?

I hadn’t looked too closely at them since they were marked as ‘info’ and not ‘warn’ or ‘error’. After rebooting the host both nodes will start up successfully, fully operational web interface and all, and run for about 15 minutes before dropping into this loop

It also doesn’t seem to be limited to just storj. While the nodes are running in this loop I’m also unable to open or create any new files even on the host OS drive; I get a warning from Debian that there are too many open files. As soon as I stop the storj containers then the OS is back to normal …

I think the answer is here in Stephen’s reply. [image] INFO http: Accept error: accept tcp XXX.XXX.XXX.XXX:28967: accept4: too many open files troubleshooting Hi! What does this error mean? I have to increase files limits for the user running the node?

Alrighty, that’s a different approach than what I’ve already tried. I’ll try applying that when I get back and will update this post with the results.

This is a bug and needs to be fixed. Storagenode updater uses the same config file as storagenode, but it does not understand (nor needs to!) half of the keys there. Instead of ignoring them, it screams in the logs. That log message shall be demoted from INFO to DEBUG, or removed altogether. It sh…

The part I’m getting hung up on with that post you linked, is that (as far as I can tell) I’m not using any sytemd service to run/manage the docker container… It’s just a basic docker install with no dedicated docker/storj user account. So it seems to me that I should be changing the NOFile limit fo…

You can try to run the container with --ulimit parameter: docker run --ulimit nofile=5000:5000

I re-ran my two nodes with that parameter and confirmed with docker -exec that it did update the container’s NOFile limits, however they are still exhibiting the same behavior and outputting the same log messages

Well, you could try raising the number higher. If not, I would recommend asking in a Docker forum for assistance as it seems to be Docker related.

Storj Community Forum (official)

StorJ nodes stuck in loop - too many open files

Node Operators troubleshooting

Knowledge May 1, 2024, 5:41pm 5

I think the answer is here in Stephen’s reply.