I all but given up on running a node after 1.101.3 was released. I am running the PC version and it would stop responding after about 20 minutes. After some amateur investigation I noticed the storj process was running away with my RAM until it reached 100% then everything would crash. With the release today of 1.102.3 all is well in Storj land even though my stats were destroyed. Thanks again for giving me one of my vices back!
Node eat ram when you have very slow hdd read write, at same time demand is high.
1 Like
Hello @Teelow,
Welcome back!
Due to a stress testing you are now know, that your setup cannot handle a load.
You need to optimize your setup. How disk is connected? How much RAM do you have? I guess you are using a Windows GUI version?
Is it a VM?
It would be nice to Stress test the node myself.
For example that I can start the node with a special flag and the node would simulate the max load that it should be able to handle and after it finishes the test all test data is deleted.
Or is it possible to test that by myself? (Would be cheaper for storj too, I think)
2 Likes
To be honest Idea is super great, that cold be some environment that emulate some stress test for node and make some scores, how it perform, then we can tweak accordingly, and share some setup components that are not good for setup. Typical tester mesure only hdd write/read separatly, but we hava random read writes combined, also RAM,CPU make sense.
That’s easy, use a storj-up
You may also use a storj-sim:
But we want to have a real customer-like experience, which, I would quote:
So, likely you are missing an opportunity, if you wouldn’t act.
That would require me to setup everything locally.
I think it would be better to make a test flag, to test how the node will handle everything under huge load or so. So CPU/ram/disk and bandwidth wise.
If I setup everything local, how can I make sure that I have “real” experience and not get the things impacted by the test setup?
I’m against the idea to flood out with not needed functionality of the storagenode binary (more resources usage, more the size, and it’s will be wasted in 99.(9)% of time) when you have a (two) much better solutions to achieve your goal.
1 Like
Just limit the concurrent connections on YAML. This will prevent the system to get requests until the ones on queue are written.
Below my settings:
how many concurrent requests are allowed, before uploads are rejected. 0 represents unlimited.
storage2.max-concurrent-requests: 10
1 Like
The default is unlimited. This option is designed for, how everyone here on the forum calls a weak devices, a potato, which cannot handle a load.
It has a drawback - because the customer get a response from your node that’s it’s overloaded and cannot handle the request, unfortunately this limit is not informing satellites to do not include your node to the suggested list, so the customer uploads may fail with the error:
or
as a result.
But is this not a bad thing? If I am a customer, I want access to my file now and not in 5-10 min. And have to try all the time increasing my cost
Yes, it’s a bad thing. But if you are a lazy SNO, you may just limit your node to handle 10 requests as for a “potato”. Or you may configure it to handle the load or at least increase the number of handled threads, it’s up on you.
I would personally think is my my setup is correct to handle a load, maybe I overprovisioned it or overcomplicated (like using VMs with overbloat OSes like Windows when you can use docker).
It would be a very good feature if the node software could adjust that number dynamically based on load. If load goes up, upload number goes down and vice versa.
These are the things I meant as Storj service quality in the other post. Obviously I don’t want to start a discussion, it’s not my aim, in fact I am a very fond of SNO and far be it from me to question the effort and the intellectual and physical resources that the Storj group makes available for everyone. However, as this last post indicates, if you are a home-based SNO with limited resources, the service offered to customers is also limited. I hope I have explained myself and not be misunderstood.
Please submit a pull request to support this, we will accept it, I believe!
You seem to keep forgetting that I am not a coder and I am not on Github.
I can only make suggestions here.
Even if you are right about limited resources for the single home setup, we have 24k nodes. Thus the customer has a pretty good quality for the service, and with a grow it will be only better.
However, the single node will become a small piece in the whole picture in this case, it’s of course can be treated as a weakness, but when we have thousands weak nodes, which CAN handle the huge load, well, this all become not so important. We just all doing our parts in the success.
1 Like
No, but you seems always forgetting that we have limited resources to implement every suggested feature.
Sorry, but I would always call to the Community, because the Community is a power of a Open Source software, in this case also rewarded materially!