How much of nodes can be run on 4 cores CPU? 3 or 4?
As we still need OS running somewhere.
Is that an 8 thread CPU?
No, i’m talking about 4 thread CPU.
threads would play a big role just like clock frequency… storj documentation recommends 1 core but lets just call that a thread… it will rarely use the cpu fully, but sometimes it might use quite a bit… depends on what it’s doing…
the capacity of the nodes would also factor into this, because the bigger the storage the more processing will be required for database purposes.
so if its just because you plan on running many nodes and want to get an early start on them, i wouldn’t worry to much about running like 16 on a 3ghz with 8threads… tho many vm software does have a max of 1 vm pr core / thread of your cpu.
and you won’t want to run 2 node pr 1 thread for years… atleast not 24tb nodes that is…
but for small 1 -2 tb nodes you might be able to do 10 on 1 thread… but storj recommends 1 pr thread, most likely for good reason… but if you plan on upgrading in the near future and … then you might be able to go crazy
so really there are many things to take into account… are you sure you want a ton of nodes anyways… there will be more overhead with multiple nodes.
I am running 5 nodes with a Dual Core Xeon (without HT). CPU usage never goes over 20%. OS is Windows Server 2019.
At the storj documentation they call “core” not “thread” so it would be interesting to know should it be “core” or can be a “thread” 
If the node will show errors when used less then 500Gb of hdd space, maybe there can be also problems with running lets say 4 nodes on 4 cores (4 threads) CPU, as OS will use some CPU as well.
most stuff seems to be pretty fluid between threads and cores… tho most often a core is slightly better than a thread…
i think some of the very modern software are starting to be better at utilizing cores, but generally i always aim for getting hyperthreading or multithreading on my cpu’s as it gives them much more performance for generally next to no added cost.
also dug into thread vs core performance a while back… there was basically no difference, tho in some computations the threads struggle or isn’t gaining any advantages at all…
this is supposedly due to the computational units inside each core of the cpu… when doing hyperthreading, each thread has its own computational units… i forget what those are named, FPU maybe… floating point units, these deal with certain kinds of math, however there are more complex computational units also, a hyper threading core will have multiple FPU’s which it then routes stuff through… and thus can do twice the work… both threads however share the higher level computational units, and thus if the computation are heavy on those, then no amount of extra threads will help…
kinda like the SPARC CPU’s which have like 10 floating point units pr core… they are immensely strong in floatingpoint, but the higher level stuff they might not be much better than a regular core.
ofc on top of this comes the clocks, because you can do a certain number of operations pr clock cycle, usually 1 or 2 but i believe there are more in some cases… the faster the clock the more computations, it’s really all you have to look at for general computing speeds comparison across different generations of cpu’s over the last 10 years…
clock and threads, ofc it might be off by 10-25% because some cpu’s have different caches and technologies… and for some specialized workloads there might be immense gains… but for general computing, you don’t need to look at much more…
but like i said, i think you can run as many nodes as you like pr core or thread… however don’t expect 10 x 24tb nodes to run on 1 thread / core.
1 core per node isn’t a hard requirement. Full nodes need almost no CPU and even very active nodes don’t need that much (it used to be higher during initial beta iirc).
So I’d say with 4 cores you would still have no problem running 16 nodes, because 15 nodes will be full most of the time. Additionally you don’t get more ingress/egress from 16 nodes vs 1 node so there really is just a small overhead in having additional nodes. If your hardware would be fine running one large node with 16 drives, it should be fine with running 16 nodes too.
So as long as the CPU utilization isn’t anywhere near 80%, i would just keep adding nodes.
I think the more limiting factor might be the bandwith for the HDDs. But even in that regard, if 15 nodes are full, they hardly need bandwith.
2 Likes
I would start with assuming a max of 4 nodes on a 4 thread cpu and see how that goes. I have a 6 core/12 thread Ryzen 5 of which 4 cores/8 threads are dedicated to another task. The other 2 cores/4 threads are used by Storj and the OS running two nodes. The load is lightweight perhaps 20% peak utilization at this point. I assume as we load up the CPU we would see performance degradation that affects the rig’s ability to quickly service requests and ends up with more cancellations as discussed in other posts.
i got 16 threads but at 2.13ghz + turbo (ofc that unsustainable)
with your ryzen 3600 which i assume it is… i think that has 6 cores and 12 threads anyways.
but thats at like near 4ghz… and my cores are so much more powerful than needed… and each of your threads are in almost all aspects closing on twice the speed…
it’s a modern computational power house the ryzen 3600… i tried to dedicate my cores to my vm’s in linux… but the system doesn’t seem to separate out the tasks, to singular cores… which is good because you really want all of them to work together instead of having 11 sitting idle and 1 working over time to keep up.
you might be able to find some extra computational power with great numa or uma configuration…
i’ve tested the successrate thing extensively, and yes there is a performance hit on upload successrates, but since then uploads successrate have been proved wholly wrong, the results they gives are basically worthless because everything seems to indicate that even a node with 13% successrate will most likely get and keep most of the remaining 87% of files also…
and so sure successrate is a measure of something of the internal processes of the cpu, because it does change depending on work loads, but to node operation it might not have any bearing at all…
so yeah long story short… upload successrates at present isn’t a measure of anything because the logs are perfectly wrong at a level where the number is practically useless.
1 Like
For reference.
Old laptop mobile i5-6300U CPU (6th gen) 2 cores / 4threads
OS: Ubuntu LTS 20.04 desktop.
Running 2 nodes on 2 USB3 HDD:
=> 10 to 20% CPU usage on heavy traffic.
=> almost nothing on usual traffic
You can run nodes on low end CPUs, Disks are the bottleneck
1 Like
Then the question why Storj at it’s minimum requirements talks about 1 core
That information is confusing. Because at the same list 500Gb minimum is very strict rule, thought that with the cores is the same, does not matter how the core loaded.
To do not have a bottleneck if your CPU is old or slow enough
If i knew that earlier, i would not buy two RPi’s 
Though that this is strict as with the 500Gb
you have 4 nodes on 1 pi?
No that’s for two different locations where two PC’s with 2 cores CPU are runing.
Thought that 1 core for OS, other core for NODE. Means it is no possible to run more nodes anymore. But now see that it is possible, as the load for these cores never been more then 20% or so.
Another reason for this requirement is to prevent people from mining or doing something else that pegs the CPU at 100% on the same system. Your node(s) would definitely not like that.
Ok. But this gets in contradiction with question RAID vs NO RAID.
6 or 8 HDD’s would nicely work as 1 node in RAID even on 2 or 4 cores CPU as well as statement, that nodes should not invest in the new equipment
I think the overhead of running multiple node processes is negligible. So 6 or 8 nodes would probably work as well on the same hardware. Most of the processing is done per transaction and the total number of transactions does not change. I have seen no noticeable increase in CPU use when adding 2 new nodes on the same system.
But it of course depends on what cores we’re talking about. Some modern chips could probably easily run 8 nodes on a single core, while low power older chips might do better having 4 cores for a single node. Things are never quite as black and white. So my advice would always be to start with one node, see how it’s doing and then decide. I think in most cases you can easily add a second and third node. But you should always keep an eye on resource usage.
1 Like
Totally agree with you. But then it should somehow explained in the form where you apply for the identity. As for now we have very strict rule for “500Gb”, and on the same list very floating “1 core” So that strict vs that floating makes things confusing.
1 Like