I have a couple of nodes running but I’m thinking to config a larger setup. What do you think is ideal for future scaling?
Every node have different ip:
Standard way. Multiple node, every node one specific hdd. Once full you never enlarge storage space (if you want to do that you must move everything in a different hdd)
Multiple nodes in a RAIDz pools. Easy to allocate space and manage every single nodes adding new hdds. In this config what do you suggest? large raidz 2 pool (if you have 24bay for example 24hdd in raidz 2, is it safe enought?)
I know that storj team advice is to separate hdd for every node and not wasting hdd for raidz but when you want to scale up will be difficult to manage a lot of nodes.
I personally use a raidz1 for newly added node for it to vet, since my raidz1 is already up doing other things.
Once vetted, I would then more it to its proper single hhd.
I have one big node running in a VM with the data stored on a 2xraidz2 pool. I use that server and that pool to run some other VMs as well.
For me, keeping a lot of single-drive nodes running would be too much of a hassle (also take up more physical space than a 4U server with 24 drive bays and would not allow me to use the remaining space for anything else). If egress goes up and my node starts having problems with performance I’ll see what I can do, but using L2ARC or getting faster CPUs right now would be very premature optimization.