From my experience, having a large number of small nodes on a single hard drive has only one drawback, but so many benefits that I wouldn’t choose otherwise. The drawback is that you’ll better script your way through administration of nodes, because you don’t want to manually do steps on all nodes separately. The benefits though…
- On restart a node walks over all stored data. This is faster on small nodes.
- Migration of small nodes between drives has smaller downtime.
- And if migration is within a single machine, you can dedicate small LVM volumes for each node, then migrate a node with zero downtime using
pvmove. - If you get a new drive for Storj, you can just migrate one of the already vetted nodes from an old drive and have the drive fill up faster.
- If you suddenly need space, you can quickly gracefully exit a single node recovering 500GB of disk space, as opposed to sacrificing a large node (because graceful exit of a larger node also takes more time). And even if you don’t have time to graceful exit a small node, sacrificing it is less of an impact. Hopefully Storj implements partial graceful exit soon, so that this use case is no longer valid.
- In case there is another operator within your IP block, more nodes gets you more traffic, because the traffic is divided equally between nodes, not between operators (unless Storj fixed this problem since I have checked the code).
Also,
smaller nodes do less I/O as well, so it should balance out.