I dont have experience with a full node. I have a spare jbod 48 3.5 hdd with 6gb backplane (netapp 4486). Connected with a lsi 9200-8e (2 port 6gb).
Do you think in the future can manage 48 nodes (hypothetically, dont wanna start a discussion about centralization). Someone can show an hdd utilization of a big node (15/20tb)?
on larger nodes egress eventually surpass ingress, but disks are pretty fast at read operations.
for a node in the 15-20TB range egress would certainly be above ingress.
a 4486 was designed to operate 48 disks, and really hdd disk speeds or iops haven’t really changed much since that time… it would work just fine for running 48 storagenodes.
might run into some bandwidth limitations when the filewalker is running on all nodes…
i mean assuming like 200MB/s read speeds, x 50 so about 10000MB/s
i guess if both ports on the HBA is used, you might be close… not sure if it works that way tho…
the dual IOM6 controllers are for failover i think, not parallel workloads …
so atleast in theory there might be a bandwidth limitation, but nothing serious… its also rare to really hit the max speeds on hdd’s anyways.
long story short, i think its fine… running a couple of netapp 4246 myself haven’t really had any issues.
you will need a half decent system to run that many nodes, like 12 cores and 64GB RAM minimum i would say.
it will ofc run on less, but as the nodes grow they will soak up more and more resources especially when doing larger tasks like running the filewalker.
1 Like
Why run 48 nodes? If it’s the same piece of hardware, barring you don’t have different subnets, why not just run one node? If you have 48 different subnets, that’s a different consideration. Though wrapping them all up on one piece of hardware for a single point of failure seems risky.
1 Like
If you do not have RAID already, run them one per disk as soon as a previous full or at least vetted.
Do not run them all at once if they are in the same /24 subnet of public IPs - the vetting could be longer in the same amount of times as a number of vetting nodes.
Actually if you start with one node and wait until it’s vetted, if you then start a second you will get a slight increase in ingress until that node is vetted. It only applies to one at a time though so if you still insist on running that many nodes, start them one at a time once the previous one is vetted. It’s not a significant increase but hey, every little bit counts right? Oh and yeah, don’t use raid. Additionally, the less drives your using the less power your sucking up to basically spin a bunch of empty drives.
1 Like
I don’t know yet if second IOM on 4486 will double the bandwidth but I don’t think . It will be useful if someone have history disk activity on a full node.