Large Nodes - Performance Tweaking

I am experimenting with some larger nodes (redundant RAID10 storage).

For one node I have mapped 70TB on the RAID array to one mount point and set up 1 node to run from this.

  • Once the node is full, it will have very many files, does this significantly increase the chance of database corruption versus a smaller node - and if so is there a way to back up the database or something in case of corruption.

  • On the other nodes I’m running, the available total size of the share does not seem to have any correlation to the amount of additional disk space used/filled each month. Is there something I could tweak to increase the numbers of files downloaded to speed up the disk filling? (It is a very powerful server that this node is running on).

Is it possible to have a node that runs on multiple IPs (say a server has 2x gigabit network uplinks, can I share the traffic between the two of them?)

Not sure if you know this but there’s no way to speed up the filling of the hard drive, The data being shared is being uploaded to over 3000 nodes around the world, Having a powerful server doesnt technically make it better for receiving data it could increase your chances to get the data first.
People are running nodes on a Pi4 which isnt powerful at all and it will still receive about the same amount of data as say a Xeon with 28cores.

Im going to say no to this one but you can have it to be redundant to pick up the second internet when one goes down but thats pretty much it.

Im currently running my node on a Free dell that i got from the trash its currently at 4.5TB of bandwidth this month, Its running on mechanical hard drives no raid nothing.

I think in your case you spend more on electricity than make some money.

1 Like

If there were, I’d be doing that on my smaller node as well and… we’d back where we started, wouldn’t we?

I think the best you could do is use round-robin DNS for load distribution, which isn’t ideal, but it’d spread traffic to some extent. However, DNS caching kind of breaks this and the load will likely not be equal across both IP’s.
Edit: I guess it’s not the absolute best, there are other smarter DNS based load balancing solutions that take into account whether a server at an IP is actually reachable, which would also cover fail over scenarios that round-robin does not. But I think that may be overkill.