What's the latest status on node storage resiliency

Seems silly with the continual increase in drive sizes if individual nodes are penalized for single fault data loss events, especially if so many copies are stored elsewhere.

I became a node operator while storj was beta and have done it mainly to promote the product. At this stage, however, I need to increase my disk storage, and the rate of return is so low I don’t think I can justify running further. I’m curious if I’m the only one in this sort of situation or this is common. If I didn’t have to factor in RAID, it would be easy for me to justify continuing.

Thoughts?

Hello @ske5074 ,
Welcome to the forum!

We do not store multiple copies since v3 is started, we use erasure coding, see

and

The RAID is an expensive option, it’s better to run several nodes, each on own HDD. This will result working as a RAID, because all nodes in one location (in the same /24 subnet of Public IPs) are treated as a one big node. So, if one disk die, you will lost only one part of the common data, but not all as in case of one node and bitrot during RAID rebuild.

See also RAID vs No RAID choice

3 Likes

Thanks - So there’s still no consensus on RAID. That’s what I was afraid of.

You are welcome!
It will be difficult to reach a consensus - everyone have an own situation. You may have a RAID already (NAS for example), or you want to build a new setup (even if we always suggest to use what you have now) and want to learn a new technology. Or you use refurbished HDDs, which already have a long history and could have a short lifetime in worse case.

However 1 node - 1 HDD is a simplest and cheaper option. The software overhead is 150MB-300MB RAM with non-SMR HDDs and native filesystem (NTFS for Windows, ext4 for Linux, APFS for macOS). With any software RAID the overhead will be greater (usually RAID requires more RAM and CPU), HDDs will wear more quickly because of additional operations to handle RAID and they will have a higher load.
Hardware RAID is not so flexible as software RAID, it has additional cost to hardware and you can find yourself with bunch of disks if the RAID card would burned-out, because to rebuild them back to the RAID you need to buy exactly the same RAID card (they often does not compatible with each other).

2 Likes

The math also doesnt work in RAIDs favor. Given the (admittedly arguable) death of RAID5, you should have 2 parity disks. So for all of the good times with RAID, you are missing out on 2 drives worth of payout. Also if you lose enough drives to kill the array, you are out all of your node(s), where with a JBOD (without parity) you only lose what died. Then you have the payout from all of the drives you have left while you replace the drives/nodes you lost.