Of course it does. Otherwise what is the difference between running a 1TB node and a 10TB node? We should all run 1TB nodes if there is no difference.
A 10TB node has approximately 10x the potential income of a 1TB node. Of course not all pieces will generate the same amount of egress traffic, but if we assume both nodes have filled up, the 10TB node is almost certainly going to be generating more egress traffic than the 1TB node unless the 1TB node got really lucky and is storing many “hot” pieces.
If I had enough space and cared about that, I’d probably run X+2 nodes on X hard drives, with the last two nodes of minimal size. If one node fails, I’d have a pre-vetted node on another drive to move and start again. This approach reduces the risk of losing time on vetting again.
If Tardigrade becomes successful, in my opinion the effect of economy of scale will likely make most of storage space be operated by large SNOs. Then these features will probably show up, whether coded by Storj or independently, as there will be money in this work.
You are correct, but there are still a lot of other scalable and redundancy measures that should be put in place to accommodate that level of scale. Just Kubernetes along with a fault tolerant storage backend can fill this to a pretty significant extent. Would really only become a valuable addition to the application once an individual storage node is storing more data than one can easily do on a small cluster of systems. Even then, this is still going to be a lot of work to abstract the separate storagenode components to be scalable for a small handful of storage nodes.
I’m not saying that this idea is not a good one, I would be fully supportive of seeing something like this as an SNO who already has nearly redundant everything in my infrastructure stack. It’s just that there are so many other measures that can be taken to accomplish a very similar end result with the exception of a redundant database layer with Storj Labs not having to put in any additional engineering.
yeah raid in the 3x6tb raid5 example the raid mitigates the somewhere between 1% and 90% chance that one drive might fail in a year… but lets call it 1.5to 6%
ofc it does reduce your theoretical max earnings by 1/3… so i really becomes an question if your drive has more than 30% chance to fail… and if 1/3 fails then you also only lost 33% not 100%
if you got 8 drives in a raid6 then you have somewhere between 4% to 16% chance of annual failure and loss of all of your data… while you would with a set of 8 drives only have 0.5 to 2% chance of loosing atleast 1 drives , which would loose you 1/8th so still less than what the 25% cost of the raid redundancy
but its getting closers, the higher we get the ratio of disks to redundant disks, the less cost… but then we end up running raid arrays that are like raidz2 / raid6 on 12 arrays…
which is just painful… but it’s very economical in its loss of capacity…
meanwhile having 12 drives you will be up to 6-24% chance of disk failure a yearand you are paying just about the same…
24 and 2 redundant and you are home… but well… arrays like that can take many months to resilver / rebuild a lost drive… so odds are you will lose drives faster during resilvering because of the added strain, and thus you end up actually going beyond what makes the raid setup safe…
so yeah for most people running seperate nodes seems to be the way to go…
tho there is something to be said for raid performance when its setup right… one has to keep in mind it gives you access to have vast dataset and manage them with thousands of users… a regular hdd is most often made to supply 1 or a few people with data over limited time spans.
