Delegate Traffic based on node type

Hi all,

I’m new here so forgive me if this is not a new suggestion but thought I would throw it out there.
Quick background, Developer by trade with a bit of operations thrown in.

No doubt, we have a lot of different people and nodes connected. From hobbyists running a node on their laptop, to Homelab types and many in between. This effects what hardware and network each user has. Some with 500GB’s and an DSL/Cable connection. Some with 70TB’s and a gigabit+ connection with again, a large mixture of hardware and network capabilities. Some people have a limited broadband pipe with sub 20mbit but a lot of storage and would do well hosting data that doesn’t require regular Egress, while others can provide high bandwidth, low latency data retrieval.

As a developer, I use AWS S3 for high availability, but at the expense of higher storage costs. If I need cheap bulk storage, Backblaze B2. The key difference between the two is performance and cost. S3 is great for performance, B2 for cold storage. I would recommend, if you want to really get the interest of Devs/Ops types, the ability to choose between storage nodes and bandwidth nodes would be great.

Nodes with low bandwidth would charge less for storage but when egress is required, charge more. Such as restoring from backup. While high bandwidth nodes charge more for storage, less for egress, but they make up the cost due to receiving high priority traffic.

Implementing such a system would make Tardigrade more viable for customer facing systems while also keeping the idea of cheap distributed storage alive. Give the customers what they need while tailoring the backend to support the wide range of infrastructure available.

3 Likes

Hey @Cyber-Hoarder welcome and thank you for such a thoughtful first post :slight_smile:

tagging @brandon our product manager to hop into the discussion!

1 Like

ack, just remembered he may be on vacation at the moment, during te holiday. OP if you also wouldnt mind, it would be really great if you can submit this idea in the idea portal, which is ideas.storj.io

Hi CyberHoarder,

Welcome to the Storj Community! Great thoughts. The network is architected to naturally play into these sorts of scenarios.

My thesis has been that by combining B2 pricing with S3-level of availability, we will be able to efficiently tackle both markets/use-cases simultaneously. I believe that an advantage of Tardigrade is minimal, and simple pricing - ie there is no need to spend days sifting through various licensing schema – as you will be paying only for what you use (in terms of bandwidth and storage).

Compared to AWS/Google Cloud/Azure storage offerings. we are half the cost half the cost, 20% more performant, globally vs regionally available, open source, and encrypted. There is also no need to replicate data across availability zones to achieve redundancy or global availability (making it even cheaper than the Big Guys).

If you want infrequently-accessed storage, you will not be paying for unused bandwidth, and pricing/feature set is highly competitive with Backblaze and lower cost alternatives.

Additionally, it is important to note that B2 and others have high egress bandwidth charges. Some of these providers claim “unlimited” egress which really isn’t unlimited. You can only transfer the data out one time, and if you do it more than that they will just block your account. With Storj you can download the file as many times as you would like.

That being said, we do also have some notion of “storage classes” on our roadmap. Today, the idea is to leverage Reed Solomon (in conjunction with Audit and Repair processes) to optimize for performance for S3-like use case, while still having SLA-driven assurance of data durability. In the future, we hope to leverage node selection to tackle compliant storage (HIPPA, GDPR, SOCs, etc.). Node selection will also play a role in CDN-type use cases, and may make it even more efficient/economical to select for cold-storage.

4 Likes

@Cyber-Hoarder you make some great points. @keleffew response is spot on with our strategy for how we believe we can make Tardigrade more viable for all sorts of customers and use cases.

2 Likes