Update Proposal for Storage Node Operators - Open for Comments

Announcements
82

I see your point, IMO it’s too complicated to have qualifications tests, these can be falsified and comprise a lot of work.

Storage Tier is a nice idea for datacenters but not what storj SNO is for.

The amount of SNO capable of providing with a part of the data the client requires is quite big, and the amount of idle time of drives is also quite significant, so search and serve a part is actually quite fast.

Let me try and put this in another perspective, if you put one file in one drive and you want speed, SSD is the obvious choice, but you could achieve almost the same result with a lot of mechanical drives in raid0, of course latency is always higher with HDD.

Storj is a lot of mechanical drives capable of giving you that part, having a an SSD does not make a world of difference for the data hoarding people do, documents, pictures, videos, backups, etc, most will not be reused for a long while.

If you need SSD like speeds for your project, you probably need something already in the server or very close to it, i.e. to serve a webpage with static content, machine learning, databases, etc.

Storj business model is good enough for years to come, the only issue for SNOs is the price for storage, cold storage is increasing everyday, it will eventually if not already is bigger than the active storage.

SNOs get a lot of the money off the Egrees, if an 18TB drive has almost no Egress and it’s full, it could be preferable for the SNO to exit that node and just readded again in the hopes to get fresh active data to bump up the egress, this can actively work against storj trying to repair and shift the data to other available SNO.

83

I understand the reasoning behind all of this but unfortunately the journey ends here for me. I’ve been running my SNO on a home server for quite a few years time (since v2 beta). But the new payouts, especially the rates proposed in Europe, don’t have the incentive I need to continue.

I’m using existing hardware and have enjoyed a 100% full and mature node which will still pay out monthly for the storage. But all disks wear out at some point. Taking into account the cost of 24/7 energy (not cheap in the Netherlands), with these payouts it will be very hard to pay for a replacement disks it fails. I know you’re not supposed to buy new hardware as SNO, but then I would just be delaying the inevitable by staying. The goals and direction of the project are clear.

What remains is just not worth the time and effort for me, someone with a non-tech day job. I’m choosing to spend my time on family and other activities. But I wish you all well. I hope Storj becomes the decentralised success it aspires to be. And a big thank-you to the community and valued members (thinking of you BrightSilence). All the best!

16 Likes
84

Sad to see you go @mdr, but we all need to make our own decisions on whether it’s still worth it. Unfortunately these changes could only lead to some exiting. As someone who runs nodes in the Netherlands as well, keep in mind that energy prices are dropping again. HDDs last quite long these days. Even with now 22 HDDs spinning, I haven’t had a failure in years. And some of those are very old now.
Should you change your mind, I’m sure the community would love to see you around here again. (I appreciate you mentioning me specifically, but this community is awesome all over)
So I’ll stick with “tot ziens”! :slightly_smiling_face:

7 Likes
85

its a text on top in the dash rn. with link to forum.v 1.78…

2 Likes
86

mm… did anyone actually come up and compare the difference if you do something like this though?

  • Having fresh data, rotating it between 2 harddisk, so that these are fresh data with egress traffic most time, potentially earning more maybe, but less stable m/o/m in returns?
  • Having fully stored data on 2 harddisk, minimal egress, but stable m/o/m returns?
  1. With only 1 internet IP
  2. Factoring that you will have to give up the withheld amount when you shutdown the node in scenario 1, since to optimize egress, you’ll likely have to skip GE and reset the node so that you can start the new node validation quickly
87

It is interesting to be an outsider watching Storj after I tested this service for video. Which btw still needs a typical CDN for TTFB.

But I think this pricing change is quite strange. It seems Storj is not quite sure what differences the product has vs a centralized service.

For comparison Cloudflare (My goto for cheap/useful) with R2 WIPES the floor in price, speed and tier 1 network. Although not yet fully distributed and only 7 locations at $0.15/gb or $15/TB no egress charges it’s much better deal. These 7 locations are extremely throughput heavy. Almost as fast as a CDN. But with the Cloudflare CDN being free with R2 you’d have to be kinda nuts to pay $7/tb for Storj. There are PB users on Cloudflare that they don’t boot. That service will soon distribute objects automatically for the same price of $0.15/gb $0 egress. For many applications there will be no need for a CDN.

There is also DO spaces object storage plus CDN Spaces Object Storage Pricing | DigitalOcean

My point is this:

$4/tb is too low for both Storj and your node suppliers. Triple or quadruple the price.

Remove egress entirely or to a much lower cost.

Why? No one could compete with that on such a distributed network as Storj. The moat is massive. Only a distributed network could even offer such a thing.

It is very unlikely of people to use egress at more than a 30% utilization rate meaning the profit on storage would be much higher than egress. .

Right now the cloud has definitely lost all zing thanks to egress. People have had enough. With more and more 1Gbps connections the moat of Storj grows larger.

I’d happily pay way more for storage to have egress free. Certainly AI and data science areas, media, big data would too.

Keeping distributed data low when that is the entire selling point is quite interesting when I care very little about the egress side of Storj, considering it’s likely the object is delivered over a very garbage residential grade network. Not a direct tier 1 carrier grade port.

I’d pay more for storage if egress was waived. It also incentivizes large data growth on the network which has a roll on effect of more storage on the network being added.

You’re competing users against each other via egress when that’s really a terrible selling point “oh yeah it’s delivered from their house network isn’t that great?!” No, not at all.

Charge for data the main selling point and differentiation.

There will be explosive growth for storage in the next 20 years. The cost of egress and connectivity will come down drastically.

I mean, wasabi has free egress and lots of locations. I think the team needs to see the writing on the wall. Businesses in the data space are shifting to free egress.

6 Likes
89

Thy don’t have free egress, they are misleading people. Any egress over the amount stored cost money. Or if you are on a free egress plan — account suspension.

5 Likes
90

I did’t studie the storage plans of other companies, but I believe Storj is the onlyone that charges only the used space, not the entire TB, even if you store 1KB of data. So at others you pay each month for 1TB, the full price, but with Storj you pay 4$/TB to 0$ per month, depending of how much you store and for how long. The egress can’t be free in a distributed network based on resources from others. There can be scenarious in witch case the providers don’t gain anything and leave.

91

Storj is definitely not the only one, most of object storage providers charge for actual space used – Backblaze, Amazon, Azure, Google Cloud Storage, etc. (except Wasabi. They change for 1TB minimum, with 3 month minimum retention charge – but they don’t tell you that up front either, do they? Ugh!)

It is, however, common for end user products – such as Google Drive, that is implemented on top of GCS, or (now defunct) Amazon Drive, implemented on top of AWS S3, OneDrive, Box, etc. to charge flat rate for “available” space regardless of actual usage, so that the vast majority of light users cover minority of heavy users abuses, and company profit.

1 Like
92

That is correct but with huge caveats, you need to use it with a CDN then it is truly free. Which was my point.

Egress is expensive for every provider. But cheap for Storj. Don’t charge for it. Build the moat. Sadly, for Storj, not only will CF beat them at this cost as 1000 servers all connected sharing bandwidth from a central data centre is cheaper than 1000 computers in people’s home. They will win the egress free race. They’re even building for it right in front of your eyes. Cloudflare servers don't own IPs anymore – so how do they connect to the Internet?

With wasabi there is free egress but only to the amount of storage. 1PB stored = 1PB egress. Because it’s not made for that.

A lot of people don’t know egress is unlimited with CF CDN + Wasabi, if you’re pushing 1PB/month you’ll likely get a call from CF asking for about $2000/4000/month. If you can’t afford that then you can’t afford the service you’re egressing.

CF R2 is truly egress unlimited and is really awesome, much faster than storj when I tested it and that was including a 200ms round trip to a different countries server. But as CF has dedicated cross continent backhaul it’s very quick.

Btw I tried Storj last night using a video I got from Steam games store. And I mean it got absolutely wiped by CF CDN and Akamai by almost 20-30x TTFB. 50ms average vs 1-3 seconds. Your egress story is not sellable. Sell scalable storage instead.

If I take Australia for example where egress is between $1/Mbps & $3/Mbps. A large provider will likely pay less but at $7/TB it would be cheaper to provide that via dedicated link from a data center.

If Storj users are delivering from home networks the bandwidth is cheap/underutilized anyway.

2 Likes
93

Better to charge by GB and forget about small files. Similar to R2. It would be $0.15/gb over 10gb even if the data is over by 1mb with R2. Don’t see an issue with that. Storing 1000 small files at 100mb but paying for 1gb isn’t an issue for anyone.

1 Like
94

The problem is that because of the design of the system, Storj has to pay GCP (or similar) egress costs because that’s where the satellites are. They can never offer anything less with the current design without losing money.

There are other distributed providers that can allow consumers to connect directly to the nodes (or nearly so, definitely not with GCP egress costs as Storj is today) which is the game changer you’re alluding to.

I was hoping this discussion would move to a protocol evolution enabling this distributed network, but it seems to just move the prices.

3 Likes
95

By GCP do you mean Google Cloud Platform? In that case that would be the highest egress costs in the market.

96

I think for video fast working from storj, we need to allow clients gateways(client side) to buffer metadata of files that it own and update it time to time, so client side just can start directly download files very fast, not waiting satellite response, i think this technology will significantly lower time to video start.

2 Likes
97

Hello @2bluesc,
Welcome to the forum!

The general traffic is going directly between nodes and the customers. Satellite is an address book, auditor, repair and payment processor.

So egress costs from compute provider applies only to upload of repair traffic.

2 Likes
98

Hey @what, welcome to the forum! These are great thoughts and ideas you are having.

Unfortunately, we don’t really like price discussions that are rooted in reality and not in a future rainbow fantasyland. We node operators spend thousands of dollars for hardware during the great subsidies period and need to make up for these upfront costs. According to us node operators, you are also forgetting the most important facts:
STORJ is faster, better, cheaper, greener, more reliable, and geo-redundant for free! It basically is the best S3 provider this world has ever seen. That is also the reason why in the not-so-distant future, our centralized S3 will be the backbone for the decentralized Web 3.0! So why are there no customers you may ask. Simple, first of all, STORJ does a bad job marketing our awesome product to these thick users and companies. It is also still early! Remember, Amazon and Uber also started small. Another problem is that STORJ is too cheap and because of that customers think it has no value. We should raise prices to 20$ for storage and egress. It would be a win-win, customers will finally ditch the inferior AWS S3, and I could get my back my hardware investment.

Btw I tried Storj last night using a video I got from Steam games store. And I mean it got absolutely wiped by CF CDN

That is probably because you are holding it wrong. Remember, STORJ is the best product the world has ever seen. It can’t be beaten by CF. You probably did something wrong. Let me guess, you used the defaults? Then it is your fault, you should have asked @Dominick in this forum for tuning recommendations. Otherwise, this is clearly your fault!
/s

my forum experience

Update Proposal for Storage Node Operators - #779 by IsThisOn
Let's talk about the elephant in the room: The Storj economic model (node operator payout model) - #947 by IsThisOn
Update Proposal for Storage Node Operators - #456 by IsThisOn
Update Proposal for Storage Node Operators - #398 by IsThisOn
Update Proposal for Storage Node Operators - #299 by IsThisOn
Let's talk about the elephant in the room: The Storj economic model (node operator payout model) - #947 by IsThisOn

1 Like
100

Hey man, I understand that you are frustrated, but this isn’t helping anything. I think it is kinda obvious things can be improved, and that Storj could be a little bit more transparant.

But I think we should also kinda accept that the situation we are currently in is just what it is due to decisions in the past. Maybe you are right with your suggestions. But changing the whole system at once is also a huge risk of people leaving the network. The one thing that is most important for Storj is not lose any data. That would rip their reputation apart. A sudden leave of operators could leave the whole system very vulnerable, a risk that maybe Storj couldn’t take according to their risk modelling.

I would also like to see more presence of Storj on this forum on the what and why they are doing things. Because from a technological interest, but also because I am interested in the future of this product. And if I, as a node operator, fit in that future.

I interpret your post as frustration about a product that you are interested in, otherwise you would’nt take the effort in reading the posts and express your ideas. But I don’t think people take posts more serious if they are so sarcastic.

Sorry for not minding my own business :man_shrugging:t2:

6 Likes
101

Hey @pietjebell! Your interpretation is semi-right.

I’m not. I have no horse in this race. I used an unused, old 8TB external SMR HDD as a test for one year. It died (not because of SMR :wink:) and went into the trash. I’m not angry but amused. Re-read my post, but this time don’t imagine someone angry but someone who makes fun of a religion.

That is 100% true. The basic idea of a decentralized storage platform based on UNUSED resources is very interesting. Like, it is an interesting mind game, not something I would bet money on. I also like to write to improve my English, because I am not a native speaker.

This is where I disagree. What I see in the Crypto space is a lot of young people who dream of an Insta lifestyle without putting into any work and dreaming of a passive income.
Maybe I can’t persuade someone from investing $ into HDD. But if this goes south, nobody can claim that there were no warning signs. If that happens, hopefully, people will take a deep look into the mirror and not blame it on the usual suspects (Biden, Jews, Global Elite, WEF, Gates, Soros, the list goes on). I think STORJ is a lot better than other projects, thanks to a technological barrier of entry and not being a get-rich-quick scheme.

Which suggestions? I didn’t make any suggestions. I simply wanted to debate claims other users made.

I think humor is by far the best way to reach people. If you are not “allowed” to make fun of your own bubble, you are most likely in a cult.

There is absolutely nothing you need to feel sorry about. You voiced your opinion in a clearly written and respectful way. That is what the internet used to be all about for decades before social media took over :slight_smile:

3 Likes
102

Is there any update on this? We’re approaching the next payout cycle and node databases still have the original payout information in them. The new payout rates have yet to be reported back. (Adding @littleskunk as well)

103

I believe it will be deployed shortly. Don’t have an exact time, but the team is working on it.

4 Likes