Announcement: Changes to node payout rates as of December 1st 2023 (Open for comment)

Hello storage node community,

We really appreciate the lively discussion the pricing announcement yesterday has engendered. We’re especially appreciative of the replies that understand what we’re trying to achieve in the long term, but we also understand that the new pricing simply doesn’t work for some storage node operators. In addition to the forum discussions, we’ve been meeting with node operators (both public network operators as well as commercial node program participants) on an ongoing basis to get feedback and share perspectives.

We wanted to share a bit more on two fronts: i) the data and details that led to our decisions, and 2) what options and steps are available to node operators.

One challenge the conversation has is a bit of asymmetrical data knowledge. While we don’t share everything about our business expenses and revenue, we do aim to be transparent (see any of our recent town halls). Above all, we aim to be reasonable and responsive, and have had multiple discussions with many storage node operators in advance of the pricing announcement we made yesterday. In addition to the already public information about the costs and scale associated with running the network, we are sharing additional data below that will provide context to the pricing announcement.

First, market conditions, and our economic situation have driven our payout rate adjustments. Storj doesn’t operate in a vacuum. While market prices for cloud storage have not changed dramatically in recent years, the market price for bandwidth and egress has dropped significantly. Wasabi, Backblaze, and Cloudflare have all introduced pricing models where a significant amount of egress is free. And, the large hyperscalers and CDNs are dropping prices as well. In order to stay competitive, we need to be able to drop prices too.

Furthermore, payouts needed to be adjusted to accommodate our current and growing customer base. As you may have gathered from our recent town halls, Storj is attracting larger and larger customers. This is a good thing overall for the health of the network. But, as you can imagine, large customers demand both significant discounts and significant guarantees around performance and security. So, large customers are generally not paying the quoted list prices.

Additionally, we are increasingly working with channel partners to sell our product. This is also a good thing overall – it means that there are more people selling Storj and offering a complete solution. However, the channel partners also need to make money to offset their costs. So, they take a portion of the list price, leaving less for Storj to use to cover expenses.

Not the least, we value our team members and need to compensate them fairly for the work that they do building and maintaining a great network. Our team members work full time to write the code, sell and market Storj, and keep the network running. They need to be paid, and paid at rates that reflect current economic reality.

Finally, we strive to give back to the communities where our customers and team members live and work, and to show by example how a company can contribute for the greater good. As noted above, we do provide a good deal of free or reduced priced data for non-profit organizations.

The bottom line for all of this is that – to maintain a healthy network, we need to adjust payouts. We did so after significant deliberation and input from the node community. We are responding to market signals and the needs of multiple stakeholders, storage node operators included. We understand that, for node operators in some markets and circumstances, this means that it may no longer be profitable to operate nodes, to provide large amounts of bandwidth, or to add new nodes.

Storj Network Data

We also want to share some internal data to show how our payout costs have gone up.

For instance, here are the totals for the amount we have spent on storage node payments since mid 2022. Taking into account minimum payment threshold, we paid the following real dollar amounts to storage node operators, listed by the month of service earnings. All of these are verifiable on the Ethereum, Polygon, zkSync Lite, and zkSync Era chains.

Month Amount
2022-01 $49,580.83
2022-02 $158,818.23
2022-03 $78,281.42
2022-04 $182,561.60
2022-05 $50,529.21
2022-06 $129,058.82
2022-07 $121,382.26
2022-08 $105,435.62
2022-09 $135,788.54
2022-10 $141,526.23
2022-11 $135,372.38
2022-12 $149,968.03
2023-01 $145,930.23
2023-02 $156,195.96
2023-03 $185,784.37
2023-04 $138,213.39
2023-05 $173,766.86
2023-06 $144,336.49
2023-07 $142,058.61
2023-08 $136,346.83
2023-09 $106,183.86
Average $131,767.61

As you can see, the average payout since the beginning of 2022 has been around $130K per month. This is the total amount we are targeting per month to ensure there is sufficient network utilization to make it economically viable to run a storage node (as first announced here), considering the size of the network we are trying to target. In advance of customer usage, we store synthetic data and use synthetic egress to maintain that level of payout. We plan to continue to use these tools to keep this average payout rate near these levels until the growth of customer data eliminates the need for the synthetic load.

There has been some discussion regarding our costs in this thread. For a previous discussion on costs you can check out this post. But if we analyze our costs just for storing data (not including bandwidth) if a customer uploads 1TB of data to our network, we don’t only store 1TB. Because there is a risk some nodes could leave or fail, we actually store closer to 2.7 TB (80/29 = 2.758…). What this means is that for each TB of data we store, we pay node operators $1.50 x 2.7 = $4.05. Our current list price per TB is $4 so we are currently paying SNOs over 100% of our revenue for data stored. Then, in addition to the SNO costs, we have costs for running the satellites as well as customer acquisition and customer support. Originally we had anticipated changing the payout rate for storage to between $0.75 - $1.00, but after listening to feedback from the community (including here, here, and here) we determined that it was better for the network to keep the storage payout rate stable, even though it meant going above our desired $1.00, and instead work on becoming profitable by more aggressively doing other things including lowering the Reed Solomon expansion factor (still to be done), lower satellite costs (work has begun, see post regarding shutdown of EU North and US 2, but more work to be done), and bringing down our heavily subsidized egress payouts.

On the egress side of things, again originally we had targeted bringing the SNO costs somewhere in the range of $1.50 - $5.00 per TB. With this current announcement we are bringing that down to $2.00 and don’t anticipate further changes in the foreseeable future. There was a lot of analysis and work that went into determining the right balance on this incentive. While competitors like Wasabi and Backblaze have raised prices, both offer some level of free egress. We frequently need to price more aggressively or bundle some egress to win against those competitors.

Our new egress pricing is admittedly on the lower end of the above range in part because of the decision to keep storage payout high, but also due to the fact that we still have significant costs for our edge services and, as noted, customer acquisition and customer support costs that need to be covered by the $5 per TB difference between the $7 we charge customers and the $2 we will be paying SNOs. With the revenue we receive from customers, we also need to cover all of our other operating and engineering expenses to run the company and improve the product.

While we aim to be supportive of our public network and want to pay rates that are sustainable, it is worth pointing out the broader environment we are in. Other projects that are similar to Storj in structure but with much larger networks, i.e. Filecoin and Chia, are currently paying lower rates for data stored with zero remuneration for egress. Those other projects have significantly higher requirements for more expensive to operate hardware than is required for a storage node in our network. In light of that, we don’t believe that the rates that will take effect in December are unreasonable in this market. We provide these data points to share where the market for storage seems to be going (which is evidenced by the continued growth of Storj nodes).

Steps available to node operators

If you are one of the many node operators who we have either talked to directly, or has expressed support for what we are doing in the forum or elsewhere, thank you! We see you!

If the new prices don’t work for you, you have three options available to you:

  • Abrupt halt of your node for any length of time, which is destructive to our overall network health (see below for details).
  • Graceful Exit, which has recently been improved and revamped and does not have the challenges past graceful exit incarnations had. Please see the details here.
  • Continue to engage in dialogue via the forum or by reaching out to us directly.

The first choice is fairly destructive to our overall network health. While the incentive design and rules of the network are optimized to encourage nodes to choose the second choice, a Graceful Exit, we have designed our network to handle abrupt network exit by failing nodes. Due to the war in Eastern Europe we have already adjusted our durability characteristics such that we could lose a substantial number of nodes and not lose a single object. However, this type of durability does result in higher amounts of repair and extra pieces, which reduces the amount of egress per piece and thus the total per TB earnings for node operators. If a large-scale abrupt exit were to happen, remaining nodes will get much more traffic, and nodes that we allow to return will come back to find less data. To underscore that last point, nodes and node operators that break our terms of service by engaging in intentional disruption of the network are explicitly subject to direct disqualification.

On the other hand, the second option is helpful to the network overall, as it provides us a strong signal, with a decent lead time. We see the number of nodes that are queued for Graceful Exit, which allows us to keep the network healthy while having a dialogue with you. This is also the only way to recover your held amount. It’s worth pointing out that Graceful Exit was designed with node operators in mind, both so that we could keep the expansion factor lower (thus increasing the ratio of egress per piece), and so that node operators had control of the (potentially) strong signals they send to satellite operators, without jeopardizing the network altogether.

As you know, we recently refactored Graceful Exit. Thanks to this new implementation, we can now offer a support-ticket driven ability to cancel Graceful Exit. If you have started a Graceful Exit and then later decide you want to remain on the network, during the next few months you can file a support ticket to have this Graceful Exit canceled. Note that this will only affect data that has yet to be transferred off your node, and all data that gets transferred off before you cancel will remain transferred. We will evaluate if adding general support for this is feasible for the long term for functionality like reducing the amount of data any storage node shares with the network. But most importantly, a partial Graceful Exit will not risk disqualification, like abrupt disconnect would.

Summary

With this new announcement, we still are passing through almost all storage margin to storage node operators, and are still paying reasonable egress rates we have carefully vetted against the market. We want to reiterate that we avoid frequent rate changes. Our previous set of price changes to egress was nearly four years ago. And, based on our read of the market, we don’t anticipate needing to reduce prices further in the foreseeable future.

Again, we want to thank you all for your patience and understanding as we go through this effort to bring our business and the network into the best possible health.

The Storj Team

17 Likes