Hello Storage Node Operators!
Important: This includes an announcement of changes to payout rates for storage nodes on three production satellites: asia-east-1, europe-west-1, us-central-1 effective July 1st
First of all thank you all for your feedback and support during this time of change. We are grateful for the discussion and your engagement on the forums and elsewhere.
Since the changes made May 1st, (announced on April 17th) we have monitored the conversations in the forum and the health of the network and want to report on the results we’ve observed based on the steps taken to date.
What we did
We announced on April 17, 2023 that there would be changes to payout rates for storage nodes on three satellites: saltlake, europe-north-1, and us2 effective May 1st.
Additionally, on June 14th, we announced that we would be sunsetting the europe-north-1, and us2 satellites (some additional information on this below and more details on this to come later).
What we observed
We were interested in determining whether the changes to the incentive model would impact the health of the network. We measured the following dimensions of network health:
| Overall node count dimensions | Measure on April 16 | Measure on June 25 | % change |
|---|---|---|---|
| Count of active nodes on saltlake | 23,275 | 22,448 | -3.6% |
| Count of active nodes on us2 | 23,275 | 22,448 | -3.6% |
| Count of active nodes on europe-north-1 | 23,275 | 22,448 | -3.6% |
| Median Daily Repair Bandwidth (TB) | Mar 1 - Apr 16 | Apr 17 - June 25 | % Change |
|---|---|---|---|
| saltlake | 16.4 | 24.1 | 46.8% |
| us2 | 0.11 | 0.10 | -11.9% |
| europe-north-1 | 11.0 | 19.5 | 76.7% |
| Nodes initiating graceful exit (GE) or going offline by satellite | Measure on April 16 | Measure on June 25 | Median +/- 1 Stdev March 01 - April 16 |
|---|---|---|---|
| Count of nodes that called for GE in the prior 30 days saltlake | 262 | 163 | 222 +/- 41.3 |
| Count of nodes that called for GE in the prior 30 days us2 | 107 | 181 | 110 +/- 17.1 |
| Count of nodes that called for GE in the prior 30 days europe-north-1 | 210 | 197 | 195 +/- 26.6 |
| Count of nodes that went offline in the prior 30 days saltlake | 607 | 438 | 723 +/- 70.2 |
| Count of nodes that went offline in the prior 30 days us2 | 613 | 448 | 626 +/- 25.8 |
| Count of nodes that went offline in the prior 30 days europe-north-1 | 608 | 443 | 617 +/- 20.1 |
This data shows a few important things:
- There was only a small decrease in the number of active nodes on the satellites, largely driven by about 1000 nodes going offline around June 8th;
- The nodes that did leave the network seem to represent a small number of node operators rather than a general exodus
- Only moderate increases in repair traffic was observed
- There was no significant increase in nodes calling for GE or going offline;
- At no time did any of these changes threaten the health of the network or the data in any way;
Based on the results above, we have decided to make an additional change to the node payout rates.
Please read the following FAQ carefully as it details the changes we are planning to make. It also hopefully addresses some of the concerns that have been posted in the forum. As always we would love to hear from you and understand your thoughts and/or concerns on these changes. You can share any thoughts you have regarding this in the forum post.
The Storj Team
Q: Why did I see such a sharp decline in my payout in April and May, when the payout changes weren’t supposed to take effect until May?
A: Many of you have noticed a sharp decline in your individual payouts in April and were concerned that this was related to the changes of the payout rates. This mostly impacted nodes and data on the us-central-1 satellite. On this satellite the amount of customer egress dipped dramatically during April. It is difficult for us and our customers to predict ahead of time how much egress traffic they anticipate using. Our customers are typically growing, but often they are also implementing updated infrastructure which changes their usage pattern on the network. Over time as we increase in customers, these changes will be less dramatic, but for egress especially, we do anticipate swings from month to month to continue in perpetuity as it is much more fluid than data stored.
Regarding egress decreases in May, part of the story is what is described above, in that customers’ behavior is unpredictable, but we also took action against some abusive accounts on the network which did decrease the amount of egress traffic on all production satellites.
Q: What will the updated payouts be beginning July 1st?
A: We have decided based on the metrics we have been following and the input from the forum, to make the following changes to node payouts. Note: the rates previously set on May 1st for non production satellites will remain in effect:
This is the new payment schedule that will take effect as of July 1st, 2023:
Q: Are you planning to add any synthetic load to the network along with these payout reductions?
A: Over the past few months there has been a large increase in the amount of data stored. This growth in the storage on the network has led us to feel that we do not need to add additional synthetic data at this time, even with the upcoming removal of the two test satellites’ data.
Q: Will I be able to see the impact of the payout changes on my node dashboard?
A: We are working on updates to the node dashboard which will allow us to show historical and future anticipated payouts using different payout rates. If there is a change to the payouts in the current month, those changes won’t be reflected in the node dashboard until the payouts have been completed for the previous month. Until we finish making changes, historical values might also not show the correct earnings. Currently, you can see payout rates per satellite if you select an individual satellite on the payout information page of the dashboard.
Q: Do you anticipate making further changes to the payouts?
A: To reiterate a bit from our last post, fundamentally, storage node operator (SNO) payments right now are being subsidized by us to grow the network, and we are currently in a position where the network is larger than it needs to be for the demand we have. We realize that dropping payment rates will slow the growth of, and maybe even continue to shrink, the network, and that is precisely what we have decided we need to do.
In doing this we are targeting greater utilization of the network. This should enable SNOs’ drives to fill faster and produce higher returns over time. We do not yet know exactly where we will land in the long term on the payout amounts but we trust that the conversations we have, the economic signals we receive, and the corresponding network statistics will inform this process for us. We will monitor network capacity, sustainability, node churn, stability, and health at each step of the process, using this information to determine what, if any, further actions are needed.
Q: What updates can be provided regarding the shutdown of us2 and europe-north-1?
A: At this point we are still finalizing the exact shutdown process and timing, and will communicate more about that as it is determined. But we do anticipate beginning to delete data from the satellites very soon.
Q: What options do SNOs have if the new payout structure doesn’t work for them?
A: If at some point in this process the payouts for a satellite do not work for you, you can of course gracefully exit the given satellite.