interesting how much here is synthetic data, someone have the answer?
just want to know about the amounts
i hope there is only client data now.
1 Like
For February, my node got $52 total, $26.2 was from test satellites (europe-north and saltlake). This is how egress looked:
855kbps (258GB) was from test satellites and 2701kbps (816GB) was from customer satellites
However, stored data was more from the test satellites:
~15TB from test sattelites and ~7.18TB from customer satellites.
So, with the proposed values, I would get $19-$28. If the test data goes away, I would get $6-$11. Now that is not really in the “worth it” level. Yeah, sure, I would get some free space on my server and since the node is running already I may keep it running, but $6 is a bit too low for me to be interested in troubleshooting or trying to keep the node up. It may become like some other things I ran, “let’s check how X is doing, oh, it has crashed a month ago, well, better restart it”.
3 Likes
Yes I totally agree with that. But it’s also important to understand, that egress also may differ from month to month. once it may be good and the other times it may be bad. Right now, I get about 10% of my overall storage as egress which means if I have 20TB storage, monthly I have around 2TB egress from it (10%). But if let’s say egress goes to 40% and that 40% would be really really stable, then no problem for me. It would still be good. And I would say most of SNOs would be also happy. But if the egress stays 10% and we have to pray for customers to download the data, for the long run it’s not sustainable. I think it would be cool to see 50% of all data stored see egress of that, that would probably give us more income than we actually have, but we would have to assume that 50% of the customers download the same amount of data they upload in a month. Which in my opinion is unreal assumption.
But as I said before, if egress goes UP, we are good. But if egress stays at it is, then RIP.
Egress criteria for SNOs to be happy:
20$ per 1TB egress → 5$ per 1TB egress
1TB egress data per month → 4TB egress data per month.
This means we are in the same spot. But the most important thing is that we have to know for sure that’s gonna be happening. We just can’t run nodes in hope that we will see at the end of the month how the egress is looking.
It also matters what are the use cases for the storage you sell. if it’s 90% cold storage, then I doubt that will happen. The best option I think is to gradually reduce egress payout, with the same amount of egress traffic hitting the network. So first reduce to 10$ per 1TB and make sure the egress has doubled for data. if that happens reduce it further and so on.
Also another idea is to have dynamic egress payouts. Let’s say your node had egress 10% of your total storage, so you get paid 10$ per TB egress, if it had like 25% egress of your total storage you get paid 5$per TB egress. Or something like that.
This way we kind of make egress payouts like storage payouts - more stable
1 Like
I can understand the idea and goal of these changes, but I feel this is absolutely not the right way to do it. Instead, (as I have posted earlier, and other users have) the idea of bandwidth should be removed.
Bandwidth doesn’t cost anything for the SNO (and if an SNO is on a limited bandwidth plan, that’s not very smart), and won’t cost anything for Storj. If the concept of bandwidth were waived, it would do more good than bad. The customers would be happy, as they are paying less.
Also, why is the concept of holding tokens even a thing? Repair shouldn’t be paid for, right now it seems like SNOs are just being paid for anything and everything. No tokens should be held (and if they are, they should be released back to the SNO eventually, not held for the cost of repair). This would make the SNO more profitable, but would also make it more attractive for the customers, if they don’t have to worry about bandwidth.
1 Like
According to https://storjstats.info traffic should be negligible.
0.5% of Egress
Summary
( 1.10 TB + 1.57 TB + 1.42 TB ) ÷ ( 1.10 TB + 1.57 TB + 1.42 TB + 81.8 TB + 697 TB + 3.55 TB ) = 0.005201
0.1% of Ingress
Summary
( 3.51 TB + 4.25 TB + 3.69 TB ) ÷ ( 3.51 TB + 4.25 TB + 3.69 TB + 1080 TB + 6910 TB + 3360 TB ) = 0.001008
but 34% of Storage capacity
Summary
( 22.5 TB + 3270 TB + 3180 TB ) ÷ ( 22.5 TB + 3270 TB + 3180 TB + 1880 TB + 6260 TB + 4250 TB ) = 0.343141
4 Likes
While I appreciate your faith in me, I have no intention to create a union like this, official or unofficial. Unions in any form usually appear when concerns aren’t being listened to and that is simply not the position I hold. Furthermore, while I do sometimes relay the sentiments of other SNO’s I might not necessarily agree with, I will first and foremost advocate for what I personally think is right. This is also why I encourage other people to join this conversation in places like the twitter space. Going over some of the points you mentioned.
This was literally mentioned in the top post as one of the things they are already looking to change. It’s clear that they understand that this is an issue and the usage of this free account doesn’t align with the original intention for it. I have no doubt this will be resolved. I would also like to point out that in these pricing discussions we’ve consistently ignored these costs (as well as the cost of test data) to begin with. As long term, these issues should be resolved separate from the payout discussion. So I think we’ve already displayed the right sentiment around this and I trust Storj will take steps to resolve these unnecessary costs.
We’re not covering any losses. We’re here voluntarily and we can leave whenever we want. Storj isn’t making us pay for anything. We are the ones who choose to stay or leave. All I ask is that if you intend to leave, say it loud, but say it smart. Explain why you’re leaving, lay out why these changes make your node no longer viable. This is really the thing that will effect the most change.
So… I don’t think I see the immense oversupply mentioned in the top post. Storj has helpfully just added stats for stored amount on storagenodes after expansion factor.
There is currently roughly:
- 30PB of free space on nodes (24PB free on vetted nodes)
- 27PB of data stored from customer satellites
- 15PB of data stored from testing satellites
Assuming that test data disappears the network is roughly 37.5% used. That’s slightly low, for sure. But given the exponential growth curve in data usage on the network, this isn’t extremely low. Also keep in mind that some nodes may present more available free space than there actually is. I’m guilty of this myself due to a quirk in my setup. Furthermore, free space is important to have room for onboarding large customers. You don’t ever want Storj to be in the position to have to say no to prospective customers because the capacity isn’t there on the network. In the end, I don’t really buy the argument of wanting to fix oversupply. This is a side effect of these changes, but not the main driver. And while it would help compensate loss of income for node operators who still have free space, it won’t help those with already full nodes.
In the end I don’t see limiting onboarding of new nodes as the highest priority. That said, I do advocate for limiting the payout drop compensations (surge payouts) only to existing nodes and letting new nodes deal with the future payout scheme from day one. This will both stop influx from new nodes, with the upside of having only new nodes that are viable using the new payout structure. And furthermore it will help Storj Labs determine the level of node acquisition using these prices as soon as possible. And find out if this would grind node acquisition to a halt.
We’ve always worked under the assumption that the customer facing satellites (eu1, us1, ap1) store customer data and the others test data. I have no reason to believe this isn’t the case. If those assumptions are correct, see the numbers I mentioned above.
At the moment I have seen insufficient evidence to support the assumption that egress is going up at all (apart from when replacing test data with customer data, which won’t be nearly enough to quadruple egress). I know that is a goal, but goals don’t automatically happen. I’m running under the assumption that things will mostly stay the same. Additionally we have to consider that the older nodes are, the older the data they hold will get, which may actually lead to less egress over time. So for now I’m not going to assume that these numbers will change significantly.
Repair workers require compute and egress charges as well. Even if nodes aren’t paid, repair will still cost money. There needs to be an incentive for node operators to run graceful exit, rather than just shutting their nodes down. The held amount is the only thing doing that… and it’s currently doing it poorly as many node operators already don’t find it enough incentive to run graceful exit. If anything, I think the held amount requirement should go up as it will in the end lower repair costs across the network. However, it should never be 75% of payout like it is now in the first 3 months.
3 Likes
Just because it is “free” for you doesn’t mean it is free for everyone. Despite the possibility to game the free egress system, traffic is not free. Usually ISP / DC / NO have to pay for outgoing traffic and incoming is free, because sender pays. It is possible if you generate too much outgoing traffic your ISP may cancel your contract or introduce a limit because you are too expensive.
Additionally there are still ISPs with specific limits in place. Which Internet Service Providers Have Data Caps? | HighSpeedInternet.com
And many more around the World.
It seems I was over optimistic with regard to my assessment of the present situation and your personality @BrightSilence or to be more precise your point of view. I am telling you, all this I read here goes into a direction of a perfect competition model and we all sooner or later will deeply regret it. I would also like to add that in my opinion your points of reference used for assumptions related to: i) the pricing / renumeration model and ii) stats related to free and utilized space on the nodes are over simplified. Or to be precise: are not taking all variables into account. I will probably make you a bit sad … 
… but I am not going to try to provide you with details why I think so. It would be useless time for me to try to understand all this in detail, mostly because I will never have all those data that Storj Inc. has (not that I am asking for such data) and because the renumeration I obtain from this hobby project does not justify it.
Unions in any form usually appear when concerns aren’t being listened to and that is simply not the position I hold.
I believe this is not correct @BrightSilence, however, formal or informal, it seems to look that you are our union’s managing director already, so may the force be with you. LOL
You’re missing or ignoring all of the above places where this is addressed. Why should I allow my 500Mbps line to be saturated with egress and my disks to be slammed with activity (reducing their life) for 0 gain? In your scenario I’m better to store as much data as I can and then set a limit in my router of 50kbps for egress, thereby saving my internet speed just for me and my disks from high activity.
6 Likes
Hmm, good point. I was speaking from my behalf but I can see how others may have different scenarios.
Bandwidth doesn’t cost anything for the SNO (and if an SNO is on a limited bandwidth plan, that’s not very smart), and won’t cost anything for Storj. If the concept of bandwidth were waived, it would do more good than bad. The customers would be happy, as they are paying less.
zz, 6 posts above you, I wrote that my datacenter charges me $10 per TB egress, or $20 for unlimited bandwidth 
Where I live, both data centers and home ISPs charge for bandwidth, they will just sometimes roll it into your 2 year contract. The cheaper the plan the more likely I am to have a set egress cost/restrictions added to my bill. There is no avoiding it where I live.
As other have mentioned before, why not have tiers for storage? In particular, what seems to be easy to change is the expansion factor.
So why not bump up the base amout to $5/TB storage for the 2.7x expansion. And let customers slide it however much they want up to 1.0x factor (or whatever the minimum is required to recreate a missing piece, might be 2.0x to have a copy of each piece? I’m not sure how the reed soloman algorithm works). With the cost/storage scaling downwards linearly. This won’t solve the overall issue, but it would help with the argument of competing with Backblaze price while also providing more geo-redundancy.
If the customer then has less important data, they get a much better price compared to Backblaze, and if they want that additional security they pay way less compared to competitors.
Regarding egress payment to node operators need to come down, fair enough, but why does it have to be below the competition? The fact that you can get insane speeds and redundancy for cheap is the pitch. But why not have it at $10/TB for customers like other companies? We are not competing on egress amounts. And this can be further reduced if we remove the reliance on S3 (the operators will be paid the same amount, and Storj earns the same amount $/TB egress but the difference in further reduction would come from less S3 traffic).
If it is a major concern, I believe others said, charge more for S3 and much less for the other connections for egress. And I don’t know how much effort will go into this part, pay operators based on if the egress is S3 or not for the customer. Essentially paying a flat % of the customers amount to operators.
If not possible, then just pay flat to operators always, and incentivise customers to use the other alternatives (be it prices or better tools/integrations)
6 Likes
I realy don’t see a good explanation from Storj team why prices for customers don’t increase? You realise that you just can’t drop the SNO payouts to much, so increase the DCS prices. Why is this so hard? You don’t trust your own product?
3 Likes
It’s similar to how RAID5/6 work, except with lots more pieces. And expansion factor of 1x would simply mean that with any piece lost the data is gone. However, you could lower the expansion factor without raising the risk of loss. This can be done by raising the total number of pieces. So instead of needing 29 out of 80 pieces to reconstruct a file, you would pick 80 out of 160 pieces. The latter would likely actually have lower risk of loss (I didn’t do the calculations). Either way, it’s not just a matter of picking the expansion factor. Raising the number of pieces per segment would help as well. Although raising that number has a negative impact on small file performance. Interestingly though, it would have a positive impact on large file performance due to higher parallelism. (assuming single file/segment transfers) This is why I have been suggesting Storj to tune these settings. Have a look at chapter 7 of the whitepaper if you want more info: https://www.storj.io/storjv3.pdf
That said, I would argue that Storj shouldn’t trade of reliability. If data gets lost, it’s not just the customer who feels the pain. It’d hurt the reputation of the network immensely. Try managing the PR when you have to explain that the data loss happened because they used a lower tier… It’d be bad. Instead, Storj should determine safe options and let the small file performance take a hit instead. While ensuring reliability of data.
The work for the node operator doesn’t change. S3 gateway just adds a layer inbetween that has to be paid for. So it’s reasonable to keep payouts to nodes the same, but charge customers for that extra layer.
7 Likes
here is a thought… if the gateway is so expensive to run… why not charge people for the service… sure one might keep some kind of free or lower performance tier.
but why not charge for large scale enterprise usages of the gateway service…
its not really StorjLabs or SNOs problem how much bandwidth some other company want to use and removing the expansion factor for them…
well that isn’t free and apparently expensive… so maybe they should be charged for that.
1 Like
First thing, thank you for posting the proposal. I was starting to worry about it, as it was already announced 3 months ago.
The proposed ranges for egress traffic are pretty wide. I wonder whether this is an attempt to differentiate payments between nodes (maybe by reliability, maybe just by maturity), or just the uncertainty of estimates. As I stated in @BrightSilence’s thread, I’d be fine with the upper bounds, or slightly less than that. Lower bounds, not really.
I’d like to compare this proposal to Hetzner’s (obviously…) offer. Their best offer is around 1.2 EUR/TBm of storage including mostly unlimited egress, with hardware and networking probably more reliable (i.e. less repairs needed) than most of SNOs. If there’s not enough SNOs at that level, Storj can always get some hardware from them.
Was going to ask the same. SNOs are (supposedly) mostly private individuals. And there is that thing called inflation that hits them more than companies. Cause companies can have cost write-offs and when operating costs are becoming a problem to the point where there is no room to maneuver, they can always increase the price of their product, shifting the increase on the customers.
I am not saying that even a small cut shouldn’t happen, but really, how it was calculated before to be any profitable long-term? Was the plan all along to lure many nodes first, then start the cuts and hope folks stay “at minimum wage”? You clearly must’ve calculated that. Or was it a blind “we hope that somewhere on the planet the running costs will be low enough to host sufficient amount of nodes for the network to work”.
Transparency? Share those calculations and plans.
--------------
About the thing that old nodes tend to keep cold data and do not get much egress. How about the network balances out the share of the old data, moving some of it from old nodes to new ones? Old nodes get room for fresher data with higher probability of egress and new just get the same amount of ingress as now, but some of it comes as a cold data from existing nodes. And yes, that egress for old nodes would be free, as I assume fresher data will be more profitable anyway. If you (totally not looking at certain individuals) have issues with that, maybe we could have an opt-in/out config “i-dont-want-to-be-paid-more: true”. How would that work? I don’t know. Free up some as new data comes? Free upfront and then data just flows? Whatever the algo, that’s just an idea to address the issue of old nodes becoming Snorlaxes.
1 Like
and where did you get those numbers.?
Cheapest I could find is ~1.27€/TBm excl. VAT (64TB server), but in regards to hardware it’s nothing more than consumer grade. Network yes, it’s a bit more than just a single ISP connection. And there are already a couple of Hetzner Servers active, as nodes and linkshares.
Right now they have a 10×10TB auction server at 107 EUR/month before VAT, and a 15×10TB server at 143 EUR/month. I’m observing their auctions fairly regularly, this is the usual price for these devices. Sorry, their UI doesn’t allow linking to a specific server on the auction, but there’s a list here: https://www.hetzner.com/sb?country=ot&drives_count_from=10&drives_size_from=10000&drives_size_to=16000&search=ECC.
They’ve got ECC, so at least this factor makes them better than regular consumer hardware.
2 Likes