Why do SNOs even concern themselves with these things? I mean I understand being curious and all, and I’m just using your post here as an example since it reminded me of many similar ones… but what business is it of ours? We’re not business partners, share holders or in any other kind of position to be making demands of Storj other than in terms of what directly effects us as SNOs such as negotiating pay rates, etc.
I’m just speaking generally here, not necessarily singling out your post.
I constantly see people demanding Storj tell us this or that, arguing about what’s right or wrong for the company. Now I have certainly voiced my opinions about things, but never have I thought Storj owes us anything other than pay for what we provide and maybe clarity on things when they directly effect us in certain cases.
I honestly don’t understand why so many of us concern ourselves with the inner workings of the company, especially when it’s pretty clear so many of us don’t even understand the basics of running a company nor have the expertise in terms of large scale data storage, finance, marketing etc. Now I’m sure there are exceptions to that, but for the most part it’s none of our business. Just because Storj tries to be transparent about as much as they do (which is great) doesn’t mean they owe it to us or should be wasting their time explaining things to us that most people won’t actually understand anyway. To be quite honest, I couldn’t even stand having shareholders with legitimate voting rights, let alone my suppliers (which is what SNOs are) demanding to know how I run my company and thinking they should have some sort of say in now it operates. The whole thing is truly laughable.
It’s about whether STORJ is viable and living up to their promises, so it affects our ideas about the viability of the whole project.
The higher the N/K-ratio, the lower the payout rate for SNO’s and the less profitable are our efforts. Just because the $4/TB paid at the front door, can be maximal $4*K/N being paid to us (disregarding the fact not always really N pieces are actualle stored, because if so they would pay more than they got at this point).
It’s fun! A real world case for many statics-interested people here around.
It’s helping the community, to find blind spots and so on.
I’m not demanding, I’m wishing for… Kind of different/nicer thing.
I don’t say they owe us it. However, since we’re participating in the project, it would be nice if they gave some insights for the reasons stated above.
I don’t think so.
We’re kind of employees or actually subcontractors of STORJ. If we suspect not to be paid out in the end, we could chose to use our wares and efforts for other projects. So the project should make sure there is some basic trust concerning their financial situation, viability of the project as a whole and so on. So this kind of information can help in these cases.
Again, our ideas don’t matter to the company except for where they do.
Not really our concern though. This is clearly where they believe they need to be to keep data safe. You know, because of Russia… and all the whales. Make sense? But many of us on here can’t seem to correlate the two. People are so concerned with the risk to Storj over the Russia issue, the whale issue, but them complain about about the N/K ratio.
Based on what Storj is openly willing to share, sure.
Fair to a point. I agree some could be useful to a degree, but many just clutter up threads bickering about things over and over again.
That’s why I said I wasn’t trying to single out your post out. Wishing / hoping is not the same as those who feel they should be entitled somehow to this information.
No we definitely are not. Although it’s a kind of unique situation due to the decentralized nature of it, we are a supplier. But even if we were, they still wouldn’t owe us this information. If you think I’m wrong, try this… when you get to work tomorrow, go to your boss and say “hey boss man, need to see your books! I need to verify the long term viability of your business to make sure I’m getting a paycheck next week or I’m not coming in. Furthermore, you need to raise your prices cuz my paychecks feelin a little skinny! Oh, and that guy over there… yeah, you gotta can his ass. He’s costing you way to much money, you don’t need him, and he’s definitely the reason you can’t pay me more.” Don’t forget to record it all and report back though! lol
Sure, therefore we’re more than once asked for ideas. Also there have been a lot of feature requests, which were implemented to improve the service or the maintenance from the SNO-side.
So, if you effectively want to clear the air on the risk of whales and Russia, then it might be an idea to show them while whales and VPN’s aren’t that much of an issue (because too few nodes to bother and most of the time too small time frame of downtime to be really a concern for durability). Cut-off of Russia might be a concern to some, but I don’t think it ever will happen; and if so, there will probably some time to prepare.
And yes, payments, download/upload speeds and risk of data loss all have to do with that N/K-ratio.
I think you can think of us as suppliers, semantic difference with subcontractors in my opinion when we’re talking about services like storage.
I don’t know where you live, but in my country every business has to file a year record of wich you can roughly estimate the financial well-being of the company. For a small payment, I can acces it from every company in the Chamber of Commerce. Besides, every company of a certain size must have a employees’ association, to which they account the state of business or changes they want to make from time to time. So yeah, in my country (NL) it works a kinda this way.
Plus: Storj is a startup which means the business is small, dynamic and they have limited resources only. They have asked the community for advice and ideas in the past (like where to find customers for Storj DCS). It is a smart approach to ask the community for advice if you don’t have resources for everything. It can help to avoid similar mistakes as they have made in the past.
Well at least I know never to do business in your countries. Try that shit here in the US and you’ll be standing in the unemployment line tomorrow. Now publicly traded companies are a different story, but unless your a shareholder you have no business opening your mouth. But either way, you still can’t just go poking around their business making demands and telling them what to do. If Storj asks for something, or someone has something of value to point out then sure, why not, no problem. I’m just referring to all the useless bickering I see in this forum all the time, mostly by people who just seem to be bent because there not getting rich hosting Storj nodes yet. I have to imagine the Storj employees that read this stuff just want to go bang their heads against the wall. Maybe it’s a generational thing, I don’t know… but hey, whatever floats your boat man.
The thing with employees participation through mandatory works council or employees sitting in the companies supervisory boards is a kind of a love-hate relationship. I think Elon Musk is just learning that right now with the German Tesla factory.
But to be fair, what might be useless from ones own perspective might be useful with another hat on.
Doesn’t matter, we’re talking about orders of magnitude differences here. You can average it out over the range all you want, but the lowest reliability is going to mostly determine the average.
This is incorrect. My calculation is not for exactly the amount of Russian pieces that lead to data loss, but that amount or more. Meaning that it doesn’t matter if the other pieces are also Russian or not. In that case the second part would be equal to *1 and might as well be left out, which I did.
Correct, but when 29 pieces are left there is still no data loss. Hence I calculate with when 28 or fewer pieces are left.
It isn’t. See my response to your second point.
Well you’re right about that, but I’m still not sure where the flaw in my thinking is. Curious.
Well that will surely be the case for newer data. But on sufficiently old data, I think the equal distribution is a fairly good estimation. And most data is old. The problem is that they’re still several orders of magnitude away from eleven 9’s. It’s going to be near impossible to compensate for that with favorable distribution. In order for the 2,04E-07 chance for 54 pieces to not be a problem overall, it would have to represent 1/10000 pieces. And that’s assuming the very rest of the range is rock solid (which we know it isn’t and there are many more below the durability threshold).
I don’t think it solves the problem.
Either way, despite our different outcomes, I think your results still make my point that in order to be able to fulfill the promise of eleven 9’s, these settings for their RS implementation are required and even not sufficient in the case of Russia going offline.
It’s industry standard to measure durability on a per year basis. But again, that doesn’t help them much. Even if Storj was 10 years old, that would get them 1 order of magnitude if you spread it out over those 10 years. Which clearly isn’t enough. We’re also not talking about dropping a bomb on western-Europe. We’re talking about a much more realistic scenario of Russia being cut off the internet.
That I agree with 100%.
Though I’m certain they are doing a lot of these calculations in house as well. It would be nice to have some insight in them.
One last note. We’re calculating with the assumption that this is the only risk factor. But if Russia going offline would put them right on the line of eleven 9’s, they’re still in trouble, because the other native risk factors in the network are still in play as well. They would have to also have a solid additional margin for that risk.
It’s for the benefit of SNOs if Storj can lower the redundancy, because there would be more money to play with and it could stop payouts from lowering further. So when those suggestions pop up, it’s good to know why they couldn’t just lower those numbers given the promises they are making to customers. What effects us as SNOs and what impacts customers is tightly related. If we want to make a good argument for one side, it would be helpful if we consider the consequences on the other side.
Furthermore, additional transparency leads to enabling outside resources to contribute to possible solutions. Isn’t that kind of the whole idea of open source? Taking the contributions where you can get them, no matter the source?
I don’t think that’s what we’ve been discussing here. But sure, the payout discussions have veered into that. I’ve mostly remained silent on the broader business scope though as I am indeed not qualified to respond to that. However, when the scope is limited to something like unit economics, it becomes easier to join in and make suggestions.
I don’t disagree, but it can’t hurt to ask, right? If they see value in sharing this kind of info, because it might enable external resources to think along and provide solutions and new information. They can share it. It’s up to them. I don’t think anyone is demanding. Chill dude.
If lawyers are watching… no we’re not!
But yeah, I agree with everything else @JWvdV says there.
That exception is the whole ballgame, my friend.
So… then it’s good to show how they are related. Is it not? We can have a more informed discussion that way and prevent baseless suggestions of just lowering the redundancy that wouldn’t be viable anyway.
Then shouldn’t you welcome the hard numbers shared here? This isn’t bickering right now. It’s analyzing and informing.
That is nothing to be proud of. That kind of secrecy gets people who rely on a company for their livelihood screwed over. It allows companies to pull the hood over everyones head and screw over employees, contractors, suppliers, customers and everyone who depends on them. Transparency is important and actually helpful for the company too.
If you’ve been corrected on this and keep saying it, it seems you aren’t listening.
Storj is already really transparent. And that transparency works both ways. The point is that they are really open to informed community input, even when they haven’t requested that input themselves. If I look through the feature requests I’ve posted, I can see that most of them have been implemented. Often with an informative back and forth between Storj and the community in the process. This includes things they never asked for, like my suggestion to stabilize reputation scores. Tuning audit scoring Refining audit containment: Refining Audit Containment to Prevent Node Disqualification for Being Temporarily Unresponsive Checking storage path availability: Make the node crash with a fatal error if the storage path becomes unavailable Adding suspension scores to the API and dashboard: Report audit suspension scores back to node databases, API, and dashboard And many more that aren’t in the suggestions area, but posted elsewhere. With this kind of back and forth being seemingly very productive for both sides, why would we not discuss these things?
That’s not te case. You should consider the binomial distribution here (assuming equal chance to get a piece, for Russian and non-Russian nodes c.q. random distribution).
If you divide 80 pieces, the chance you end up with >51 pieces on Russian nodes, is the combined chance of getting 52-80; this should be exactly the same chance of ending up with <29 (so 0-28) pieces on non-Russian nodes.
Take the chance from 80 random distributed pieces exactly 52 end up on Russian nodes, which is the same of 28 pieces on non-Russian nodes as stated above:
This literally says: If I have to distribute 80 pieces, on how many ways I choose 52 out of these 80 (is the same as choosing 28 out of these 80, because it’s the inverted one). This is being multiplicated with the chance I divide 52 of the same pieces over Russian nodes and 28 on non-Russian nodes.
So yeah, this is the calculation. If you think it’s otherwise, you’re the one that should prove why. I would underline there is a flaw…
The order of magnitude doesn’t matter if you don’t know to what fraction of the files it’s applicable to. Consider the case you promise 99% availability of 100 pieces, you can make up 1 lost piece with 99 of 100% availability. So it’s about the weighed mean of chances. So, the lowest reliability isn’t determining the average in this case. Because 99% is more near the 100% availability of the 99, than the 0% availability of that one piece. That’s exactly the same for STORJ and the reason I’d love to know the audit scores.
It depends. Also the risk of Russia being cut-off of the world-wide internet has a certain chance. And I’m inclined to think it’s less than a percent and even if it’s happening, there will be a certain time to safe a lot of the data. In risk calculation, you have also to appoint a chance to the expected and even unexpected events.
Besides, you make the assumption of equal distribution here. I really doubt it. Even for older data.
I would say, there should be some way to estimate it from the repair ingress and repair egress on newer nodes. Assuming a normal distribution, if we know that 54 is really the repair threshold.
That’s why I also assumed a 95% retrievability per se. In order to compensate a little bit for the known and unknown factors that could influence this.
In my opinion your flawing here in logic.
If, for example in year 10 Russia would being cut off from the internet, then the logic would be as follow:
STORJ is an increasingly growing company, so the average age of pieces would be <5 years, assume 4 years. Which have all 100% availability. So it’s not 1 order of magnitude difference, but less.
Assume order E-8 of pieces is being lost, then it’s probably skewed towards Russian customers which STORJ is also bound to stop business with. Since those were uploading from Russia and Russian nodes had a benefit in winning those races. Some Russian companies might even have enforced it by geographical borders. How much it is, I don’t have a clue…
We still have RU nodes on our excluded country list. The way this works is that RU nodes will still get selected for an upload but the repair checker and repair worker will count these nodes as unhealthy but retreviable with a bonus rule to not replace them.
So let’s say a segment has just 60 nodes remaining and magically 40 of them are stored on RU nodes. The repair checker would queue this up with just 20 healthy pieces remaining. That is below the minimum threshold but luckily the repair worker is smart enough to still download from all 60 nodes. The target is to upload enough pieces to end up with 80 healty pieces. So in this example the repair worker will select 80-20=60 new nodes. The repair worker will not remove the 40 RU nodes. They would come on top as bonus pieces.
Ending up with only 20 healthy pieces is an extreme example to explain how it works. In reality such a segment would be queued up for repair earlier long before. The overall idea of this feature is that we could take all RU nodes offline and there would be no repair work caused by that event because our system already factors in that risk and does the nessesary repair work up front.
That’s exactly what I meant, a little bit of transparency is worth a lot of speculation and bickering. Thanks @littleskunk!
Does STORJ.INC also have a blog or something, in which they reflect on the matters they encounter? I mean, we’re here all for a variety of reasons. Some of us are just here to earn some money, others to have a project that increases their skills, enthusiasm about the project, curiosity, a mix of previous, or other. Just like topics: what are the statistics, what are the ethic considerations (e.g. if we have a zero knowledge storage, aren’t we responsible if it’s being used by Chinese government for the persecution lists or child porn), considerations at the moment there is need for a change (like the recent lowering of payouts), and so on.
