Yes, it’s wrong. It was true before we started to show the score instead of percentage
these are all gret comments will pass them along internally
1 Like
As an additional thought, if we were to have percentages going from 0% to 100%, I guess it would require some relabeling. For instance:
Only realized recently that @BrightSilence actually already thought of that there:
I’ll link the above thread to my initial post.
1 Like
Well I’m obviously going to have to vote for this one. I’ve actually implemented this in the earnings calculator for audit failures and suspension already. 0% being everything is good and 100% being suspension or disqualification. Additional benefit is that with suspension you can go over 100% and you can see how close you are to recovering from suspension mode as well.
I still have uptime as a 100% for perfect uptime though. But I could flip that around too if needed. I definitely would if the web dashboard is going to follow the same principle.
@Alexey asked me previously to align with the web dashboard, but given the small amount of space I had in the layout of the earnings calculator and the likelyhood of confusion when it says suspension 100%, I decided against that at the time.
So yeah, I’m using (1 - score) * 250. Would be great if the web dashboard did the same thing.
3 Likes
Your version is confusing, well, at least to me.
I think it would be best if the dashboard showed a bargraph indicator that goes from zero to 100, with the current score and the threshold for bad things marked on it. The numeric score could be displayed ads well.
1 Like
The formula may be confusing, but it’s literally the numeric representation of what you’re describing.
1 Like
It probably is, it’s just more confusing than the actual score with the threshold shown together. Probably because it essentially is a “percentage of percentage”. Or maybe I am not used to thinking like that.
1 Like
There’s nothing confusing about being suspended at 100% and being fine below that. No matter how you calculate that score.
2 Likes
Well the score wasn’t a percentage to begin with. So it was always kind of wrong to show the score as a percentage. By making this change the percentage actually shows how close you are to being suspended or disqualified, which is why the percentage sign is correct in that setup.
I haven’t heard it be called more confusing before though. That’s interesting. I think that might be in part because you’re already familiar with the underlying actual score. I would argue that the end user shouldn’t worry about the underlying score at all and simply think of their reputation as the scores as displayed.
So yeah, I agree with @kevink, but I still find it interesting to hear the other side.
2 Likes
Basically, to me, the real score is kind-of like percentage. I know it’s a bit more complicated than that, but IIRC in the short term it does behave like a percentage, I fail x% of audits, my score becomes 100-x (or 1-x/100). Then there is a lower limit after which bad things happen.
Uptime score is actually a percentage though, again, with a lower allowed limit. It is measured in a more complicated way, but it behaves kind-of like that.
Your version, to me is confusing because I am trying to understand what it means and it’s a bit difficult.
The new score is 100-(current_score-0.6)/0.004. Or, the inverted percentage difference between the current score and the lower limit. I guess it’s OK, it just makes me do mental math that I would not need to do otherwise.
Then again, I know what the actual score means, maybe if I didn’t, your version would be better.
I’s probably the inverted part that gets me, it kind-of feels like if the gas gauge in my car did not show the gas level, but instead showed how much gas I have used compared to the tank, given that, say, 20% remaining is the lower limit. I fill up the tank, the needle is at zero, then as I’m driving, it climbs up, until it shows 100%, at which point I know that I’m almost out of gas.
It would probably be less confusing to me if was not inverted - started at 100% and went down to zero at the point of “bad things happening”.
1 Like
This is actually nowhere close. The score drops much faster. And you can’t fail anywhere near 40% of the audits. Which makes the percentage sign very misleading. I’d be surprised if a node could survive long term with 2% missing data.
I get that the inversion is confusing. I opted for that because there is no good naming for the suspension score that doesn’t sound like it isn’t a bad thing. Unknown audit score is used in the code, but that’s a seriously confusing name. Since the name is a negative, the score should reflect the negative. So I’m calling it a suspension score and a disqualification score.
If you want proof for the confusion. There have been many topics from people being scared of something being wrong because their suspension score showed 100%. I listed a few here: Earnings calculator (Update 2022-08-24: v12.0.0 - Now compatible with node version 1.62+ - Detailed earnings info and health status of your node, including vetting progress) - #322 by BrightSilence
But there are many more. A negative name should imply negative effects if the score is high.
That’s where the gas analogy falls apart. The gas gauge represents how much gas you have, but the suspension score doesn’t represent how much suspension you have.
I get where you’re coming from and I would be fine with 100 being perfect and bad things happening at 0. But we would need a different and better name for the suspension score in that case.
2 Likes
The score is a ratio between alpha and alpha+beta, so it;s kind-of liek a percentage, even though it does not represent the percentage of audits.
How about “critical audit score”, “non-critical audit score” and “uptime score”? High score = good, low sore = bad. Critical - the one that results in DQ, non-critical, the one that results in suspension.
As for the analogy, I guess I am used to 100% being “good”, be that uptime, the amount of gas I have, battery charge level etc, though I guess I could look at it as the amount of passes on a tape = high number means the tape is almost worn out, though I would probably still instead look at the “wear” remaining.
1 Like
No need to enter percentages …
Scale of course from 100% to 60% for the suspension
1 Like
I like those better, but they don’t immediately make the consequences and differences clear.
Actual gauges are a good idea. That solves the question of whether high or low is good.
1 Like
Yeah, on the graphical dashboard, a bar indicator going from zero to 100, with the limt ant current score marked on it would be obvious to anyone.
1 Like
Like this
or that
1 Like
Yeah, also, either marking the limit or, I guess, the end of the bar could be the limit.
1 Like
Gauges or other bars to visually represent scores are alright, but I would keep the percentage displayed also, as it is more precise which is particularly important for the audit score, as it becomes very quickly (way too quickly sometimes) an urgent issue that needs action/checking.
Colors on bars/gauges could reflect that too I guess: green could be a very small portion only, suggesting that anything below 90 (for instance) should worry the user.
1 Like
Update: The audit score recently evolved:
Nodes now get disqualified when dropping below an audit score of 96% which makes this number even less precise, but at the same time closer to the % of authorized loss before DQ (Nodes with more than 4% of data loss are DQ). So in a way, it is now more intuitive…
Even though I still think it could be reworked to pan from 0 to 100% visually on the dashboard, if we want to keep its current way of working maybe a simple relabeling for this score (like “Stored data health”) with a tooltip detailing the 96% threshold could be enough?
1 Like
i fully agree with @Pac on this.
the dashboard should be easy to understand for new SNO’s and others…
requiring near zero knowledge…
the 4% allowed now, being equal to the spectrum of 0 to 100%
should be easy to do, make sense and work in API.
1 Like