Intro to the Web Dashboard

Node Operators getting started
1

I made a video to introduce the Web Dashboard. As a new SNO, it took me a while to understand what I was looking at, so I thought maybe I could ease the path for new SNOs. Feedback welcome; if there are errors, I’d like to correct them before too many folks see it!

7 Likes
2

That’s awesome! Nice work.

I don’t think there is any reason to rerecord as there aren’t any really wrong things in there. But it seems you have it unlisted while waiting for feedback, so I’ll just mention what I can say and leave it up to you.

  • It seems the video starts halfway into a sentence?
  • Status does show offline if the node can’t be reached by the satellites, so it does also confirm your port forwarding is working.
  • Last contact only refreshes when you refresh the page, but what it actually means is the last time a satellite was able to contact your node. So if after a refresh it doesn’t show a low number, your node has lost contact. It takes a while for it to actually show offline, so this could be an early indicator.
  • The minimum version shows what version you can still run and be considered online. Lower than that will no longer work at all. However, if you are 2 minor versions behind, you’ll stop receiving data. So you need to make sure you are always up to date and you can’t really wait until you reach the version displayed there.
  • Satellites: I think you mostly got this correct, but your wording may have suggested the satellites do the dividing up of segments into pieces. This is actually done by the uplink ran by customer. The satellite is more like the traffic tower telling the customer to where they can upload it.
  • Split between repair upload and normal upload. Your node is in a bit of a strange situation there is my bet. It’s likely vetted on one of the larger satellites that has a lot of repair, but not so much incoming traffic, while your node is still being vetted on other satellites. This skews the numbers quite a lot. There is a lot of repair traffic at the moment, but the split on more mature nodes still has more than a third being normal upload. And in recent months that was often 50/50. I think in the future we can expect this to be more normal upload than repair usually. Either way, data is data, it doesn’t matter where it came from.
  • Trash: While at the moment all deletes go to trash, that isn’t normally the case. This was temporarily done to work as a fallback for some other feature still in development. But after that’s done it’s only really used if your node missed a delete because it was offline or something like that. The amount you’re seeing is ok I think. It is not included in used and is also not paid.
  • Online score: I’ll not dive into the minute detail here as it is… a lot. But one thing you said is only true for new nodes. Your online score recovered pretty fast because you didn’t have 30 days of history yet. Since the score is based on the past 30 days, every new day changed the percentage for you. But for nodes older than 30 days, a lower online score only recovers after it has aged out of that 30 day time frame. so if you drop to say 90%, even if you stay online perfectly after that, it will stay at 90% until the down time is more than 30 days old.
  • Other scores: While disqualification only happens below 60%, the suspension scores and audit scores really show a problem with the functioning of your node if they drop at all. Especially the audit score, which if it drops means your node is unable to prove to satellites that it’s holding the data it should be holding. This should always be immediately investigated.

None of these things are major and like I said, I don’t think it requires rerecording. But you may think differently. Anyway, great vid!

8 Likes
3

Wow, thank you for the detailed and insightful feedback! I think I will re-record, I already did it 2-3 times, and once it’s all set up it’s not to hard to redo. Much appreciated.

5 Likes
4

I figured you might be one of us perfectionists, it’s a curse really. :sweat_smile:

Let me know if I can clarify more, I’m happy to help.

3 Likes
5

Well, I haven’t re-recorded yet. But I wanted to reply on this point: I don’t think the issue is what you suspect. When I recorded the video I had not yet been fully vetted on any of the nodes. (I am now, on saltlake, but only in the last few days.)

Separate point, I’m (mildly) concerned about how long vetting is taking…while that one is complete, the percentages for the others are 35, 33, 30, 6, and 7 (with no movement at all on the 6 and 7 satellites, ap1 and europe-north, in the last week or so). I’m almost six weeks in, so based on what I’ve read I’d expect vetting to be almost complete all around.

I’m curious - am I now drawing from the 95% pool from saltlake, but still confined to the 5% pool for the others? Or do I have to wait to be vetted by all six before shifting to the other pool?

6

This.

I wouldn’t worry, vetting periods vary a little depending on traffic and how many nodes there are in vetting. It also speeds up over time. So most of the satellites are actually quite a bit passed the halfway point with these numbers.

If I calculated correctly, the best approximation is actually:
progress = ln(2x+1)/ln(201)
(don’t ask how I got there…)

Plugging in your numbers:
35 => 80%
33 => 79%
30 => 78%
6 => 48%
7 => 51%

I might need to correct this a little for audits that are specifically initiated to target unvetted nodes, depending on how those work. So I may need to tune this is little bit, but I might actually include this formula in the earnings calculator to give a better idea of progress than the raw number of audits.

Edit: Ok, looked it up and actually, audit now picks nodes at random and then audits a random piece. This was done to ensure all nodes get audits. But, since every piece has on average about 66 pieces and only 1 piece is determined this way, the other on average 65 pieces audited still get selected relatively randomly and based on the amount of data stored you get hit more often. So, my calculation was correct for roughly 65 out of 66 pieces or 98.5%. The other 1.5% is linear. It hardly seems worth it to correct for that given the small difference, but I did the work, so I might as well post it.
New formula:
progress = 98.5% * ln(2x+1)/ln(201) + 1.5%*(x/100)
Plugging in your numbers:
35 => 80%
33 => 79%
30 => 77%
6 => 48%
7 => 50%
In the rounded percentages only 2 changed… so yeah, basically wasted effort to add in the linear element. Ok, that’s enough math for today.

Edit2: The above needed adjustment, to account for the fact that the percentages of what’s linear vs logarithmic would work for a node with an average amount of data, but of course nodes in vetting have a below average amount. I’ll spare the actual calculation and just say that I used estimates from the earnings estimator combined with the average data stored per node from the new public stats (thanks Storj Labs!) to calculate that a 90%/10% split is more representative. Though we are now stacking estimates on estimates…

New formula:
progress = 90% * ln(2x+1)/ln(201) + 10%*(x/100)

Plugging in your numbers:
35 => 75%
33 => 75%
30 => 73%
6 => 44%
7 => 47%

This actually feels more accurate to what I’ve seen in the past. And the conclusion remains the same for the most part.

If this is after 6 weeks, I would expect the highest scoring ones to take about 2 weeks more. And the lowest scoring ones maybe 6-8 weeks more. Do report back if that was accurate. As I think showing progress numbers that are linear would be a great improvement in the earnings calculator.

Edit3: Since this is based on some estimates and given that the following gives pretty much the same line (with at most a 2% deviation), I will probably simplify it to the following if I’ll implement it in the earnings calculator.

progress = ln(x+1)/ln(101)

Which generalizes to:

progress = ln(x+1)/ln(n+1)
With n being the number of required audits.

Which ironically, is where I started this whole story with just a guess on what the curve might be like before I started to complicate things with actual calculations.

2 Likes
7

Very nice Video, well explained and a very pleasant voice :wink:

2 Likes
8

Hey @o1eal,
thanks for the nice video, this sure will be helpful to someone!

You proposed to check the cause of any score dropping below 100% which I think creates more panic than neccessary for new SNOs - I’d rather consider being above 97% not concerning at all.

Also when you re-record, you might want to blur out your node ID + wallet address instead of not showing the entire screen. It’s just a bit confusing hearing someone talk about a menu that is out of view :wink:
(Or you could replace these values with some other characters using your browser’s dev tools)

1 Like
9

I’ve updated the earnings calculator with non-linear vetting progress. See if that gives you a better indication? And perhaps even more important, alleviates your concerns regarding vetting progress? I’d welcome any feedback in that topic!

1 Like
10

Thanks! I posted my results on the script’s page, in case you’d like to compare the old and new for a node presently in the midst of vetting.

As for my “concerns,” I should emphasize, they really are very mild – more curiosity than worry about how long it takes me to start earning ~$9/month :slight_smile:

But, I suspect that the strange ratio I have – far more repair data than “regular” data – might mean that my own vetting will take longer. Are audits based on both repair and regular data, or just regular data?

11

Yes, they are based on all data what you have. There is also no difference between customers’ and test data.

1 Like