I have some discrepancies between the earnings calculator and the web dashboard regarding the age of some nodes. In all cases your script reports first contact joined in the last one or two days of the month whereas the dashboard puts in in the next month.
Is the GUI not taking the data from the same database/table like you do?
Iāve seen something like this on my own node as well. I can only guess as to what the cause is. Iām pulling the join date from reputation.db. Itās reported back by the satellites. I then simply calculate the number of months between now(UTC) and that date. I can only speculate on what happens on the dashboard, but I think they may be calculating the number of paystubs in heldamount.db instead. This will probably be confirmed if we donāt see that month number on the web dashboard go up on June 1st, but only after payout is done. I think that was the case this month as well, but Iām not entirely sure.
It does result in the earnings calculator saying us-central-1 is in month 16 right now, while I havenāt received my held amount return on that sat. Iām not sure how to work around that without displaying the wrong month number for almost half a month. And honestly I think itās mostly an edge case when a node has made contact with a satellite but not received any data until the next month.
With the upcoming updates on the SNOboard, this will also become visible as they plan on showing the join date and month number right next to each other.
Hrmm, actually, the web dashboard does something even weirder.
There are so many things weird/wrong here, Iām not entirely sure where to begin.
const now = new Date();
Not entirely sure, but I think this would use local date, not UTC.
const secondsInMonthApproximately = 2628000;
An average number of seconds in a month is usedā¦ why? This will lead to rounding errors.
return Math.ceil(differenceInSeconds / secondsInMonthApproximately);
Number of months is calculated as the number of āapproximateā months between join and now rounded up. Which is simply wrong as it shouldnāt be counting a time difference, but the number of calendar months.
If you join on January 31st and it is now February 1st, itās still your second month, but that one day difference rounded up would be counted as 1 month with this method.
The strange part is that itās so simple to do it right:
(utcnowYear - joinYear) * 12 + (utcnowMonth - joinMonth) + 1
No need to bother with number of seconds in a month.
So Iām left to conclude the calculation on the web dashboard is currently not correct. That doesnāt really solve the problem entirely though. As I joined us-central-1 on 2019-02-28, Iām technically in month 16 now and so I should have received half my held amount back with the latest payout, but I didnāt. I donāt really feel like digging through more code right now to find out how the satellite determines this month number, especially since I donāt have access to satellite dbās to check my data there anyway. But needless to say, this may need some future fine tuning.
@littleskunk: Iām not entirely sure who to point this discrepancy in calculation out to. Perhaps you can confirm my findings and/or send them on to the correct person?
weāre passing this along to the internal team.
dealing with Date() is one thing I really dont miss from my coding days lol.
thanks for raising this point!
Small bug fix today. In a previous release I switched the calculation of hours per month to a static 730 hours which Storj uses to determine payout per month. However, this variable was also used to calculate how far along we are in the month. Since weāve passed the 730 hours this month (May has 31 days and is longer than the average month), the estimate by end of month was displayed as lower than actual earnings so far. That obviously canāt be correct. This version switches back to using the actual calculation for calculation of month progress, so we can never be more than 100% done with the month.
These are great findings. Please open a github issue or maybe even a PR with your code change.
This is the correct behavior. Lets say you join on the 31th and you get some download traffic. At the end you might get a 1 cent payout. ā We count that month even if you receive 0 payout.
For your held amount you might have received it in month 12 already. In the first version of our new payout calculation was a bug that we fixed quickly but it āaffectedā about 50 storage nodes that joined very early. Have you check the last few month in the storage node dashboard API? Watch for the disposed value. That should tell you when it was payed and how much.
I think weāre in agreement. The current calculation on the web dashboard would still say youāre in month 1 on February 1st, but it should be month 2 at that point. My initial explanation perhaps wasnāt clear.
I didnāt even have to look. Iāve stared at that heldamount.db for waay more time than I care to admit to get all the features built in to the earnings calculator. Despite that I looked again just to be sure, but there I donāt have any disposed values other than 0. Iām inclined to think the joined_at date for this satellite is not correct and is more likely to be the issue. The reason is that all other satellites have an exact timestamp, while this one has 2019-02-28 00:00:00+00:00. I also remember I didnāt get a payout in that first month, which could be the reason it is not counted by the satellite. At this point I donāt think itās important enough to look into it further before the next payout happens. Iām pretty sure it will be included in that one anyway.
Iāll create an issue as I donāt currently have a development environment set up to quickly test the changes. That and I havenāt worked with typescript before. Itās always easier to read a new language than to write in it. 
Edit: Done https://github.com/storj/storj/issues/3902
We wiped us central at some point. To make sure the node age for the held amount calculation is correct we used the payout list from a previous month. In that payout list we had only the day without a timestamp. Your node age should be correct.
Letās wait for the current payout and if you still donāt see it I can dive into the data we have on the satellite side.
What does that mean?
Where does it show the surge payouts? Perhaps I didnāt get any?
Instead of underlining text I get ANSI codes, like this:
e[4m
Payout and held amount by satellite:e[0m
Thanks, very useful script and I agree, it should be included in the dashboard.
They are reasons for payout to be withheld. Sanctioned countries refers to countries which are sanctioned by the US government and there are trade limitations that prevent Storj from being allowed to pay anyone in that country.
The 1099 form is a tax form that is required for Storj to pay a SNO more thanā¦ I believe something around $600. But I donāt remember the exact amount. This is only applicable to US residents.
Disclaimer: I am not a lawyer and donāt live in the US so Iām not familiar with the applicable details. This is just my understanding.
Iām aware that this doesnāt work correctly in powershell or cmd. Try the new windows terminal. It should work with that. Andā¦ itās a pretty nice improvement on powershell or cmd anyway and lets you use both as well as WSL from within a single app.
This update has taken a little longer than I would like. The reason is that my nodes are on the docker release and the delayed update roll out made it take longer until my nodes had the required data for this update. Most important added feature is the suspension score. Over the past few weeks many nodes have been suspended and at least for now, the score this is based on isnāt displayed anywhere. The calculator will now show this score so you know exactly how close to suspension you are or how close you are to recovering from suspension.
This score has been added as part of a larger rework of the scoring display. The new scores work different from before. The new scores display how close you are to either disqualification or suspension. 0.0 is a perfect score. 100% means you are suspended or disqualified. This makes it easier to know how close to trouble you are without having to know what the threshold is for disqualification or suspension. The downside is that it deviates from what the web dashboard shows.
Another new feature is that the calculator now has a WARNING status when any of the percentages for disqualification or suspension go over 5%. This indicates your node may need some attention as itās failing audits.
Let me know what you think about this new system!
Note: This update requires storagenode version v1.6.3+
I would like to recommend a space of 1 line after every satās rows to make it more readable.
us-central-1 18
Status: Suspended @
europe-west-1 15
Status: OK
europe-north-1I have a bit of a selfish reason why I donāt want to do that. The calculator wonāt fit in my tmux pane anymore if I do that. But this screenshot is messy because it has examples for every situation. Itās much easier to read if your node is performing normally. I also updated the screenshot in the top post with a normal situation screenshot. I think that looks fine, but let me know what you think.
Note Iāve already posted this error to GitHubā¦
Getting error when running the latest version of the calculator:
Traceback (most recent call last):
File "/home/bill/source/storj_earnings/earnings.py", line 283, in <module>
for data in con.execute(query):
sqlite3.OperationalError: no such column: audit_unknown_reputation_scoreThis error is occurs when the V9.2.2 V9.3.0 earnings.py is run against the pre- V1.6.4 databases.
You need to wait until your node is updated to the latest version.
EDIT: didnāt actually check the version, just used the title. Sorry for any confusion.
OK, Just need to wait a while thenā¦ 
Whoops, my bad for not updating the topic title. Should be correct now.
Responded to this on Github already. But just to be sure.
Looks great! As always, very appreciative of your contributions!
Hi, good afternoon.
Iām getting an error in running the program. Iāll leave you two captures below. The first one shows the files that you have to copy to another folder, and I think that there is some file missing that you have to copy, because when I make the copy manually of ALL the files that are inside the node, the problem disappears.
Which is the file that I have to incorporate to give us the error?
The paystubs table is in the heldamount.db file. So youāre not missing a file. But if you copy the files, itās best to do that with the node stopped. Otherwise you could be missing data thatās in the .db-wal files. So stop the node first, copy, then start the node again and run the script on the copy. That will likely fix it, but please report back if it doesnāt.
Edit: Btw, if youāre running a windows GUI node itās safe to run the script on the live dbās without copying. But definitely donāt do that if youāre using docker as it could corrupt the databases on docker nodes on windows or macos.
