This changed. Check the announcements
2 Likes
You are correct. Must have missed that post.
A link would be helpful. I can’t find it.
1 Like
I wanted to wait until all satellites had reported back paystub data before posting this.
Payout and held amount by satellite:
SATELLITE MONTH JOINED HELD TOTAL EARNED HELD% HELD PAID
us-central-1 15 2019-02-28 2.67 USD 0.9324 USD 0% 0.0000 USD 0.9324 USD
SURGE (160%) 1.4918 USD 0% 0.0000 USD 1.4918 USD
Received: 13.49504785($1.48)
europe-west-1 12 2019-05-31 41.63 USD 2.8724 USD 0% 0.0000 USD 2.8724 USD
SURGE (160%) 4.5958 USD 0% 0.0000 USD 4.5958 USD
Received: 42.10082444($4.63)
europe-north-1 1 2020-04-18 0.00 USD 0.0091 USD 75% 0.0068 USD 0.0023 USD
SURGE (160%) 0.0145 USD 75% 0.0109 USD 0.0036 USD
Received: 0.0342706($0.00)
asia-east-1 11 2019-06-10 0.69 USD 0.9630 USD 0% 0.0000 USD 0.9630 USD
SURGE (160%) 1.5408 USD 0% 0.0000 USD 1.5408 USD
Received: 14.58129379($1.46)
saltlake 3 2020-02-11 14.71 USD 8.9106 USD 75% 6.6830 USD 2.2277 USD
SURGE (160%) 14.2570 USD 75% 10.6927 USD 3.5642 USD
Received: 15.2805705($1.68)
stefan-benten 14 2019-03-03 187.51 USD 40.4020 USD 0% 0.0000 USD 40.4020 USD
SURGE (160%) 64.6433 USD 0% 0.0000 USD 64.6433 USD
Received: 588.28261914($64.70)
_____________________________________________________________________________________________________+
TOTAL 247.20 USD 54.0895 USD 6.6898 USD 47.3997 USD
SURGE (160%) 86.5432 USD 10.7036 USD 75.8396 USD
All seem great with small fluctuations that can easily be explained by token value changes. I used the etherscan feature to get value at time of transfer, which isn’t 100% correct.
Except saltlake seems a little low
I’m definitely not complaining, just trying to find out why.
My first instinct was maybe some discrepancy in the stored data calculation as I had seen small differences there before.
So I compared the paystub data to storage usage data from the node.
That’s definitely not it.
Next check, do I get $1.50 per TB.
Looks like Storj uses the average of 730 hours per month despite April having only 720 hours. Fair enough, that’ll even out over time and that number is close enough to 1.50 for me. So that’s not it either.
Let’s move on to bandwidth. Normal egress traffic, lets compare GB downloaded from my node between paystub and bandwidth usage in the nodes db.
Hey… that doesn’t look right? The paystub seems to have a significantly smaller amount.
Did I do something wrong here?
A 0.245TB difference is 4.90 USD, with 160% surge 7.84 USD. 25% paid out would be 1.96 USD.
Add that to the 1.68 USD that was paid out and you get 3.64 USD. Close enough to the initial calculation.
QED
Queries for reference:
select usage_get/1000/1000/1000 from paystubs
where hex(satellite_id) = '7B2DE9D72C2E935F1918C058CAAF8ED00F0581639008707317FF1BD000000000'
and period = '2020-04'
select sum(amount)/1000/1000/1000 from bandwidth_usage_rollups
where hex(satellite_id) = '7B2DE9D72C2E935F1918C058CAAF8ED00F0581639008707317FF1BD000000000'
and action = 2
and interval_start >= '2020-04-01 00:00:00+00:00'
and interval_start < '2020-05-01 00:00:00+00:00'
Edit: My node ID in case someone wants to look into it:
12aYrWFmJqrmhN3zgkvANBTsj2DdLwf2aZC8T5t7CrNazHahKXW
I don’t mean to say you have to look into it for my node specifically, but if it’s indicative of larger problems this example might help.
2 Likes
I think we have incorrect data on the satellite side.
I opened dashboard…
Looks like I should send payout for Stoj Lab 
That’s due to a different issue. Held back amount is shown including surge, while everything else is shown without. Held back without surge would be 2.19 for you, which is roughly 75% of the total.
Yes, I agree, its different problems. But I think the root cause is the same (It was not tested) 
Unless you mean software still in development, it’s really not.
The issue you’re pointing out is simply a matter of displaying the wrong numbers on the dashboard. It doesn’t impact actual payout calculation. And it can be fixed afterwards by simply fixing the dashboard.
The issue I pointed out is a difference in bandwidth usage numbers between the nodes own accounting and what the satellite reported back on the paystub. This isn’t just a reporting issue, the administration of bandwidth usage is wrong on one end or the other even before looking at the dollar amounts. It’s an issue in the core calculation, rather than in the display of information. This can’t be fixed afterwards, payouts have gone out already based on this.
I don’t care about the few bucks, that is compensated several times over by the generous surge payouts we’ve been getting. I care about pointing out something that may be going wrong so the underlying issue can be fixed.
Edit: Did you edit your comment to add that (it was not tested) bit or am I seeing things? Either way, not tested is almost certainly not true. But it slipped through testing.
1 Like
Yes, I completely agree with you, sorry but I reported another issue in my post (it does not reply to your post with these findings in accounting). I also agree then is not a critical bug, but it should be fixed.
Yes, I added immediatly (5-7s) for prevent misunderstanding about what I mean
You are a bit too late to complain about that one. It is explained here:
Yes a translation is needed 
1 Like
I will redirect that to our level 2 support. At the moment it is hard for me to find the time to check it all myself. Worst case I have to do it myself later this week.
2 Likes
@BrightSilence fantastic bug report, thank you! Having details like that makes it so much easier to dig into problems.
I tracked down your bandwidth and payment data on saltlake, and you are entirely right: there is a clear discrepancy. The reason for it is that that satellite is having some innocuous-looking database trouble which is causing queries on accounting data to take absurdly long. This means that it is way behind (from what I understand, weeks or months) on processing the bandwidth claims submitted by storage nodes. In the case of your node, the payment system was only able to see the bandwidth usage up to April 8. That’s why the total get bandwidth reported was so low.
The good news is that you’ll still be paid for the rest of the bandwidth used, once the system processes it. We hadn’t noticed that saltlake was behind at all in order processing, so thank you for letting us know about the problem and about the need for some more monitoring checks in that area.
I’ll also be making a ticket on our internal Jira to rearrange the way we store pending bandwidth claim data, so that this should stop being an issue.
11 Likes
Thanks for the quick and detailed response! It’s good to see it will be fixed automatically, though at the cost of some held back amount, since it will now be paid out in the 50% held back period instead of 75%. So I guess that’s only good for SNOs then. A delay is better than a miscalculation, so I’d say this is good news. I guess this satellite has been pretty busy over the past few months. Catching this stuff on saltlake I think means it’s doing its job. 
10 Likes
sort of makes me feel alittle better that it’s not only my server that cannot keep up with the network lol
3 Likes
Time for a new update, here are my findings.
During payouts the STORJ value fluctuated quite a bit. As a result etherscans estimates were all over the place. I used 0.14 instead, since that was what it was actually hovering around during these payouts. Unfortunately I can’t be exact without the exchange rate used for the actual transaction. I kindly ask to include this in the databases with a future update.
Let’s start with held amount payouts. It looks like this confirms what I noticed earlier and the returns for held amount actually happen in month 16, not in 15. us-central-1 paid out the held amount in month 16 and stefan-benten in month 15 did not.
Next up, Saltlake payout is higher than expected. This includes compensation from missed payout last month.
So, last month I missed 1.96 in payouts. But to make it complicated a different held back percentage applies this month and there was a surge payout last month. This month I got 2.63 more than expected.
My guess is that surge is not applied to this correction (Which… I don’t care, just want to know what happened) and 50% held back is applied (instead of 75% last month). Based on last months numbers that should be 4.90*50%=2.45. Close enough.
I was lucky enough that this node still had all separate payouts this month. But you can see how doing these checks would be much more complicated without that. Please include exchange rate and STORJ amounts in the dashboard (and API/node databases), so we have a complete picture.
I’ve started 2 additional nodes during May, but they don’t have significant enough income to warrant a deep dive. Additionally, most of their payouts were actually merged into a single payout, so can’t really do these exact checks.
tl;dr:
- Only one significant “issue”. Payout of 50% held back amount is happening with the payouts of month 16 instead of 15
- Please include exchange rate and STORJ amounts on the dashboard, API and node databases
8 Likes
Thank you for doing this work to simply the SNO’s overview of the payouts…
I totally agree that this process in being able to manually verify the payment data from logged data from node or otherwise… is a critical point both for good measure / transparency / trust
all in all it all boils down to (TBm + Egress)*surge = payout
i wouldn’t want to dig through it all, but would be very nice if my external logs easily could also verify the node numbers and payouts… which is why i started logging it … or i did and then i broke my OS and didn’t get it finished yet… ofc right now i’m getting like 4$ payouts… so difficult to really feel cheated out of anything lol
the 15 vs 16 is just a matter of how you count… if its after 15 months or if you count 0 then you will end up at 16 ;D
1 Like
Pay is LOW. That is a fact.
We can’t even afford the gasoline for the lambo.
1 Like