Here are the two nodes at home for month July, same subnet.
July (node1) = 1.06TB
July (node2) = 1.07TB
2.45TB total including repair ingress. Still think it’s optimistic?
I don’t know why the node on the other IP isn’t seeing the same traffic. It’s possible there is another node in that subnet, though the chances of that happening are very small right now. But these ones do seem to confirm my numbers.
Look at my earned compared to your calculator. $11.37 vs $16.64.
It’s an estimator. You happened to have started your nodes in the slower months, which made vetting take a little longer and you’re perhaps a little behind the estimator as a result of that. It’s never going to be perfect. Do you really think the difference you just posted is that big? To me that looks more like a confirmation I am on the right track. And your own data confirms at least one of the estimations I made.
1 Like
Goes back to the same issue where you are over estimating profits like storj’s original estimator. Your are in the very ideal environment being near geographically where all the test data is uploaded, high data connections, high end hardware, 1 node per subnet, starting the node in ideal months, etc… You are not accounting for the fact that not every will have these advantages. Which is why I’ve said your estimator is very optimistic.
Are you serious right now? The original calculator said you would make $177 a month with a 1TB HDD on a 1gbit connection in the first month.
Connection speeds are an input, if you list them correctly they would be taken into account. I’m not going to argue with you anymore. You said the 2.5TB ingress was optimistic, but your own data proves you wrong on that point.
If you want to keep claiming it’s optimistic, please show which inputs on the right you disagree with and back it up with actual data. Your criticisms are meaningless if it just remains speculation. So tell me which inputs you disagree with and show the data from your nodes throughout June and July, which is what these estimates are now mostly based on.
2 Likes
I guess someone can’t take any critisism. I’ll leave this discussion as it is here when we get more data points from other SNOs as I think you feel too emotionally attached to have a proper discussion.
A proper discussion provides evidence. Please provide some and I’ll be more than happy to adjust.
I understand that it is quite disappointing to see $16 vs $11 and that this can create some frustration.
And you would like to see even better estimations created.
Maybe in another few months of gathering data, we could come up with “even more realistic earnings estimator” ;D
This is just an estimator, it may be misscalculating - as “estimator” implies.
On the other hand I have to say that there is a world of difference between Bright’s calculator vs original.
Original is giving estimations that are greatly exaggerated like 100 of times… maybe even 1000’s - For me when I calculated my “estimations” - it gave me earnings in multiple thousands of dollars per year - i could earn a living in my country just by buying 10TB HDD and semi cheap network connection 
This estimator’s variance is (from your example) +/-50%? not +/-1000%.
Imagine your disappointment if you bought hardware based on estimations estimating that in one month it will return its value… and then, after a month, you earned ~$5 (my node earned $5 last month, fortunately I didn’t buy any hardware 
)
I think it is much more realistic especially that it accomplishes one more job - it deters people from recklessly buying hardware for 100’s of dollars and then being disappointed by “real life” earnings.
P.S. Full disclosure - I might have taken some part in Bright’s estimator creation by participating in one of the threads discussing real time bandwidth utilization (I have this little silly hope in my head
) - so I might be biased to defend “realistic earnings estimator”
P.S.S. And traffic estimations can change dramatically when new venues of utilization become available for STORj. What means that there may be adjustment needed to earnings estimates - again gathering and analyzing data
2 Likes
That topic showed that traffic is really consistent across nodes. So yes, it definitely helped inform this estimator. With every post on performance I adjust my assumptions and calculations. A lot of actual data has gone into this, which is exactly why I don’t put much value in opinion without data to back it up. I understand that most of this adjustment and fine tuning happens quietly in the background. But I didn’t just make up these numbers. They’re based on actual data from my nodes and verified by reports from others here on the forums. Sure if I see an average of 2.4TB ingress I might round it up to 2.5TB. I had it set to 2TB for a long time until that no longer was close to real traffic patterns. I don’t want to get more detailed than +/- 500GB because it suggests a level of accuracy an estimator can never achieve in a volatile environment.
So many arguments these days consist of one side throwing accusations without evidence, forcing the other side to run around providing actual proof that those accusations are wrong. I don’t want to play that game anymore, because it takes absolutely no effort to make that accusation and it takes quite a bit of effort to prove it wrong each time. I’ll put in my time when you’ve put in yours.
You want to make assertions, please back them up with evidence or they are not worth my time. (Journalists and politicians should also take that approach more often)
4 Likes
I just noticed that one of those nodes is actually 2.5 months old, the other 3 months almost exactly. Meaning you don’t even have 3 whole months on one of the nodes. Which means the expected earnings for that month would be somewhere between the estimates of month 2 and 3. Which are $7.25 and $16.64 respectively. There is some exponential growth going on so you can’t just go with the average, I estimate the expected pay for that node would be around $10. Average that with the expected $16.64 for the node that is 3 months old and you get around $13. This explains almost the entire difference you see. Yes, you made me do the work of disproving your baseless statements anyway. I actually care about the estimator being accurate. But it seems it totally is.
Unless of course you still want to complain about a difference between $11.37 and $13.
3 Likes
I guess 4 dollars more is too optimistic I think @Brightsilence should make his “estimator more realistic” maybe it should show you make even less so that your going to be even more disappointed. Compared the the original earnings calculator that storj had at the start made you feel like you were going to get rich every single month. Brightslience did and has been doing above and beyond what he has to do for this project to help all us SNOs. His estimate is pretty spot on.
7 Likes
This estimator is f*****g awesome.
Real - 24.00$ (17h to finish the day and the month), maybe finish on 24.70$
Estimated - 25.56$
4th month
2 Likes
Thanks for the feedback, I’m glad to see it’s working well for you! Let me know if you see any consistencies in the future. This month we’ve seen a shift with a little less ingress and more egress. It’s never entirely stable, so I try to make adjustments to represent an average of recent months.
Ps. You might want to remove the f-bomb from that post. Posts with that are known to be flagged by community members. Which I personally find pretty ridiculous, but it would still be hidden if it’s flagged.
3 Likes
Hey all, I kind of want to share a thought that is still incomplete. I keep an eye on network behavior to make sure this estimator is as accurate as possible. But fluctuations in traffic patterns make that pretty hard from time to time.
Lower ingress
In recent months we’ve seen lower amounts of ingress. For a while it averaged out to about 2TB per month on nodes fast enough to deal with that kind of traffic. But last month came out to about 1.2TB for me and this month is just over 400GB so far, so probably between 600GB and 700GB by the end of the month.
Delete percentage
The delete percentage is based on estimates and indirect information. Initially I looked at the total amount of ingress on my node since the last network wipe and compared that to what was currently stored. I compared that to the average amount stored over that period and noticed that about 5% was deleted. I think this is still fairly accurate, but it fluctuates a lot as well. As an alternative to that calculation I looked at the number of pieces stored on my node vs the number of pieces deleted based on the logs. This came out to just under 10% of pieces are deleted each month. Now it seems these tend to be the smaller pieces, so it’s not 10% of data stored.
Potential max storage
The soft limit of the amount of data your node will eventually store is a function of both ingress and delete percentage. At some point the absolute size of deletes will match ingress and the size of your node won’t increase anymore. This soft limit number is really sensitive to small changes to both of the inputs mentioned here.
Ingress / Delete% = MaxPotential
At the numbers the estimator currently uses: 2TB / 5% = 40TB
Using worst case numbers: 600GB / 10% = 6TB
Now I can confirm that worst case is definitely not the case, since my nodes store a total of 16.5TB and size is still growing on average. Though pretty slowly now.
My best estimate is that my nodes have actually seen about 3% of the stored size deleted. Which is also lower than earlier estimates. Even though it’s 10% of the pieces stored, it’s only 3% of the space used.
My best estimate for the soft limit going by this months stats: 600GB / 3% = 20TB
This means the speed at which nodes fill will slow down as they approach 20TB. Now this is obviously no guarantee that future months will be similar. It could speed up again with new customers or testing or it could slow down more because of more nodes coming on board. But this effect is something that should be considered when spending money on HDD purchases.
Preliminary conclusion
It seems right now it’s probably not a good idea to buy much more than 20TB of space.
Changes to the earnings estimator may happen if current traffic patterns hold up. Until then, keep in mind that things have slowed down a little in recent months and estimations may be a little optimistic.
2 Likes
This month, I’m running about 750 GB on 3.5 TB full node.
On lack of analytics
It might be nice to have a third party opt-in analytics network. Node operators submit a signed public key along with a hash of their node ID… to attest that they are a node operator.
The analytics service could be very simple, just pull the daily bandwidth numbers each day at local time 02:00 … and have a secondary port open which queries the analytics using an authentication token of some kind.
Rate limiting on the data could be set to one query per 4 hours or something like that to ensure that DDoS isn’t a problem.
Low speed chatter between nodes could result in node operators being able to determine their local success rates along with local bandwidth comparison. Those data could then be fed into a localized earnings estimator.
1 Like
Are you sure you’re talking about ingress here? Since you mentioned it’s a full node. Also ingress has been really consistent across nodes in the past.
750 GB Egress… now 785 GB.
about 50 GB Ingress.
Sorry for the confusion. Half paying attention sometimes results in incorrect reporting…
2 Likes
Quick followup on my previous post. It seems like we’re settling into a situation where 1TB ingress per month is starting to be the new normal. Unfortunately that means I had to cut that parameter in half. There is also an upside though. Since I now have a full node as well, I have a better idea of the level of deletes we can expect (on a full node, ingress = deletes as it only fills up space that has been freed up). This seems to be quite a bit lower than 5% estimated earlier. It’s actually about half that amount right now. This fortunately means the maximum potential of a node is still around 40TB. It just takes longer to get there.
Changes are reflected in the sheet now. This should help it fit new node behavior better.
5 Likes
yeah, 2 years lol… Hopefully by then we have more ingress again.
I actually had 0.6TB ingress last month and 0.75TB the month before, so 1TB feels optimistic to me.
1 Like