How can I know if the payout is correct if I can’t verify it by myself?
I don’t understand that question. You can look up your payout history and break it down by satellite. The rest is just a bit of math.
The payout history for a month is only available when all payouts are made. For example the payout history of last June is only available some days ago. And it doesn’t show the correct fluctuations in avg disk usage per day, which I use to evaluate and optimize my nodes. My node receives more data, the ingress is great, but the avg disk space just went down and down. Is that logical?
My serious question: Is it that hard to fix it? technically speaking?
While I agree that the most important thing is the payout, for me having more frequent reporting of the usage is also important.
A lot of these statistics that we have from the nodes, while some people classify them as useless, are good indicators to the health of the nodes and the node software. It is true that they can at times generate a lot of noise, (e.g. the multiple “my average dropped 60% on a single day” topics), but they also serve to detect any problems that may be occurring (e.g. test data being overwritten).
As satellites have their own dashboard and statistics for a whole load of things regarding the network, SNOs want statistics relating to what is going on with their nodes, and maintaining accurate stats is important.
At the end of the day SNOs are here to profit from their investment into the project (even if its not monetary investment via “use what you have”, it still requires time from the SNOs to run, and time is precious). Therefore, SNOs will want as much transparency as they can feasibly get.
Im not saying that all stats issues are a high priority, but I have a hard time dismissing them as well. I know this could just been your personal opinion and not reflect at all the stance of the whole Storj team, but to most of us SNOs, Storjlings are the main visibility we get of the team and therefore most things said by them are usually “seen” like a whole team opinion, rather than a simple invidivial expressing their views.
This is not a rant, just an opinion, and keep up the good work.
→ Grafana
I feel like we are having this conversation once per month. The dashboard is the simplest data source you can get. Don’t expect that data source to ever be as accurate and detailed as a grafana dashboard. And there is no way we can compete with grafana. This is out of question.
The beginning of the conversation was the storage node dashboard. Common you can export the data to excel, fill the gaps and run the math. It is not that hard. This is a low priority at best.
My point in this thread is that you are using the wrong data source. If you have trust issues we can meet at the end of the month and run the math together. All the data you need for that is available. If your goal is to find out how healthy your node is we can do that as well. In both cases there is no need to do any code changes. Well at least not in the areas you call a priority. I can see that one of my nodes is a week behind with TTL deletes and that is not a problem of the satellite. These kind of issues are currently high priority. So lets hunt down these and stop complaining about some incorrect numbers on the storage node dashboard.
3 Likes
Of course, as I said in my reply, it was not a rant. I do believe Storj does good work, please do not misunderstand the message I was trying to convey.
I also use grafana as a dashboard, and even that has its own limits, given that it relies on the information provided by the node software.
As I said, I do not consider this a high priority, but the fact that the dashboard does not correctly show information, even if its just because there is a gap and the graph does not handle it properly, is an issue, no matter how you see it, which is why I objected against a dismissive tone to the issue. I never said it was the most important issue and that it needed to be fixed right now and ignore everything else. No, it should just be in the backlog for whenever there is time to resolve it.
Apologies if my message was misunderstood, but I am not complaining about incorrect numbers being displayed, I was voicing my opinion that simply this issue should not be dimissed. I will not complain about the number being wrong because the team knows its an issue, and resolves it whenever they are able to by running the corresponding tasks.
Currently an issue in the forums (which multiple users have already said) is that it is becoming very hostile. SNOs are hostile towards Storjlings, and Storjlings can only deal with so much. I am not blaming anyone in specific, if anything its mostly the SNO’s fault if you ask me, but everyone has to put in their part to avoid conflicts over simple misunderstandings when people are just trying to voice an opinion without any crisicism in mind.
1 Like
Yes it is logical. For my own node it looks like this:
Prevous month I started with 30 TB and because of all the juciy test data uploads I finished the months with 60TB. So on average 45TB and my payout is in that area. This math isn’t accurate because I didn’t take snapshots with accurate numbers. It is just out of my memory what node size I was used to vs what I see now already 2 weeks into the new month.
This month I would expect my paid space to go down. The testdata uploads are not full speed anymore as they have been last months. I would expect a reduction of up to 20% or so. Currently maybe even a bit more because I can see that my success rate is getting worse and try out different settings to concer that. Anyway the point is peak size was 60TB and I would expect the month to finish with 50TB or so. Depending on which tricks I can find to improve my success rate I might be able to continue growing the following month. I am using the test uploads to benchmark my setup. So decreasing the size of my nodes is fine as long as I gain more knowledge that will help me to outgrow it in the long run.
Depends on what your target is. I would say 95% success rate are easy to archive. There are still issues that are harder to fix but given that my payout is increasing I could just ignore them. Now my target is 98% success rate and that is getting a challenge that requires a higher time investment. Is that time investment worth it? Most likely no. I do it more for the knowledge gain and to break my own records.
1 Like
It is easy to add additional information to the grafana dashboard. So if you are missing something I am sure the community can team up and I am happy to provide some code links. Most of the time it is just a single line of code to add data to the metrics endpoint. Now you might ask why I don’t do that code change myself. My goal would be to enable the community to do it on there own and these one line code changes are the best to practice it.
1 Like
Let the community fix it, huh? Why not, indeed?
So I can get the satellite code, fix it, run my own satellites that would be more respectful to SNOs and make the community happier. Is that it?
Not sure where you are reading that but yea it is open source and you can setup your own satellite.
2 Likes
Ok, here’s the solution then. I’m currently building my pet hosting setup. Once I get some time I’ll try to setup a satellite. Anyone interested to try it?
1 Like
Maybe you both should setup a satellite and compete against each other. Lets see who can run a satellite in the 10+ PB range. Are you planning on uploading test data or do you have some customers waiting to get onboarded on your satellites?
2 Likes
Running a satellite means burning money for a long time, maybe forever. So sorry, I will stick with running nodes. This way I can have fun while at least having a small chance to ROI my hardware. If storj is not running out of money…
It’s not a lot of work to add more logging, see my PR here for an example. One more line of code and the same information would be exposed through the debugging interface live.
(btw, Storj folks, it would be nice to have it reviewed 
)
Probably a bit more involved if you would like to expose information that is only available on satellites, because then you’d have to add some protocol for satellite—node communication…
Honestly, this sounds like a fun thing to do. And only approx. 30kUSD to run on Hetzner together with the nodes, for 3 USD/TB storage with no egress costs. I’m setting up a competitor right now!
1 Like
Yep. I also noticed reviews are a bit slow. It will take some time to address that. Please keep the PRs comming and I will show them around in the company and ask everyone how they feel about that and what needs to be changed to get faster reviews.
I see you did your homework. That number sounds about right. That last part might have been sarcasm? Just in case you want to do this it might be cheaper to not pay public nodes and instead join the storj private cloud programm. 10PB might be a bit abicious for that but if you control all the nodes yourself you might be able to reduce the storage expansion factor and get lower than 3 USD/TB.
Edit: I do have a bit of extra space that I could offer for a lower price in order to bootstrap other satellites but the moment my storage gets full that might change. Feels more like a time limit and then I would expect equal payouts.
2 Likes
Yep!
Curiously, Hetzner themselves offer storage boxes at 2 EUR/TB, though they’re not object storage, not as redundant, and apparently quite slow.
So I guess for 3 USD/TB a company with non-trivial storage needs could get it at Hetzner with Storj software quite easily with a lot of benefits, while having storage colocated with compute. We do need to wait until Hetzner rolls their own object storage though—because apparently they are working on that—and check how competitive it would be.
At the same time I checked AWS. A similar setup there would cost 27 USD/TB, which compared to regular S3 frequent access tier at 20 USD/TB doesn’t work out that well. And S3 doesn’t have the drawbacks of Hetzner’s storage boxes. If, however, it was possible to set up, let say, half of the nodes on spot instances, then suddenly the cost is down to ~15 USD/TB, which starts being competitive with S3 frequent access tier.
2 Likes
How does Grafana help when data from the satellites are missing or the data provided from the storagenode is inaccuarate?
1 Like
1 Like
Grafana has all kinds of math functions that can be used to fill the gaps. I don’t see any inaccurate data but that might be related to my dashboard. The data source isn’t limited to satellite and storagenode data. It would be possible to show os data instead if that turns out to be more accurate for you.
1 Like