My nodes are reliable. They’ve been working for about 6 years with only short maintenance breaks and will continue to work as long as it’s profitable for me.
If there are enough of you who can host data for free, I don’t see a problem. You can do that. Storj is happy to accept volunteers. I see from your posts that there are many volunteers, which is a good thing. A volunteer-based business has much lower costs.
Yes, it’s a simple investment that has paid off. Will I not receive compensation for fixed costs? Oh well, I’ll have to leave to avoid debt. You have the right to provide space for free; I don’t dispute that.
I pursue profit because it allows me to pay taxes and buy food. I work to have money I can use for my own purposes. I don’t do it for a cause. Storj doesn’t exist for a cause either. The company exists to make money. I exist to make money too. Without money I will die because I cannot even buy food.
Storj has the right to reduce payouts to $0. That’s their choice. It’s obvious that many people will stay for the cause, which is understandable. This is how religions, among other things, operate. Unfortunately, many people operate solely for profit, including me.
You can’t run a serious business with volunteers alone—forget that idea.
The option of not paying operators is a double-edged sword. If you don’t pay, the service could shut down at any moment, and you might lose your entire business.
Another possibility is that, somehow, a payment freeze plan is introduced until the current situation is resolved—and who knows if there might be a surcharge for staying active and supporting StorJ.
Only StorJ’s internal staff knows if it’s possible to get out of this slump; they know the actual numbers of how many customers and operators there are. We also have to take into account any binding contracts that may exist within StorJ Select.
To be clear, I’m not accusing/blaming/attacking you personally, or anyone else for that matter. I understand your circumstances, (maybe too well), and obviously rational node operator will take profit if there is any to make. The whole design is based on nodes acting in self-interest, and being byzantine.
My statements are strictly from the perspective of running a stable network, and not self-serving operators’ interests. Yes, I’m also an operator, and yes, you are right, in HCOL area where I live, storj payouts at any rate are laughable and negligible, so yes, I do that for other, non-monetary reasons. This has no bearing on validity of my claims. From that perspective, storj needs to configure and tune incentives to right-size the capacity, minimize churn, and increase stability. Payout rate is one of the knobs that easiest to turn, and that is now out of whack…
In this case it’s the other way around. If you pay to make it profitable, people for whom it stops being profitable for any reason whatsoever will leave abruptly. People who don’t rely on storj to take their kids through college will take the electricity subsidy however small it is. They simply have no reason to leave, unlike “profit seekers” who run a “business”
You know, people turn to insults when they run out of arguments.
The implication being I’m a fool, or are you reflecting?
This is viewing world in binary – make profit or work for free. There is the whole range in between. Especially, since no work here is involved. Congrats, you killed a straw man.
RE: berries> we had a…particular individual, who claimed that drives are like berries you pick off a plant, hence free (=reuse what you have).
The most dangerous SNO is the one that can’t react to a failing drive, because he has no money aside to replace it. Cue the ZFS discussions and why it’s the holy grail of storage.
I feel that every time I post on this forum. People dont’ search and I guess discourse could be smarter to suggest better existing posts.
This also assumes profit - you imply profit shall be enough to buy a drive at least. This is wrong.
You already have a server running. You already have a drive. If it dies – you already have to replace it. Storj does not change that. You don’t have any meaningful overhead of running nodes outside of melodramatic notion that extra 20 IOPS will decimate useful life of the disk…
Yes. Especially if you bough disk for storj, you are not going to replace it. If this disk also hosts your data – you will.
Some storj representatives here used to actively discourage using fault tolerant arrays, which I always shrugged at and ignored – because that recommendation is not only wrong in the grand scheme of things (I guess it’s technically correct that node does not need to be fault tolerant on the network scale), but also contradicts “don’t bring new capacity” mantra – if you could power off the disk – why woudl you keep it running.
This. The big loss is whatever the primary purpose was for the drive. The fact that you also lost your Storj node that was helping offset costs… is secondary.
The network was designed so any node can leave for any reason: and the repair system has been handling any node that left for any reason… for years. SNOs going offline isn’t dangerous: we don’t even notice.
If it hosts my data, it’s not the only place I’d keep my data. I would replace the drive right away, or I could wait for a month until my paycheck arrives. Or the next month, if my car breaks down. This is what you don’t want to realize: If it’s running for someone’s own use, they are free to replace it as and when they see fit. If it hosts a node, the clock starts ticking the moment it goes offline. Sure you can leave it offline for days on end, but that means that one piece is missing. Multiply that by a number of SNOs barely scraping by until next work paycheck (not storj payout) and that’s when you run into the risk of a missing piece.
Well, this is a bad and rather contrived example. If you have data in two places, that means you need data to be in two places. If one place perishes – you need to fix that, regardless of whether your car breaks down. Again, whether you run nodes has no bearing on this. If you can’t afford a driver for yourself – obviously you are not going to buy one just for the node.
And you are free to ignore the clock. I do. I don’t go out of my way to host nodes. Once we had power outage for 7 days. I slept just fine.
This is accounted for in the design.
I strongly suggest reading (or re-reading) the whitepaper:
we expect most nodes to be rational and some to be Byzantine, but few-to-none to be altruistic.
So, what you essentially say is that the only “reliable” nodes are those where the operators choose to run them for almost free?
Running the node takes at least some effort, even if someone would use the server that’s on all the time and the drives that are on all the time anyway. Setting it up, checking if it still runs once in a while etc. That should be worth at least something. Either the project is cool (looking for aliens, trying to cure cancer) or it is profitable to do it, otherwise, why do it?
For some reason you seem to think that people who do it for money are somehow worse. But then Storj company itself also does it for money, including all the people working for it.
It was meant to make profit, otherwise why do it? The less money I make from this the less interested I am to keep it running. Now, I may be someone who would keep the node running way past the time it would not make sense anymore, but others may decide to shut them down earlier.
After all, I shut down my crypto miners once what they mined was less that the money I paid for electricity.
At some point it does not become worth it. How much time you would spend looking after the node if all you got was $0.1/month? It’s not zero, right?
At least for me, the effort I spend on making sure the node runs OK is proportional to the money I get for it. When I got more, I checked graphs a few times per day, trying to see problems before they become too big, trying to figure out how to optimize the node so it can do more IOPS and so on. Now I do that infrequently. I still have the monitoring scripts I wrote, but don’t really bother updating them. My monitoring still sends me SMS if the node needs updating or it stops running and I still react to them, but probably not as fast as I used to.
I know what happened to other stuff I ran (and that includes Storjv2). I see that I don’t make any money on it, so I stop caring about it. I may notice that the server or VM crashed a month after it happens and such.
I do not have a hard cutoff like “pay me $x/month or I shut it down” like the others, but as the x goes down, the likelihood of the node not working and me not caring about it goes up.
Why should someone feel guilt about leaving? I mean this is purely for profit endeavor. It’s not like Storj is an animal shelter or something like that where people are expected to work for (almost) free in order to achieve some good goal. This is just for money - Storj charges customers money, node operators charge Storj money.
That is exactly what I was talking about. Some people trying to push through foolish decisions like reducing already small payouts without any tangible benefit to the network. They don’t care how this will affect the network which, incidentally, provides paid services to its clients.
No. I say nodes run by people who don’t have threshold of profitability are more stable long term. They don’t have a major risk factor.
The server or nodes? Either way, the script checks it for me. Which I setup 5 years ago. 0 work from my side.
I’m serious, the only hour of work for the node I spent in last few years is what go dropped FreeBSD 13.3 and I needed to login and upgrade jail to 13.4. This issue is self-inflicted, and most operators will never have it.
Yeah, but it’s so little that noise level.
You are projecting. I never said anything about character of people who do it for money. Operators are expected to be rational. And if it makes money – of course it woudl be silly not to capture that. My point was from network perspective – network should nto want operators that do it for profit – because they start building out glass castles. That shatter as soon as profitability ends. Those who don’t expect profit but are content with subsidy – those nodes will continue running. That’s pretty much my whole point.
Sure. This is technically true statement, but the amount of money we are talking about are so minuscule that I could just donate the remainder. My nodes now pay me $100. If they start paying $10 instead – it would still make sense to run them. But this is way beyond the pay reduction storj is considering. And if it went under $10 – they are already running. I’ll just keep them running. Because, as I said before, I don’t mind donating resources that otherwise will go to waste. Yes, including whole 5 extra watts of energy the nodes impose. Some people are money driven. Some are not.
You did way more than necessary. Nodes are self-sufficient and self-contained. Operator’s job is to run the process and forward the port. Everything else is not operators job, including any monitoring. What are you monitoring? You cannot act on the results, so why waste time?
If you don’t notice that your sever crashed for a month – maybe you don’t need to run that server 24/7? Maybe it’s a backup server you need once a week?
Power it off.
Again, it’s not black and white. It’s a spectrum. From operators perspective:
Major Profit: Node pays enough to justify buying new Dell servers in pallets → bad, has threshold “I abruptly leave if I stop making money”.
Minor Profit: Node pays enough to add new disks to your existing running server → better but still shit. Node can leave for the reasons that have nothing to do with storj or operators - such as energy cost increased, or HDD prices skyrocketed.
Subsidy: Node pays enough to offset some electricity but not enough to buy new disks → Sweet spot. Node keeps running no matter what, because the alternative is zero. Upfront cost – Zero. All payment is pure profit (minus 5 watts)
Donationware: Run nodes for free for warm and fuzzy feeling of helping reduce carbon emissions for no payment in return. Many people oppose that, they think I want Storj to do that – but I never suggested that.
I suggest the “Subsidy” band. The band of stability. Right now storj is in Minor Profit band. This causes avoidable turnover accompanied by performative door-slamming posts on the forum. It’s also wastes a little bit of money - -but that’s secondary.
What customers pay has no bearing on what operators should be paid. It’s not USSR.
That’s the sweet spot. Enough profit to keep growing, or absorb fluctuations in costs. Those are the nodes that are the hardest to leave.
Of course it does, but not the way you phrased it: The cost of storage provided to storj should be lower than the cost of storage provided by storj to the client. That doesn’t mean we have to absorb storj’s costs. If storj decides that they’ll keep providing their S3 gateway for free, I don’t have to take that into consideration as a supplier to the company. They need to figure out a way to do that on their own. In other words: if storj decides that they’ll provide storage to their customers for free, I don’t have to provide that same storj for free to storj. You might, and there’s nothing wrong with that. Just don’t generalize it for the rest of us.
I’m kind of in the same boat. I have Xeons with ECC memory and UPSes. I have about 35TB usable capacity but I have another 20TB or so of drives on standby, and more on the way. I even have SSDs for caching. I would certainly like to get more data.
I recall that one of the reasons that the earlier payouts were so high was to encourage network growth. IMO that was probably not needed, because SNOs like me already had tons of spare capacity ready to go, there was just no reason to waste power on installing drives that wouldn’t have filled.
I ran Storjv2 on separate servers and yes, near the end of the v2 network I did forget to check on the server and it stayed crashed for a month.
As I said, my level of effort depends on the amount of money. It’s not like “oh they won’t pay me so I won’t take care of it”, just that I am less interested if I get paid less.
Subsidy and donation nodes can also stop for the reason that the operator got bored or the node crashed and the operator forgot to check.
The major and minor profit operators announcing that they are going to leave is normal IMO. I mean, what did Storj expect? Every time the payout rates are reduced, somebody is going to leave, because to him it is no longer worth it (and he does not want to just donate the resources). Let’s say Storj announces that the new rate is $0.1/TB - even more people would leave.
Upfront cost is the effort required to set it up.
During the performance tests I noticed that my server was a bit slow, so I tried to figure out how to make virtualized IO faster (which I did). I then later redid my array from 3xRAIDZ2 to 9xmirror to further improve performance. I was too late with this though, since the testing stopped and that customer chose Select AFAIK.
At one time the filesystem of the data drive could no longer be expanded because there were too many inodes, I had to rsync all files to another filesystem.
There are also other threads on this forum with my various findings - initially the node used 1 core, which slowed it down and so on.
Also, Storj did not send emails about an offline node, so I had to set up the notifications myself.
So ar I managed to keep the node running. Would it have run the same if I did not look at it at all? Maybe, maybe not. After one update the node did not start anymore because it moved the executable files to the data directory which was mounted with noexec.
I did try to notice problems and at least write to the forum about them.
Yes, my node is already running, I do not have intention of shutting it down. If I just found out about Storj right now, I probably would not start a node. The operation seems complicated and there’s not enough money in it to bother.
If I get a second IP I will start a new node though, since I can just clone the VM now and change the identity.
By the way, for a long time, the kind-of recommendation (no, it was not an official recommendation) from Storj was to buy a raspberry pi and use that to run the node. Only when they reduced the rates by a lot did the “you have it online anyway” part came.
Also, running the node imposes some requirements. For example, I cannot take the server offline for too long (I can migrate my other VMs I want to keep online to another server, but only this server has the space for the node), I cannot make the node shrink to then use the space for my own stuff (though there’s enough free space there for it to not matter to me).
So, you have about 60TB of data, probably over at least 3 nodes with separate IPs. $100 can indeed be worth it. $10 maybe. I get ~$30 for my one node. If it went down to $3 I really would stop caring about it.
First of all there are culture differences. What looks like blackmail to us might be normal communication to them. Sure that doesn’t mean we have to copy this behavior. My hope is that they will notice this culture difference as well and are willing to meet us somewhere in the middle. After all the situation is simple. Payout rate might drop. This forces all of us to decide if we continue or call graceful exit. Even if we are mad the right thing to do would still be to call graceful exit to not burn any bridges.
Second the repair system should be able to handle this. You will be surprised how much effort is needed to get some pieces below the minimum threshold. That is not an easy task. If halve of the pieces are stored on nodes that are operated by people like you and me than all they can do is cause some panic looks on some of my coworkers followed by praying that your and my nodes stay online for a few more hours to repair the most at risk segments and a day later it would be mostly over. Sure I don’t want to test that out but I have seen how powerful the repair system is. It will kick in in just hours, identify the segments with the lowest piece count and repair them first.
Also, it’s just information.
There are three possible answers to an announced pay reduction:
“I will keep working.”
“I will quit.”
Silence (and then doing one of the above)
Everyone has a threshold of how much $/month or $/TB they need in order for it to be worth it to keep the node running. Even though it looks like any amount is better than zero, running the node still takes at least some effort. At the very least I would have to log in to the crypto exchange, press “sell”, then press “withdraw” and pay the withdrawal fee. Someone could do it for fun, ignoring the money, but how many node operators are like that? Storj does not even have a cool goal, like searching for aliens or a cool graphics display you can look at.
So, I do not think it is wrong for node operators to say that if the payouts go down, they will quit. Worse would be if node operators quit without any warning. Storj announces the payout cut and the day after the last payout with old rates, a bunch of nodes shut down without any warning at all.