I very recently found out about storj and today i decided i am gonna give it a go.
A few questions came up though. Am i better of making 10x 5tb nodes or one 50tb one?
It will be a lot easier for me to maintain with small nodes. backup wise etc.
Or is there a sweet spot for storage? I am on 1000/1000 id love to get a hint on how much storage would be optimal for 1000/1000. (I have plenty of storage but i don’t want to provision more then what is being used.
Are there any way you could say schedule downtimes? So that your rank don´t get tanked just because some maintenance was due.
Depending on who you ask, you will get different answers. But the advise from Storj is to run one node per physical HDD. This should in the long term be more profitable to you. I agree with that assesment.
If you already have an array and you plan to use available space, you should know that there is no advantage to running multiple nodes on the same storage device (whether that is an array or individual disk). Nodes on the same network are treated as one node for traffic allocation, so running multiple nodes does not get you more traffic or data.
Final note, there is absolute consensus that you shouldn’t be using RAID0 or JBOD or similar multi disk options without redundancy, as the loss of one disk would have you lose your entire node.
If the question is to run 10x5TB nodes or a 50 TB node made up of 10x5TB drives, then my suggestion is to run 10 nodes of 5TB (with each node being a single 5TB HDD). My reasoning is that with 10 HDD drives acting as a single node then the chance of losing a drive and losing your entire node is far worse than having 10 nodes and only losing a single node.
Remember, payments are based on node reliability and it takes months for each node to get a good reputation and become profitable. Here is a good post on profitability:
As Storj just recently went into production status the demand for storage by paying customers is just starting to grow. If you put together 100TB of storage who knows how long it would take to be used? To lower your risks you might want to start with a few lower cost HDD drives and add nodes as you fill each node. Also due to the need for reliability consider quality disks, reliable networking, power backup.
My personal experience is that I started with a single 10TB HDD (Western Digital ‘Red’) which is consumer class and as it filled up and I learned more about Storj, I decided to add 14TB Western Digital HC530 HDDs (Data center class drives that are designed to run 24x7 with low power consumption). Also remember you lose about 10% of your capacity for headroom so I get about 13 TB per drive. My breakeven for one 14 TB drive is about 15 months and 19 months for two. We are in it for the long game.
Hi! Thanks for the responses. It´s more a question of is there an advantage to run multiple instances or one big one. But since i wont get more traffic, or actually i have 3 IP addresses but more then 3 would be no benefit then. More then that there is no point then.
The setup disk wise is a 34x 6TB SAS Drives drives in RAID60 plus hotspares, and that wont really change since this is used for some other stuff but 80% free so might aswell put it to good use. Drives are not what will tank my uptime. Or the hardware for that matter.
So I guess my best option is to start at 5 TB and expand when it starts to get full till i run out if space.
Any ideas on how long the data can be unavailable before it gets tagged as lost? If we have a node with 50tb and that has multiple disk failures and goes down. It would be at least 1 day to actually restore that data, but if you get suspended due to data not being available for 1 day. then there is no point for me to actually do backups of that data.
I think the most important thing here is availability, sure storage and bandwidth is a necessity. But the uptime is what’s going to be the struggle. So time to go on ebay and get some new UPSes. Even though we haven’t seen a power outage in my area for about 2 years now.
For Link failure. Say main connection goes down, we automatically switch over to the 4G backup connection. So data is not unavailable but bandwidth goes from 1000Mbit to about 40Mbit, I am assuming it comes down to luck if we get an audit then or not.
Yes availability is indeed important. Being down for 1 day should definitely be avoided. The ToS lists a maximum downtime of 5 hours per month. However, a new uptime checking system is being worked on and I do believe it will be a bit more lenient. That said, 1 day would probably be too much. There is no advantage of running multiple instances on the same /24 subnet. You mentioned having 3 IP’s, if they are in the same /24 subnet they would still share traffic, so keep that in mind.
Going down in bandwidth temporarily is much less of a problem. So as long as that 4G is stable, you should be fine with that failover.
In rare cases there may be an advantage of running multiple nodes on the same IP and the same array.
That is, if someone else in the same /24 also has a node, running more nodes gets you more data.
well according to the documentation the current limit allowed on each node is 24tb, so you will need to make multiples no matter how you look at it…
performance wise you are most likely better of collecting it all into one big chunk… one has to assume that even running multiple nodes and during realworld workloads they will not always see the same traffic and thus splitting data out over 50 drives would mean if one nodes data is wanted you only have one drive to deliver it, and ofc then comes the whole redundancy issue…
i would say with that many drives… maybe one pool of 6x8 drive raidz2 and then two global hotspares… or that would be how i would set it up… and then maybe a sizable l2arc to help manage all that space… or a shitload of RAM
ofc with 50 drives you need to be able to get access to more of the same type as they start to fail… since a 2% AFR would put you one drive short per year… thus you either need to maintain the number of drives or make it a smaller pool and keep some as replacements.
This is not true, it’s listed as the max recommended spec for some reason, but there is no upper limit for node size. My guess is they mentioned it as mac recommended because it may not be very likely to fill up. But we’ll have to see whether that’s the case in real use.
As for the rest of the comment, I think it’s moot because of this.
I think RAID60 is a pretty decent option with this amount of drives.
Can you elaborate on what you’re basing this on and why you think that running multiple smaller nodes over the same timeframe would have a different effect?
Which calculator are you referring to? The Storj provided earnings estimator is really unrealistic in this aspect. It will not give you a real world view of what to expect.
I mostly agree, except up to around 50mbit it does make a difference. Beyond that it’s negligible.
Well the point about ISP bw has no advantage cant be true.
I am talking about unlimited connections no traffic caps. But perhaps Storj simply wont use it?
Perhaps i will not be assigned more traffic if i have a 5tb 50/50 connection then on a 5tb 1000/1000 connection?
The gigabit line would be able to push 6.7tb a day, while the 50mbit one would “only” be able to do 336gb a day. Perhaps Storj does does not check your capabilites on how much data you can transfer simply beacuse there is no need for it at the moment?
yeah and even tho 50/50 might be enough, it might affect latency at high utilization… tho in my limited experience with running nodes, i’ve never seen more than about 60mbit peaks once for a few seconds.
ofc with test data the flow is kinda stable, with real world activity we should expect it to be much more unstable that then having higher bandwidth might be very useful… but for now … i tend to agree with direktorn for now on mostly test data workloads…
because its the central controller for that which decides how much data is directed at a node… and afaik there isn’t anything that would take into account if a node was swamped on bandwidth or not…
so bandwidth is irrelevant so long as you have enough… atleast thats how it seems…
also if you compare across nodes of the same ages, their data ingress and egress is insanely close, i assume this is also why that if you have downtime, then the allocated data during that downtime seems to ingress after you get back online… maybe not all of it… but there sure seems to be a clear effect of it during extended downtime.
The backup is useless, it will be outdated on time of creation. If you restore from the backup, the node will be disqualified for missed pieces since backup.