Is that a persistent L2ARC… or a special metadata device? I like the idea of a metadata device that’s also able to store all your small files… but I also like the idea of a L2ARC that can fail and nothing terrible happens. Decisions decisions…
Well since I’m on the defending side of that (full disclosure: I’m a Cloudflare partner, helped with the early stages of their loadbalancers, before they were offered to the public), when a significant attack happens, a lot of alarm bells start going off. The “victim” starts notifying upstream providers of the attackers. Since that amount of traffic (ie the infamous 2.5Tbps attack targetting Google) doesn’t go unnoticed long, the ISPs have a vested interest in stopping it as soon as possible. Let’s take the SEA-ME-WE3 cable for example (full disclosure again: worked for one of the 92 owners of that cable). That cable, which is the longest underwater cable in the world as far as I know, has an online capacity of 2.3Tbps. It’s very likely half of the world (Australia to UK) would notice it if it suddenly became saturated.
The attacker’s upstream see the notices on their abuse@ email (they are technically forced to act upon them, except some “attacker friendly” ISPs) and they put the IPs on monitoring. As soon as they are convinced that there is indeed an attack, they cut them off. The victim can null-route IPs on its end to keep the servers online in the meantime.
Back to storj: the incoming data per second isn’t the problem. The packets per second are. If we are talking about that much traffic, it’s more likely routers would start crashing left and right, long before any drive reliability issues come into play. This is a valid concern. Worrying about drives failing because they store TTL data isn’t.
The Storj Drives (see previous reply) have a 550TBW/year and 2.5M hours of MTBF. Taking everything to face value, that means if you write 550TB per year, that drive should last for 285 years. As you can see, those are statistical “guidelines”. It doesn’t mean that if you write 551TB in a year, the drive will fail immediately.
Besides that, everyone (on the “OMG TTL IS DEATH!” camp) seems to think that as soon as the new client signs, all existing data will be immediately converted to TTL data and no other data types will ever be uploaded. There will be normal data (ie data uploaded and deleted at a random future interval by the client), forever data (never to be deleted as long as the account is paid for), low TTL data (I’m guessing nothing less than a week), medium TTL data (30 days), and long-term TTL data (3 months). Do I have any proof of this? No, but I can confidently say that others will use the network as they have been using it so far.
You shouldn’t. You should sell your payout the moment you receive it. This isn’t an investment.
5 Likes
9 posts were merged into an existing topic: Get paid out in Fiat
You are laughing, but few years ago nobody thought that it HDDs have any kind of non-trivial wear on mechanical operations except for statistical puff of smoke at the end of the bathtub curve. Now we start seeing HDD producers trying to actually quantify this wear.
Will customers negotiate between themselves whose turn it is to generate the peak load?
1 Like
I’m not seeing that, I’m seeing everything “good” have a 550TBW/year rating, from HDDs to SSDs. The rest (ie leftovers on the production line) ~300TBW/year.
They might decide to test out peak load at the same time. But we have that situation already just on a scale that storage nodes don’t care. Every now and than there was an increase in uploads for a few days. All the nodes are happy and the moment the peak load is over they start a new thread complaining about the reduced traffic. Nothing changes on that side.
4 Likes
A post was merged into an existing topic: Get paid out in Fiat
I don’t want to entice for selling your STORJ as fast as possible. But is there some service out there which takes all the tokens getting sent to a for example specified ERC20 address, and converting them immediately to a desired token like USDT or ETH without further actions?
Yes, but also really different places not VPN ones.
1 Like
Then you should have FATAL errors in your logs, if the log level at least error.
Is this satellite success rate-based rate limiting running on all satellites already or just on Saltlake?
All satellites are running it. The other satellites are running choiceofn 2 at the moment. We might want to increase that to choiceofn 3 or 4. There hasn’t been a decision yet.
Storj select is still using the old random node selection. It has some additional constrains that the new node selections don’t allow yet. Additional code changes are in progress to unify it. We basically took a short cut and implemented the bare minimum for the new node selections fully aware that we have to exclude storj select for the time and work on that problem later.
How does the new node selection work in case of multiple SNOs behind a /24? Is it a case of “winner-takes-most”?
If an SNO is running fast nodes and another SNO is running potato nodes (both with plenty of free space), does the new system pick the faster nodes more often?
Both of them will get 50% of the normal rate. The fast node with a 100 % success rate would get a 6 times higher selection rate but multiplied with the intial 50% that would be only 3 times the rate. If the other node on the same subnet is full it will go up to 6 times the rate.
Same math for the slow node just in reverse.
I wonder if that’s set up now? Looking at traffic graphs… the last 24h looks very similar to the day before it. For me it looks like peak traffic starts around 10am EST… and ends 2am EST… then it lowers for around 8h in-between.
At least that’s the pattern for the last 2 days. We’ll see if it spikes around 10am EST again today…
If I understood that right, does that mean that during the “initial” selection the faster nodes in a /24 aren’t pitted against the potato nodes in that /24, just an even split between the two groups as was the case so far? It’s only “natural selection” down the road, when the faster nodes will “stand out” more in the selection process?
Sounds about right. I don’t know if the time of the day the traffic spikes is also the same hour these customers would have their spike. So maybe don’t count to much on that. The entire curve might shift by a few hours (±12 hours :D). The point of uploading our test files in a similar pattern is that it will send more files to nodes with enough bandwidth and encourage growth on these nodes. A flat line would give a bigger incentive to nodes with limited bandwidth but that are not the best nodes for these customers later. So the idea behind the sin curve is to set the right incentive to get the nodes that can deal with that best.
2 Likes
That’s actually pretty smart (and a bit sneaky 
)! Uploading data at a constant rate sounds easiest (especially if you’re trying to maintain a specific reserved capacity)… but yeah it wouldn’t reward higher-performance nodes. Clever!
1 Like
What’s the ratio of the min/max bandwidth of the current test? My traffic graph has a flat region. Is the test like that or does that mean that my node is not fast enough?
I don’t think I can follow. So let me explain it in a different way.
Current RS numbers are 16/20/30/38 and choiceof 6 (both might get changed in the futures to match the required throughtput with the remaining number of nodes)
This combination of RS numbers and choiceof factor will select 38x6 nodes in total. Now each subnet has an equal chance to make it into this selection group. If there are 2 nodes on the same subnet they get 50% of that rate. The 2 nodes on the same subnet will never get selected at the same time.
The rest with the choiceof comparison happens after that. The fast and the slow node need to be part of the initial selection group to get to the comparison part. At that moment the success rate will change the outcome. The slow node might get matched up against a fast node and kicked out of the final set of 38 nodes. The fast node on the other hand has kind of a garantee to always be part of the final set and because of the 6 times bigger initial selection it gets 6 times more uploads. With the second node on the same subnet the initial rate gets cut in halve.
In production the RS numbers are still 29/35/65/110 and choiceof 2. Similar math just with a maximum factor of 2
4 Likes