A post was merged into an existing topic: Get paid out in Fiat
I donât want to entice for selling your STORJ as fast as possible. But is there some service out there which takes all the tokens getting sent to a for example specified ERC20 address, and converting them immediately to a desired token like USDT or ETH without further actions?
Yes, but also really different places not VPN ones.
1 Like
Then you should have FATAL errors in your logs, if the log level at least error.
Is this satellite success rate-based rate limiting running on all satellites already or just on Saltlake?
All satellites are running it. The other satellites are running choiceofn 2 at the moment. We might want to increase that to choiceofn 3 or 4. There hasnât been a decision yet.
Storj select is still using the old random node selection. It has some additional constrains that the new node selections donât allow yet. Additional code changes are in progress to unify it. We basically took a short cut and implemented the bare minimum for the new node selections fully aware that we have to exclude storj select for the time and work on that problem later.
How does the new node selection work in case of multiple SNOs behind a /24? Is it a case of âwinner-takes-mostâ?
If an SNO is running fast nodes and another SNO is running potato nodes (both with plenty of free space), does the new system pick the faster nodes more often?
Both of them will get 50% of the normal rate. The fast node with a 100 % success rate would get a 6 times higher selection rate but multiplied with the intial 50% that would be only 3 times the rate. If the other node on the same subnet is full it will go up to 6 times the rate.
Same math for the slow node just in reverse.
I wonder if thatâs set up now? Looking at traffic graphs⊠the last 24h looks very similar to the day before it. For me it looks like peak traffic starts around 10am EST⊠and ends 2am EST⊠then it lowers for around 8h in-between.
At least thatâs the pattern for the last 2 days. Weâll see if it spikes around 10am EST again todayâŠ
If I understood that right, does that mean that during the âinitialâ selection the faster nodes in a /24 arenât pitted against the potato nodes in that /24, just an even split between the two groups as was the case so far? Itâs only ânatural selectionâ down the road, when the faster nodes will âstand outâ more in the selection process?
Sounds about right. I donât know if the time of the day the traffic spikes is also the same hour these customers would have their spike. So maybe donât count to much on that. The entire curve might shift by a few hours (±12 hours :D). The point of uploading our test files in a similar pattern is that it will send more files to nodes with enough bandwidth and encourage growth on these nodes. A flat line would give a bigger incentive to nodes with limited bandwidth but that are not the best nodes for these customers later. So the idea behind the sin curve is to set the right incentive to get the nodes that can deal with that best.
2 Likes
Thatâs actually pretty smart (and a bit sneaky 
)! Uploading data at a constant rate sounds easiest (especially if youâre trying to maintain a specific reserved capacity)⊠but yeah it wouldnât reward higher-performance nodes. Clever!
1 Like
Whatâs the ratio of the min/max bandwidth of the current test? My traffic graph has a flat region. Is the test like that or does that mean that my node is not fast enough?
I donât think I can follow. So let me explain it in a different way.
Current RS numbers are 16/20/30/38 and choiceof 6 (both might get changed in the futures to match the required throughtput with the remaining number of nodes)
This combination of RS numbers and choiceof factor will select 38x6 nodes in total. Now each subnet has an equal chance to make it into this selection group. If there are 2 nodes on the same subnet they get 50% of that rate. The 2 nodes on the same subnet will never get selected at the same time.
The rest with the choiceof comparison happens after that. The fast and the slow node need to be part of the initial selection group to get to the comparison part. At that moment the success rate will change the outcome. The slow node might get matched up against a fast node and kicked out of the final set of 38 nodes. The fast node on the other hand has kind of a garantee to always be part of the final set and because of the 6 times bigger initial selection it gets 6 times more uploads. With the second node on the same subnet the initial rate gets cut in halve.
In production the RS numbers are still 29/35/65/110 and choiceof 2. Similar math just with a maximum factor of 2
4 Likes
Thinking about RS ratios scrambles my brain. So this may sound like nonsense. But is this right?
- Long ago the network uploaded 80 pieces, with 29 needed to recover: so thatâs where a 2.7 expansion factor came from?
- Then at some point 29/65 was configured⊠which is more like 2.2x (so the network can fit a bit more customer data in the same raw SNO space)
- Now youâre testing 16/30, which is about 1.9x⊠so the network capacity will go up again.
And as that expansion factor goes down⊠it gives Storj-the-company a bit more breathing room: since you live in the margin between the $4/TB/m charged to the customer and the $1.5/TB/m for SNOs. But it does increase the risk of unrecoverable data. But youâve been running the network for years and carefully watching things so understand you can still meet your recoverability/availability guarantees?
If so: nice job! I really want to see Storj run with a profit every month, for many months, before the treasury tokens get too low. Good luck!
The flat part is showing the limit of the network and not the limit of your node. We still need some fine tuning. We have to balance 2 objectives. Hit the peak load (=no flat part in the curve) and upload enough data every day to reserve enough capacity. We figured out that we can hit the peak load by increasing the choiceofn factor or increasing the long tail but we can still change these params a few days before it goes live. The other part with reserving enough space requires time. So that has higher priority at the moment. I am still running the math and the data points donât make sense yet. The dashboard for management is saying the upload rate is too low and we donât reserve enough space. Our upload tool is saying we are hitting the target. So there could be some hidden bugs that we still need to find. Best case it is just a bug in our math and the dashboard is just off. Worst case the dashboard is correct and we might see some extensive garbage collection. We donât know yet and while we keep investigating we donât worry about not hitting the peak load.
Beside that at least some of the target customers have a decent tollerance around the peak load. Imagin the flat part would be 3 hours every day. What would happen on the side of the customers is that the uploads will lag behind. So instead of an fast upload they kind of queue up a bit. Initially just by a few seconds but later we might be talking about minutes. The moment the current load reduces enough the queue will clean up and all files have been uploaded. So we can cut off the peak load to some extend and these customers wouldnât have a problem with that. Ofc we want to give best service. We are going to communicate with these customers about what we see on our side and how it looks on their side.
We are working on a few plans to increase throughput. One of the plans is that we rent out a few servers ourself to fill the inital gap. So if your nodes canât take the load we might just add 1000 nodes ourself and wait for the network to keep growing over time. We would reduce this surge capacity over time and donât intend to run it for long. Just some surge capacity for times of rapid grow.
2 Likes
Maybe to clarify this part a bit. With your nodes I mean the entire network and not any single node. By increasing the choiceof factor we would increase the load for the fast nodes. That might sound like a great choice for the nodes that benefit from that but it also has some downsides. Your single node might be very happy about filling the free space in just 2 weeks but what do we do after that? To maintain the maximum throughput it is better to add more nodes instead of dialing in more and more on the fast nodes.
So at the end of the day I expect both to happen. We will increase the choiceof factor and also add surge capacity. That sounds like the best of both worlds.
The ideal situation would be us getting enough space reserved. At that point this kind of gets trivial. At that point we donât need to guess which choiceof factor, long tail or surge capacity we need. We can simply test it with the remaining nodes. At that moment we have to hit the peak load just once and the TTL will make sure it will only get better and not worse from that point on.
3 Likes
Looks like your graph only updates once every hour or once every two hours.
Or⊠to make sure your own burst-capacity doesnât get filled⊠could something like the repair system constantly be trying to empty the Storj burst nodes into the regular network? So for those few important hours per day theyâre soaking up the extra data⊠but the rest of the day theyâre slowly pushing those pieces off of themselves so theyâre ready for the next surge? Kinda the opposite of a regular SNO: youâre trying for those nodes not to hold data long-term.
Itâs like the rest of the SNOs are old lead-acid batteries⊠and youâd be running a bank of capacitors: speedy when you need it but they never have to hold a lot. Clever!
2 Likes