There are 12,700 subnets with nodes - how is traffic distributed?

[dzeto]

Hi, there are currently around 12,700 subnets running Storj.

If all of them are receiving data (assuming each subnet has at least one active node with a free space), that would mean each subnet gets about 6.29 TB per month, or roughly 200 GB per day, based on the total 80 PB of traffic over 30 days. The 80 PB figure is just a rough number from Storj stats page.

However, 75 PB of that traffic was in the US, and only 5 PB in the EU.

Does that mean US nodes are getting significantly more data than EU ones?

Because my EU nodes don’t reach 200 GB of traffic per day on any clean EU subnet. I wonder where all this traffic shown on the Storj stats page actually goes.

Thanks!

[jammerdan]

@dzeto

I wonder where all this traffic shown on the Storj stats page actually goes.

I guess this answers your question:

@nerdatwork

I hope you remember it also counts Storj Select data too.

@Ottetal

• Vivint is doing great. But Vivint is still on StorJ Select - so this is not directly relevant to the average SNO

[Alexey]

However, 75PB not all US1 Select only space. There are customers who uses a similar amount of data.

[jammerdan]

Maybe you want to read more about the Vivint ride and how it (sadly) turned out for the SNOs. It started with great hopes and testing and tweaking for the then unnamed customer here Updates on Test Data

@Bryanm

We don’t expect that this load is temporary, and may in fact be the new normal.

And found an unfortunate end for the SNOs here:

@Bryanm

We had anticipated that this customer would upload to the public network, and as such we did crucial performance testing and scaling on SLC with the assistance of our community. These tests were crucial to our winning this business and we appreciate the input and support we received. As we were working with this publicly traded company, at the last minute they mandated the use of the Select tier for their data, and they have already begun onboarding data to that product. We are saddened that this was the outcome and that our expectations that this deal would lead to growth on the public network didn’t materialize.

@littleskunk

The customer decided last minute that the public network doesn’t work for them. We even explained to them that storj select is running the same code so there is no difference. I can say in this instance we did our best to push them using the public network but their response was no way we need storj select or we have to bail out.

[Roxor]

If most of the paid customer data originates in the US, and customer software prefers the fastest nodes it knows about… and Satellites also prioritize those fast nodes (when routing new data) - then I’m not surprised US-based SNOs get more traffic/day.

All else being equal: I’d expect US nodes to win more races as that customer data doesn’t have the added latency of bouncing back and forth across oceans.

[Toyoo]

@dzeto

Does that mean US nodes are getting significantly more data than EU ones?

Probably, but data you quote is not enough to make this conclusion.

Do not mistake location of a satellite with location of a customer, or traffic. us1 uses nodes in EU, and eu1 uses nodes in the States. We can probably say that American customers will prefer us1, and European eu1, but if they’re global companies, that doesn’t mean they will generate traffic only from their countries, even if they pick a single satellite.

On top of that, Storj has recently introduced a storage tier that actively rebalances pieces across the world, so that download performance is good wherever you are. We are actually observing elevated repair traffic for about two months now. We don’t know what rules exactly are implemented for this storage tier, and there are more nodes in Europe than in America, but this still suggests data should rebalance at least a bit.

[Roxor]

@Toyoo

Do not mistake location of a satellite with location of a customer, or traffic. us1 uses nodes in EU, and eu1 uses nodes in the States. We can probably say that American customers will prefer us1, and European eu1, but if they’re global companies, that doesn’t mean they will generate traffic only from their countries, even if they pick a single satellite.

At the same time: a customers preference in satellite doesn’t change the result that local SNOs will be lower latency, and win more races. Like a US-uploading-customer, using the EU satellite, will still get provided with a diverse set of possible nodes to upload too… which will likely include US nodes local-to-them… and those US SNOs will likely capture more of those uploads than EU nodes.

If the end uploads are always direct from customer-to-SNO… then no matter what-satellite-recommends-what-nodes: local=faster. In some sense the satellite is just making sure they’re not all local (especially in the case of the new “Production Cloud Global” tier Toyoo mentioned)

[agente]

Any feedback from USA node operators? montly ingress/egress?

[Roxor]

I’m seeing about the same numbers as last year. A node growing around 3TB/year sounds normal: and 4TB+ would be great!

[EasyRhino]

I wonder how many clients use US1 even though they are not in US.

[Julio]

RE: US1
Last quarter my stats = no real growth. Last two months, significant repair data: +12%, +17%, and currently this month so far +50%! more egress than ingress. Ingress can surge 6-7% monthly but will delete 5-7% in the same month on a daily basis, ie: ~net 0 stored.

The churn of data seems to be about 27% (last month) of stored amount at the moment.

So, if you have 10TB @ $1.49 = $14.90.
Data churn = 2.7TB INGRESS (27%) x 150% = 4.05TB EGRESS/REPAIR x $2 = $8.10
TOTAL: $23.00

This amounts to extra pay of about 54%-8% = +46% more than normal! I am happy they are increasing their RS shards while keeping SNOs well fed, even without much growth.
Normal repair/egress seemed to be about 6-8% / month over the last year, thus the 8% correction shown in the calc. above.

2 cents,

Julio
P.S. That’s only a somewhat accurate end of month estimate, if this repair data continues.