Nodes shrinking

[Roberto]

As long as statistics are common, there will never be transparency on how much data is on Global and how much is on Select, they should be divided

[Alexey]

No, but EU Select and AP Select have much less impact up to no existence.

[Vadim]

Ingress amount on this stats are not really corelated with real life. When was low Ingress days last week, stats shown that Ingress was same high like before.

[littleskunk]

@jammerdan

It is not just about wanting constant growth. It is about Storj telling us there is constant growth. They keep saying they are operating in one of the biggest and fastest growing markets, that there is rapid revenue growth, that there is constant growth, that there is a strong sales pipeline with petabyte customers and that almost no existing customers leaving them instead they put more and more data into Storj. Source: Current and recent town halls.

I can confirm one more time that the network is growing. The missconception in this thread is that a growing network would mean all nodes are growing. It doesn’t even mean that the nodes on average are growing. Your node might be shrinking while my node is growing. This is totally fine.

@jammerdan

Unfortunately Storj does not disclose for which of their product offering this holds true.

What you are looking for was disclosed here: Data Cleanup Plan - Update

Please note that this was for the public network only and doesn’t include storj select. It is exactly what you wanted to see. Just that it doesn’t line up with your node and so you can’t verify it that easy.

[Vadim]

Does Storj stiil deleting this old data? I see very normal ingress but i have every week 25-30tb in trash.

[littleskunk]

@Vadim

Does Storj stiil deleting this old data?

What do you mean with old data? Storj deletes unpaid data. Most of that is comming from expired free trial accounts. A small amount also from paid accounts with unpaid invoices. Most of the steps are fully automated by now so it will be more like an ongoing monthly cleanup.

@Vadim

I see very normal ingress but i have every week 25-30tb in trash.

That is most likely paid customer data. If the customer deletes or overwrites files they do end up in trash. More customers with short living data means a higher percentage of trash.

[Roxor]

@littleskunk

Your node might be shrinking while my node is growing. This is totally fine.

@littleskunk

More customers with short living data means a higher percentage of trash.

When all the test data was going out: if I remember correctly part of the tuning was to give satellites a bit of a memory as to what nodes were fast: to steer uploads towards them: so customers saw better performance?

If so, then as we see more short-lived data: that means more and more opportunities for new data to be sent to “fast” nodes. So nodes that may be higher-latency (such as those behind VPNs, or in geos far-from-the-customer (like US customers talking to European nodes)) will still feel the effects of deletion of old/free/unpaid data… but will naturally receive less new data to offset it.

So it’s kinda winner-takes-most: fast nodes can soak up most of the network growth. And those SNOs won’t be in here complaining . But that’s how it should be: customers should always use fast nodes first and only the slower nodes if the fast nodes are full and rejecting uploads.

[Julio]

@Roxor

satellites a bit of a memory as to what nodes were fast

Just keep swimming, just keep swimming - … 10 minutes later. Just keep swimming…

2 cents,
Julio

[Alexey]

@Roxor

When all the test data was going out: if I remember correctly part of the tuning was to give satellites a bit of a memory as to what nodes were fast: to steer uploads towards them: so customers saw better performance?

The test data is only on Saltlake, and it’s so small that you can think it’s deleted completely. But likely you mean the unpaid data. It’s an ongoing regular process - some accounts expiring, some stopped to pay:

@littleskunk

Storj deletes unpaid data. Most of that is comming from expired free trial accounts. A small amount also from paid accounts with unpaid invoices. Most of the steps are fully automated by now so it will be more like an ongoing monthly cleanup.

@Roxor

so customers saw better performance?

The speed for the customer depends on the number of nodes which it’s used to upload or download, which in return depends on a R/S numbers (they now can be individual on per-customer basis), the used backed on the nodes and their closeness to the customer (regarding the response time). The latency to the satellite likely doesn’t make any difference for uploads/downloads, but may affect the listing.

@Roxor

So nodes that may be higher-latency (such as those behind VPNs, or in geos far-from-the-customer (like US customers talking to European nodes)) will still feel the effects of deletion of old/free/unpaid data… but will naturally receive less new data to offset it.

I think so too.