Yes, kind of expected. Your node win not all uploads until the success (the piece is stored on your node).
My ingress (router traffic) to growth ratio is far below 50%. I guess the average is more like 20%. For the oldest (largest) nodes it is almost zero. Most of the new data these days seems to have a lifespan of one week or less so deletes offset the incoming data.
Yeah it’s like a slow trickle for a couple days… then a bloom filter comes out that wipes almost all progress over the next 12 hours. Rinse and repeat, over and over. Less than 10% growth over the last 90 days.
Can’t complain though: staying flat is better than going backwards
Exactly.
Well it is not exactly what I would consider as an exponential growth.
Do they grow?
Only small nodes are growing (slowly). 7/9tb nodes are going to staying flat
It because 7/9tb nodes, are old, and have lot of data form free accounts, that storj still deleting time to time. fresh nodes dont have this data.
I have a different perspective on the current situation. From what I see, today’s ‘new’ traffic consists of a much higher proportion of data with an expiration date compared to the past. A significant portion of what comes in today will be deleted in 30 days or more. So, whether it’s old or new node, growth eventually stabilizes and stops at a certain point.
This is a histogram of file age (days) in the trash folder of my oldest node.
There is no indication of still ongoing free account deletes.
We also have old customers, who stores and access their data from time to time. They are paid account, and I believe they have a significantly more data in the network.
depends. My full nodes doesn’t grow, of course. But which still has a free space and has deletions as well is still slowly growing, the net grow is about 300GB/mo.
It could be possible. You can check the TTL database.
How can I check the new TTL data? What format is used for those *.dat files?
Do you mean a hashstore? Then I have no answer, this is an experimental feature, so no tools for that.
I also not sure is there an information in the stat (on the debug port - see Guide to debug my storage node, uplink, s3 gateway, satellite), you may check. I heard that there should be some metrics related to hashstore.
No. The database file piece_expiration.db seems to be no longer in use for TTL. There is a new folder piece_expirations with a number of *.dat files.
Ah, yes, it was a different fix when we have had an issue when not all records were inserted into a SQLite database under the load.
I would ask the team, is there a way to read it.
Meanwhile you may roughly estimate the amount of TTL data by the file size, also see their expiration date in the filename.
I got an answer