Yes, kind of expected. Your node win not all uploads until the success (the piece is stored on your node).
My ingress (router traffic) to growth ratio is far below 50%. I guess the average is more like 20%. For the oldest (largest) nodes it is almost zero. Most of the new data these days seems to have a lifespan of one week or less so deletes offset the incoming data.
Yeah it’s like a slow trickle for a couple days… then a bloom filter comes out that wipes almost all progress over the next 12 hours. Rinse and repeat, over and over. Less than 10% growth over the last 90 days.
Can’t complain though: staying flat is better than going backwards ![]()
Exactly.
Well it is not exactly what I would consider as an exponential growth.
Do they grow? ![]()
Only small nodes are growing (slowly). 7/9tb nodes are going to staying flat
It because 7/9tb nodes, are old, and have lot of data form free accounts, that storj still deleting time to time. fresh nodes dont have this data.
I have a different perspective on the current situation. From what I see, today’s ‘new’ traffic consists of a much higher proportion of data with an expiration date compared to the past. A significant portion of what comes in today will be deleted in 30 days or more. So, whether it’s old or new node, growth eventually stabilizes and stops at a certain point.
This is a histogram of file age (days) in the trash folder of my oldest node.
There is no indication of still ongoing free account deletes. ![]()
We also have old customers, who stores and access their data from time to time. They are paid account, and I believe they have a significantly more data in the network.
depends. My full nodes doesn’t grow, of course. But which still has a free space and has deletions as well is still slowly growing, the net grow is about 300GB/mo.
It could be possible. You can check the TTL database.
How can I check the new TTL data? What format is used for those *.dat files?
Do you mean a hashstore? Then I have no answer, this is an experimental feature, so no tools for that.
I also not sure is there an information in the stat (on the debug port - see Guide to debug my storage node, uplink, s3 gateway, satellite), you may check. I heard that there should be some metrics related to hashstore.
No. The database file piece_expiration.db seems to be no longer in use for TTL. There is a new folder piece_expirations with a number of *.dat files.
Ah, yes, it was a different fix when we have had an issue when not all records were inserted into a SQLite database under the load.
I would ask the team, is there a way to read it.
Meanwhile you may roughly estimate the amount of TTL data by the file size, also see their expiration date in the filename.
I got an answer
