My pc asked me to switch the timezone. guess storj ingress beams my pc through time and space 
thats genius! why didnt i think of that 
2 Likes
Thatās how Storj increased MB/sā¦ they kept the MB the sameā¦ but made the seconds take less time 
1 Like
Iām seeing a lot of uncollected garbage pile up again as a result of this testing.
I have large GC runs ongoing now. I can think of a few reasons why this might happen. It could just be that I get to keep fewer pieces than I thought and they didnāt finish fast enough. But that seems to be a large amount if itās just that. It could also be that some data got deleted early on the satellite end. @littleskunk could you comment on that if you know?
Yep, noticed the same. Gave up on participating in races few days ago because of that, no value for me in it.
They said deleting the forever-free data would take several monthsā¦ so maybe all this GC is just those small batches of deletions going through every few days?
If this test is the new normal shouldnāt there be always like 25% of storage used for trash? I mean its 30 days TTL and once it is moved to trash it will be there for another 7 days at least.
Or is TTL data not moved to trash? 
TTL data is directly deleted: no vacation in the trash folder 
2 Likes
This month looks really good data wise. Although I lost contact to 4 nodes which are located in another country (burned potato :D) last month. My 3 other nodes already got 5.6 TB ingress in the last 6 days. In the past with all nodes working my maximum ingress was a little more than 4TB per month.
So, good job on the Test and related updates to the storj software and satellites.
1 Like
Here is the fix: https://review.dev.storj.io/c/storj/storj/+/13456
1 Like
Just double-checking my inference. GC is catching piece within the 24 hours it takes until expiration cleans them up? This will now be reduced to 1 hour?
I thought bloom filters were always based on several days old snapshots. How could it even delete data within that 24 hour period?
I forgot to give an update on the test results from yesterday.
It looks like we are already running the RS numbers with the lowest storage expansion factor and the lowest long tail. There isnāt much I can do to make it more efficient without reducing the throughput below our target.
We found out that the more we fill the network with data the lower the throughput will get. It will fall below our target over time. We have some variables that allow us to get the performance above the target again by increasing the long tail or the choiceof factor.
Now our hope is also that by keeping up the load we can motivate more nodes to join the network. At the current rate we would overshoot the required capacity at some point so it is time to change the load pattern to a more dynamic one that hits the target throughput for a few hours per day and reduces it a bit for the rest of the day to match the expected usecases as good as possible. That will be the moment you can start planning to add more nodes
8 Likes
Check your trash folder. For some satellites I had 3 bloom filters in the last 7 days. So they must have changed it.
1 Like
Is it normal to have 2 bloom filters for a single satellite that are 30 mins apart as per their timestamp ?
My nodes have lost almost half of data from the peak. Your 30 days TTL test data makes it not more shrinking at best.
I guess it needs some real growth to motivate me. Or at least a signed contract from one of those big customers you talking about.
3 Likes
If you keep up load that disappear in 30 days youāll demotivate SNOs for the extra effort (networking and caching, especially if you have multi nodes) for the same earnings. You need to fill that nodes with data without TTL or with a growing path including TTL to motivate SNOs.
Are you going to grow data uploaded in this way?
3 Likes
That would be the wrong set of SNOs. We need SNOs that join the network accepting the TTL data because that will be uploaded later on.
2 Likes
But if our disks is not getting filled, and our disks is just getting hammered and we dont get payed for this why would we expand? Can you eleborate on this?
1 Like
I canāt speak for everyone in the room. I can only speak for myself. For my own nodes I did the math. I donāt know why your math ends up with 0 payout but for my node the outcome is a decent payout that I am getting from this usecase. Luckily my setup is already perfect. I have way too much hard drive capacity and now I can make use of that. I have enough memory to not worry about garbage collection. It takes just minutes. The only variable that limits my payout is my bandwidth. I ordered a faster internet connection and soon my ISP will just flip the switch on his side. I am fine with the new situation and I am happy to take the money that you donāt want. Or maybe you havenāt done the math yet.
2 Likes
I thought Storj was going to try to maintain a certain amount of reserved-capacity (like 5PB or something, just to guess a number) and then any other growth would be natural/customers? So essentially Storj is pre-paying SNOs for some space nowā¦ to make sure they have time to add disks before natural demand catches up?
So yes that would mean SNOs grow faster for awhileā¦ but will level-off laterā¦ so every day fresh reserved-capacity ingress only covers reserved-capacity TTL deletes.
If thatās the plan, or close to it, Iām fine with it.
1 Like
This forum really needs a dislike button.
8 Likes