I would like to discuses about trash holding time.
SNO have to hold 7 days data that not paid. I understand that there was times when there was tests of some deletion processors and to not lose data there was need of this, but it was long time ago. So why we still hold deleted data 7 days? node is full and 100gb deleted in trash for 7 days not give to enter any new data. if I count all deleted data in my nodes it will be several TB not paid data.
May be 1-2 days will be enough, if there is good reason and it is just make any sense? if clients want to have some convenience of error from deletion, may be it can be a feature that need to pay for possibility to restore deleted file? But then it should be paid.
3 Likes
If I understand correctly, this is not for customers. This is for recovery in case of satellite bugs, like when a satellite sends a wrong bloom filter. This is scary enough, as in case of bad bloom filters it could impact significant amount of data.
It may be terabytes, but at your scale it’s probably still less than 1% of data you store?
Frankly, I’m more worried about the superfluous I/O that delayed deletion requires…
5 Likes
@Toyoo is right. This isn’t for customers. There was already a bug in server side copy that caused a race condition, resulting in bloom filters being sent out that didn’t contain new pieces, making the nodes delete them incorrectly. In scenarios like that, Storj Labs has 7 days to become aware of the issue, identify the pieces that need to be restored and write the code to restore them. 7 days is not a lot of time to do that. I don’t see a way to lower that and not risk data loss in such scenarios.
5 Likes
Then a I wrote to before it will be nice feature for clients also. clients also make errors delete sometimes big amount of data exigently, possibility to restore it good feature, that can make additional income.
1 Like
end user trash = normal stored data on node operator side, becouse if he would like to recover it, the data has to be there. As soon as the end user (or his service provider actually) empty trash, the data can be deleted on node operator side.
node operator trash = deleted, but recoverable data from Storj point of view as Toyoo and BrightSilence wrote
2 Likes