Deleted pieces will be sent to trash by default

Hi SNOs,
The next software upgrade includes a change that will by default send piece deletions to the trash rather than immediately removing them from disk. This is a temporary precaution to avoid failing audits for pieces that were deleted between satellite database backups if we have to restore from a previous backup. Each node still has the option to turn this configuration off, but we highly recommend you keep it on. It is very unlikely that we will have to restore from a satellite database backup (we have never had to restore from backup thus far), but in the event we do, this will be the only way for you to avoid audit failures and potential disqualification.
Again, we understand this is not an ideal solution but we will be implementing a better solution in the coming months.
You may check out the change here

8 Likes

Just a question - If a database backup was restored would the piece deletions be moved out of trash and back into the original storage location?

Then you have to pay (fo trash storage) for your bad ideas.

Why do you want to be paid twice for storage ?

Why not store the list of recent deletes in tardigrade? Then they can be re-applied after the backup is restored

Yes that is our short term solution while we work on a better solution.

That is what the better solution would do. Storage nodes just keep the delete message and if the satellite requests an audit for a piece that was deleted the storage node can proof it with a signed delete message. Problem right now is that we didn’t had the time to implement it.

3 Likes

Why twice? Deleted = not paid. Trash = free storage.

Don’t all databases support online/hot backups these days: do you need to bring the satellites down for them? (if the problem is that backups aren’t timely: isn’t the solution just to perform them more often?)

I have no problem with having a bit more trash hanging around: I think most of us still have tons of free space :slight_smile:

2 Likes

Hot standby replicas are not backups, they are redundancy. There is a big difference.

Redundancy is for uptime – your hardware is malfunctioning or someone broke the primary config, so we switch to the replica.

Backups are for disaster recovery.

If a human accidentally runs a too-broad update/delete operation on the database, replication is going to happily ship the update to the standby and it won’t help you recover your mistake. Other forms of data loss could also be replicated to the standby replica.