Drop Bloom filters, bring back direct deletes

[jammerdan]

@Alexey

And how is it supposedly should be deleted on others?

Yes, that is exactly the question, of how to keep going the deletion even in the background.
I have made suggestions for that:

1. Customers uplink sends out as many deletion requests as it is fine with. Could be 0, 1 … n
2. Customers uplink passes the request to at least 1 node and node passes it on p2p: 1, 2, 4, , 16, … This is where we would need definitely an acknowlede by the node so the request does not get lost.
3. The satellite sends out the deletion requests on behalf of customers uplink
4. The satellite puts the information which pieces to delete into a log and nodes download this log
5. Always: Final clear-out with Bloom filter

@Alexey

And here is a timeout again, the node could be hang, offline, overloaded or restarted.

Yes, but we may not even need the acknowledgement from the node. I think we should have it definitely if it is p2p. But maybe not even then as long as the satellite knows about the file deletion request. The nodes that have successfully received the request will delete the piece and send orders to confirm deletion so the satellite knows what has been deleted and what is still there. Those that have not received the deletion request from uplink will receive deletion request or deletion log from the satellite and if they miss even that final delete with a Bloom filter.

@Alexey

It still adds a latency, when the node didn’t ack the request or ack it too late

Most important ack is from satellite. Then subsequently it may be enough that customers uplink spits out as many requests to nodes as it is fine with without waiting for an acknowledgment.
But even if we wait for 1 (the fastest) acknowledgement, it would be as fast as if downloading a file today. I don’t know what the average time to first byte is on downloads today. But an acknowledgement from a single node should not take longer.

@Alexey

But how it would know about this nuance, if the information that this segment also has more than a one metadata record is not yet written to the database, and the deletion request just received an information that there only one metadata record?

Yes I see that.
Let me first quote this:

@BrightSilence

It would be much better if they ran some logic on the satellite to determine which segments are fully deleted and still send that list to nodes.

I don’t know how quickly the satellite can determine which pieces to delete or when it would be safe to do so. Ideally it would pass the correct list to the customers uplink so that it would be safe to delete anything that uplink has received. If that is not possible and we rely on the satellite to coordinate the deletion anyway, then I would say the satellite should know which pieces can be deleted and which pieces need to stay and either send out requests or prepare the logs accordingly.