This would solve some of the concerns from my original post, which is great.
At least for the BFs that are currently not processing there would be no piling up as only the last Bloom filter would be saved as suggested. That’s really good.
So the only questions left would be:
- Could the newest BF somehow be merged with the one that is currently processing?
- Shall the current process be exited, the current Bloom filter deleted and restarted only with the new one?
- Should at leas when the long running retaining gets interrupted and restarted (maybe after update or so) the process continue only with the newest Bloom filter?