Yes, as far as I know. However, there is not a lot of deletions for some reason. The last BF was at Jan 28 on my nodes.
2025-01-28T02:07:03Z INFO piecestore New bloomfilter is received {"Process": "storagenode", "satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "creation": "2025-01-27T14:57:30Z"}
2025-01-28T07:39:13Z INFO piecestore New bloomfilter is received {"Process": "storagenode", "satellite": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "creation": "2025-01-26T01:57:58Z"}
2025-01-28T20:34:04Z INFO piecestore New bloomfilter is received {"Process": "storagenode", "satellite": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "creation": "2025-01-24T20:26:19Z"}
Interesting. My node with hashstore enabled (AP1 is the only satellite on that node) does not have that text in the log file, But my other nodes do. This node has data saved in both the piecestore and hashstore because it is not doing active migration, just passive migration and new data writes.
I think each storage backend uses own wordings for typical for that backend’s chores.
I believe that you need to search for “compact”. I think they are mutually exclusive, so piecestore BF processing likely disabled due to migration to a hashstore backend.
And also I think we never tested the slow migration + BF processing. On Select they are separate processes running in the different time.
Yeah, AP1 doesn’t provide a huge amount of ingress, but interestingly, it’s still more than SLC—and SLC does send out bloom filters to my nodes. Meanwhile, I haven’t received a BF from AP1 in over a month now. So it’s a bit odd that AP1 isn’t sending any.
AP1 might be a small satellite but percentage wise the amount of uncollected and unpaid trash must be huge now. What ist the reason for AP1 not sending bloom filters @Alexey ?
Can’t this BF be segmented on time periods?
Like generate a partial BF only for pieces older than 3 years; than generate one for pieces 2 to 3 years old; one for 1 to 2 years old; one for pieces under 1 year.
More likely, BF for subdirectories 22…2z, then another for 3a…3z, etc. Would require node code changes though.
But, this process is only bound by RAM because the implementation is deliberately simple. There is a proof-of-concept piece of code that does it with rather insignificant RAM usage.
And they sent a second bloom filter just 11 hours later. Looking good. The first bloom filter resulted in 17.49GB of trash. The second brought it up to 19.93GB.
2025-02-11T02:48:59Z INFO piecestore New bloomfilter is received {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "creation": "2025-02-10T22:57:52Z"}
2025-02-11T13:55:47Z INFO piecestore New bloomfilter is received {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "creation": "2025-02-11T02:28:59Z"}