I was casually reading storage node source code, wanting to check the details of bloom filter implementation, when I noticed one thing in the file walker process when doing GC. Currently this process goes like:
- For each 2-letter subdirectory of the satellite’s blobs directory… (
walkNamespaceInPath
)- For each file in that subdirectory… (
walkNamespaceWithPrefix
)- lstat() it,
- construct the BlobInfo object, which simply records the piece ID as decoded from the file name, storage version (also decoded from the file name) and the result of lstat(),
- call the walkFunc, which for GC is defined here:
- check the file’s modification time, as returned by lstat() above,
- check if the piece exists in the Bloom filter.
- For each file in that subdirectory… (
Now, lstat() needs to fetch data from the file’s inode, which is an additional I/O operation for each file. But checking the Bloom filter is just an in-memory operation which costs almost nothing. It would probably make sense to reverse the order of these checks so that lstat() is only called if the piece does not exist in the Bloom filter. And given the expectation that it is rare that a piece does not exist in the Bloom filter, this reversion should actually eliminate most of random I/O done during the GC process. The only I/O left would be:
- directory scans, which can be made sequential with defragmentation,
- actual garbage file deletions, which ought to be rare.
Now, I don’t know golang enough to be perfectly sure of this analysis. Besides, it’s 7am here, and I haven’t slept this night. But if I’m right, this change should make the file walker process for collecting garbage a breeze. What do you think?