On the impact of two-level directory structure for storing blobs

Well, yes. But back then (when v3 entered public testing) there were a lot of unknowns and somebody in the chat even told me that the node could be used to store lots of very small files (I do not remember what size they said, but I remember that it was something like 10 bytes or whatever) and I also did not know about the inode count limit of ext4. So yeah, learning experience. At least the node was 3TB or something at the time when I ran into the problem, so it did not take very long to rsync it to the new virtual disk.
It turns out that the minimum piece size the node stores is 512B (with the occasional 10B piece a few times per day), so the defaults are more than enough.

Could be, I only remember it from 10 years ago when with thick provisioning lvm snapshots would make a serer with SSDs slow down to a crawl, which was “fun” because we used the snapshots for backups, so if backups do not complete overnight and there was a snapshot remaining in the morning customers got upset. ZFS fixed that.
I do not use lvm anymore, unless it’s on some old server that was installed with lvm and is still used. If I want snapshots and store a lot of data I’ll use zfs and if I don’t need snapshots and for system drives I can use ext4 (on top of md raid if it’s not a VM).

Slows down all regular I/O, i.e. uploads, downloads, file walkers, etc. With default lvmthin parameters—quite a lot. If you choose a large chunk size, then it might actually be workable, at the cost of disk space obviously.

1 Like