I have DBs and logs on a separate ZFS SSD mirror with other settings 
You are right, thanks - will update settings sometime.
2 Likes
Need some input from the ZFS masters here with their special devices.
I currently have a few nodes on single-disk ZFS setups (i.e. no redundancy because itās just some linux ISOs and the Storj node).
I have 2 SSDs that I can set up in some sort of mirror (i.e. LVM/mdadm/) and just partition it so that each single-disk pool gets a chunk of of the mirror.
However, I am anticipating at some point that these disks will fail and I will end up replacing with likely higher capacity drives, or potentially move the disk to a different system, etc. Donāt really want to deal with contiguity requirement for partitions. Flexibility is somewhat important here.
Iāve been messing around and it seems you can use zvols on ZFS mirrors as well as flat files as the special vdev. Thus I can move around these āblock devicesā with ease, create snapshots, use remaining space for other files, revert back to simple partition setup, etc at the cost of additional logistical and file system complexity.
Thus I think zvols may be a good middle ground. My single disk vdev pools are setup with ashift=12, recordsize=128K, but Iām not sure what would be an appropriate value to set for volblocksize if I wanted to use zvols. As far as I understand there will be read amplification if I use too large of a block size.
If we take this example output from zdb thatās not from my pool:
5948872 164102062592 32285892096 74362949632 12500 5.08 0.33 Metadata Total
There are 5948872 blocks and allocated size is 74362949632, so the average block size for metadata is around 12KiB and thus Iād choose something like 16K for volblocksize (which happens to be the default value)?
Edit: If I use anything other than 128K or 4K for volblocksize, it will say One or more devices are configured to use a non-native block size. Expect reduced performance. Right now setting it to 4K to I guess mimic a SSD with 4K blocks.