Continuing the power consumption reduction journey.
I’ve started seeing the boot-up HBA/Backplane negotiation issues again, inspire of the updated backplane firmware – looks like it made it more stable, but did not eliminate the issue completely.
Scouring the interwebs; it appears there were quite a few issues lurking around mpr driver, 9500 chipset, FreeBSD pre-13.2 mpr driver issues, and the sorts. I’m using TrueNAS 13.3, which is supposed to have driver with the fixes, and the symptoms were different anyway, so it was quite puzzling.
Since the issue manifested only on boot, and in case of successful boot it worked fine, it still seemed to me some sort of early boot instability between the backplane and hba. So, I found a direct attach backplane, (SAS, not even SAS2, let alone SAS3 – plenty for HDDs anyway), BPN-SAS-826A, for $25, shipped, and replaced my fancy SAS3 expander NVME supporting backplane with this ancient crap.
I had to install second HBA-9500 8i, because I need 12 ports, so extra 4 watts, but savings on removing the smart backplane contributed another good chunk of saved power. (Why do I have two 8i and not one 16i?- because I bough the first one to use with expander backplane, and then the second one thinking the first one was defective. But then kept both because this is not sellers’ fault, why should I be returning perfectly fine hardware)
Anyway, now we are at 151W at idle-ish.
Edit. It’s in fact 147 in steady state. Splendid!
Pretty impressive. I’ll test it through a few cold boots to ensure it’s stable, and if so – power savings + stability --can’t ask for more.
Edit. Solid like an iceberg.
The only downside – boot process is much longer now. Before, smart backplane would bring each HDD up to speed as the machine was booting. Now nothign happens until driver loads in FreeBSD, and then each disk spins up in groups… so the boot is a bit prolonged.
TLDR:
- Backplane firmware update is not enough to cure 9500 HBA compatibility issues with it
- Dumb backplane consumes much less power, and should be more stable.