Failed to add bandwith usage

It’s simple and reliable enough. You may also add nodes while you have resources. If one disk die, you will lose only this small amount of common data. In case of array without redundancy (which you seems would like to suggest), the whole node will be lost, i.e. all common data will be lost.
So you do want to have a redundancy. This is mean that at least one disk will be wasted for redundancy.

By the way, usually the chocking drive could froze the whole system and zfs is not a panacea, the independent disks will likely continue to work in this case unlike zfs:

And also - how easy to extend a redundant array and rebalance it in zfs (to increase performance as usually happen in the legacy RAID)?