Extensible software production-ready RAID in Linux?

My #1 goal with RAID, is redundancy from failure. My #2 concern is scalability. Because ZFS’s RAID doesn’t yet allow you to add a disk to the raid pool, and BTRFS RAID6 isn’t stable what are my options?

  • It seems like the best compromise I can see is Raid 1 (mirroring) going to Raid 10 Mirror/Striping. Assuming each drive is $300, If I do this, it’s my understanding I’ll be able to scale from 2 disks ($600), then 4 disks ($1200), then 6 ($1800) disks. Each time creating a bigger raid 10 volume. However, with this method I’ll have to always maintain 50% disk parity.
  • Whereas, if I wanted to spend the $1,800 up front and launch with Raid-z2 I’ll be able to suffer any two failures of drives. And rather than having 3-drives usable, I’ll have 4-drives usable. But that such a solution will always be substantially less performant?

Did I summarize these options correctly? Is there a production ready scalable Raid-6 solution that’s better than ZFS? I’d be great be able to buy three drives for raid6, and scale up one drive at a time re balancing as I scale.

Asked By: Evan Carroll

||

Yes, your summary is correct.

And yes, 3 mirrored vdevs will give you better performance than a 6-drive raidz2 (roughly triple the IOPS – IOPS scale with the number of vdevs). This might not matter much if you’re mostly doing large sequential reads & writes (e.g. video files), or if the pool is mostly intended for archiving/backing-up data….but if it’s for a heavy database workload, then the more IOPS you can get, the better.

BTW, another option that might be worth considering is to buy 3 drives at a time and end up with two striped 3-drive raidz1 vdevs in your pool. Same capacity as the raidz2, and better performance (because there are two vdevs) – the only downside is that it’s raidz1 rather than raidz2 (so you can only afford to lose a maximum of one drive from each vdev).


Personally, I like to use mirrored vdevs as much as possible (certainly on boot/OS pools, which I build with SSD or, these days, NVME drives), and only use raidz for bulk storage (especially when performance isn’t critical, although I usually improve that by having lots of RAM for ARC and using some partitions on my SSD or NVME drives as SLOG and/or L2ARC for HDD pools). If I could afford it, I’d use mirrors for everything but it basically means paying double for storage.

Answered By: cas

Synology boxes use LVM to provide a RAID array you can just add disks to – but I don’t know the details of how this works under the hood. It’s probably something similar to this. Meanwhile md allows you to add disks. In both scenarios, you lose a lot of the benefits of using ZFS.

Answered By: symcbean
Categories: Answers Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.