Configuring storage pools with ZFS

(Optional+Recommended) By-Passing the LSI Controller

• [Dangerous!] Clear the controller configuration (it will wipe ALL configuration, will cause data loss)

/opt/MegaRAID/MegaCli/MegaCli64 -CfgClr -a0

• [Optional] For old LSI controllers, we have to create a RAID0 per each disk (for some reason it does not work without RAID0, with newer controllers this is not needed)

/opt/MegaRAID/MegaCli/MegaCli64 -CfgEachDskRaid0 -a0

Map your storage disks by-partuuid, by-path or by-id

• Devices can be used with the human-friendly device name (i.e. sda, sdb, etc.). As Linux can remap this it may not map with the correct physical disk, and can carry problems if a different mapping is being performed.
• The use of by-partuuid, by-path or by-id is recommended instead of the use of device names.

/dev/disk/by-path -> This is the default method that will be used at PIC

/dev/disk/by-partuuid

/dev/disk/by-id

• For instance, for dc106.pic.es which is a SuperMicro X8DT3 we have:

[root@dc106 ~]# ls -lthra /dev/disk/by-path/
total 0
drwxr-xr-x 2 root root 2.2K Jun 25 14:25 .
drwxr-xr-x 7 root root  140 Jun 25 14:25 ..
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:18:0 -> ../../sds
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:0:0 -> ../../sda
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:1:0 -> ../../sdb
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:19:0 -> ../../sdt
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:14:0 -> ../../sdo
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:2:0 -> ../../sdc
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:17:0 -> ../../sdr
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:15:0 -> ../../sdp
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:31:0 -> ../../sdaf
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:28:0 -> ../../sdac
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:26:0 -> ../../sdaa
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:25:0 -> ../../sdz
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:23:0 -> ../../sdx
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:22:0 -> ../../sdw
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:12:0 -> ../../sdm
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:10:0 -> ../../sdk
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:35:0 -> ../../sdaj
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:32:0 -> ../../sdag
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:27:0 -> ../../sdab
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:13:0 -> ../../sdn
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:3:0 -> ../../sdd
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:6:0 -> ../../sdg
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:29:0 -> ../../sdad
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:24:0 -> ../../sdy
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:21:0 -> ../../sdv
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:16:0 -> ../../sdq
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:34:0 -> ../../sdai
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:33:0 -> ../../sdah
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:20:0 -> ../../sdu
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:5:0 -> ../../sdf
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:4:0 -> ../../sde
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:30:0 -> ../../sdae
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:9:0 -> ../../sdj
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:7:0 -> ../../sdh
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:8:0 -> ../../sdi
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:11:0 -> ../../sdl

Links of Interest

• Gluster On ZFS