Difference between revisions of "Configuring storage pools with ZFS"

From Public PIC Wiki
Jump to navigation Jump to search
(Created page with " == Links of Interest == * [http://www.gluster.org/community/documentation/index.php/GlusterOnZFS Gluster On ZFS]")
 
Line 1: Line 1:
 +
== (Optional+Recommended) By-Passing the LSI Controller ==
  
 +
* '''[Dangerous!]''' Clear the controller configuration (''it will wipe ALL configuration, will cause data loss'')
 +
/opt/MegaRAID/MegaCli/MegaCli64 -CfgClr -a0
 +
 +
* ''[Optional]'' For old LSI controllers, we have to create a RAID0 per each disk (for some reason it does not work without RAID0, with newer controllers this is not needed)
 +
/opt/MegaRAID/MegaCli/MegaCli64 -CfgEachDskRaid0 -a0
 +
 +
== Map your storage disks by-partuuid, by-path or by-id ==
 +
* Devices can be used with the human-friendly device name (i.e. sda, sdb, etc.). As Linux can remap this it may not map with the correct physical disk, and can carry problems if a different mapping is being performed.
 +
* The use of '''by-partuuid''', '''by-path''' or '''by-id''' is recommended instead of the use of device names.
 +
:'''/dev/disk/by-path''' -> '''This is the default method that will be used at PIC'''
 +
:''/dev/disk/by-partuuid''
 +
:''/dev/disk/by-id''
 +
* For instance, for dc106.pic.es which is a SuperMicro X8DT3 we have:
 +
[root@dc106 ~]# ls -lthra /dev/disk/by-path/
 +
total 0
 +
drwxr-xr-x 2 root root 2.2K Jun 25 14:25 .
 +
drwxr-xr-x 7 root root  140 Jun 25 14:25 ..
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:18:0 -> ../../sds
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:0:0 -> ../../sda
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:1:0 -> ../../sdb
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:19:0 -> ../../sdt
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:14:0 -> ../../sdo
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:2:0 -> ../../sdc
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:17:0 -> ../../sdr
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:15:0 -> ../../sdp
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:31:0 -> ../../sdaf
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:28:0 -> ../../sdac
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:26:0 -> ../../sdaa
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:25:0 -> ../../sdz
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:23:0 -> ../../sdx
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:22:0 -> ../../sdw
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:12:0 -> ../../sdm
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:10:0 -> ../../sdk
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:35:0 -> ../../sdaj
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:32:0 -> ../../sdag
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:27:0 -> ../../sdab
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:13:0 -> ../../sdn
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:3:0 -> ../../sdd
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:6:0 -> ../../sdg
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:29:0 -> ../../sdad
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:24:0 -> ../../sdy
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:21:0 -> ../../sdv
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:16:0 -> ../../sdq
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:34:0 -> ../../sdai
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:33:0 -> ../../sdah
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:20:0 -> ../../sdu
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:5:0 -> ../../sdf
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:4:0 -> ../../sde
 +
lrwxrwxrwx 1 root root  10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:30:0 -> ../../sdae
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:9:0 -> ../../sdj
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:7:0 -> ../../sdh
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:8:0 -> ../../sdi
 +
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:11:0 -> ../../sdl
 +
  
 
== Links of Interest ==
 
== Links of Interest ==
 
* [http://www.gluster.org/community/documentation/index.php/GlusterOnZFS Gluster On ZFS]
 
* [http://www.gluster.org/community/documentation/index.php/GlusterOnZFS Gluster On ZFS]

Revision as of 13:24, 25 June 2015

(Optional+Recommended) By-Passing the LSI Controller

  • [Dangerous!] Clear the controller configuration (it will wipe ALL configuration, will cause data loss)
/opt/MegaRAID/MegaCli/MegaCli64 -CfgClr -a0
  • [Optional] For old LSI controllers, we have to create a RAID0 per each disk (for some reason it does not work without RAID0, with newer controllers this is not needed)
/opt/MegaRAID/MegaCli/MegaCli64 -CfgEachDskRaid0 -a0

Map your storage disks by-partuuid, by-path or by-id

  • Devices can be used with the human-friendly device name (i.e. sda, sdb, etc.). As Linux can remap this it may not map with the correct physical disk, and can carry problems if a different mapping is being performed.
  • The use of by-partuuid, by-path or by-id is recommended instead of the use of device names.
/dev/disk/by-path -> This is the default method that will be used at PIC
/dev/disk/by-partuuid
/dev/disk/by-id
  • For instance, for dc106.pic.es which is a SuperMicro X8DT3 we have:
[root@dc106 ~]# ls -lthra /dev/disk/by-path/
total 0
drwxr-xr-x 2 root root 2.2K Jun 25 14:25 .
drwxr-xr-x 7 root root  140 Jun 25 14:25 ..
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:18:0 -> ../../sds
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:0:0 -> ../../sda
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:1:0 -> ../../sdb
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:19:0 -> ../../sdt
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:14:0 -> ../../sdo
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:2:0 -> ../../sdc
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:17:0 -> ../../sdr
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:15:0 -> ../../sdp
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:31:0 -> ../../sdaf
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:28:0 -> ../../sdac
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:26:0 -> ../../sdaa
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:25:0 -> ../../sdz
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:23:0 -> ../../sdx
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:22:0 -> ../../sdw
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:12:0 -> ../../sdm
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:10:0 -> ../../sdk
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:35:0 -> ../../sdaj
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:32:0 -> ../../sdag
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:27:0 -> ../../sdab
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:13:0 -> ../../sdn
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:3:0 -> ../../sdd
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:6:0 -> ../../sdg
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:29:0 -> ../../sdad
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:24:0 -> ../../sdy
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:21:0 -> ../../sdv
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:16:0 -> ../../sdq
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:34:0 -> ../../sdai
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:33:0 -> ../../sdah
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:20:0 -> ../../sdu
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:5:0 -> ../../sdf
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:4:0 -> ../../sde
lrwxrwxrwx 1 root root   10 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:30:0 -> ../../sdae
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:9:0 -> ../../sdj
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:7:0 -> ../../sdh
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:8:0 -> ../../sdi
lrwxrwxrwx 1 root root    9 Jun 25 14:25 pci-0000:05:00.0-scsi-0:2:11:0 -> ../../sdl

Links of Interest

Retrieved from "http://pwiki.pic.es/index.php?title=Configuring_storage_pools_with_ZFS&oldid=6"