RH: Multipathing NetApp LUN on RHEL 5.3

 

  1. Make sure you have the packages needed by RHEL, otherwise install them.

    rpm -q device-mapper
    rpm -q device-mappermultipath
    yum install device-mapper
    yum install device-mapper-multipath

  2. Install QLogic Drivers if needed, or utilize RHEL drivers. In my case I am using HP C-Class blades with Qlogic HBA cards. HP drivers can be found at the HP site, driver is called hp_sansurfer. I am utilizing RHEL built in drivers, but you can install the HP/QLogic drivers as follows:

    rpm -Uvh hp_sansurfer-5.0.1b45-1.x86_64.rpm

  3. If Qlogic HBA, install the SanSurfer CLI, this is very useful program for doing things with QLogic HBA cards, it can be downloaded at QLogic website, install as follows:

    rpm -Uvh scli-1.7.3-14.i386.rpm

  4. Install NetApp Host Utilities Kit, the package is a tar.gz file, you can find it at the now site http://now.netapp.com.Open it and run the install shell script

    netapp_linux_host_utilities_5_0.tar.gz

  5. Once Everything is installed on the host, create the LUN and ZONE it from the NetApp, Brocade(SAN Fabric),Host

    To find your WWPNs, use the scli as follows:
    # scli –i all
    // Use the WWPN numbers for the iGroup and Brocade Aliases

  6. Once it has been Zoned and mapped correctly, verify if your RHEL host can see it.

    // Rescan HBA for new SAN Luns

    # modprobe -r qla2xxx
    # modprobe qla2xxx
    // Check the kernel can see it
    # cat /proc/scsi/scsi
    # fdisk –lu

  7. Utilize NetApp tools to see LUN connectivity

    // Check your host and utilities see the LUNs
    [root@server ~]# sanlun lun show
    controller: lun-pathname device filename adapter protocol lun size lun state
    NETAPPFILER: /vol/servervol/serverlun /dev/sdf host6 FCP 100g (107374182 400) GOOD
    NETAPPFILER: /vol/servervol/serverlun /dev/sda host4 FCP 100g (107374182 400) GOOD
    NETAPPFILER: /vol/servervol/serverlun /dev/sde host6 FCP 100g (107374182 400) GOOD
    NETAPPFILER: /vol/servervol/serverlun /dev/sdc host5 FCP 100g (107374182 400) GOOD
    NETAPPFILER: /vol/servervol/serverlun /dev/sdd host5 FCP 100g (107374182 400) GOOD
    NETAPPFILER: /vol/servervol/serverlun /dev/sdb host4 FCP 100g (107374182 400) GOOD

  8. Utilize NetApp tools to check multipathing, not set yet

    [root@server ~]# sanlun lun show -p
    NETAPPFILER:/vol/servervol/serverlun (LUN 0) Lun state: GOOD
    Lun Size: 100g (107374182400) Controller_CF_State: Cluster Enabled
    Protocol: FCP Controller Partner: NETAPPFILER2
    Multipath-provider: NONE
    ——— ———- ——- ———— ——————————————— —————
    sanlun Controller Primary Partner
    path Path /dev/ Host Controller Controller
    state type node HBA port port
    ——— ———- ——- ———— ——————————————— —————
    GOOD primary sdf host6 0c —
    GOOD secondary sda host4 — 0c
    GOOD secondary sde host6 — 0c
    GOOD secondary sdc host5 — 0d
    GOOD primary sdd host5 0d —
    GOOD primary sdb host4 0c —

    Time to configure multipathing

  9. Start the multipath daemon

    # service multipathd start

  10. Find you WWID, this will be needed in the configuration if you want to alias it.
    Comment out the blacklist in the default /etc/multipath.conf, otherwise you will NOT see anything.

    #blacklist {
    # devnode “*”
    #}

    // Show your devices and paths, and record the WWID of the LUN
    # multipath -v3


    ===== paths list =====
    uuid hcil dev dev_t pri dm_st chk_st vend/pr
    360a98000486e576748345276376a4d41 4:0:0:0 sda 8:0 1 [undef][ready] NETAPP,
    360a98000486e576748345276376a4d41 4:0:1:0 sdb 8:16 4 [undef][ready] NETAPP,
    360a98000486e576748345276376a4d41 5:0:0:0 sdc 8:32 1 [undef][ready] NETAPP,
    360a98000486e576748345276376a4d41 5:0:1:0 sdd 8:48 4 [undef][ready] NETAPP,
    360a98000486e576748345276376a4d41 6:0:0:0 sde 8:64 1 [undef][ready] NETAPP,
    360a98000486e576748345276376a4d41 6:0:1:0 sdf 8:80 4 [undef][ready] NETAPP,

     

  11. Now you are ready to configure /etc/multipath.conf

    Exclude (blacklist) all the devices that do not correspond to any
    LUNs configured on the storage controller and which are mapped to
    your Linux host. There are 2 methods:
    Block by WWID
    Block by devnode
    In this case I am blocking by devnode since I am using HP and know my devnode RegEx
    Also configure the device and alias(optional).
    The full /etc/multipath.conf will look like this:

    defaults
    {
    user_friendly_names yes
    max_fds max
    queue_without_daemon no
    }
    blacklist {
    ###devnode “*”
    devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*”
    devnode “^hd[a-z]”
    devnode “^cciss!c[0-9]d[0-9]*” # Note the cciss, usual in HP
    }
    multipaths {
    multipath {
    wwid 360a98000486e57674834527533455570 # You found this
    alias netapp # This is how you want to name the device in your host
    # server LUN on NETAPPFILER
    }
    }
    devices
    {
    device
    {
    vendor “NETAPP”
    product “LUN”
    getuid_callout “/sbin/scsi_id -g -u -s /block/%n”
    prio_callout “/sbin/mpath_prio_ontap /dev/%n”
    features “1 queue_if_no_path”
    hardware_handler “0”
    path_grouping_policy group_by_prio
    failback immediate
    rr_weight uniform
    rr_min_io 128
    path_checker directio
    flush_on_last_del yes
    }
    }

  12. Restart multipath and make sure it starts automatically:

    // Restart multipath
    # service multipathd restart
    // Add to startup
    # chkconfig –add multipathd
    # chkconfig multipathd on

  13. Verify multipath is working

    //RHEL tools
    [root@server scli]# multipath -l
    netapp (360a98000486e576748345276376a4d41) dm-2 NETAPP,LUN
    [size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
    \_ round-robin 0 [prio=0][active]
    \_ 4:0:1:0 sdb 8:16 [active][undef]
    \_ 5:0:1:0 sdd 8:48 [active][undef]
    \_ 6:0:1:0 sdf 8:80 [active][undef]
    \_ round-robin 0 [prio=0][enabled]
    \_ 4:0:0:0 sda 8:0 [active][undef]
    \_ 5:0:0:0 sdc 8:32 [active][undef]
    \_ 6:0:0:0 sde 8:64 [active][undef]

    //NetApp utilities Tool
    [root@server scli]# sanlun lun show -p
    NETAPPFILER:/vol/servervol/serverlun (LUN 0) Lun state: GOOD
    Lun Size: 100g (107374182400) Controller_CF_State: Cluster Enabled
    Protocol: FCP Controller Partner: NETAPPFILER2
    DM-MP DevName: netapp (360a98000486e576748345276376a4d41) dm-2
    Multipath-provider: NATIVE
    ——— ———- ——- ———— ——————————————— —————
    sanlun Controller Primary Partner

    state type node HBA port port
    ——— ———- ——- ———— ——————————————— —————
    GOOD primary sdb host4 0c —
    GOOD primary sdd host5 0d —
    GOOD primary sdf host6 0c —
    GOOD secondary sda host4 — 0c
    GOOD secondary sdc host5 — 0d
    GOOD secondary sde host6 — 0c

  14. Now you can access the LUN by using the mapper

    [root@server scli]# ls -l /dev/mapper
    total 0
    crw——- 1 root root 10, 63 Sep 12 12:32 control
    brw-rw—- 1 root disk 253, 2 Sep 16 10:54 netapp
    brw-rw—- 1 root disk 253, 0 Sep 12 16:32 VolGroup00-LogVol00
    brw-rw—- 1 root disk 253, 1 Sep 12 12:32 VolGroup00-LogVol01

  15. Format it to your liking and mount it

    # mkdir /mnt/netapp
    # mkfs -t ext3 /dev/mapper/netapp
    # mount /dev/mapper/netapp /mnt/netapp/
    //verify it mounted
    # mount


    /dev/mapper/netapp on /mnt/netapp type ext3 (rw)

  16. If you want it to be persistent after reboots put it on /etc/fstab and make sure multipathd start automatically.

    # cat /etc/fstab


    /dev/mapper/netapp /mnt/netapp ext3 defaults 0 0

  17. If possible reboot to check it mounts correctly after reboots.