Clicky Web Analytics

Clicky

Oct
2
Sat
Posted By Ritesh Chhajer on Saturday, October 02, 2010
7648 Views 1 Comments


EMC Powerpath basically provides multipathing and improves I/O over SAN.

LUN Mapping:
When dealing with CLARiiON array, database backup would be taken at LUN level using Navisphere CLI(naviseccli) snapview that would allow to create a copy of LUN using snapshots. 

That's why it becomes highly imperative to have mapping of LUN to physical device and ASM diskgroup.
So here is the script with comments before each command to explain it's usage.
This is to be run as root.

diskgroups.sql
set termout off
set echo off
col diskgroup format a30
col disk format a30
set pages 0
set head off
set feedback off
spoo diskgroups.out
select adg.name "diskgroup",ad.name "disk" from v$asm_diskgroup_stat adg,v$asm_disk_stat ad where adg.group_number=ad.group_number;
spool off
exit

dg_power_lun.sh
#!/bin/bash

#Set Oracle Home
ORACLE_HOME=`grep ASM /etc/oratab | cut -d: -f2`
export ORACLE_HOME

#Set Oracle SID
ORACLE_SID=`grep ASM /etc/oratab | cut -d: -f1`
export ORACLE_SID

export PATH=$ORACLE_HOME/bin:$PATH

echo "Enter SYS Password"
stty -echo
read passw
stty echo
# Generate diskgroups.out
sqlplus -s "sys/$passw@connstring as sysdba" @diskgroups.sql

#Set Heading of the output
printf "\n%-15s %-15s %-25s %-15s\n" "Disk Group" "LUN" "Power Path" " ASM Disk"
printf "%-15s %-15s %-25s %-15s\n" "---------------" "-------------" "-------------" "-------------"

# This is for proper indentation
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")

#Loop through the file generated by diskgroups.sql 
for i in `cat diskgroups.out`
do
# Disk Group Name
v_dg=`echo $i|awk '{print $1}'`

#Disk Name
v_disk=`echo $i|awk '{print $2}'`

#Query the disk to ensure that it's ASM disk
#v_asmdisk=`/etc/init.d/oracleasm querydisk -d $v_disk | awk '{print $2}'| sed 's/\"//g'`

#Get the major,minor numbers so that they can be used to get the device names
v_minor=`/etc/init.d/oracleasm querydisk -d $v_disk | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk '{print $1}'`
v_major=`/etc/init.d/oracleasm querydisk -d $v_disk | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk '{print $2}'`

# Get the device name based on the major,minor recieved from the above disk
v_device=`ls -la /dev | awk -v v_minor=$v_minor -v v_major=$v_major '{if ( $5==v_minor ) { if ( $6==v_major ) { print $10}}}'`
v_device1=`ls -la /dev | awk -v v_minor=$v_minor -v v_major=$v_major '{if ( $5==v_minor ) { if ( $6==v_major ) { print $10}}}'|sed 's/1//g'`

# Get the LUN information from powermt - PowerPath Management Utility
v_lun=`powermt display dev=$v_device1|grep "Logical device"|awk '{print $4 $5}'|cut -f2 -d'['|cut -f1 -d']'`

#Display
printf "%-15s %-15s %-25s %-15s\n" $v_dg $v_lun "/dev/$v_device" $v_disk
done

IFS=$SAVEIFS

Sample output
# ./dg_power_lun.sh
Enter SYS Password

Disk Group      LUN             Power Path                 ASM Disk
--------------- -------------   -------------             -------------
DG_SYS          LUN1            /dev/emcpowery1           DISK1
DG_SYS          LUN2            /dev/emcpowerba1          DISK2
DG_ARCH         LUN3            /dev/emcpoweran1          DISK3
DG_ARCH         LUN4            /dev/emcpoweray1          DISK4
DG_ARCH         LUN5            /dev/emcpowerab1          DISK5
DG_ARCH         LUN6            /dev/emcpowerax1          DISK6
DG_DATA1        LUN7            /dev/emcpoweraq1          DISK7
DG_DATA1        LUN8            /dev/emcpowerao1          DISK8
DG_DATA1        LUN9            /dev/emcpowerak1          DISK9
DG_DATA1        LUN10           /dev/emcpoweral1          DISK10


Max Sector Size:
On RHEL5.3, if you are using EMC Powerpath, check out the max sector size.
Basically, the way I/O is handled has changed in RHEL5 from RHEL4. 
Let us say emcpowerx is one of my device and I want to check the max sector size.
# cat /sys/block/emcpowerx/queue/max_sectors_kb
512

This means max sector size is limited to 512K so I cannot issue 1M I/O.

Do the following:

1. Upgrade Powerpath
# rpm -qa|grep EMC
EMCpower.LINUX-5.3.1.00.00-111

# powermt version
EMC powermt for PowerPath (c) Version 5.3 SP 1 (build 111)

#rpm -U EMCpower.LINUX-5.3.1.00.05-001.rhel5.x86_64.rpm

# rpm -qa|grep EMC
EMCpower.LINUX-5.3.1.00.05-001

# powermt version
EMC powermt for PowerPath (c) Version 5.3 SP 1 HF 05 (build 1)

2. Set up max_sectors_kb = max_hw_sectors_kb
% vi setmaxsectors
#!/bin/sh

PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

DEVS=`find /sys/block -type f -name "max_hw_sectors_kb" -exec echo {} \;| awk -F/ '{print $4}' `

for device in $DEVS
do
        MAX_HW=`cat /sys/block/$device/queue/max_hw_sectors_kb`
        echo "$device -> $MAX_HW"
        echo $MAX_HW > /sys/block/$device/queue/max_sectors_kb
done

3. Copy setmaxsectors file to /etc/init.d and reboot
# cp setmaxsectors /etc/init.d
# chkconfig --add setmaxsectors
# chkconfig --level 2345 setmaxsectors on
# reboot

4. Verify
# cat /sys/block/emcpowerx/queue/max_sectors_kb
32767

This can provide upto 20% I/O boost for ASM thereby the underlying Oracle database.

I/O stats
powermt - EMC Powerpath management utility can be used to check the I/O and latency
It has got bunch of options. Few examples:
# powermt display DEV=emcpoweray every=5
# powermt display latency DEV=emcpoweray every=5

Few useful naviseccli commands:
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 getlun -capacity
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 getsp
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 getdisk
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 getrg
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 getcache
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 -createsnapshot 1 -snapshotname DB-LUN1
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 snapview -startsession $sess_name -snapshotname $lun_list -consistent
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 snapview -listsessions
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 reserved -lunpool -list
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 snapview -lunpool
naviseccli -User myaccount -Password mypassword -h mycxname -Port 2163 -Scope 0 snapview faults -list

If you don't want to use CLI, use GUI:
https://mycxname:2163/

I found following documents on EMC quite useful to understand how the snapview snapshots work and the CLI:

http://www.emc.com/collateral/software/white-papers/h1349-emc-clariion-snapview-snapshots-snap-sessn-knwldgbk-wp.pdf

http://www.emc.com/microsites/clariion-support/ax100/pdf/069001181.pdf

My sincere thanks to Ruiping Sun for all his valuable inputs.

Categories

Rants & Raves Minimize

  • Gravatar
    Storage Guy Thursday, October 21, 2010 at 1:07 PM

    Good to see wonderful scripting and excellent storage information coming out from an Oracle DBA. Always a pleasure to work with you.

  • askdba.org Friday, October 29, 2010 at 12:31 AM
    via pingback
    October 2010 Blogroll Report | AskDba.org Weblog

  • Recommended Oracle DBA Books Minimize

         

    Tag Cloud Minimize


    Archive Posts Minimize
     
    Monthly
      Yearly

      Disclaimer:
      This posting is provided "AS IS" with no warranties, and confers no rights. You assume all risk for your use.

      This posting has nothing to do with my present or past employer.