HP 3Par Upgrade Part 2 – Hosts

Over the past few days I have been completing all the pre-upgrade host checks for the 3Par OS upgrade to ensure a successful upgrade. Here are the steps I’ve taken:

 

1 Check compatibility of components – This is to ensure that you are running a tested configuration of components that have been proven to work together by HP. There are 2 ways to go about this. Firstly you can use SPOCK.  This site contains all the compatibility information you will need to complete your own checks. Or you can complete a host worksheet and return it to HP who will then verify the compatibility of all your components and firmware versions. The components you need to check are fairly standard to any SAN upgrade – Server OS, multipath software, HBA’s and fabric switch firmware versions.

In my case as this is the upgrade of our largest datacentre I did both. My checks matched up with HP’s with only once cluster requiring a HBA driver upgrade. This upgrade is done so onto the next stage

2 Check load balancing is set to round robin –  This is a requirement for any Windows servers running 2008 and using the native MS MPIO driver. As I have over 40 hosts to check I didn’t want to have to visit this manually. So I managed to get a script to do it, here is how:

I used the Microsoft command line application mpclaim to view the multipath configuration. Specifically I ran mpclaim –s –d from the command line.

To run the command line on multiple servers remotely without having to logon I used psexec. You can download it from here . Here is an excellent article on how to use it: psexec guide

In this case I used it in the following way

A Choose the server you want to run the script from and create a folder on it called C:scripts. Copy psexec to this folder

B in C:scripts create a file called 3par_servers.txt. Populate this with a list of the servers you wish to check for multipath configuration

C Also in C:scripts create a batch fie called mpclaim.bat and enter the following command line into it mpclaim –s –d

D Finally, open a command line from the machine you wish to run the script on, change directory to C:scripts and then enter c:Scripts>psexec -c -f @C:scripts3par_servers.txt C:scriptsmpclaim.bat

E You should then see the window populate with the information you require. An example of the output is below:

\Server1

C:Windowssystem32>mpclaim -s -d

For more information about a particular disk, use ‘mpclaim -s -d #’ where # is the MPIO disk number.MPIO Disk   System Disk LB Policy   DSM Name

——————————————————————————-

MPIO Disk5   Disk 6       RR           Microsoft DSM

MPIO Disk4   Disk 5       RR           Microsoft DSM

MPIO Disk3   Disk 4       RR           Microsoft DSM

MPIO Disk2   Disk 3       RR           Microsoft DSM

MPIO Disk1   Disk 2       RR           Microsoft DSM

 

Check LB policy appears as RR for all volumes.

3 Preventing LUN’s being marked as offline following reboot – On the first Windows Server 2012 or Windows Server 2008 reboot following an HP 3PAR array firmware upgrade (whether a major upgrade or an MU update within the same release family) the Windows server will mark the HP 3PAR LUNs offline but the data remains intact. To prevent this it is recommended that KB2849097 is applied to all attached Windows 2008/2012 hosts

 

It is essentially a PowerShell that changes the registry value to 0 for HKLMSystemCurrentControlSetEnumSCSI<device><instance>DeviceParametersPartmgr. The value is responsible for the state of HP 3PAR LUNs following an array firmware upgrade and a 0 indicates they stay online.

 

Windows Server 2008/2012 requires the PowerShell execution policy to be changed to RemoteSigned to allow execution of external scripts you can control this through a GPO. Or again amend through a PowerShell command

 

I got our PowerShell guy to look into if there was a way to the script against all hosts remotely but didn’t have much luck with this. It’s something I will have to look into for future upgrades, but on this occasion I had to log into each host individually and run the script.

So once you have ran the script in KB2849097 you can check its set the registry value as expected value by running: the PowerShell commands

 

Get-ItemProperty –path “HKLM:SYSTEMCurrentControlSetEnumSCSIDisk*Ven_3PARdata**Device ParametersPartmgr” -Name Attributes

 

The value returned should then be 0

 

4 VM’s on ESX running pass through disks. The following wasn’t relevant to our environment but if you are running ESX with raw device mappings check out KB2754704 and KB2821052

 

That’s it host checks complete! Onto the next stage

If you missed the first part of this series catch it here:

HP 3Par Upgrade Part 1 – Planning

 

 

 

 

 

 

Pre Upgrade Clean Up – Stage 1 3 Par Cage Cabling

Before progressing to the next phase of the upgrade plan for 3.1.2 MU3 a couple of outstanding issues needed resolving. First of all HP had noticed the cabling of the disk shelves was incorrect, the bottom two shelves should have been connected via DP-1 but were using DP-2. No changes have been made to cabling so it must have been incorrect since installation.

We used the cabling guide for 2 node systems http://h20628.www2.hp.com/km-ext/kmcsdirect/emr_na-c03607590-4.pdf to check HP findings. The cabling setup is specific to the number of cages you have, the combination of large or small cages used and the number of nodes. We are using a 2 node system with 2 large cages and 5 small. The correct cabling is show in the diagram below:

In our case the two lowest cages on the red loop were connected via DP-2 and we managed to move them to DP-1 with no down time as service continued through the other loop. To check it was now running through the correct port I ran

Showcage –d

I then saw as below that link_speed was displayed correctly against DP1 rather than DP2.

 

———Cage detail info for cage7 ———

 

Position: —

 

Interface Board Info           Card0           Card1

     Firmware_status         Current         Current

         Product_Rev             320c             320c

State(self,partner)           OK,OK           OK,OK

VendorId,ProductId         HP,DCS1         HP,DCS1

         Master_CPU             Yes               No

           SAS_Addr 50050CC10EEEA83E 50050CC10EEFD63E

Link_Speed(DP1,DP2) 6.0Gbps,Unknown 6.0Gbps,6.0Gbps

 

PS PSState ACState DCState Fan State Fan0_Speed Fan1_Speed

ps0     OK     OK     OK       OK   HiSpeed   HiSpeed

ps1      OK     OK     OK       OK   HiSpeed   HiSpeed

 

I checked over my connected systems for any issues, plus ran a checkhalth and and showalert –n to check for any issues. All looked good 🙂