HP 3PAR StoreServ 7000

Installation, Startup and

Break Fix

Lab Guide

HP ServiceOne
Rev. 14.1
Course : 00722456
HP 3PAR StoreServ 7000
Installation, Startup and
Break Fix

Lab Guide

HP ServiceOne
Rev. 14.1
Course : 00722456
 Copyright 2013 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice. The only warranties for
HP products and services are set forth in the express warranty statements accompanying
such products and services. Nothing herein should be construed as constituting an additional
warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
This is an HP copyrighted work that may not be reproduced without the written permission of
HP. You may not use these materials to deliver training to any person outside of your
organization without the written permission of HP.
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix
Lab Guide
November 2013
Table of Contents

Table of Contents ................................................................................................... 1

HP 3PAR StoreServ 7000 Installation,Startup, and Break Fix Lab Activities ........... 3
Objectives ....................................................................................................... 3
Requirements .................................................................................................. 3
Prerequisite ..................................................................................................... 3
Lab 0: Remote Lab Access..................................................................................... 5
Objectives ....................................................................................................... 5
Lab Environment ............................................................................................. 5
Lab 1: HP 3PARStoreServ 7000 configuration and locating system components ... 7
Objectives ....................................................................................................... 7
Introduction ..................................................................................................... 7
Exercise 1: Best Practice configuration ........................................................... 8
Exercise 2: Locating system components ........................................................ 9
Exercise 3: Interconnect Node....................................................................... 12
Exercise 4: Getting familiar with the Backend (SAS) cabling ......................... 13
Lab 1 Activity debrief ..................................................................................... 18
Lab 2: HP 3PARStoreServ 7000 Installation ......................................................... 19
Objectives ..................................................................................................... 19
Introduction ................................................................................................... 19
Exercise 1: Install a Virtual Service Processor ............................................... 22
Exercise 2: Installing a Storage System using HP 3PAR SmartStart ............. 29
Exercise 3: Configure the Storage System using HP 3PAR SmartStart ......... 41
Exercise 4: Install the HP 3PAR Management Console using HP 3PAR
SmartStart ..................................................................................................... 55
Lab 2 Activity debrief ..................................................................................... 56
Lab 3: Configure the Storage System (CPG, VV, Host) ........................................ 57
Objectives ..................................................................................................... 57
Introduction ................................................................................................... 57
Basic Functions ............................................................................................. 58
Virtual Copy (Snapshot) ......................................................................... 77
Dynamic Optimization............................................................................. 80
Thin/Thick Conversion ............................................................................ 81
Adaptive Optimization............................................................................. 83
Scheduling Tasks ................................................................................... 87
Lab 3 Activity debrief ..................................................................................... 89
Lab 4: CLI commands .......................................................................................... 91
Objectives ..................................................................................................... 91
Introduction ................................................................................................... 91
Different access methods to the CLI command set ........................................ 92
CLI command examples ................................................................................ 93
CLI Commands to get specific information .................................................... 96
Lab 4 Activity debrief ................................................................................... 100
Lab 5: System upgrade ...................................................................................... 101
Objectives ................................................................................................... 101
Introduction ................................................................................................. 101
Drive Enclosures and Disk Drives Upgrade ................................................. 102
Rev. 14.1 L -1
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Rebalance the storage system .................................................................... 104

System Tune using GUI (MC) ...................................................................... 105
Verify a System Tune process (optional) ..................................................... 108
Lab 5 Activity debrief ................................................................................... 109
Lab 6: HP 3PAR Service Processor Onsite Customer Care (SPOCC) usage ..... 111
Objectives ................................................................................................... 111
Introduction ................................................................................................. 111
Exploring the SPOCC functionality .............................................................. 112
Maintenance Mode ...................................................................................... 117
HP 3PAR InForm OS Update/Patch using SPOCC...................................... 119
Service Processor Update/Patch using SPOCC .......................................... 130
Guided Maintenance ................................................................................... 134
Lab 6 Activity debrief ................................................................................... 136
Lab 7: Node Rescue........................................................................................... 137
Objectives ................................................................................................... 137
Auto Node to Node Rescue ......................................................................... 137
Service Processor (Physical) Node Rescue ................................................ 143
Lab 7 Activity debrief ................................................................................... 144
Lab 8: Manage Events and Alerts ....................................................................... 145
Objectives ................................................................................................... 145
Introduction ................................................................................................. 145
Managing Events......................................................................................... 146
Managing Alerts .......................................................................................... 148
Managing Alerts/Events using CLI ............................................................... 150
Lab 8 Activity debrief ................................................................................... 151
Lab 9: De-installation of a 3PAR Storage System ............................................... 153
Objectives ................................................................................................... 153
Introduction ................................................................................................. 153
Connect the Maintenance PC to a Controller Node ..................................... 154
De-Install a HP 3PAR Storage Server ......................................................... 154
Lab 9 Activity debrief ................................................................................... 157
Lab 10: Installing Storage Software Manually ..................................................... 159
Objectives ................................................................................................... 159
Introduction ................................................................................................. 159
Setup a HP 3PAR StoreServ 7000 storage system without SmartStart or the
Service Processor wizard ............................................................................ 160
Setup the Virtual Service Processor using Moment Of Birth (MOB) ............. 168
Install HP 3PAR OS Files ............................................................................ 182
Add the Storage System to the Service Processor ...................................... 183
Lab 10 Activity debrief ................................................................................. 185
Lab 11: Configure Storage System ..................................................................... 187
Objectives ................................................................................................... 187
Introduction ................................................................................................. 187
FC Port Configurations ................................................................................ 188
Multipath (MPIO) Configurations ................................................................. 192
Host, CPG and Volume Configurations........................................................ 194
Lab 11 Activity debrief.................................................................................. 195

L -2 Rev. 14.1
 Lab Guide

HP 3PAR StoreServ 7000 Installation,Startup, and

Break Fix Lab Activities

Objectives
After completing this lab, you should be able to:
 Install and configure HP 3PAR StoreServ 7000 Storage System using SmatStart
 Use HP 3PAR Management Console to perform a Customer Orientation session
 Use SPOCC to access the Service Processor to perform monitor and
maintenance activities
 Upgrade the Storage System (hardware/software)
 Manage Events and Alerts
 Perform Guided Maintenance activities
 Deinstall a Storage system
 Install HP 3PAR StoreServ 7000 Storage System without using SmatStart

Requirements
To complete this lab, you will need:
 Access to the lab environment
 One HP 3PAR StoreServ 7000 Storage System, one Windows host and one ESX
server connected to the same fabric
 A system to run VMware vSphere Client
 A predefined zone configuration on the FC switches

Prerequisite
To successfully complete the labs, you have to be able to:
 Manage a Windows system
 Use the VMware vSphere Client to manage a virtual machines

Rev. 14.1 L -3
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

L -4 Rev. 14.1
 Lab Guide

Lab 0: Remote Lab Access

Objectives
After completing this lab, you should be able to:
 Access the remote lab
 Identify the components of the different groups of the remote lab environment

Lab Environment
The following graphic shows the lab environment for a single lab group

The teacher assigns each student to a lab group.

Connect to the Virtual Lab using the access page https://virtual-labs.onsite.de

The login credentials depend on your group assignments and are provided by your
instructor.

Rev. 14.1 L -5
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

L -6 Rev. 14.1
 Lab Guide

Lab 1: HP 3PARStoreServ 7000 configuration and

locating system components

Objectives
After completing this lab, you should be able to:
 Verify if a given HP 3PAR StoreServ 7000 Systems configuration is configured
according to “Best Practice” or Supported Configurations outside the Best
Practice guidelines
 Use the Part Locator function of the HP Partner Service Media Library to
familiarize yourself with the different system components

Introduction
HP 3PAR Storage Systems provide tremendous flexibility to scale by adding
Controller Nodes, Drive Enclosure, and Disk Drives. In order to ensure best results,
“Best Practice” guidelines have been developed to reflect recommended best
practices to configure HP 3PAR systems.
In this lab, you have to demonstrate the correct cabling of the HP 3PAR StoreServ
7000 system.
You will use the provided documents HP 3PAR 7000 Installation Guide, HP 3PAR
StoreServ 7000_7450 Storage Cabling Configuration Guides, HP 3PAR StoreServ
7000 Storage Detailed Configuration Guide and HP 3PAR StoreServ 7450 Storage
Configuration Guide to retrieve the required information.

Rev. 14.1 L -7
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Exercise 1: Best Practice configuration

Prepare yourself by reading the provided manuals (~10 Minutes). Join the breakout
room of your group (video and audio) and discuss in your group. Try to find the correct
answers for the exercise questions.
Use HP 3PAR StoreServ 7000 Storage Detailed Configuration Guide and HP 3PAR
StoreServ 7450 Storage Configuration Guide.

Answer the following question:

Can you mix SSD and HDD drives in a HP 3PAR StoreServe 7450 configuration?

____________________________________________________________________

Is a NL-only configuration supported for HP 3PAR StoreServe 7400?

____________________________________________________________________

How many SSD devices are supported in a 4node HP 3PAR StoreServe

7400configuration?

____________________________________________________________________

Can you mix SFF and LFF enclosures on the same node pair?

____________________________________________________________________

What is the minimum drive quantity per node pair for SAS Fast Class, SAS SSD and
SAS Nearline in a HP 3PAR StoreServe 7200 configuration?
____________________________________________________________________
What is the minimum drive quantity per node pair in a HP 3PAR StoreServe 7450
configuration?
____________________________________________________________________
Which drive types can be in the same column of drives in the M6720 enclosure?

____________________________________________________________________

L -8 Rev. 14.1
 Lab Guide

Exercise 2: Locating system components

You can work on your own in this exercise.
To must familiarize yourself with the different system components, use the Part
Locator function of the HP Partner Service Media Library.
HP Services Media Library (SML) is a tool to provide information that is easy to
access, “just-in-time,” and which contains unique rich media elements that are
available to service personnel.
Examples of rich media elements include:
 Video clips
 Animations
 Audio
The library does not serve as a single point of access for product or training
information, but rather, it provides quick access to service information that has been
created in rich media formats.
If you prefer, you can download the SML user interface and media elements for a
product from the partner portal and install and run it on your computer.
1. Connect to the virtual lab page https://virtual-labs.onsite.de and log in with your
group credentials.

2. To access HP Partner Service Media Library, use the link on the resources page of
the virtual lab environment.
3. Sign-in using your HP Passport.
4. Select 1. Product category Storage, 2. Product family 3PAR StoreServ Storage,
and 3. Product series HP3PAR StoreServ 7000 Storage.

Rev. 14.1 L -9
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

5. Use the Physical Reference Manual and the Part Locator and at the bottom of this
page to identify all the different system components.

L -10 Rev. 14.1

 Lab Guide

Rev. 14.1 L -11

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Exercise 3: Interconnect Node

Join the breakout room of your group (video and audio). One student should drive the
session, by sharing his/her desktop. Use the tools of the virtual classroom to draw the
correct cabling into the provided diagram. Discuss the interconnect issue in your
group.
In a two-node configuration, the connection between the nodes is provided by the
backplane.
Connection between nodes 0 and 1 and nodes 2 and 3 require four interconnect
cables, which are directional and must be installed correctly for the storage system to
function.
The cables are labelled as A and C. The A end connects to nodes 0 and 1 and the C
end connects to nodes 2 and 3.

IMPORTANT:
The interconnect cables are directional and must be installed correctly
for the storage system to function.

Use the following worksheet to show the proper cabling of a four-node configuration.

Worksheet:

L -12 Rev. 14.1

 Lab Guide

Exercise 4: Getting familiar with the Backend (SAS) cabling

Join the breakout room of your group (video and audio) and discuss the cabling in
your group. One student should be driving the session by sharing his/her desktop.
Use the tools of the virtual classroom to draw the correct cabling in the provided
diagram.
On the HP 3PAR StoreServ 7000 systems, Drive Enclosures are daisy chained from a
SAS port on one node to the next Drive Enclosure in the chain, and from the last
enclosure in the chain to the corresponding SAS port on the other node in the same
Node Enclosure. This assures redundant paths from each node to each Drive
Enclosure on each chain.
Alerts will be posted noting the condition if the limit is exceeded. This is also flagged in
“checkhealth.”
Use the following worksheet to show the proper cabling of the given configuration.

Rev. 14.1 L -13

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Worksheet Group 1
2N4S (2 Node 4 Enclosure Small)

L -14 Rev. 14.1

 Lab Guide

Worksheet Group 2
4N6L (4 Node 6 Enclosure Large)

Rev. 14.1 L -15

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Worksheet Group 3
2N3L3S (2 Node 3 Enclosure Large 3 Enclosure Small)

L -16 Rev. 14.1

 Lab Guide

Worksheet Group 4
2N1S2L (2 Node 1 Enclosure Small 2 Enclosure Large)

Rev. 14.1 L -17

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Lab 1 Activity debrief

Use the space below to record your Key Insights and Challenges from the Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

L -18 Rev. 14.1

 Lab Guide

Lab 2: HP 3PARStoreServ 7000 Installation

Objectives
After completing this lab, you should be able to:
 Set up the Virtual Service Processor
 Use SmartStart to configure the Service Processor
 Initialize the storage system software
 Add a storage system to the Service Processor

You will be working with your lab partners for this exercise.

Introduction
In this lab, you will learn how to complete the installation of a storage system by
setting up and configuring the storage system software and service processor
software.
During this exercise, you will use a Virtual Service Processor and SmartStart.
Before starting the setup task, make sure that the all hardware components are
correctly installed and powered on.

From the following list, obtain the addresses and names of the systems assigned to
your group before proceeding with the setup and configuration procedures.

Note:
To avoid dual IP addressing problems or duplicate names in the lab
environment, please use only the listed names and IP configuration.

Join the breakout room of your group (video and audio). One student should drive the
session by sharing his/her desktop. For each exercise, a different student should drive
the exercise.

Rev. 14.1 L -19

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

HP 3PAR Virtual Service Processor

SP ID: SP000<StoreServ serial
ID format used for the physical and virtual SP. The literal “SP000” string number>
followed by StoreServ serial number (7 character)

Hostname VirtualSPn (n=lab group number)

Permanent IP address 192.168.0.15 (group 1) 192.168.2.54 (group 5) 192.168.2.94 (group 9)

(IPv4 only) 192.168.0.26 (group 2) 192.168.2.64 (group 6) 192.168.2.104 (group 10)
192.168.0.37 (group 3) 192.168.2.74 (group 7) 192.168.2.114 (group 11)
192.168.0.48 (group 4) 192.168.2.84 (group 8) 192.168.2.124 (group 12)

Subnet mask 255.255.255.0

Gateway IP address 192.160.0.13 (group 1) 192.168.2.52 (group 5) 192.168.2.92 (group 9)
192.168.0.24 (group 2) 192.168.2.62 (group 6) 192.168.2.102 (group 10)
192.168.0.35 (group 3) 192.168.2.72 (group 7) 192.168.2.112 (group 11)
192.168.0.46 (group 4) 192.168.2.82 (group 8) 192.168.2.122 (group 12)

DNS settings: not applicable

 Domain name
 DNS servers
Maximum supported speed and duplex setting: Auto negotiate
 Auto negotiate or
 Custom duplex (half or full) and
 Custom speed (10, 100, or 1000 MB/s)

Proxy server settings for Remote Support (if applicable): not applicable
 Protocol
 Address and port
 Authentication information (optional)

Date and time: Enter manually

 Enter manually
 Use NTP server information

Time zone region Europe/Berlin

New password to assign setupusr (for use with the system setup wizards) setupusr
New password to assign 3parcust (to access SPOCC) 3parcust

L -20 Rev. 14.1

 Lab Guide

HP 3PAR StoreServ 7000 Storage System

Serial number 1620502 (group 1)
NOTE: This 7-digit number is located on the back of your HP 3PAR storage 1620501 (group 2)
system next to the power switch for the node enclosure PCM, and begins with 1620500 (group 3)
16 (for example, 1624635). 1619183 (group 4)
1619186 (group 5)
1619185 (group 6)
1619184 (group 7)
1619182 (group 8)
1629551 (group 9)
1629550 (group 10)
1629547 (group 11)
1629549 (group 12)
Name P7400-n (n=lab group number)

IP address 192.168.0.211 (group 1) 192.168.2.205 (group 5) 192.168.2.209 (group 9)

(IPv4 only) 192.168.0.222 (group 2) 192.168.2.206 (group 6) 192.168.2.210 (group 10)
192.168.0.233 (group 3) 192.168.2.207 (group 7) 192.168.2.211 (group 11)
192.168.0.244 (group 4) 192.168.2.208 (group 8) 192.168.2.212 (group 12)

Subnet mask 255.255.255.0

Gateway IP address 192.160.0.13 (group 1) 192.168.2.52 (group 5) 192.168.2.92 (group 9)

192.168.0.24 (group 2) 192.168.2.62 (group 6) 192.168.2.102 (group 10)
192.168.0.35 (group 3) 192.168.2.72 (group 7) 192.168.2.112 (group 11)
192.168.0.46 (group 4) 192.168.2.82 (group 8) 192.168.2.122 (group 12)

Date and time: Copy from SP

Time zone region Europe/Berlin
New password to assign 3paradm (to access the storage system) 3pardata

Rev. 14.1 L -21

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Exercise 1: Install a Virtual Service Processor

A virtual Service Processor is included with the 3PAR Base Operating System. Each
HP 3PAR StoreServ 7000 requires its own Service Processor.
The Service Processor can be installed on a VMware system (or Hyper-V system,
later in 2013), provided by the customer, meeting the following specifications.
 VMware vSphere 4.1, 5.0, or 5.1
 Server listed in VMware Compatibility Guide and with the following features
o 2GB RAM (minimum)
o 320GB free disk space
o DVD ROM or DVD RW
o 1Gb Ethernet port

1. Connect to the virtual lab environment using the credentials assigned to your lab
group.
2. Connect to the server assigned to your group. (For example, S1-P7000-Grpx,
where x is your group number)

3. Open the Desktop.

4. Open the ClassFile folder on your desktop and right-click the
3PAR_Virtual_SP_4.1_MU2_P05_Software_QR482-11007.iso file to mount the
ISO image.

5. Start the HP 3PAR vSphere Client (use the link on your desktop).
6. Use the listed IP address and login with the username root and the password
HP-P7000.
L -22 Rev. 14.1
 Lab Guide

7. In the VMware vSphere Client window, select File > Deploy OVF template to
import the VSP (VM) OVF file.

8. On the Source page, click Browse to import the OVF from its location (mounted
ISO image), then click Next.

9. On the OVF Template Details page, verify the OVF template, and then click
Next.

Rev. 14.1 L -23

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

10. On the Name and Location page, type a name for the VSP (Virtual SPn, n=lab
group number), then click Next.

11. On the Disk Format page, select Thin Provision, then click Next.

L -24 Rev. 14.1

 Lab Guide

12. On the Ready to Complete page, review the deployment settings and select the
Power on after Deployment checkbox, then click Finish.

13. A Deployment Completed Successfully message displays after a few minutes.

14. Click Close.

15. To verify that the system has started, right-click the VSP in the virtual machine list,
select Open Console, and check that the system is ready to login.
Press Enter to get login prompt.

Rev. 14.1 L -25

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

16. The Lab environment does not have DHCP enabled. To set the VSP IP address,
login as setupusr without a password.

17. Enter IP address, Netmask and default gateway according the following table
and type Y.

L -26 Rev. 14.1

 Lab Guide

Group VSP IP-addr Netmask Gateway

1 192.168.0.15 255.255.255.0 192.168.0.13
2 192.168.0.26 255.255.255.0 192.168.0.24
3 192.168.0.37 255.255.255.0 192.168.0.35
4 192.168.0.48 255.255.255.0 192.168.0.46
5 192.168.2.54 255.255.255.0 192.168.2.52
6 192.168.2.64 255.255.255.0 192.168.2.62
7 192.168.2.74 255.255.255.0 192.168.2.72
8 192.168.2.84 255.255.255.0 192.168.2.82
9 192.168.2.94 255.255.255.0 192.168.2.92
10 192.168.2.104 255.255.255.0 192.168.2.102
11 192.168.2.114 255.255.255.0 192.168.2.112
12 192.168.2.124 255.255.255.0 192.168.2.122

Note:
Press Enter, “any key” doesn’t work.

18. Press Ctrl-Alt to release the cursor and continue to setup the VSP with
SmartStart.
Rev. 14.1 L -27
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

19. In the VMware vSphere Client window, select the Summary tab for the IP
address to display (in the General section, IP address field).

20. Make a note of the IP address shown on your system.

_________________________________________

21. Exit the console window and the vSphere Client.

L -28 Rev. 14.1

 Lab Guide

Exercise 2: Installing a Storage System using HP 3PAR

SmartStart
Before you begin installing the storage system software using HP 3PAR SmartStart,
verify that:
 You are running on a Windows 2008 R2 or 2012 system.
 All components are installed.
 The storage system setup is verified, the storage system is powered on, and
the LEDs are functioning correctly.
 Either one of the following has been done:
o The Service Processor is installed and powered on, and you have
assigned an IP address, or
o The VSP is installed and connected, and you have determined the
temporary IP address.
 The SP connection is established.

Starting SmartStart
1. Connect to the server assigned to your group (for example, S1-P7000-Grpx).
2. Locate the SmartStart ISO image in the ClassFiles folder.
3. Mount the ISO image.

4. Open SmartStart application, and click Agree to the End User License
Agreement.

Rev. 14.1 L -29

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

5. Read the Welcome page and start SmartStart wizard Step 1, Prepare to
Configure. Do this to verify that you have the information you need and all
preliminary tasks are complete.

L -30 Rev. 14.1

 Lab Guide

Set up the Service Processor

6. Use SmartStart wizard Step 2, Set up Systems, to setup the Service Processor
and your new storage system.

7. Click the Set up the Service Processor link in SmartStart wizard Step 2, Setup
systems.
This setup was previously called MOB (Moment of Birth).
8. If you are setting up a VSP, type the SP IP address obtained in the VMware
vSphere Client window Summary tab.

9. Select Continue to this website if a Certificate Error is shown.

10. In the User Name text box, type setupusr and leave the password blank. You will
change this password in the SP Setup wizard.

Rev. 14.1 L -31

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

The Welcome page shows information needed to complete the wizard and setup the
VSP.
Note:
Refer to Lab 2 Introduction on page 18 of this lab guide for your group
individual parameter.
11. Click Next.
12. Enter the seven digits StoreServ Serial Number and click Generate SP ID.

13. Notice the generated Service Processor ID and click Next.

14. Review the IP settings and make corrections, if required.

L -32 Rev. 14.1

 Lab Guide

15. Do not Enable DNS Support and then click Next.

16. Do not Enable proxy server for Remote Support, and then click Next.
17. Enter the System Support Information as highlighted in the following screen shot.

Rev. 14.1 L -33

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

18. To configuring your time zone, use the current date and time. As for Time Zone
region, select Europe/Berlin. To proceed, click Next.

19. Set passwords for user setupusr and 3parcust to be the same as the username
(setupusr and 3parcust). To proceed, click Next.

20. The Summary page shows all options selected and cannot be modified. To
modify, click Prev. To proceed, click Next.

L -34 Rev. 14.1

 Lab Guide

21. Monitor the progress.

22. Click Finish to close the Service Processor setup screen.

Rev. 14.1 L -35

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

L -36 Rev. 14.1

 Lab Guide

Initialize the Storage System

The next step is to initialize the storage system before you can configure it. The
Storage System Setup wizard verifies your storage system configuration, initializes
the storage system, and runs initialization tests on the storage system. This process
was previously called “ootb” (out of the box)
23. Click the choice, Set up the storage system.

24. Supply the permanent SP IP address of your group and click OK.

25. Continue to this website if you get the Certificate Error and use the setupusr
account to login to start the Storage System Setup Wizard.

Rev. 14.1 L -37

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

The HP 3PAR Storage System Setup Wizard should be displayed. The Welcome
page shows information needed and the estimated time to complete the wizard and
set up the storage system. Click Next.

Note:
The required information to configure the storage system is listed at the
beginning of this lab

26. On the Enter Storage System page, verify the 7-digit number Storage System
Serial Number and click Next.

27. On the Verify Storage System page, verify that the correct storage system is
displays and then click Next.

28. On the Configure Networking page, complete the required fields. Type the
name and IP address for your new HP 3PAR storage system (listed at the
beginning of the Lab2 and then click Next.

L -38 Rev. 14.1

 Lab Guide

29. On the Configure time page, select Copy time options from the Service
Processor and click Next.

30. On the Change Password page, type a new password for the storage system
username 3paradm. (use 3pardata as new password)

31. The Verify Configuration page shows all options selected and cannot be
modified on this page. To modify, click Prev. To proceed, click Next.

Rev. 14.1 L -39

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

The Progress page shows the setup process and operation status.
Note:
Setup system volumes can take a long time. For some systems in the
lab up to 10 min, depends of the disk type and capacity.

32. After the Storage System Setup wizard initialized the storage system, click Finish
in the Setup Progress and Results page to exit the Storage System Setup wizard.

33. Close the browser window.

The wizard continues to run tests in the background after you have exited the
Storage System Setup wizard. If these tests discover any issues, the HP 3PAR
Management Console displays details in the Alert tab relevant to the issue.

L -40 Rev. 14.1

 Lab Guide

Exercise 3: Configure the Storage System using HP 3PAR

SmartStart
In order to configure hosts and storage using HP3PAR SmartStart, you have to
connect to the storage system first.
1. Select Step 3, Connect to the storage system to connect a storage system.
2. Click the Connect to the storage system link

3. Enter IP Address or Name, the User Name, and Password of your storage
system and then click OK. (3paradm/3pardata)
4. Wait until the system is listed as a connected storage system and the System
State is Normal. This may take a while.

Rev. 14.1 L -41

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Host configuration

5. Select Step 4, Configure hosts, to configure the Fibre Channel or iSCSI host
connection.
6. Click the Configure FC host link.

The Fibre Channel Host Configuration screen will be displayed. The Welcome page
shows information needed to complete the wizard and what will be done during each
step. Click Next.

L -42 Rev. 14.1

 Lab Guide

Configure the Fibre Channel connection to the SmartStart host (localhost).

7. Enter localhost as Host address and click Connect. If the Result area shows that
the system is connected to localhost, click Next.

8. On the Validate FC HBA Setup page click Validate.

9. You should see two HBAs.

10. Make a note of the WWN of the two HBAs of your system.

_____________________________

_____________________________

Rev. 14.1 L -43

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

11. Click Next.

12. On the Validate the Fibre Channel Connection page, select your storage
system and click Validate.

13. If validation failed, configure the embedded FC-port (0:1:1) with the connection
type Switch. Click Configure.

14. Validate the configured port and configure the embedded FC-port (1:1:1) with the
connection type Switch. Click Configure.

15. Validate the configured ports and make sure that at least two connections to the
storage system are valid and then click Next to proceed.

L -44 Rev. 14.1

 Lab Guide

16. On the Configure Multipath I/O page click Configure.

17. Verify the progress and make sure that multipath is set to enabled, round-robin
load balancing is set, and a device ID is created.
18. Check the box to enable Reboot when finished. Click Next to proceed.

19. On the Install and start Host Explorer page, notice the message that there is no
Host Explore available at this time for Windows 2012; click Next to proceed.

Rev. 14.1 L -45

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

20. Verify all items on the Summary page have a check mark and then click Finish.

21. The host will now be rebooted and you will lose the connection to your Window
server. Wait few a minutes and reconnect to the Windows server.

L -46 Rev. 14.1

 Lab Guide

Storage configuration

22. Start SmartStart, by mounting the ISO file and then running smartstart.exe.
23. Connect to the storage system and login.

24. Select Step 5, Configure Storage, to configure the storage system.

Notice the configuration steps provided.

25. Click the Create host link in SmartStart wizard step 5 Configure storage.
Rev. 14.1 L -47
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

26. Read the Welcome page and click Next.

27. On the Host Settings page, specify as Name your Windows system name (for
example, S1-P7000-Grpn); select Windows 2012 as Host OS, leave all other
fields unchanged and click Next.

28. On the Fibre Channel Settings page assign the WWNs of your Windows system
and click Next.

29. Skip the iSCSI Settings page by clicking Next.

30. Review the Summary page and click Finish.
L -48 Rev. 14.1
 Lab Guide

31. Click the Create virtual volumes link in SmartStart wizard step 5 Configure
storage.

32. Read the Welcome page and click Next.

33. On the Configure Virtual Volume screen, enter a name Volume_1_Grpn (n=your
lab group number) for the volume.
34. Do not check the Export Volume(s) after creation box.
35. Create a Fully Provisioned volume of 10 GB using any FC_rx CPG as User
CPG.
36. Leave all other fields unchanged and click Next.

Rev. 14.1 L -49

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

37. Review the Summary page and click Finish.

38. Click the Export virtual volumes link in SmartStart wizard step 5 Configure
storage.

L -50 Rev. 14.1

 Lab Guide

39. Read the Welcome page and click Next.

40. Select the previously created volume and host and click Next.

41. Review the Summary page and click Finish.

42. Click the Add virtual volumes to Windows link in SmartStart wizard step 5
Configure storage.

Rev. 14.1 L -51

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

--

43. Read the Welcome page and click Next.

44. On the Connect to host system page, leave localhost as Host address and click
Connect.

45. If connected, click Next.

46. Wait until the virtual volume information is retrieved.

47. Select the previously created volume. Specify Volume_1_Grpn as the disk label
and E: as the drive letter. If drive letter E is not available, use the next higher
available drive letter and click Next.

L -52 Rev. 14.1

 Lab Guide

48. Review the Summary page and click Finish.

49. Wait until the device is initialized and mounted.

Rev. 14.1 L -53

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

50. Verify the created volume is seen on the Windows system.

L -54 Rev. 14.1

 Lab Guide

Exercise 4: Install the HP 3PAR Management Console using HP

3PAR SmartStart
1. Select Step 6 Install the Management Console of the HP 3PAR SmartStart
page to install the graphical user interfaces of the HP 3PAR storage system.
2. Click the Install the HP 3PAR Management Console.

3. Do not change any default setting in the installation wizard. Press Install to start
the installation, wait until the installation is finished.

4. To verify the installation, double click the HP 3PAR Management Console icon on
the Desktop to start the management console and connect to the Storage
System.

Rev. 14.1 L -55

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Lab 2 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

L -56 Rev. 14.1

 Lab Guide

Lab 3: Configure the Storage System (CPG, VV,

Host)

Objectives

After completion of this lab, you will be able to:

 Verify the storage configuration using the HP 3PAR Management Console
 Perform basic configuration of CPGs, VV and Hosts
 Demonstrate how to create snapshots, convert volumes (Thin/Thick), use
Dynamic Optimization, Adaptive Optimization and Schedule tasks

You will be working with your lab partners for this exercise
Join the breakout room of your group (video and audio). One student should drive the
session by sharing his/her desktop. For every exercise, a different student should
drive the exercise.

Introduction

In this lab, you will learn the basic configuration tasks required for customer
orientation as part of the system installation.
In the previous lab, you used SmartStart to configure a storage array. In this lab, you
will perform a similar configuration, but will use the HP 3PAR Management Console.
The main focus of this lab is to make you familiar with the Management Console.

Rev. 14.1 L -57

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Basic Functions

Note:
In the previous lab, a host, CPGs and a volume were already created
using SmartStart. You will unexport the volume, delete the host and
delete the volume to get the array in a fresh state before you go
proceed with the labs.

First, remove the volume and host created during SmartStart setup
configuration.
1. Start the 3PAR Management Console (GUI) and connect to your HP 3PAR array.
(user: 3paradm, password: 3pardata)

2. Select your Storage System, select Provisioning and Virtual Volumes. Select
the Exported icon. Identify the Virtual Volume that was created with the
SmartStart Wizard. Right Click this Volume and select Unexport.

L -58 Rev. 14.1

 Lab Guide

3. Select Yes on the popup page and click Delete on the Warning popup to
unexport the volume.

4. Select your Storage System, select Provisioning and Virtual Volumes. Select
the Unexported icon. Identify the Virtual Volume you have created with the
SmartStart Wizard. Right Click this Volume and select Remove.

Rev. 14.1 L -59

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

5. Select OK on the popup page and click Delete on the Warning popup to remove
the volume.

6. Select Hosts. Select the Host which was created during the SmartStart
installation. Right click the host and select Remove Host. Select OK to finish the
operation.

L -60 Rev. 14.1

 Lab Guide

7. It is NOT necessary to delete the CPGs created during the SmartStart setup. If
your group decides to delete the CPGs, you can do it from Provisioning  CPGs
 right click the CPG  select Remove. If the Remove button is greyed out, then
there is a Volume still associated with the CPG.

Note:
During storage system setup, some CPGs are created automatically.
For example, FC_r5, NL_r6 and so on.

Rev. 14.1 L -61

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Systems Tab operation

8. Start the HP 3PAR Management Console (GUI) and connect to your HP 3PAR
array (user: 3paradm, password: 3pardata) if you have not already done this.

9. Upon connection, you “arrive” on the Systems tab, which is shown on the lower
left hand corner of your GUI.

What are the different tabs you see in the lower left hand corner?
_________________________________________________________________
_________________________________________________________________
_________________________________________________________________

L -62 Rev. 14.1

 Lab Guide

10. Select View, Preferences…. on the upper left corner.

11. Explore the preferences you can set. Select the Advanced tab and select Show
logical disks. Click Apply and then OK to close the window.
12. Select the double arrow located on the lower left hand corner to modify the
available tabs. Select Add or Remove Buttons and verify that Layout Grid is
marked as available. If not, mark Layout Grid.

Rev. 14.1 L -63

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

13. While in the Systems tab, open up the tree under your HP 3PAR array. Click the
various components and answer the following questions:
What is your array serial number?
_________________________________________________________________

How many nodes do you have?

_________________________________________________________________

How much memory do you have?

_________________________________________________________________

How many drive cages do you have?

_________________________________________________________________

What type of drives and how many of each do you have?

_________________________________________________________________

Are there any licenses installed (Software tab)?

_________________________________________________________________

14. While in the Systems tab, right-click your array and select Set License.

L -64 Rev. 14.1

 Lab Guide

You have the option to use the License File option and point to the file location or
you can use the License Key option and copy/paste the file contents (without the
description) in the field provided.
Enter the license key (see the ClassFile folder for the license file of your system)
even if the system is already licensed.

15. While in the Systems tab, click Physical Disks and notice the type of disk of your
system.

16. Then, right-click any one of the available hard drives under the Physical Disk tab
in the centre window pane. Notice the option to locate a single drive.

Rev. 14.1 L -65

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Hosts Tab operation

17. To create a new host select the Hosts tab, open up the tree on your array and
right-click Hosts… . Select Create Host.

18. The wizard to create a host is the same as the one used using SmartStart. You
should see a Welcome page and the Host Settings page. The Welcome page
explains the process and gives details on the important components.
How do you bypass this page if you no longer wish to see it displayed?
_________________________________________________________________

19. On the Host Settings page, specify as Name, your Windows system name (for
example, S1-P7000-Grpn), then select Windows 2012 as Host OS. Leave all
other fields unchanged and click Next.
20. On the Fibre Channel Settings page, assign the available WWNs and click
Next.
21. Skip the iSCSI Settings page and click Next.
Note:
It is not supported have Fibre Channel and iSCSI links in a single Host
configuration.

22. Click Next to see a summary. Then click Finish.

L -66 Rev. 14.1

 Lab Guide

You now have a host you can provision and export volumes to.

Rev. 14.1 L -67

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Provisioning Tab operation

23. Select the Provisioning tab; click CPGs on your storage system, and the CPGs
tab, to see the CPGs already created (if any).

24. To create new CPGs, right-click CPGs and select Create CPG.

25. Notice any information advised on the Welcome page and click Next.
26. Create one or two CPG’s, depending on your array configuration and available
disk types.
Enter the name of the CPG; for example FC_R5_Group_X (Where X is your
Group number). Typically, a good practice is to use a name that identifies your
CPG drive type and RAID type.
Under Allocation Settings, set the Device Type to FC and RAID Type to RAID 5.
Leave the default Set Size and Device RPM.
L -68 Rev. 14.1
 Lab Guide

Note:
Notice that the message at the bottom may state that the system does
not have enough space.

27. Check the box for: Estimate free system space based on the current
Allocation Setting selections.
How much free space do you have?
_________________________________________________________________
28. Check the box for Show advanced options and click Yes if you get asked to
continue.
What are the additional options do you have now?
_________________________________________________________________
_________________________________________________________________
_________________________________________________________________

Rev. 14.1 L -69

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

29. Change Availability to Magazine and notice the change to the estimated free
system space.

L -70 Rev. 14.1

 Lab Guide

30. Explain why we have now free space!

_________________________________________________________________
_________________________________________________________________
_________________________________________________________________

31. Click Next.

Because Show advance options is selected you see an optional Disk Filter
menu. This allows you to limit the CPG to specific physical drives.
Do we recommend using this option for typical customers? Why?
_________________________________________________________________
_________________________________________________________________
_________________________________________________________________

Rev. 14.1 L -71

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

32. Click Next to see the Summary page. Verify all the fields are correct and click
Finish.
During the process of creating CPGs, you might possibly get one of the following
messages:

L -72 Rev. 14.1

 Lab Guide

If you want to create RAID 0 or Raid 5 for NL disk, you have to change system
parameter(s).

Note:
To change the default parameter, you need to use the setsys CLI
command. Refer to the HP 3PAR Command Line Interface Reference
Guide.

33. Create one additional CPG, if you have additional drive types in your array. (You
should have at least 2 CPGs created.)

34. To create a Virtual Volume, click Virtual Volumes on your storage system, and
the Virtual Volumes tab, to see the Virtual Volumes that are already created.

Note:
The admin and .srdata volumes are system volumes and created
automatically during the system setup. You cannot export nor delete
this volumes.

35. To create new Volumes, right-click Virtual Volumes and select Create Virtual
Volume... .

Rev. 14.1 L -73

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

36. You will see a Welcome page. Click Next.

37. On the Configure Virtual Volume page, specify a name for your virtual volume.
(Volume1_Win2012_Grpx where x is your group number).
38. Check the Export Volumes(s) check box underneath the Name field.
39. Select Fully Provisioned.
40. Enter a small size for your virtual volume. It is recommended, for lab purposes, to
use an “odd” or “unusual” size (like 7GB) so you can identify it easily when you
export it to your host.
41. As User CPG and Copy CPG, select one of the CPGs created previously.

L -74 Rev. 14.1

 Lab Guide

42. Check the Show advanced options at the bottom of the page.
43. Click Next. You will see some additional advanced menu options including Zero-
Detect, depending if you have selected “Thin provisioned” or “Fully provisioned”
for the volume.

Thick Provisioned Thin Provisioned

Rev. 14.1 L -75
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

44. Click Next to see a summary and click Finish.

45. Because you checked the Export Volume(s) after creation box previously, a
wizard will automatically appear which allows you to export your volume to your
desired host.
46. You will first see the Welcome page. Click Next.
47. Your volume will be automatically selected on the left Virtual Volume box.
On the right hand side, in the Export To box, select your Windows host.

48. Check the Show advanced options box.

What additional advanced options do you see?
_________________________________________________________________
_________________________________________________________________
_________________________________________________________________

49. Click Next to see the Summary page and then click Finish.
50. Logon to your server and verify that your volume exists (Server Manager  File
and Storage Services  Disks).
51. Bring the disk online, initialise the disk, and create a volume.

L -76 Rev. 14.1

 Lab Guide

Virtual Copy (Snapshot)

1. While still in your Provisioning tab, click Virtual Volumes under your array tree.
Then, highlight your newly created volume and select Create Virtual Copy.

2. A new window will appear. Next to Destination Name, you can enter a name of
the snapshot volume or you can choose an automated naming format. Select
Volume name followed by the current count of VVs created.

Rev. 14.1 L -77

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

3. Check the Show advanced options box.

What additional options are available?
_________________________________________________________________
_________________________________________________________________

4. No need to apply any advanced options. Just click OK to finish. You now have a
newly created snapshot of your virtual volume.

L -78 Rev. 14.1

 Lab Guide

5. Check the characteristics of the created snapshot.

Rev. 14.1 L -79

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Dynamic Optimization

1. While in the Provisioning tab, click your array tree and highlight Virtual Volumes.
Then, right-click your virtual volume. Select Tune Virtual Volume.

2. Select your volume and choose a different CPG for the User Space.

3. Click OK and monitor the tune progress and verify the change.

L -80 Rev. 14.1

 Lab Guide

Thin/Thick Conversion

You will now perform an online Thick to Thin conversion of the virtual volume that you
have created.
1. When a volume has a snapshot, you need to delete the snapshot before
converting or you need to keep the original volume. If you decide to keep the
original volume, the volume needs to be renamed. This can be specified in the
wizard.
2. Delete the snapshot by right clicking the snapshot and selecting Remove…

3. While you are in the Provisioning tab, click your array tree and highlight Virtual
Volumes. Right-click your virtual volume and select Convert Virtual Volume.

Rev. 14.1 L -81

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

4. A window appears and you now have some options. Select Fully provisioned to
Thin. Move the volume you want to convert to the Virtual Volume to Convert
box.
At the bottom, you have an option to move it to a different CPG.
What function is this similar to?
_________________________________________________________________

5. Click OK, and monitor the conversion task. When the conversion completes,
check the volume information. It should reflect the new provisioning type.

L -82 Rev. 14.1

 Lab Guide

Adaptive Optimization

Prior to 3.1.2, Adaptive Optimization was configured in System Reporter. Now, with
this latest release, you can configure AO on the array from the Management
Console.

1. While in the Provisioning tab, click your array tree and highlight AO
Configurations. Then right-click and select Create AO Configuration.

2. Click Next at the Welcome page.

3. You will see a Configure AO page where you would enter the AO parameters.
Give your AO configuration a name. For example, AO_Config_GroupX.
4. Choose Balanced for the mode setting.
5. Enter your CPG’s into the tiers fields. Not all three tiers need to be populated.
Since you only have two CPG’s so far, you can simply fill any of the two tiers
given.
Note:
Tier 0 is your highest tier. Therefore, you can enter your fastest CPG
here. Or you can leave it empty in case you plan to add a faster CPG in
the future.

Rev. 14.1 L -83

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

6. When done, click Next.

7. Now you will be able to specify when you want AO to run. You will now create a
schedule to run AO. Assume you want AO to run once a day during the weekday
only. You want AO to only analyze the work day in order to most accurately reflect
real production workload. Assume you want to analyze from 8am to 8pm only,
which is a 12 hour window.
Check the Start or schedule AO configuration box.
8. Select Create Schedule.
9. In the Max Run Time field, enter 12 hours.
10. Check Analyze Only.
11. In the Schedule box, give this schedule a name. For example, Weekday_8am-
8pm.

L -84 Rev. 14.1

 Lab Guide

12. In the Recurrence page, choose Advanced.

13. A new calendar pattern window will appear on the Custom Recurrence page.
Select:
0 in the Minutes field.
8 PM in the Hours field.
All in the Months field.
Monday, Tuesday, Wednesday, Thursday, Friday in the Days of Week
field
14. Click OK.

Rev. 14.1 L -85

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

15. Click Next to see a Summary.

16. Click Finish.
17. Verify that the AO Configuration was created.

L -86 Rev. 14.1

 Lab Guide

Scheduling Tasks
In this part of the lab, you now will schedule tasks through the 3PAR Management
Console.

1. Click the Task and Schedules tab on the lower right hand corner of the 3PAR
Management Console. Then, highlight your 3PAR array on the upper left hand
side. Then, click the Schedules tab.

Notice the System Task(s) used for housekeeping.

2. Now, click the Schedule button on the top bar. You should notice that you have
several different functions that you can schedule using the GUI.
Select Create Virtual Copy.

3. On the Create Virtual Copy page, select your volume in the drop-down list.
4. As Destination Name, select the drop down list and then select Volume name
followed by the current count of VVs created.
5. Enter a name for the schedule task. For example, “ Daily_Virt_Copy “

Rev. 14.1 L -87

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

6. Change the Recurrence to Daily, and the time to a minimum of 15 minutes

ahead from your current time.

7. When done, click OK.

8. After the specified time, verify that the scheduled snapshot of your volume
occurred.
Notice that the time of the RAID array is not always the same time as the time of
your Windows host, but it should be synchronized with the SP time.

L -88 Rev. 14.1

 Lab Guide

Lab 3 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Rev. 14.1 L -89

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

L -90 Rev. 14.1

 Lab Guide

Lab 4: CLI commands

Objectives
After completing this lab, you should be able to:
 Identify the different methods of CLI access to the controller nodes
 Install and use the CLI interface
 Explore the given HP 3PAR StoreServ 7000 Storage Systems
 Identify service relevant information

You will be working with your lab partners for this exercise.
Join the breakout room of your group (video and audio). One student drives the
session, by sharing his/her desktop. For each exercise, a different student should
drive the exercise.

Introduction
In this lab, you will get an overview about the different access methods to the CLI
command set of the controller nodes.
You will use CLI commands to discover the assigned HP 3PAR Storage System and
identify service relevant information.
Use the HP 3PAR Command Line Interface Reference guide for command details.

Rev. 14.1 L -91

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Different access methods to the CLI command set

The primary method to use the CLI command set is the HP 3PAR Command Line
Interface. To install this interface, start the setup for the given operating system on the
HP 3PAR CLI and SNMP CD. If the CLI interface is installed, start the interface and
connect to the controller node by specifying the IP address and username/password
of the controller node.

If the HP 3PAR Command Line Interface is not installed:

1. Locate the HP 3PAR CLI and SNMP ISO-image in the ClassFile folder on the
desktop.
2. Mount the ISO-image using the Virtual CloneDrive.
3. Install the HP 3PAR InForm OS Command Line Interface by accepting the
default setting.
4. Start the HP 3PAR InForm OS Command Line Interface.
5. If you get prompted for a system, enter the IP address of your assigned 3PAR
storage system.
6. The username/password you have now to specify are the same as you would
use to access the system using IMC, the graphical interface
(3paradm/3pardata).

If you have PuTTY or any other terminal emulator program supporting SSH on
your system, you can use these tools to connect directly to the IP address of
the storage controller and to then log in.
To use this method:
1. Start PuTTY from the desktop.
2. Enter the IP Address of your storage system and open a SSH session.
3. Login using the username and password of your 3PAR system
(3paradm/3pardata).

You can also access the CLI interface via the Service Processor using the
serial port or IP address.
To use this method:
1. Start PuTTY from the desktop
2. Enter the IP Address of your Service Processor and open a SSH session.
3. Login using the username and password of your Service Processor (3parcust).
4. Click 7 and press the Return key.

Note
By using this method, an internal account is used to login to the storage
system with service specific capabilities (3parservice).

L -92 Rev. 14.1

 Lab Guide

CLI command examples

As you go through the CLI command lab:
 Enter each command and describe the results below.
 Concentrate on those commands that might help you in diagnosis and
troubleshooting the array.
 Consider how the output of the commands assists you while servicing your
customers.
 Use the HP 3PAR Command Line Interface Reference Guide for more details.

Command
showsys
_________________________________________________________________
help showcage
_________________________________________________________________
showcage
_________________________________________________________________
showcage –d
_________________________________________________________________
showcage cage0
_________________________________________________________________
help showpd
_________________________________________________________________
cmore help showpd
_________________________________________________________________
showpd –i
_________________________________________________________________
showpd –e
_________________________________________________________________
showpd –failed
_________________________________________________________________
showpd –degraded
_________________________________________________________________
showpd –failed –degraded
_________________________________________________________________

Rev. 14.1 L -93

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

showpd –p –cg 0
_________________________________________________________________
showpd –p –cg 0 –mg 0
_________________________________________________________________
checkhealth
_________________________________________________________________
checkhealth –detail
_________________________________________________________________
showbattery
_________________________________________________________________
showbattery –log
_________________________________________________________________
showbattery –i
_________________________________________________________________
showalert
_________________________________________________________________
showalert –all
_________________________________________________________________
showalert –all –oneline
_________________________________________________________________
showeeprom
_________________________________________________________________
shownode
_________________________________________________________________
shownode –d
_________________________________________________________________
shownode –i
_________________________________________________________________
showport
_________________________________________________________________
showport –i
_________________________________________________________________
showversion
_________________________________________________________________

L -94 Rev. 14.1

 Lab Guide

showversion –b
_________________________________________________________________
showversion –a
_________________________________________________________________
showpatch
_________________________________________________________________
showinventory
_________________________________________________________________
showlicense
_________________________________________________________________
showlicense –raw
_________________________________________________________________

Rev. 14.1 L -95

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

CLI Commands to get specific information

Please answer the following questions. For questions that require you to issue
commands, include the command in your answer. You will be using the HP 3PAR
Command Line Interface Reference Guide as reference.

1. What command would you use to change the InServ name?

_________________________________________________________________
2. What is the command to determine the type of drive cage installed in the InServ?

_________________________________________________________________
3. What is the command to determine which node is the master node in the cluster?

_________________________________________________________________
4. What command displays the version of InForm OS installed?

_________________________________________________________________
5. What command displays which patches are installed?

_________________________________________________________________
6. What command displays applied licenses?

_________________________________________________________________
7. What command would you use to shut down the highest number node?
Note: Do not shutdown any node in the virtual lab environment!
_________________________________________________________________
8. What command would you use to shut down the array?
Note: Do not shutdown any system in the virtual lab environment!
_________________________________________________________________
9. What command would you use to reboot a node in the array?

_________________________________________________________________
10. What command would you use to reboot the entire array?

_________________________________________________________________

L -96 Rev. 14.1

 Lab Guide

11. What command displays what slots contain control memory in a node?

_________________________________________________________________
12. What command displays what N:P:S a drive cage is connected to on the A loop?

_________________________________________________________________
13. What command reports the most recent alert and severity?

_________________________________________________________________
14. What command reports the model and type of the internal node disks?

_________________________________________________________________
15. What command reports the vendor of the installed physical disk?

_________________________________________________________________
16. What command displays the batteries’ status information?

_________________________________________________________________
17. What command displays the details regarding the Data DIMM?

_________________________________________________________________
18. What command displays how many spare chunklets are available (free) on a
physical disk?

_________________________________________________________________
19. What command displays the disks installed in cage 1 only?

_________________________________________________________________
20. What is the command to display sparing algorithm and the command to change it
to minimal?

_________________________________________________________________

Rev. 14.1 L -97

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

21. What command reports the physical disk temperature?

_________________________________________________________________
22. What is the correct port mode for ports connected to drive cages?

_________________________________________________________________
23. What is the command to determine if the physical disks are running current
firmware?

_________________________________________________________________
24. What command would you use to capture the license information for restoration
purposes?

_________________________________________________________________
25. What command would you use to install the license?

_________________________________________________________________
26. What is the command you execute prior to performing any service action?

_________________________________________________________________
27. What command would you use to identify if there is a failed drive?

_________________________________________________________________
28. What is the command to find the manufacturers of the HBAs located in the node
and how do I locate the HP part number?

_________________________________________________________________
29. What command would you use to acknowledge the most current alert?

_________________________________________________________________
30. What command would you use to see the events of the last hour?

_________________________________________________________________

L -98 Rev. 14.1

 Lab Guide

31. What information is contained in the event log?

_________________________________________________________________
32. What command must you execute after completing a service activity?

_________________________________________________________________

Rev. 14.1 L -99

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Lab 4 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

L -100 Rev. 14.1

 Lab Guide

Lab 5: System upgrade

Objectives
After completing this lab, you should be able to:
 Identify steps to add disk to an existing HP 3PAR StoreServ 7000 System
 Identify steps to add disk chassis to an existing HP 3PAR StoreServ 7000 System
 Rebalance the storage system
 Upgrade a two node P7400 to a four node system

Introduction
In order to upgrade an HP 3PAR StoreServ 7000 System, you must make sure that,
after the upgrade, you will maintain at least the same availability level and that the
system is balanced. To achieve this, you have to follow the configuration guidelines of
3PAR storage systems.
After an upgrade, you have to tune the storage system so existing volumes can also
benefit from the new hardware.
Use the HP 3PAR StoreServ 7000 Storage Service Guide: Service Edition located in
ClassFile folder as reference.

Note:
Due to the fact that you don’t have physical access to hardware, you
have to use the videos of the HP Partner Services Media Library to see
the manual operation required to install and replace hardware
components.

Rev. 14.1 L -101

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Drive Enclosures and Disk Drives Upgrade

You can work on your own in this exercise.
1. Use the HP 3PAR StoreServ 7000 Storage Detailed Configuration Guide and the
QuickSpecs to verify the minimum supported increment of disk drive of any type
for your system.

2. Review the required steps for how to install a drive using the HP 3PAR
StoreServ 7000 Storage Service Guide: Service Edition and check the FRU
videos of the Partner Services Media Library (PSML).

L -102 Rev. 14.1

 Lab Guide

3. Use the PSML for details of how to handle the hardware components.

After physically installing new disk drives, it is necessary to redistribute the data
of the existing volumes.

Rev. 14.1 L -103

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Rebalance the storage system

Join the breakout room of your group (video and audio). One student drives the
session, by sharing his/her desktop. For each exercise, a different student should
drive the exercise.
With Dynamic Optimization you can change the virtual volume’s parameters, RAID
levels, and set sizes by associating the virtual volume with a new CPG.
Dynamic Optimization enables you also to update the layout of volumes to take
advantage of the current system configuration. For example, when a system is
upgraded by adding nodes, cages, or physical disks, the initial volume and logical disk
layouts may no longer be optimal for the new system configuration. Updating the
system layout optimizes the use of all physical resources in the system at a given
time.
In previous releases the re-tuning process was manual; retuning an entire system
could take several days (mostly just waiting for tunes to finish).
Now the TuneSys command can be thought of as an enhanced and automated
version of Dynamic Optimization, performing VV/LD re-layout.
Phase 1 – Internode tuning:
 Used when new nodes are added.
 Redistributing existing LDs for VVs across all nodes evenly.

Phase 2 – Intranode tuning:

 Used when new capacity added to a node pair.
 Redistributing LDs across all disks, including newly added disks evenly.

Phase 3 – CPG consistency checks & associated LD re-tunes:

 Redistributing chunklets of existing LDs across all disks in a CPG, including
newly added disks and or LDs attached to newly added nodes.

Each phase consists of:

 Analyzing the running system for imbalances or discrepancies.
 Activating a tune controller, this runs and monitors the tunes.
 Before starting each tune, TuneSys waits for chunklet cleaning to complete in
order to maximize the space available for the tune.

Administration & best practices:

 Tunesys requires no direct administration.
 Tunesys is “fire and forget” – no user interaction is needed once it starts.
 The default options for -nodepct, -chunkpct and -diskpct should be OK for
most systems.
 It is possible to perform a dry run of tunesys to see what tuning will be
suggested for the current configuration.

Note that dry run analysis is limited to inter-node tuning.

L -104 Rev. 14.1

 Lab Guide

System Tune using GUI (MC)

1. Select the Provisioning Action and then select your System. Tune System will
appear on the task bar.

2. Clicking Tune System will bring up the “Tune System” wizard. Select Analyze
only and click OK.
Note:
Select the Analyze only checkbox if you want to view an analysis
instead of running the tuning task. Running the tuning task can take
several hours depending on what have to be tuned.

Rev. 14.1 L -105

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

3. Note the task ID.

4. Click OK and right-click the task and select Show Details.

L -106 Rev. 14.1

 Lab Guide

Rev. 14.1 L -107

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Verify a System Tune process (optional)

This lab is written without defining the details of every step on purpose. This is to
encourage a group discussion and this way students can demonstrate what they have
learned in the previous steps.
During this lab, you create a VV that does not use all the disks of the system. After
verifying that not all disks are used by this VV, you allow it to use all disks and start the
system tune again.
Finally, you verify that the tune task is running successfully.
 Create a CPG with an active filter, so that not all available disks are used. (For
example, not the drives in cage 0)

 Create and export a VV by using this CPG.

 Start a system tune with the default settings as DryRun.
 Check the task details.
 Start a normal run of tune system with the default settings.
 Check the task details and note the steps during the tune (this can take up to 30
minutes).
 When tuning is done, check the Layout Grid of the VV you have created.
 Verify that the chunklets used by the VV are only on disks you have selected for
this volume.
 Modify the CPG so that all disks of this type can now be used.
 Start the tune system again, with default settings.
 After the tune task has finished, verify that chunklets from all disks are now used
for this VV.
Note: if you do not see a change, check to see if your volume is initialized and
formated in windows. If you have a thin provisioned volume, copy some data onto the
disk.

L -108 Rev. 14.1

 Lab Guide

Lab 5 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Rev. 14.1 L -109

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

L -110 Rev. 14.1

 Lab Guide

Lab 6: HP 3PAR Service Processor Onsite Customer

Care (SPOCC) usage

Objectives

After you have completed this lab activity, you will be able to:
 Access the Service Processor using the Web
 Explore the SPOCC functionality
 Update/Patch HP 3PAR InForm OS using SPOCC
 Using Guided Maintenance

Join the breakout room of your group (video and audio). One student is driving the
session, by sharing his/her desktop. For each exercise, a different student should
drive the exercise.

Introduction
SPOCC is a suite of service tools applications that provide a web-based user interface
for support of the Service Processor (SP) and the HP 3PAR array. It provides a
vehicle to review logs and files, to store various types of support documentation, and
to manually record product configuration details that are not directly available from the
product.
Use the HP 3PAR Service Processor Onsite Customer Care (SPOCC) User’s Guide,
HP 3PAR OS Upgrade Instruction as reference.

Rev. 14.1 L -111

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Exploring the SPOCC functionality

1. Use the HP 3PAR Service Processor Onsite Customer Care (SPOCC) User’s
Guide to make yourself familiar with the structure and functionality of the web
interface SPOCC.

2. Use the Web browser to connect to the Service Processor and login as user
3parcust and the password 3parcust.

L -112 Rev. 14.1

 Lab Guide

3. Select Files:
Explore the file structure but do not delete or change anything.

4. Select Support:
Notice the two support activity pages; InServ and Service Processor.

5. Select Update:
Notice the order of updating the Service Processor and HP 3PAR OS. A lab will
follow to update HP 3PAR OS.

6. Select Notify:
Notice the starting page to setup local notification (details will follow on how to
setup notification during the class).

Rev. 14.1 L -113

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

7. Select Report:
How many files are in the transfer queue of your system?

8. Select Setup:
Which utilities are provided?

9. Select SPmaint:

L -114 Rev. 14.1

 Lab Guide

Which SP Maintenance menu item you have to use to add or remove a storage
system from the Service Processor?

Note:
For selected functions such as upgrading, patching the HP 3PAR OS or
using Guided Maintenance, dedicated labs will follow.
10. Select Help:
Get help information for a selected subject.

Rev. 14.1 L -115

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

SPOCC should be used as primary interface to the Service Processor. If for any
reason you are not able to use the Web access, you can use PuTTY to connect to the
Service Processer via SSH and have access to SPmaint.
If you use a physical Service Processor you can also connect to the Service
Processor via the serial port.
11. Use PuTTY to connect to the IP address of your Service Processor and login as
user 3parcust and the password 3parcust.

12. Walk through the menus and sub menus to get familiar with the structure but do
not change anything unless you are asked to do so.
Put a focus on:
 Execute a CLI command
 Network configuration
 Connection Portal Control
 Display SP Version
 Health Check
 Reboot an InServ cluster/node

L -116 Rev. 14.1

 Lab Guide

Maintenance Mode
13. To set the Service Processor in Maintenance Mode, use the Web browser to
connect to the Service Processor and select SPmaint. Click 3. InServ
Configuration Management.

14. Select Modify to get the Service Processor InServ Info page.

15. Click the On button of the Maintenance Mode and specify how long this mode
should be enabled, and then click Update InServ Info.

Rev. 14.1 L -117

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

16. Go to Support to verify that the Maintenance Mode is enabled.

L -118 Rev. 14.1

 Lab Guide

HP 3PAR InForm OS Update/Patch using SPOCC

HP 3PAR InForm OS update packages include updates for components (the storage
system OS, BIOS, cage, and drive firmware). Individual component updates are
applied if the current software and hardware configuration is operating below the
version levels included in the update package.
HP 3PAR InForm OS patches are always performed online, no reboot is required.
HP 3PAR InForm OS updates may either be performed online or offline. Online
updates allow the hosts connected to the system to continue I/O activity. For offline
updates, host I/O activity must be stopped.
The following types of updates are supported:
 Simple Update - used when the OS maintenance level changes. An update is
applied to all the nodes and nodes reboot, one at a time, automatically.
 Advanced Update - used when the OS maintenance level changes. An update is
applied to all the nodes and user needs to reboot the node, one at a time,
manually. Advanced update is only used in cases where a customer chooses to
validate their environment before rebooting the next node. The duration of the
pause between node reboots must be limited to 30 minutes.
 Offline Update - used when the customer requests an offline update, or when the
system is operating several levels below the update package version. An offline
update requires that all hosts connected to the Storage Server be shut down
before the OS update. All nodes are updated and then rebooted at the same time.
The system is unavailable to the hosts during this operation.
Each InForm OS update/patch has their specific pre-installation requirements which
are described in the related InForm OS installation instruction document.
Never initiate any InForm OS update without studying the Installation Instructions
for the specific update.
If the Service Processor (SP) software level does not meet the requirements for the
InForm OS update, the SP software must be updated before continuing with the
InForm OS update installation.

With version 3.1.2 MU2 or later, you use the same procedure to install an update or to
apply a patch for HP 3PAR OS.
 Use the Web browser to connect to the Service Processor and login as user
3parcust and the password 3parcust.
 Click Update from the left column menu of the SPOCC homepage.

Currently, there is no other HP 3PAR OS version available newer then build 3.1.2.422
(MU2). To give you an example how to install an upgrade, you must downgrade the
storage system to version 3.1.2.322 (MU1). The steps are the same as you would
upgrade the system.

Rev. 14.1 L -119

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

1. Start the Web browser and connect to your Service Processor and login using the
3parcust account and select Update.

2. Click 2 (Update HP 3PAR OS).

3. Click 1 (Prerequisites) and notice were you can find the required documents. Use
Return button to go back to the Update menu.
4. Click 2 (Validate update readiness). Use Return button to go back to the Update
menu.

L -120 Rev. 14.1

 Lab Guide

5. Click 3 (Load Update). Select ISO Image and use Browse… go to the
ClassFiles folder and select the
3PAR_OS_Release_3.1.2_MU1_Software_QR482-10007 ISO image and
click Load.

6. Wait until the update (in this case, it is actually the downgrade) is loaded and
then click Yes to start to install this “update.”
Rev. 14.1 L -121
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

7. Set the installation type Online and select 3.1.2.322 as the update to install.
Click Continue.
Note:
To downgrade the system you have to select the installation type
Offline.

8. Read the message and click Install.

L -122 Rev. 14.1

 Lab Guide

9. Monitor the progress of the update and wait until you see the message Update
succeeded. Notice the build number change and click Close.

Now you are ready to upgrade the storage system from 3.1.2.322 (MU1) to 3.1.2
(MU2).
This time, you do not have to load the update because it is part of the Service
Processor installation image.
Rev. 14.1 L -123
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

10. Click 4. Install HP 3PAR OS update to start the upgrade.

11. Select 3.1.2.422 as update to install, and leave the installation type Online
selected. Click Continue.

12. Read the message and click Install.

13. Monitor the progress of the update and wait (up to 10 min on a two node
system) until you see the message Update succeeded. Notice the build
number change.

L -124 Rev. 14.1

 Lab Guide

Due to the fact that there is no patch higher then P11 available, you can reinstall
the patch P11 just to see how the process works to install a patch.

Rev. 14.1 L -125

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

14. Click 3 (Load Update). Select ISO Image and use Browse… go to the
ClassFiles folder and select the
3PAR_OS_Release_3.1.2_MU2_P11_Software_QR482-10022 ISO image and
click Load.

15. Wait until the update (a patch, in this case) is loaded, and click Yes to start to
install this update.

L -126 Rev. 14.1

 Lab Guide

16. Select 3.1.2/P11.patch as the update to install and leave the installation type
Online selected. Click Continue.

Rev. 14.1 L -127

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

17. Read the message and click Install.

18. Monitor the progress of the update and wait until you see the message Update
succeeded. Notice the build number. At this time you will not see a change
because the already installed patch was reapplied.

L -128 Rev. 14.1

 Lab Guide

Rev. 14.1 L -129

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Service Processor Update/Patch using SPOCC

To update Service Processor software or to apply a patch is very similar to doing
the upgrade or to patch HP 3PAR OS.
Even here, you do not have an update or patch for the Service Processor. You will
just reapply the current version to see the steps and the output you get during the
installation. The Service Processor version will not change during this lab.
1. Start the Web browser and connect to your Service Processor and login using the
3parcust account and select Update.

2. Click 1 (Update Service Processor).

L -130 Rev. 14.1

 Lab Guide

3. Click 1 (Prerequisites) and notice were you can find the required documents. Use
Return button to go back to the Update menu.
4. Click 2 (Load Update). Here you can load the ISO image to the Service
Processor.

5. Go back to Update and click 3 (Install SP Update) to see how you can select and
start the upgrade. Click Continue.

Rev. 14.1 L -131

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

6. Read the message and click Install.

7. Monitor the progress in the TPD/SP Loading/Update – Log Window.

L -132 Rev. 14.1

 Lab Guide

8. Finally you will see the SP Update Successful message.

Rev. 14.1 L -133

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Guided Maintenance
Maintenance activities have been scripted so that engineers do not need to use
maintenance manuals or enter any commands.
System status is derived at the start of each script and appropriate command output is
presented. Images relate to the specific system configuration (where possible) and
two types of maintenance log files are created and transferred to 3PAR Central at the
conclusion of each script.
This lab will show you how you can access the Guided Maintenance Page of the
Service Processor in the lab environment.

Important:
Be aware that you don’t have physical access to the hardware so
you are quite limited in the way to use the live procedures.
In all of the procedures, the component to replace is powered off
by the procedure. The component cannot be powered on
remotely; manual intervention is required!

Guided Maintenance for the HP 3PAR StoreServ 7000 system covers only the
replacement of nodes or node components.

L -134 Rev. 14.1

 Lab Guide

Use the HP 3PAR StoreServ 7000 Storage Service Guide: Service Edition for a
detailed description about the required steps to replace any of the storage system
components.

You are working by your own in this exercise.

Reading exercise (~30 minutes)
Use the HP 3PAR StoreServ 7000 Storage Service Guide: Service Edition to
familiarize yourself with the service activity to replace:
 Disk Drives
 Controller Nodes
 Disk Enclosure
 I/O Modules
 Power Cooling Modules
 Battery inside a Power Cooling Module
 Controller Node Internal Components

Rev. 14.1 L -135

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Lab 6 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

L -136 Rev. 14.1

 Lab Guide

Lab 7: Node Rescue

Objectives
Node Rescue is used to restore the internal disk of a node. This can be the case
if the disk in a failed node is questionable and replaced by a blank disk. The
rescue can be performed as:
• Auto Node to Node Rescue
• Service Processor (Physical) Node Rescue

Join the breakout room of your group (video and audio). One student is driving
the session, by sharing his/her desktop. For each exercise, a different student
should drive the exercise.

Auto Node to Node Rescue

Auto node to node rescue is started automatically when a node is removed and
replaced. There must be at least one node available in the cluster.

Note:
The automatic node rescue is the default and preferred procedure.
When performing auto node to node rescue, there may be instances
where a node is to be rescued by another node that has been inserted
but has not been detected. If this happens, issue the CLI command,
startnoderescue –node <node nb>.
Before you can issue this command, you must have the rescue IP
address. This is the IP address that is allocated to the node being
rescued and must be on the same subnet as the SP.

If a Node Rescue operation is running use the showtask -d <id> command to

view detailed status of the task.
To manually initiate a Node to Node Rescue, you have to halt the node which you
like to rescue and connect to a console port of another controller node.
1. To halt a node login to SPOCC.
2. Select SPmaint and click 4. InServ Product Maintenance.

Rev. 14.1 L -137

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

3. Click Halt an InServ cluster/node.

4. Confirm the popup message.

5. Select node 3 (or 1) to halt and click -- halt node/cluster -- ( serial port is
connected to node 0).

6. Confirm several popup messages.

7. Click SPmaint and then Execute a CLI command.

L -138 Rev. 14.1

 Lab Guide

8. Submit the shownode command to verify that node 3 (1) is not part of the
cluster.

9. Start PuTTY from the desktop.

Note:
It is a good practice to enable session logging when using PuTTY to
track command history or to read command output which would be
bigger than the window buffer.

10. Load the stored session StoreServ 7000 - Serial.

11. Verify that logging of all session output is enabled.

Rev. 14.1 L -139

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

12. Login using the Console username and password of your storage system.
(console/cmp43pd)
13. The 3PAR Console Menu will appear.

14. Select option 5, Perform a Node-to-Node rescue.

15. Enter number of the node 3 (or 1) which is halted.
16. Type yes to confirm.
17. The output will be similar to this.

L -140 Rev. 14.1

 Lab Guide

18. In SPOCC  SPmaint  Execute a CLI command enter the showtask command
to get the task ID of the rescue task.

19. Enter showtask –d <id> to get task details.

Rev. 14.1 L -141

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

20. Repeat this command until the rescue task completed (few minutes).

21. Use shownode to verify that the node is back.

L -142 Rev. 14.1

 Lab Guide

Service Processor (Physical) Node Rescue

The SP node rescue should be used only in cases when the storage system
includes a physical SP and no nodes remain in the cluster or when all nodes are
down.
The virtual lab environment is set up without a Physical Service Processor.
Use the HP 3PAR StoreServ 7000 Storage Service Guide: Service Edition for
details about the Service Processor to node connection and the required steps to
perform this type of rescue.

Rev. 14.1 L -143

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Lab 7 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

L -144 Rev. 14.1

 Lab Guide

Lab 8: Manage Events and Alerts

Objectives
After completing this lab, you should be able to:
 Manage Events using the Management Console
 Manage Alerts using the Management Console
 Manage Alert/Events using the CLI

Join the breakout room of your group (video and audio). One student is driving the
session, by sharing his/her desktop. For each exercise, a different student should
drive the exercise.

Introduction
The system tracks system events internally. Some of these events require action from
the system administrator, so they trigger alerts that are displayed for the administrator.
Events are logs created by the system which allow you to view the day-to-day health
of your system.
Alerts are generated by the system when a system event requires immediate operator
action/attention. An alert may be triggered by a single event, a combination of events,
or by repeated occurrences of the same event.
The system administrator is responsible for responding to alerts.

Rev. 14.1 L -145

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Managing Events
Events provide an overview of the health of the storage systems.
To view the events:
1. Open the HP 3PAR Management Console
2. In Manager Pane, click Events & Alerts and in the Management Tree select the
Storage Systems you wish to view.

3. In the Management Window, click the Events tab.

4. Select the time frame for which you want to view events, and then click Get
Events.

L -146 Rev. 14.1

 Lab Guide

5. To view events of a specific timeframe, click Advanced and specify the timeframe.
6. Take one event of your system and describe the information of the different field:
 Severity
_________________________________
 Time
_________________________________
 Description
_________________________________
 Type
_________________________________
 Node IP
_________________________________
 Component
_________________________________
 Message
_________________________________

7. What is the reason for this event?

_______________________________________________________________
Rev. 14.1 L -147
HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Managing Alerts
Alerts are system events that require the immediate attention of the
administrator/operator and might also require manual intervention. Most system alerts
are generated automatically without any need for user configuration. Alerts can be
also triggered by thresholds.
To view system alert information:
1. In the Manager Pane, click Events & Alerts.
2. In the Management Tree, click the Storage Systems node or a system node.
3. In the Management Window, click the Alerts tab.

4. Right-Click the alert and list the actions

___________________________________
___________________________________
___________________________________
___________________________________
5. Choose one alert and analyze the fields and messages.
What is the reason for this alert?

________________________________________________________________
________________________________________________________________
________________________________________________________________

6. Follow the link to alert actions under Recommended Actions.

L -148 Rev. 14.1
 Lab Guide

7. At the HP Storage Systems Guided Troubleshooting web site, follow the link for
your product. (HP StoreServ  HP StoreServ 7000)
8. At the bottom of the HP 3PAR product page, click the link for HP 3PAR Alert
Messages.
9. At the bottom of the Alert Messages page, choose the correct message code
starting with the first four characters of the alert.
10. On the next page, the message code series is narrowed by choosing the next digit
in the code.
11. On the next page, select the message code that matches the one that appeared in
the alert.
12. The next page shows the message type based on the message code selected and
provides a link to the suggested action. Follow the link.
13. On the suggested actions page, scroll through the list to find the message state
listed in the alert message. The recommended action is listed next to the message
state.

Rev. 14.1 L -149

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Managing Alerts/Events using CLI

As part of the CLI Command lab exercise you have seen some of the CLI commands
to manage Alerts and Events.
Review the commands to:
 View Alerts/Events for a specific system.

______________________________________________________________

 Manage Alerts/events

______________________________________________________________

 Get details of a specific Alert/Event.

______________________________________________________________

L -150 Rev. 14.1

 Lab Guide

Lab 8 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Rev. 14.1 L -151

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

L -152 Rev. 14.1

 Lab Guide

Lab 9: De-installation of a 3PAR Storage System

Objectives
After completing this lab, you should be able to:
 Connect the Maintenance PC to a Controller Node
 De-install a HP 3PAR Storage System

Join the breakout room of your group (video and audio). One student is driving the
session, by sharing his/her desktop. For each exercise, a different student should
drive the exercise.

Introduction
In this lab, you will de-install a 3PAR system.
This is in preparation for the installation lab.
After this, all data on the volumes are no longer accessible and the whole
configuration of the 3PAR is deleted.
This process is also called Back in the Box process.

Note
This is a data destructive process!

Rev. 14.1 L -153

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Connect the Maintenance PC to a Controller Node

The Console port of Node 0 is already connected in the virtual lab environment. You
can easily use PuTTY to connect to this port.
In a customer environment, you have to physically connect the Maintenance PC to the
console port. Use the Installation Guide to get details of how to get physically
connected to the console

De-Install a HP 3PAR Storage Server

To de-install a HP 3PAR Storage System, you have to connect to the console port of a
storage node. In most cases, you only log in via the serial port if a specific
management function is not available via the Service Processor, or the normal
management interface. One of the specific functions is de-installing the system (Back
in the Box).
Take care when selecting options while in 3PAR Console Menu. Some options are
customer data destructive and not reversible. Never execute a command or menu
option without fully understanding the result.

1. Start PuTTY and connect to the console port and verify that logging of all session
output is enabled.

2. Login using the Console username and password of your storage system.
(console/cmp43pd)
3. The 3PAR Console Menu will appear.

L -154 Rev. 14.1

 Lab Guide

4. Select option 6. Set up the system to wipe and rerun ootb.

Note
The system initialisation is call oobt (out of the box) procedure

Note
Do not select option 8, it will take a very long time because it has to
perform a secure system wipe of the data on the array.

5. Review the warning message and enter yes to proceed.

6. This process may take approximately five minutes. Do not interrupt this process.

Rev. 14.1 L -155

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

7. Close the PuTTY session and save the session protocol of this session.

The storage system is now (nearly) in the same state as a new system from the
factory.

L -156 Rev. 14.1

 Lab Guide

Lab 9 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Rev. 14.1 L -157

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

L -158 Rev. 14.1

 Lab Guide

Lab 10: Installing Storage Software Manually

Objectives
After completing this lab, you should be able to:
 Setup a HP 3PAR StoreServ 7000 storage system without SmartStart or the
Service Processor wizard
 Setup the Virtual Service Processor without SmartStart
 Install HP 3PAR OS Files
 Add the Storage System to the Service Processor

Join the breakout room of your group (video and audio). One student is driving the
session, by sharing his/her desktop. For each exercise, a different student should
drive the exercise.

Introduction

Use the Installing HP 3PAR Storage Software When HP 3PAR SmartStart is

Unavailable section of the HP 3PAR StoreServ 7000 Storage Installation Guide if you
cannot use SmartStart, but an initialized Service Processor is available.

Use the Installing Storage Software Manually section of the HP 3PAR StoreServ 7000
Storage Service Guide Service Edition as reference if you have to setup the storage
system without assistance of the Service Processor. Use the same document as
reference if you have to setup the Service Processor using Moment Of Birth (MOB).

Rev. 14.1 L -159

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Setup a HP 3PAR StoreServ 7000 storage system without

SmartStart or the Service Processor wizard
To set up and configure the storage system software and SP manually, you must
execute these scripted procedures from a laptop after powering on the storage
system.
You need a connection from the Management System to the Console port (Mfg) of a
highest number Controller Node. Use a RJ45 cable and the grey connector (PN 180-
0055). The grey connector has to be plugged into the serial port of the Management
Server or to a USB-to-serial adapter, if required.
The COM1 port of the Window server in the virtual lab is already connected.

1. Start PuTTY and connect to the console port and verify that logging of all session
output is enabled.

2. Login using the Console username and password of your storage system.
(console/cmp43pd)
3. The 3PAR Console Menu will appear.

L -160 Rev. 14.1

 Lab Guide

4. Select option 1. Out Of The Box Procedure.

5. Type yes to confirm that you wish to continue.

6. Verify the number of controller nodes in the system, then type c.

Rev. 14.1 L -161

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

7. Set up the time zone for the operating site. Select 8 for Europe, 16 for Germany
8. Enter 1 to confirm that the time zone and location information are correct.

9. Accept the actual time and date, or type the date and time in the format
<MMDDhhmmYYYY>, and then press ENTER.
10. To confirm the date setting, type y and press ENTER.

11. Name the storage system using up to 31 alphanumeric characters for example
P7400-x where x is your group number. Type yes and press ENTER to confirm
the name.

12. Verify the OS version is correct. Type c and press ENTER to continue.

L -162 Rev. 14.1

 Lab Guide

13. Verify the number of drives in the storage system. Type c and press ENTER to
continue.

14. The system prompts you to begin the system stress test script. Type y and press
ENTER. The system stress test continues to run in the background as you
complete the system setup.

Rev. 14.1 L -163

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

15. Enter 1 to specify an IPv4 address.

16. Specify the IP-address for the array. (See the table at the beginning of the Lab 2).
17. Specify the Netmask (press enter to use the default 255.255.255.0).
18. Specify no gateway.
19. Type auto for the network speed.
20. Press enter to use no NTP server.
21. Press enter to use no DNS server.
22. Type no to allow non-encrypted ports.
23. Review the summary and type y, to confirm that the settings are correct.

L -164 Rev. 14.1

 Lab Guide

24. Select default “D” as spare chunklet selection algorithms.

25. Verify the licenses shown (if any) and press ENTER.

Rev. 14.1 L -165

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

26. Type n to disable the CIM based management API.

L -166 Rev. 14.1

 Lab Guide

27. Verify that the OOTB successfully completes.

Rev. 14.1 L -167

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Setup the Virtual Service Processor using Moment Of Birth (MOB)

Important:

This procedure is not intended for customer use and should be used
only if SmartStart or Setup Wizards cannot be run.

Install the Virtual Service Processor like the lab exercise 3.

1. Open the ClassFile folder on your desktop and right-click the
3PAR_Virtual_SP_4.1_MU2_P05_Software_QR482-11007.iso file to mount the
ISO image.

2. Start the HP 3PAR vSphere Client (use the link on your desktop).
3. Use the listed IP address and login with the username root and the password
HP-P7000.

4. Right-click the existing Virtual Service Processor; Power Off and delete this VSP
from disk.

L -168 Rev. 14.1

 Lab Guide

5. In the VMware vSphere Client window, select File > Deploy OVF template to
import the VSP (VM) OVF file.

6. On the Source page, click Browse to import the OVF from its location (mounted
ISO image), then click Next.

Rev. 14.1 L -169

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

7. On the OVF Template Details page, verify the OVF template, and then click
Next.

8. On the Name and Location page, type a name for the VSP (Virtual SPn, n=lab
group number), then click Next.

9. On the Disk Format page, select Thin Provision, then click Next.

L -170 Rev. 14.1

 Lab Guide

10. On the Network Mapping page, click the field below Destination Networks and
map your virtual machine to networks in your lab group inventory (Group<n>
network), then click Next.

11. On the Ready to Complete page, review the deployment settings and select the
Power on after Deployment checkbox, then click Finish.

12. A Deployment Completed Successfully message displays after a few minutes.

13. Click Close.

Rev. 14.1 L -171

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

14. To verify that the system is started, right-click the VSP in the virtual machine list,
select Open Console, and check that the system is ready to login.

15. The Lab environment doesn’t have DHCP enabled. To set the VSP IP address
login as setupusr without a password.

16. Enter IP address, Netmask and default gateway according the following table
shown and type Y.

L -172 Rev. 14.1

 Lab Guide

Group VSP IP-addr Netmask Gateway

1 192.168.0.15 255.255.255.0 192.168.0.13
2 192.168.0.26 255.255.255.0 192.168.0.24
3 192.168.0.37 255.255.255.0 192.168.0.35
4 192.168.0.48 255.255.255.0 192.168.0.46
5 192.168.2.54 255.255.255.0 192.168.2.52
6 192.168.2.64 255.255.255.0 192.168.2.62
7 192.168.2.74 255.255.255.0 192.168.2.72
8 192.168.2.84 255.255.255.0 192.168.2.82
9 192.168.2.94 255.255.255.0 192.168.2.92
10 192.168.2.104 255.255.255.0 192.168.2.102
11 192.168.2.114 255.255.255.0 192.168.2.112
12 192.168.2.124 255.255.255.0 192.168.2.122

17. Press Ctrl-Alt to release the cursor and continue to set up the VSP with
SmartStart.
18. In the VMware vSphere Client window, select the Summary tab for the IP address
to display (in the General section, IP address field).

Rev. 14.1 L -173

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

19. Verify that you see the IP address.

20. Go back to the Console of the VPS and use the username root without a
password to login.

Note:
This works only the first time to enable the SP to be configured.

21. Enter yes if you are ready to configure the SP.

22. Take the default no because this is not a Secure Site.

L -174 Rev. 14.1

 Lab Guide

23. Select 2 because SP Mode is no longer supported for new systems.

24. Select 1 to continue with new site installation.

25. For the 3PAR Serial Number of your system, see the table shown previously.

Rev. 14.1 L -175

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

26. No uninitialized system will be detected; continue setting up the SP.

27. Accept the default host name.

28. Enter the IP address, netmask, and gateway of your SP; see the table above.

L -176 Rev. 14.1

 Lab Guide

29. Take the default none for DNS.

30. Specify that no Policy Manager will be used, that remote access will be allowed,
and confirm the setting.

31. This system is a Production system and no proxy server is required. Confirm the
setting.

Rev. 14.1 L -177

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

32. Type none if asked for NTP server.

33. Configure the firewall in permissive mode.

34. Confirm the network settings.

L -178 Rev. 14.1

 Lab Guide

35. Enter the country code of 53 for Germany.

36. Enter 8 (Europe) as location, 15 (Germany) as country and confirm the settings.

Rev. 14.1 L -179

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

37. Accept the defaults for the current date and time, and confirm the settings.

38. Enter y to use the DEFAULT Site key files

39. Notice the message about Inserv configuration and press ENTER.

40. The network and connectivity test starts. If the test fails, review the network
settings. But you will also have the option to continue in spite of errors.
41. Enter DE as country code of Germany.

42. Enter the support information as seen on the summary page and agree it is
correct.

L -180 Rev. 14.1

 Lab Guide

43. Do not load a Customer Documentation CD.

44. Complete MOB in spite of the error.

At the end, you will be asked to shut down the Service Processor, but instead, you
should reboot the Service Processor using the following command:
/sbin/shutdown -r now

The Service Processor is now configured. Use SPOCC or the SPmaint menu for
further configuration tasks.

Rev. 14.1 L -181

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Install HP 3PAR OS Files

HP 3PAR OS files must first be installed on the SP in order to communicate and add a
storage system.
The steps are the same as if you update the HP 3PAR OS or Service Processor files.
Before you install the HP 3PAR OS files, verify if the required files are already
installed.
1. Use the Web browser to connect to the Service Processor and login as user
3parcust with the password 3parInServ.

Note:
During manual setup of the Service Processor the 3parcust password
will not be changed. 3parInServ is the current default password for the
3parcust account and maybe changed in the next releases of SP.

2. Click SPmaint from the left column menu of the SPOCC homepage.
3. Click 1 (SP Control/Status).
4. Click 1 (Display SP Version).

If the HP 3PAR OS version of your storage system is not listed, perform a Load
Update of the required version of the HP 3PAR OS software. Refer to the HP
3PAR OS update lab.

L -182 Rev. 14.1

 Lab Guide

Add the Storage System to the Service Processor

After successfully completing the Service Processor Setup, you must add the storage
system to the configuration database of the SP. Adding the storage system permits the
SP to communicate, service, and monitor the health of the system.
1. Use the Web browser to connect to the Service Processor and login as user
3parcust with the password 3parInServ.
2. Click SPmaint from the left column menu of the SPOCC homepage.
3. In the SPmaint, select 3 InServ Configuration Management.

4. In the InServ Configuration, select Add New InServ.

5. Enter the IP address of the storage system, the user 3paradm and the password
3pardata and press Add IP Address to add the storage system.

Rev. 14.1 L -183

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

6. After successfully adding the system, you are finished with this lab.

L -184 Rev. 14.1

 Lab Guide

Lab 10 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Rev. 14.1 L -185

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

L -186 Rev. 14.1

 Lab Guide

Lab 11: Configure Storage System

Objectives

After completion of this lab, you will be able to:

 Configure the Fibre Channel Host ports
 Verify MPIO settings

You will be working with your lab partners for this exercise.
Join the breakout room of your group (video and audio). One student will drive the
session by sharing his/her desktop. For every exercise, a different student should
drive the exercise.

Introduction

In this lab, you will learn the configuration tasks required to configure Fibre Channel
host port..
In the previous lab, you have used SmartStart to configure a storage array which also
includes the setup of host ports. In this lab, you will perform a similar configuration but
you do it using the HP 3PAR Management Console.
The main focus of this lab is to make you familiar with the Management Console.

Rev. 14.1 L -187

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

FC Port Configurations
1. Start the 3PAR Management Console (GUI) and connect to your HP 3PAR array.
(user: 3paradm, password: 3pardata)

2. Select your Storage System, select Systems and the Ports icon. Click the Fibre
Channel tab.

3. Right click the port with the position 0:1:1 and select Configure… .

L -188 Rev. 14.1

 Lab Guide

4. Set the Port Configuration to Connection Mode: Host and Connection Type:
Point.

5. Set the same Connection Mode and Type also for port position 0:1:2, 1:1:1 and
1:1:2. Be aware that not all ports are connected to the fabric on all systems. Loss
sync indicates that there is no cable connected.

Rev. 14.1 L -189

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

6. While in the Systems tab, select the Summary tab to get an overview of all port
and how they are configured. The Type column identifies the SAS, FC, iSCSI and
Remote Copy ports.
Have a look at the results shown and the numbering convention. They are
numbered X:Y:Z

 X : the Node number from 0 to 3

 Y : depending on the Port Type:

 0 for SAS (DP1 and DP2)
 1 for the embedded FC ports
 2 for the optional installed HBA (iSCSI in the screen shot above)
 3 for the embedded Remote Copy Port

 Z : the port number starting with one

L -190 Rev. 14.1

 Lab Guide

Rev. 14.1 L -191

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Multipath (MPIO) Configurations

In the previous labs, SmartStart was used to configure MPIO on the windows host. If you
are not using SmartStart, you need to configure MPIO before you export and use a
volume. For Windows, you can use the MPIO configurator under Administrative tools or
the command line: "mpclaim -r -I -d "3PARdataVV" command. In both cases a
reboot is necessary.
Refer to the HP 3PAR Windows Server 2012 and Windows Server 2008 Implementation
Guide for reference

After the reboot, check also that the default MPIO policy “Round Robin” is selected.

L -192 Rev. 14.1

 Lab Guide

Rev. 14.1 L -193

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

Host, CPG and Volume Configurations

If time permits, use Lab 3 Configure the Storage System (CPG, VV, Host) to create a
Host, CPGs, and Volumes, and export the Volumes to verify the operation of the
manually installed system.

L -194 Rev. 14.1

 Lab Guide

Lab 11 Activity debrief

Use the space below to record your Key Insights and Challenges from Lab Activity.

Challenges Key Insights

NOTES

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Rev. 14.1 L -195

HP 3PAR StoreServ 7000 Installation, Startup and Break Fix

L -196 Rev. 14.1