EMC Host Connectivity Guide

for VMware ESX Server v2.x

P/N 300-002-304
REV A02

EMC Corporation
Corporate Headquarters:
Hopkinton, MA 01748 -9103
1-508 -435 -1000
www.emc.com

Copyright 2001-2005 EMC Corporation. All rights reserved.

Updated March 2005

EMC believes the information in this publication is accurate as of its publication date. The
information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." EMC CORPORATION
MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE
INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an
applicable software license.
Trademark Information

ii

EMC Host Connectivity Guide for VMware ESX Server v2.x

Contents

Preface ............................................................................................................................ ix
Chapter 1

Introduction
Purpose of this Document .............................................................. 1-2
Related Documentation .................................................................. 1-2
VMware ESX Server v2.x Features ................................................ 1-3
VMotion ..................................................................................... 1-3
Raw Device Mappings ............................................................. 1-3

Chapter 2

Installation Information for VMware ESX Server v2.x

with Symmetrix and CLARiiON Arrays
Overview........................................................................................... 2-2
VMware ESX Server Options.................................................. 2-4
VMkernel Parameters .............................................................. 2-8
Driver Configuration in VMware ESX Server v2.x ................... 2-23
Rescanning the SCSI Bus .............................................................. 2-26
Persistent Binding .......................................................................... 2-30
LUN Spanning................................................................................ 2-34
Setting Up and Installing Virtual Machines............................... 2-37
Additional Notes............................................................................ 2-45

Chapter 3

VMware ESX Server v2.x and Symmetrix Storage Array

Fibre Channel Environment
VMware/Symmetrix Environment............................................... 3-2
Host and Hardware Connectivity .......................................... 3-2
Symmetrix Connectivity .......................................................... 3-2
Booting the VMware ESX Server v2.5 from the Symmetrix ...... 3-6

EMC Host Connectivity Guide for VMware ESX Server v2.x

iii

Contents

Fabric Zoning ................................................................................... 3-8

Required Storage System Configuration...................................... 3-8
Useful VMware ESX Server Utilities and Functions .................. 3-9
Addressing Symmetrix Devices .................................................. 3-10
Fabric Addressing .................................................................. 3-10
SCSI-3 FCP Addressing ......................................................... 3-11

Chapter 4

VMware ESX Server v2.x and CLARiiON Storage Array

Fibre Channel Environment
VMware/CLARiiON Environment .............................................. 4-2
Host and Hardware Connectivity.......................................... 4-2
CLARiiON Connectivity ......................................................... 4-2
Booting the VMware ESX Server v2.5 from CLARiiON
Storage Arrays .......................................................................... 4-5
Fabric Zoning ............................................................................ 4-7
Required Storage System Configuration .............................. 4-7
Useful VMware ESX Server Utilities and Functions ........... 4-7
Failover Functionality in VMware ESX Server v2.x for Attach
to CLARiiON Storage Arrays ........................................................ 4-9
Manual Registration of VMware ESX Server v2.x for Attach
to CLARiiON Storage Arrays ...................................................... 4-13
Adding the VMware ESX Server Host to a Storage Group ..... 4-20
Manual Trespass on CLARiiON Arrays to Recover the
Original Path .................................................................................. 4-21

Index ................................................................................................................................ i-1

iv

EMC Host Connectivity Guide for VMware ESX Server v2.x

Figures

2-1
2-2
2-3
2-4
2-5
2-6
2-7
2-8
2-9
2-10
2-11
3-1
3-2
4-1
4-2
4-3

VMware ESX Server Options ...................................................................... 2-4

Startup Profile ............................................................................................... 2-5
Security Settings ........................................................................................... 2-7
VMkernel Parameter Listing ....................................................................... 2-9
Storage Management > Disks and LUNs Tab ........................................ 2-37
Storage Management > Disks and LUNs > Core Dump Partition ...... 2-38
Access Mode ................................................................................................ 2-39
Add Virtual Machine Dialog .................................................................... 2-40
Status Monitor > Virtual Machines .......................................................... 2-40
VMware Remote Console > Connect to Virtual Machines ................... 2-41
Configuration Editor, Hardware, Connection ....................................... 2-43
One Host, Two Switches, and One Symmetrix Array ............................ 3-4
One Host, Two Switches, and Four Symmetrix Arrays .......................... 3-5
One Host, Two Switches, and One CLARiiON Array ............................ 4-3
One Host, Two Switches, and Four CLARiiON Arrays ......................... 4-4
Connectivity Status dialog ........................................................................ 4-15

EMC Host Connectivity Guide for VMware ESX Server v2.x

Figures

vi

EMC Host Connectivity Guide for VMware ESX Server v2.x

Tables

2-1
3-1
4-1

VMkernel Parameters, Descriptions, and Default Values .................... 2-12

Symmetrix SCSI-3 Addressing Modes .................................................... 3-12
Useful VMware ESX Server Utilities and Functions ............................... 4-7

EMC Host Connectivity Guide for VMware ESX Server v2.x

vii

Tables

viii

EMC Host Connectivity Guide for VMware ESX Server v2.x

Preface

As part of an effort to improve and enhance the performance and capabilities

of its product line, EMC from time to time releases revisions of its hardware
and software. Therefore, some functions described in this document may not
be supported by all revisions of the software or hardware currently in use.
For the most up-to-date information on product features, refer to your
product release notes.
If a product does not function properly or does not function as described in
this document, please contact your EMC representative.
This guide describes the features and setup procedures for VMware
ESX Server host interfaces to EMC Symmetrix and CLARiiON
Storage Array systems.
Audience

This guide is intended for use by storage administrators, system

programmers, or operators who are involved in acquiring, managing,
or operating Symmetrix, CLARiiON, and host devices.
Readers of this guide are expected to be familiar with the following
topics:

Symmetrix or CLARiiON system operation

VMware ESX Server operating environment

EMC Host Connectivity Guide for VMware ESX Server v2.x

ix

Preface

Organization

Here is an overview of where information is located in this guide.

Chapter 1, Introduction This document is meant to assist in the
installation and configuration of VMware ESX Server v2.x attached to
EMC CLARiiON and Symmetrix storage systems.
Chapter 2, Installation Information for VMware ESX Server v2.x with
Symmetrix and CLARiiON Arrays This chapter covers installation
information for the VMware ESX Server v2.x with Symmetrix and
CLARiiON arrays.
Chapter 3, VMware ESX Server v2.x and Symmetrix Storage Array Fibre
Channel Environment This chapter lists Symmetrix Fibre Channel
support information specific to the VMware ESX Server environment.
Chapter 4, VMware ESX Server v2.x and CLARiiON Storage Array Fibre
Channel Environment This chapter lists CLARiiON Fibre Channel
support information specific to the VMware ESX Server environment.

EMC Host Connectivity Guide for VMware ESX Server v2.x

Preface

Conventions Used in
This Guide

EMC uses the following conventions for notes and cautions.

A note presents information that is important, but not hazard-related.

CAUTION
A caution contains information essential to avoid damage to the
system or equipment. The caution may apply to hardware or
software.
Typographical Conventions
EMC uses the following type style conventions in this guide:
AVANT GARDE

Keystrokes

Palatino,
bold

Palatino,
italic

Dialog box, button, icon, and menu items in text

Selections you can make from the user interface,
including buttons, icons, options, and field
names
New terms or unique word usage in text
Command line arguments when used in text
Book titles

Courier,
italic

Arguments used in examples of command line

syntax.

Courier

System prompts and displays and specific

filenames or complete paths. For example:
working root directory [/user/emc]:
c:\Program Files\EMC\Symapi\db

Courier,
bold

User entry. For example:

symmpoll -p

Options in command line syntax

EMC Host Connectivity Guide for VMware ESX Server v2.x

xi

Preface

Where to Get Help

For questions about technical support, call your local sales office or
service provider.
If you have a valid EMC service contract, contact EMC Customer
Service at:
United States: (800) 782-4362 (SVC-4EMC)
Canada:

(800) 543-4782 (543-4SVC)

Worldwide:

(508) 497-7901

Follow the voice menu prompts to open a service call and select the
applicable product support.
If you are located outside the North America, call the nearest EMC
office for technical assistance.

Sales and Customer

Service Contacts

For the list of EMC sales locations, please access the EMC home page
at:
http://www.emc.com/contact/

For additional information on the EMC products and services

available to customers and partners, refer to the EMC Powerlink Web
site at:
http://powerlink.emc.com

Your Comments

xii

Your suggestions will help us continue to improve the accuracy,

organization, and overall quality of the user publications. Please send
a message to [email protected] with your opinions of
this guide.

EMC Host Connectivity Guide for VMware ESX Server v2.x

Invisible Body Tag

Introduction

This document is meant to assist in the installation and configuration of

VMware ESX Server v2.x attached to EMC CLARiiON and Symmetrix
storage systems.

Purpose of this Document ................................................................1-2

Related Documentation.....................................................................1-2
VMware ESX Server v2.x Features ..................................................1-3

Introduction

1-1

Introduction

Purpose of this Document

This document is meant to assist in the installation and configuration of
VMware ESX Server v2.x attached to EMC CLARiiON and Symmetrix
storage systems.

Related Documentation
For the EMC Support Matrix (ESM) for supported server and HBA
combinations, go to:
http://powerlink.EMC.com/

For Navisphere information, refer to the Navisphere Manager

Administrators Guide.
For VMware specific documentation, such as the VMware ESX Server
Release Notes, ESX Server Administration Guide, and the ESX Server
Installation Guide, go to:
http://www.vmware.com/support/

1-2

EMC Host Connectivity Guide for VMware ESX Server v2.x

Introduction

VMware ESX Server v2.x Features

VMware ESX Server is similar to Linux, but is its own operating
system. VMware GSX Server is an application that runs on top of a
standard Linux or Windows operating system.
VMware ESX Server consists of virtualization software that provides
server consolidation and allows several instances of operating
systems running as virtual machines on one physical machine.
Currently, VMware ESX Server installs and operates as a 32-bit
operating system on Intel-based and Opteron-based servers. The
operating systems qualified to run as virtual machines include 32-bit
versions of Linux, Windows, and NetWare.

VMotion

VMotion is part of VMware Virtual Center, a systems management

and provisioning tool used by system administrators. VMotion
technology allows the migration of virtual machines from one
physical server to another physical server. For instance, VMotion
could be used to migrate virtual machines from one server to another
so that maintenance may be performed on the original server. This
allows the operating systems running in the virtual machine
partitions to continue running their workloads.

Raw Device
Mappings

VMware ESX Server v2.5 introduced a new technology called raw

device mapping. Essentially, raw device mapping is a SCSI
pass-through technology that allows Virtual Machines to pass SCSI
commands directly to the physical hardware.
Note that using raw disk mappings in Shared mode with Fibre Channel
HBAs is not supported. The HBAs must be allocated solely to the Virtual
Machines in order to use raw disks.

VMware ESX Server v2.x Features

1-3

Introduction

1-4

EMC Host Connectivity Guide for VMware ESX Server v2.x

Invisible Bo

Installation Information for

VMware ESX Server v2.x with
Symmetrix and CLARiiON Arrays

This chapter covers installation information for the VMware ESX

Server v2.x with Symmetrix and CLARiiON arrays.

Overview .............................................................................................2-2
Driver Configuration in VMware ESX Server v2.x .....................2-23
Rescanning the SCSI Bus.................................................................2-26
Persistent Binding ............................................................................2-30
LUN Spanning..................................................................................2-34
Setting Up and Installing Virtual Machines.................................2-37
Additional Notes..............................................................................2-45

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2-1

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Overview
The fibre channel HBA driver functions as a device driver layer
below the standard VMware SCSI adapter driver. The Fibre Channel
interface is therefore transparent to the VMware disk administration
system.
Note: VMware ESX Server is NOT Linux and, therefore, requires its own
EMC-qualified drivers, not the standard EMC-qualified Linux drivers.

Install VMware ESX Server from the CD. The VMware ESX Server
installation is based on a modified Red Hat v7.2 v2.4.9 kernel.
However, VMware ESX Server is NOT Linux.
VMware recommends partitioning the internal hard drive into four
partitions. For example:
/boot

50 MB

primary

swap

256 MB to 1 GB

primary

2.5 GB

primary

/vmimages

4 GB

Note that a swap partition is required by both the service console and the
VMkernel. The reference to the swap partition in this example is for the
service console. The swap partition and core dump partitions for the
VMkernel can be created using the GUI after the initial installation of the
service console OS.

As stated in the Preface, EMC does not recommend fabric boot of the
VMware ESX Server itself over the fabric, but we do support fabric boot with
VMware ESX Server v2.5. The Virtual Machines are expected to boot over the
fabric using the QLogic or Emulex HBAs.

Refer to the ESM for specifics on which operating systems are

supported as Virtual Machines.
You may customize the installation according to your server and the
amount of memory and hard disk space you have available.

2-2

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

The following are a few recommendations for the installation:

Use static IP addresses.

Set the hardware clock when prompted.

Create at least one user account other than root.

Note that emacs, samba, and NFS are not enabled by default in the
Console OS.
Reboot the system after completing the installation.
For VMware ESX Server v2.1.x installations, when the system
reboots, you will be prompted with two options in the LILO boot
prompt:
linux-up,linux

Assuming the system on which you are installing has multiple CPUs,
the default boot image will be linux. This is expected and
acceptable to use.
For VMware ESX Server v2.5 installations, when the system reboots,
you are prompted with three options in the LILO boot prompt:
esx, linux-up,linux

The default boot image for VMware ESX Server v2.5 is esx.
You can now log in to the VMware server using IE or Netscape with
the host name or IP address. This allows you access to the VMware
host using the Management User Interface (MUI).
After you log into the host, the first dialog is the VMware ESX Server
Status Monitor dialog.
The Status Monitor dialog provides a summary reporting of the
system CPU and processor usage by the Virtual Machines as well as a
visual status report of the Virtual Machines themselves.

Overview

2-3

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
VMware ESX Server Options
Select the Options tab. The Options dialog provides you with a list
of actions that can be performed to configure or modify the VMware
ESX Server, as shown in Figure 2-1.

Figure 2-1

VMware ESX Server Options

From the Options tab, select the Startup Profile menu. The Startup
Profile creates the VMware ESX Server boot configuration and allows
you to specify whether the server resources should be allocated to the
Service Console, the Virtual Machines, or shared between the two.
Fibre Channel and SCSI HBAs can be allocated for use with the
Virtual Machines only or can be shared between the Virtual Machines
and Service Console. In the case of booting from the fabric, the FC
HBAs must be shared so that the Service Console and the Virtual
Machines can access the boot device.
Networking cards cannot be shared between the Service Console and
the Virtual Machines so they must be allocated either to the Service
2-4

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Console or to the Virtual Machines. Therefore, a minimum of two

NICs must be installed in the system.
For an example of the Startup Profile dialog, see Figure 2-2.

Figure 2-2

Startup Profile

After the changes appropriate to your system are made, save the
configuration by clicking OK.
After saving the configuration changes, reboot the host so that the
changes can take effect.

Overview

2-5

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
After the VMware host reboots, proceed with the VMkernel
configuration.

The Network Connections must be set up according to the

network interface cards (NICs) being used and the customer
environment. Note that the server must have at least two NICs.
Because NICs may not be shared, one must be allocated to the
Console and at least one must be allocated to the Virtual
Machines.
It is not necessary to reboot the VMware ESX Server into the esx kernel
in order for network configuration changes to take effect.

2-6

The Security Settings selected are entirely dependent upon the

customers desires. VMware recommends that customers use the
High Security setting so all connections into ESX will be
encrypted. If FTP and Telnet are required, the Medium Security
setting will need to be selected. The Security Setting dialog is
shown in Figure 2-3 on page 2-7

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Figure 2-3

Security Settings

Swap Configuration creates, modifies, and activates swap files

that allow your virtual machines to use more memory than is
physically available. Whether or not you want to enable a swap
file depends on the amount of memory in the server. It is
recommended that a swap file be created so that the virtual
machines will be able to use more memory than is actually
available in the server.
When booting from an internal hard drive, it is recommended that the
swap file resides on the internal device, rather than a SAN-attached
device.

Overview

2-7

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

If required, the SNMP agent is enabled in the SNMP

Configuration section.

The Advanced Settings section allows you to modify the

VMkernel parameters. Currently, EMC tests with and supports
the majority of the default VMkernel parameters. The table
below shows the parameters, their descriptions, and the VMware
and EMC default settings. It is highly recommended that the
EMC default parameters are used when attached to EMC Storage
Arrays.

VMkernel Parameters
A number of parameters define the behavior of the VMkernel.
Currently, EMC recommends that the majority of the parameter
values remain the same as the VMware defaults. The exception at
this time is the Disk.MaxLun parameter.
These values can be viewed and modified using the MUI and in the
subdirectories within the /proc/vmware/config directory.
To view the listing of directories in the /proc/vmware/config
directory, change to the directory and perform a listing:
cd /proc/vmware/config
ls la
dr-xr-xr-x
2 root
dr-xr-xr-x
2 root
dr-xr-xr-x
2 root
dr-xr-xr-x
2 root
dr-xr-xr-x
2 root
dr-xr-xr-x
2 root
dr-xr-xr-x
2 root
dr-xr-xr-x
2 root
dr-xr-xr-x
2 root
dr-xr-xr-x
2 root

root
root
root
root
root
root
root
root
root
root

0
0
0
0
0
0
0
0
0
0

Jan
Jan
Jan
Jan
Jan
Jan
Jan
Jan
Jan
Jan

18
18
18
18
18
18
18
18
18
18

06:07
06:07
06:07
06:07
06:07
06:07
06:07
06:07
06:07
06:07

Cpu
Disk
FileSystem
Irq
Mem
Migrate
Misc
Net
Numa
Scsi

These directories provide the types of parameters that are listed in

each. For instance, the /proc/vmware/config/Disk directory
contains the parameters pertaining to the Disk subsystem. Any
changes made to the parameters in this directory will be specific to
Disks. For example, Disk.MaxLUN is part of the
/proc/vmware/config file located in the Disk directory and the
parameter itself is the MaxLUN setting.
If desired, changes to the values of these parameters made by
performed using the MUI. Log into the server via the MUI and select
the Options tab. In the line for Advanced Settings | Service Console
2-8

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Settings, select Advanced Settings. You will then be presented with a

window that contains the parameters and their values and
descriptions of the parameters.
The parameters are listed in alphabetical order. An example of the
MUI dialog listing the beginning of the parameter list is shown in
Figure 2-4.

Figure 2-4

VMkernel Parameter Listing

The values of the VMkernel parameters also can be set at the

command line. This is done by echoing the /proc filesystem. As
mentioned previously, the parameters are ultimately listed in the
/proc/vmware/config directory. The following example
demonstrates the steps to view and change the value of a particular
setting.

Overview

2-9

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
In this particular example, the LogMultiPath parameter is being
enabled.
[root@l82bi199 /]# cd /proc/vmware/config/Scsi
[root@l82bi199 Scsi]# ls
ConflictRetries LogAborts LogMultiPath PassthroughLocking
[root@l82bi199 Scsi]# cat LogMultiPath
LogMultiPath (Log path state changes) [default = 0]: 0
[root@l82bi199 Scsi]# echo 1 >> LogMultiPath
[root@l82bi199 Scsi]# cat LogMultiPath
LogMultiPath (Log path state changes) [default = 0]: 1

If the parameter is changed by echoing the /proc at the

command line, this change will not persist through a reboot.

Although it is not recommended, if you prefer to use the

command line, you may modify the /etc/rc.local file so that
the changes you have via echoing /proc at the command line will
be run upon a reboot.

The /etc/rc.local file is that script that contains your

initialization parameters and is run after all of the standard
system init scripts. It is recommended that you append your
modifications to the end of the file.
For example, see the end of the /etc/rc.local file for the
addition of the /proc modifications:

[root@l82bi199 root]# more /etc/rc.local

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.
# BEGINNING_OF_VMWARE_RC_DOT_LOCAL
if ( uname -a | grep -q vmnix ); then
R="VMware ESX Server 2.5.0"
else
R="Linux"
fi
arch=$(uname -m)
a="a"
case "_$arch" in
_a*) a="an";;
_i*) a="an";;
esac
NUMPROC=`egrep -c "^cpu[0-9]+" /proc/stat`
if [ "$NUMPROC" -gt "1" ]; then

2-10

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

SMP="$NUMPROC-processor "
if [ "$NUMPROC" = "8" -o "$NUMPROC" = "11" ]; then
a="an"
else
a="a"
fi
fi
# This will overwrite /etc/issue at every boot. So, make any changes
you
# want to make to /etc/issue here or you will lose them when you reboot.
echo "" > /etc/issue
echo "$R" >> /etc/issue
echo "Kernel $(uname -r) on $a $SMP$(uname -m)" >> /etc/issue
cp -f /etc/issue /etc/issue.net
echo >> /etc/issue
#Echo the /proc filesystem to enable the
#multipath logging. This will ensure that
#the change persists through reboots.
echo 1 >> /proc/vmware/config/Scsi/LogMultiPath
# END_OF_VMWARE_RC_DOT_LOCAL

By default, the maximum number of LUNs scanned per disk array by

ESX v2.x is 8. Depending upon the EMC storage array configuration
and the devices that are being allocated to the VMware host, it is
recommended that the following parameter be changed in the
VMkernel configuration to accommodate the LUNs allocated to the
host.
For example:
Disk.MaxLun == 128 (The default is 8.)

If the Symmetrix or CLARiiON Storage Group has LUNs allocated to

the VMkernel addressable as greater than eight, then this parameter
must be increased. The DiskMaxLun parameter can be as large as 255.
However, the maximum number of LUNs that can be managed by a
single VMware ESX Server is 128. The VMware ESX Server will
simply stop scanning the SCSI bus once it has seen 128 actual LUNs.
Please note that increasing the Disk.MaxLun parameter may slow down the
SCSI scanning time.

In VMware ESX Server v2.1.x, the VMkernel is configured with

support for sparse LUNs by default. The Disk.SupportSparseLun
Overview

2-11

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
parameter allows the VMkernel to scan past non-existent LUNs, that
is, allows LUN skipping. When attaching to EMC storage arrays, it is
strongly recommended that the Disk.SupportSparseLun parameter
be set at the default of 1.
The following table lists the VMkernel parameters with their
corresponding descriptions and the VMware default and EMC
recommended values.

Table 2-1

VMkernel Parameters, Descriptions, and Default Values

VMware
Default
Value

EMC
Recommended
Value

1000

1000

Cpu.ConsoleMinCpu
Min percentage of CPU 0 to dedicate to console [0-100]

Cpu.ConsoleOSWarpPeriod
Period in milliseconds [0-100]

20

20

Cpu.CreditAgePeriod
Period in milliseconds [500-10000]

3000

3000

Cpu.IdlePackageRebalancePeriod
Usec between chances to rebalance idle packages (0 to disable, 100000 max)

541

541

Cpu.MachineClearThreshold
Machine clears per million cycles to trigger quarantine

100

100

Cpu.MigratePenalty
Penalty in milliseconds [0-2000]

100

100

Cpu.MigratePeriod
Milliseconds between opportunities to migrate across CPUs

20

20

Cpu.PreemptPenalty
Penalty in milliseconds [0-2000]

10

10

Cpu.Quantum
Quantum in milliseconds [1-1000]

50

50

VMkernel Parameter and Description

Cpu.BoundLagQuanta
Number of global quanta before bound lag [1-100]
Cpu.CellMigratePeriod
Milliseconds between opportunities to migrate across cells

2-12

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Cpu.RunnerMovePeriod
Milliseconds between opportunities to move currently-running vcpu

200

200

Cpu.SharesPerVcpuHigh
Shares per vcpu for high CPU priority [100-10000]

2000

2000

Cpu.SharesPerVcpuLow
Shares per vcpu for low CPU priority [100-10000]

500

500

Cpu.SharesPerVcpuNormal
Shares per vcpu for normal/default CPU priority [100-10000]

1000

1000

Cpu.SkewSampleUsec
Interval between vsmp skew tests

500

500

Disk.DelayOnBusy
Delay in milliseconds for completion of commands with a BUSY status

400

400

[empty]

[empty]

128
Default: 8

2000

2000

Disk.MaxVCNotReadyTime
The number of seconds a VC path can report NOT READY status before being declared dead

60

60

Disk.PathEvalTime
The number of seconds between FC path evaluations

300

300

Disk.ResetLatency
Delay in milliseconds between reset thread wake-ups

1000

1000

VMkernel Parameter and Description

Cpu.SkewSampleThreshold
Number of skew samples allowed before co-deschedule (0 to disable skew checking)

Disk.MaskLUNs
LUNs to mask from kernel. Format: <adapter>:<target>:<comma separated LUN range list>
Disk.MaxLUN
Maximum number of LUNs per target that we scan for
Disk.MaxResetLatency
Delay in milliseconds before logging warnings and spawning new reset worlds if a reset is
overdue or taking too long

Disk.ResetMaxRetries
Max number of bus reset retries (0=infinite)

Overview

2-13

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Disk.ResetOnFailover
Issue a SCSI reset when failing over to an alternate HBA

Disk.ResetOverdueLogPeriod
Delay in seconds between logs of overdue reset

60

60

Disk.ResetPeriod
Delay in seconds between bus resets retries

30

30

1800

1800

Disk.ResetThreadMax
Max number of reset handler threads

16

16

Disk.ResetThreadMin
Min number of reset handler threads

Disk.RetryUnitAttention
Retry all SCSI commands that return a unit attention error

[empty]

[empty]

Disk.SchedNumReqOutstanding
Number of outstanding commands to a target with competing worlds

16

16

Disk.SchedQControlSeqReqs
Number of consecutive requests from a VM required to raise the outstanding commands to
max

128

128

Disk.SchedQControlVMSwitches
Number of switches between commands issued by different VMs required to reduce
outstanding commands to CONFIG_DISK_CIF

Disk.SchedQuantum
Number of consecutive requests from one World

Disk.SectorMaxDiff
Distance in sectors at which disk BW sched affinity stops

2000

2000

Disk.SharesHigh
Shares for high disk priority [100-10000]

2000

2000

VMkernel Parameter and Description

Disk.ResetThreadExpires
Life in seconds of an inactive reset handle thread

Disk.SANDevicesWithAPFailover
SAN devices with Active/Passive Path Failover. Format: <Device Model Id>:<Device Model
Id>:...

2-14

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Disk.SharesLow
Shares for low disk priority [100-10000]

500

500

Disk.SharesNormal
Shares for normal/default disk priority [100-10000]

1000

1000

Disk.SupportSparseLUN
Support for sparse LUNs if set to one

Disk.UseDeviceReset
Use device reset (instead of bus reset) to reset a SCSI device

Disk.UseLunReset
Use LUN reset (instead of device/bus reset) to reset a SCSI device

Disk.UseReportLUN
Use the REPORT LUN command to speed up scanning for devices

FileSystem.LockRetries
Maximum number of retries when encountering file system lock

15

15

FileSystem.RemoteFSEnabled
Enable remote VMFS support

Irq.BestVcpuRouting

Irq.IRQMaxLoadPct
Maximum percentage of a CPU's resources that should be devoted to interrupts

70

70

Irq.IRQRebalancePeriod
Time in ms between attempts to rebalance interrupts

500

500

Mem.AdmitHeapMin
Free heap space required to power on VM, in KB [256-10240]

1024

1024

Mem.AllocHighThreshold
Threshold (in MB) at which we start allocating memory above 4GB

768

768

Mem.BalancePeriod
Period in seconds [1-120], 0 to disable

15

15

VMkernel Parameter and Description

Irq.RoutingPolicy
Policy for interrupt routing, 1 for idle-routing, 2 for random, 0 to disable moving IRQs

Overview

2-15

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Mem.CtlMaxBSD
vmmemctl limit for BSD VM, in MB [0-1024]

256

256

Mem.CtlMaxLinux
vmmemctl limit for Linux VM, in MB [0-1024]

768

768

Mem.CtlMaxNT4
vmmemctl limit for Windows NT4 VM, in MB [0-192]

128

128

Mem.CtlMaxNT5
vmmemctl limit for Windows 2000 or Windows 2003 VM, in MB [0-2048]

2048

2048

Mem.CtlMaxPercent
vmmemctl limit as percentage of VM max size [0-50]

50

50

Mem.IdleTax
Idle memory tax rate [0-99]

75

75

Mem.MinFreePct
Minimum percent of memory that should be kept free

Mem.SamplePeriod
Period in seconds [1-180], 0 to disable

60

60

Mem.ShareCOSBufSize
Specify number of MPNs to be used by COW P2M buffer [2-8]

Mem.ShareScanTotal
Total page scans in pages/sec [1-10000], 0 to disable

200

200

Mem.ShareScanVM
Per-VM page scans in pages/sec [1-1000], 0 to disable

50

50

Mem.SharesPerMBHigh
Shares per MB for high memory priority [1-25]

20

20

Mem.SharesPerMBLow
Shares per MB for low memory priority [1-25]

Mem.SharesPerMBNormal
Shares per MB for normal/default memory priority [1-25]

10

10

90000

90000

VMkernel Parameter and Description

Mem.SwapAbortTimeoutMs
Swap timeout(ms) for aborting commands on a SAN[1000-1200000]

2-16

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Mem.SwapFileOnSAN
Enables swap files on the SAN [0-1]

Mem.SwapIORetry
Retry timeout for swap I/O in seconds [1-2678000], 0 to disable timeout

Mem.SwapShared
Set the max ref count of a swappable shared page [0-100]

Migrate.Enabled
Enable hot migration support

Migrate.MemChksum
Checksum VM's memory while migrating

Migrate.MinReservation
Reserve a percentage of a CPU for use by migration helper worlds[0, 100]

30

30

Migrate.NetTimeout
Timeout for migration network operations

10

10

Migrate.PageInProgress
Time in seconds after which a pagein will be killed if there is no progress[5-1800]

15

15

Migrate.PageInTimeout
Time in seconds to wait for pagein to finish [10-1800]

180

180

Migrate.PreCopyLeftMB
Maximum modified memory left over after pre-copy is done

16

16

Migrate.PreCopyMinProgressMB
Minimum reduction in modified memory after a pre-copy iteration

16

16

Migrate.TSMaster
Pseudo-synchronize clocks for migration to/from this machine[0, 1]

Misc.BlueScreenTimeout
Timeout in seconds, 0 is no timeout

Misc.ConsoleShareXfer
Transfer shares to console on RPC calls

[empty]

[empty]

VMkernel Parameter and Description

Misc.CosCorefile
VMFS to write the Service Console core file to [must be symbolic filesystem name]

Overview

2-17

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Misc.DebugMemEnable
Enable memory debugging, 0 to disable

Misc.EnableHighDMA
Enable DMA above 4GB

Misc.KVMapGuardUncached
Use guard pages around uncached kvmap mappings, [0-1]

Misc.LogToFile

Misc.LogToSerial

Misc.LogWldPrefix

Misc.MemAdmitMapEntriesMin
Free KVMap entries required to power on VM, [0-100]

30

30

Misc.MemMapEntriesLow
Point at which to start conserving KVMap entries, [0-1024]

200

200

Misc.MinimalPanic

Misc.ProcVerbose
Option unused

[empty]

[empty]

115200

115200

Misc.TimerHardPeriod
Hard timer interrupt period in microseconds

1000

1000

Misc.TimerMinGuestPeriod
Minimum period for guest timer callbacks in microseconds

100

100

Misc.VmkperfPerWorld
Should performance counters be maintained per-world [0-1]

Net.ClusterHaltCheck
1 to check for clustered tx/rx packets on halt

VMkernel Parameter and Description

Misc.PsodOnCosPanic
PSOD VMkernel on Console OS panic / oops [0-1]
Misc.SerialBaudRate
Misc.SerialPort

2-18

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Net.CopiesBeforeRemap
Copies before remapping, 0 to disable

10

10

Net.DisableWatchdog
Disable network watchdog timeout handler

Net.MaxMallocPackets
Maximum number of packets that can be allocated via the memory allocator

256

256

Net.MaxPrivateSKBs

256

256

Net.MaxRecvPackets

64

64

Net.NotifySwitch
Broadcasts an arp request on net handle enable [0 = disabled, 1 = enabled]

Net.PktsDeferXmitIntIfKeeping
Defer xmit completion interrupts if this many packets still pending

Net.PktsPerXmitInterrupt
Packets to transmit before raising completion interrupt

Net.ProcVerbose
Verbose procfs output for networking, 0 to disable

Net.RXClusterCPUSamplePeriod
CPU %idle sample period (msec) [1-10000]

1000

1000

Net.RXClusterDelayOff
Weighting factor for deactivating clustering

10

10

Net.RXClusterDelayOn
Weighting factor for activating clustering

Net.RXClusterDelayTDec
Decrement timer freq this often

1000

1000

Net.RXClusterDelayTInc
Increment timer freq this often

Net.RXClusterTMaxFreq
Max timer freq as a power of two [1-13]

10

10

VMkernel Parameter and Description

Overview

2-19

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Net.RXClusterTMigrateDelay
Timer CPU migration intervals [1-100]

Net.RXClusterTMigrateThresh
Timer CPU migration threshold (delta %idle)

20

20

Net.RXClusterTMinFreq
Min time freq as a power of two [1-13]

Net.RXClusterThreshOff
Interrupts/sec to deactivate clustering

2000

2000

Net.RXClusterThreshOn
Interrupts/sec to activate clustering

4000

4000

Net.RXClusterTimerCPU
Which cpu to run the timer on, 0 for migratory timer

Net.RecvClusterDynAdjRate

50

50

Net.RecvClusterDynQueueMax

20

20

Net.RecvClusterOffCount

30

30

Net.RecvClusterOnCount

40

40

NetRecvClusterQueueMax

10

10

Net.RecvClusterTimeoutCount

Net.RecvClusterTimeoutMS

10

10

Net.RecvCopyLength

200

200

36864

36864

Net.SwitchFailoverBeaconInterval
NIC Teaming switch failover beacon interval [1-60]

Net.SwitchFailoverThreshold
NIC Teaming switch failover threshold, a positive number to enable switch failover policy

Net.TCPIPLog
TCP/IP logging, 0 to disable

VMkernel Parameter and Description

Net.SwitchFailoverBeaconEtherType
NIC Teaming switch failover beacon Ether Type [0x6000 - 0x9000]

2-20

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Net.TcpSegmentationOffload
Allow TSO, 0 to disable

Net.VlanTrunking
1 to enable 802.1Q VLAN Tagging by VMkernel

Net.XmitClusterOffCount

30

30

Net.XmitClusterOnCount

40

40

Net.XmitClusterQueueMax

10

10

Net.XmitClusterTimeoutCount

Net.XmitClusterTimeoutMS

10

10

Net.XmitInterruptIfStopped
Raise an interrupt if a xmit packet is returned and the guest has stopped transmitting

Net.XmitInterruptTimeout
Timeout in milliseconds to check for completed transmits

10

10

Net.XmitMinWinZeroCopyLen
For windows guests the minimum size of packets to try to avoid the copy

256

256

Net.ZeroSpeedLinkDown
Zero speed means link down, 1 to enable it

Numa.AutoMemAffinity
1 to set mem affinity automatically based on cpu affinity, 0 to disable

Numa.Debug
Level of NUMA scheduler debugging information to print [0-4]

Numa.MigImbalanceThreshold
Minimum deviation in owed ms between nodes, per second, to trigger migration

75

75

Numa.PageMig
1 to permit NUMASched to manipulate page migration, 0 to disallow it

Numa.Rebalance
1 to use NUMASched rebalancer, 0 to disallow it

2000

2000

VMkernel Parameter and Description

Numa.RebalancePeriod
Frequency of NUMA node rebalancing, in milliseconds

Overview

2-21

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
Table 2-1

VMkernel Parameters, Descriptions, and Default Values (continued)

VMware
Default
Value

EMC
Recommended
Value

Scsi.ConflictRetries
Maximum number of retries when encountering reservation conflict

Scsi.LogAborts
Log timing of aborted or reset I/Os

Scsi.LogMultiPath
Log path state changes

Scsi.PassthroughLocking

VMkernel Parameter and Description

It is not necessary to reboot the VMware ESX Server in order for any
changes to the parameter values to take effect.

2-22

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Driver Configuration in VMware ESX Server v2.x

VMware ESX Server is NOT Linux and, therefore, requires its own
EMC-qualified drivers, not the standard EMC-qualified Linux
drivers.
Refer to the EMC Support Matrix for the latest supported drivers.

VMware ESX Server v2.x contains QLogic and Emulex drivers native
to the esx kernel. At the time this document was written, the drivers
qualified for VMware ESX Server v2.1x are the QLogic v6.04.02 driver
and the Emulex v4.02q driver. The drivers qualified for VMware ESX
Server v2.5 are the QLogic v6.07.02 driver and the Emulex v2.01g
driver.
These drivers are the defaults for their respective HBA families and
are contained within the VMware ESX Server v2.x kernel.
No modifications need to be made to the VMkernel in order to use
these drivers. There is no need to follow the documentation to
download, compile, and install the EMC-qualified Linux drivers. The
source code for VMware ESX Server v2.x is not included in the
VMware distribution so the Linux drivers may not be compiled.
Additionally, the Linux Emulex and QLogic drivers need to be ported
from Linux to VMware ESX Server in order for the Linux-specific
drivers to be usable in VMware ESX Server.
The drivers should be enabled in the Startup Profile menu. The
Startup Profile may be modified if the HBAs are added to the system
after the installation and initial configuration of VMware ESX Server.
The command vmkpcidivy can also be used to create a boot
configuration for the VMkernel. When using vmkpcidivy
interactively, you may specify whether your storage controllers are
allocated to the Service Console or to the Virtual Machines or shared.
The boot configuration created with vmkpcidivy will automatically
include the default QLogic and Emulex drivers included in the
kernel.
An example of the output produced when running vmkpcidivy in
interactive mode is as follows:
[root@l82bi199 /]# vmkpcidivy -i
Checking for existing VMnix Boot Configurations.

Driver Configuration in VMware ESX Server v2.x

2-23

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
The following VMnix kernel images are defined on your system:
Boot image configuration: vmnix
Image file: /boot/vmlinuz-2.4.9-vmnix2
Memory: 384M
Service Console devices:
Ethernet controller: BROADCOM Corporation NetXtreme BCM5703 Gigabit Ethernet
(rev 02)
RAID storage controller: Compaq Computer Corporation Smart Array 5i/532 (rev
01) (shared)
VM devices:
Ethernet controller: BROADCOM Corporation NetXtreme BCM5703 Gigabit Ethernet
(rev 02)
Fiber storage controller: QLogic Corp QLA231x/2340 (rev 02)
Ethernet controller: Intel Corporation 82557 [Ethernet Pro 100] (rev 08)
RAID storage controller: Compaq Computer Corporation Smart Array 5i/532 (rev
01) (shared)
Type in the name of the boot image configuration you wish to configure
or type "new" to create a new image [vmnix]:
Selected configuration was 'vmnix'.
Configuring your setup.
been provided.

Enter in the options that you want.

Defaults have

Name of this configuration [vmnix]:

Now allocating memory to the Service Console. The memory that is not allocated
to the Service Console will be allocated to the VMs. Be sure to indicate a
number smaller than you actually have on your system or you may not be able to
run any VMs.
Your system is reporting approximately 2,559 MB of memory.
Note: you cannot allocate more than 800 MB to the Service Console.
If you plan to run 4 virtual machines or fewer, 128 MB should suffice.
Set this to 192 MB for 8 virtual machines, 272 MB for 16 virtual
machines, 384 MB for 32 virtual machines, or 512 MB for more than
32 virtual machines.
Amount of memory (in megs) to allocate to the Service Console [384]:
Enable hyperthreading? (Only works with 8 or less physical processors) [y]:
Now to divide the SCSI controllers and network adapters. In this process,
you will select whether to give the previous devices to the Service Console or
the virtual machines that you will be running. In order to ensure optimal
performance for your virtual machines, we suggest you allocate as many devices
as possible to the virtual machines, leaving only the ones necessary to
configure and maintain the Service Console.
Type 'c' to allocate to the Service Console or 'v' to allocate to the virtual

2-24

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

machines, or 's' to share between both.

RAID storage controller Compaq Computer Corporation Smart Array 5i/532 (rev 01)
running on bus 1 device
3 function 0 (cciss.o) [vmhba4] [s]:
Ethernet controller BROADCOM Corporation NetXtreme BCM5703 Gigabit Ethernet (rev
02) running on bus 2
device 1 function 0 (bcm5700.o) [] [c]:
Ethernet controller BROADCOM Corporation NetXtreme BCM5703 Gigabit Ethernet (rev
02) running on bus 2
device 2 function 0 (bcm5700.o) [vmnic0] [v]:
Fiber storage controller QLogic Corp QLA231x/2340 (rev 02) running on bus 6 device
1 (qla2300_604.o) [v
mhba2] [v]:
Ethernet controller Intel Corporation 82557 [Ethernet Pro 100] (rev 08) running
on bus 6 device 2 function
0 (e100.o) [vmnic1] [v]:
Configuration complete.
Commit changes (y/n)? [y]:

Driver Configuration in VMware ESX Server v2.x

2-25

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Rescanning the SCSI Bus

For use in the VMkernel itself, rather than using the standard Linux
/proc file system to obtain information on the HBAs, refer to the
/proc/vmware/scsi directory. This directory contains entries for
each HBA in the system allocated to the Virtual Machines. Each HBA
is denoted by vmhbaN (where N indicates the file for each adapter
owned by the Virtual Machines).
You may confirm that the QLogic or Emulex driver is loaded in the
VMkernel with the vmkload_mod command. For example, the
following command may be issued on this host to list the QLogic
driver:
root@l82bi199 /]# vmkload_mod -l
Name
R/O Addr
Length
vmklinux
0x4dd000
0xf000
nfshaper
0x4ec000
0x1000
qla2300_604 0x4ed000
0x19000
bond
0x506000
0x2000

R/W Addr
0x12414a0
0x1297f28
0x1298f30
0x150e1d8

Length
0x53000
0x1000
0x22000
0x2000

ID
1
2
3
4

Loaded
Yes
Yes
Yes
Yes [

Assuming that the devices have been allocated to the VMware host
and that the VMkernel has been configured appropriately, the
Storage Management dialog in the MUI reports the devices that are
allocated to the host.
You may also view the device listing at the command line. This may
be done with the following commands:
1. more /proc/scsi/qla2300/N

where N is the sequential value of each QLogic HBA installed in

the system, beginning with the number after the last host adapter
number entry in the file. (The first host adapter number entry
begins with zero.) This is the same as it would be for standard
Linux.
2. ls la

/proc/vmware/scsi/vmhbaN

where N indicates the file for each adapter owned by the

VMkernel and, subsequently, the Virtual Machines
After the host is added to the Storage Group, the SCSI bus needs to be
rescanned.

2-26

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Rescanning the SCSI bus may be accomplished in one of the

following ways.
For VMware ESX Server v2.1.x:
1. It is recommended that the MUI be used to rescan the SCSI bus.
Simply go to the Storage Management dialog and click on the
Rescan SAN option to rescan the SCSI bus.
2. You may reboot the host, but since this is disruptive, it is not
recommended.
3. At the command line, rescanning the SCSI bus is a two-step
process for QLogic HBAs. First, you will need to force the QLogic
HBAs to rescan the SCSI bus first and secondly; the VMkernel
will need to perform the rescan.
Please note that this step is required both when using the command line
and when using the MUI.

a. Issue the following command first:

echo scsi-qlascan > /proc/scsi/qla2300/<hba_id>

For example:
echo scsi-qlascan > /proc/scsi/qla2300/2

b. Next, you will need to have the VMkernel rescan the SCSI bus.
If you are using the command line, issue the following
command:
vmkfstools s <vmware_hba_id>

For example:
vmkfstools -s vmhba0

If you are using the MUI, then go to the Storage

Management dialog and click on the Rescan SAN option
to rescan the SCSI bus.
4. At the command line, you only need to rescan the VMkernel
using Emulex HBAs.
If you are using the command line, issue the following
command:
vmkfstools s <vmware_hba_id>

Rescanning the SCSI Bus

2-27

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
For example:
vmkfstools -s vmhba0

If you are using the MUI, then go to the Storage Management

dialog and click on the Rescan SAN option to rescan the SCSI
bus.
5. For either QLogic or Emulex HBAs, you may also remove and
reinsert the driver from the VMkernel to initiate a rescan of the
SCSI bus.
For QLogic:
vmkload_mod l
vmkload_mod u qla2300_604
vmkload_mod /usr/lib/vmware/vmkmod/qla2300_604.o vmhba

For Emulex:
vmkload_mod l
vmkload_mod u lpfcdd
vmkload_mod /usr/lib/vmware/vmkmod/lpfcdd.o vmhba
It is recommended in the case of using either the QLogic or Emulex HBAs
that the Rescan SAN option in the MUI be used to rescan the SCSI bus.

For VMware ESX Server v2.5:

1. It is recommended that the MUI be used to rescan the SCSI bus.
Simply go to the Storage Management dialog, and click on the
Rescan SAN option to rescan the SCSI bus. This process calls the
cos-rescan.sh.
2. If the command line is preferred, you may use the
cos-rescan.sh script to initiate the rescan.
For example:
[root@l82bi199 /]# cos-rescan.sh vmhba2
Rescanning vmhba2...done.
On scsi1, removing:
On scsi1, adding:

3. You may reboot the host, but since this is disruptive, it is not
recommended.

2-28

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

4. If booting from an internal hard drive rather than from the SAN,
you may also remove and reinsert the driver from the VMkernel
to initiate a rescan of the SCSI bus. This is pertinent for both
QLogic and Emulex HBAs.
For QLogic:
vmkload_mod l
vmkload_mod u qla2300_604
vmkload_mod /usr/lib/vmware/vmkmod/qla2300_604.o vmhba

For Emulex:
vmkload_mod l
vmkload_mod u lpfcdd
vmkload_mod /usr/lib/vmware/vmkmod/lpfcdd.o vmhba

In VMware ESX Server v2.5, the vmkfstools command should not be

used to rescan the SCSI bus; otherwise, the Service Console will not
see or report the changes made.

Rescanning the SCSI Bus

2-29

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Persistent Binding
EMC highly recommends that persistent binding be enabled in
VMware ESX Server v2.x. When persistent binding is enabled, the
target IDs are assigned to specific storage array devices. This
assignment persists through reboots so the association remains the
same. This is particularly helpful when using raw devices.
Persistent binding may be enabled or disabled using the MUI or the
command line.
If using the MUI, select the Options tab, and then select the Storage
Management tab. Under Storage Management, there are three
options: Disks and LUNs, Failover Paths, and Adapter Bindings.
To view the current information for your system, select the Adapter
Bindings option.
An example of the output from the Adapter Bindings dialog is
below:

2-30

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

To view and reset the persistent binding from the command line,
follow the steps below.
1. Log in as root.
2. To view the current persistent binding entries for the HBAs in the
system, look at the pbindings file.
An example of what the pbindings file may look like is as
follows:
[root@l82bi199 /]# more /etc/vmware/pbindings
6.1.1.qla2300_607 = "scsi-qla1-tgt-0-di-0-port=50060160082012bb;scsi-qla1-tgt-1di-0-port=50060168082012bb;scsi-qla1-tgt-2-di-0-port=5006016010601270;scsi-qla1tgt-3-di-0-port=5006016810601270;"
6.1.0.qla2300_607 = "scsi-qla0-tgt-0-di-0-port=50060168082012bb;scsi-qla0-tgt-1di-0-port=50060160082012bb;scsi-qla0-tgt-2-di-0-port=5006016010601270;scsi-qla0tgt-3-di-0-port=5006016810601270;"

3. To remove the existing persistent binding settings, either the

pbindings file may be deleted or the pbind.pl script may be run.
The pbind.pl is located in the /usr/sbin directory and may be
run as root from any directory location.
To delete the file, use the following command:
rm f /etc/vmware/pbindings

To remove the settings using the pbind.pl script, use the D

switch. This switch deletes the persistent binding information for
all of the adapters within the system.
pbind.pl -D

For example:
[root@l82bi199 /]# pbind.pl -D
Deleting ALL bindings

4. Reboot the host.

5. After the system reboots successfully, log in and again run the
pbind.pl script. This time use the A switch which will add the
persistent binding information for all of the adapters within the
system.
pbind.pl A

Persistent Binding

2-31

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
For example:
[root@l82bi199 /]# pbind.pl -A
+-----------------------------------------+
| Setting up the following bindings...
|
+--------+---------+----------------------+
| Host | Target |
World Wide Name
|
|
01
|
000
|
50060160082012bb
|
|
01
|
001
|
50060168082012bb
|
|
01
|
002
|
5006016010601270
|
|
01
|
003
|
5006016810601270
|
|
00
|
000
|
50060168082012bb
|
|
00
|
001
|
50060160082012bb
|
|
00
|
002
|
5006016010601270
|
|
00
|
003
|
5006016810601270
|
+-----------------------------------------+

6. Edit the vmware initialization script. This script is run when the
system is booting up and starts and manages the services needed
by the VMkernel.
vi /etc/init.d/vmware

7. Search for the line containing the text

save_san_persistent_bindings. The code should be located
approximately around line 896 in the file.
896
897

# Save the SAN persistent bindings for the next boot

save_san_persistent_bindings

8. Comment out the line save_san_persistent_bindings.

This is only a temporary change. When this line is commented
out, the adapter to target bindings will not be saved and
therefore, will not persist through reboots.
896
897

# Save the SAN persistent bindings for the next boot

#save_san_persistent_bindings

9. Save the file and reboot the host again.

10. Once the system is rebooted, the persistent binding line will need
to be uncommented.
Change to the /etc/init.d/vmware directory again:
vi /etc/init.d/vmware

Uncomment the line for save_san_persistent_bindings and

save the file.

2-32

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

11. Now run wwpn.pl to associate the targets, HBA WWPN, and
vmhba.
[root@l82bi199 /]# wwpn.pl -v
WWPN 1.02 Copyright VMware 2003
Display WW port names and VMHBA information for fibre channel adapters
For each vmhba here are the corresponding Qlogic and Emulex WW Port Names
Adapter WWPN
PCI (decimal)
vmhba1: 210000e08b0910a7 (Qlogic) 6:1:0 /proc/scsi/qla2300/0
vmhba1:0: 50060168082012bb scsi-qla0-port-0=0000000000000000:50060168082012bb;
vmhba1:1: 50060160082012bb scsi-qla0-port-1=0000000000000000:50060160082012bb;
vmhba1:2: 5006016010601270 scsi-qla0-port-2=0000000000000000:5006016010601270;
vmhba1:3: 5006016810601270 scsi-qla0-port-3=0000000000000000:5006016810601270;
vmhba2: 210100e08b2910a7 (Qlogic) 6:1:1 /proc/scsi/qla2300/1
vmhba2:0: 50060160082012bb scsi-qla1-port-0=0000000000000000:50060160082012bb;
vmhba2:1: 50060168082012bb scsi-qla1-port-1=0000000000000000:50060168082012bb;
vmhba2:2: 5006016010601270 scsi-qla1-port-2=0000000000000000:5006016010601270;
vmhba2:3: 5006016810601270 scsi-qla1-port-3=0000000000000000:5006016810601270;

12. The persistent binding is now reset to the most recent

information.

Persistent Binding

2-33

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

LUN Spanning
When a volume is spanned, the volume is extended to include
multiple VMFS-2 disks or partitions. A physical extent is defined by
the VMFS-2 disks or partitions that compose a logical spanned
VMFS-2 volume.
There are some restrictions to using spanned volumes. They are:

Spanning volumes is supported only with VMFS-2 volumes, not

with VMFS-1 volumes.

The maximum number of files may not be changed.

A spanned volume may not be removed.

Data on the physical extents are lost when they are added to the
VMFS-2 spanned volume. Therefore, it is strongly recommended
that only newly created partitions or newly added devices are
used for spanning volumes.

A VMFS-2 volume may span up to a maximum of 32 LUNs.

To span a volume using the MUI, select the Options tab then select
Storage Management. Select the Disks and LUNs category. In the
Disks and LUNs category, you are provided with a list of the devices
that are assigned to the host. Select the one that you want to expand.
The configuration window for the device you chose will be shown on
your screen and will provide you with the specifics on the selected
device plus the devices that may be added to the selected device to
create a spanned VMFS-2 volume.
See the figure that follows for an example using the device
vmhba1:2:13.

2-34

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

After you have spanned a VMFS-2 volume, you may view the
changes immediately because the /vmfs directory is updated
dynamically. The command vmkfstools should be used to list the
information on the particular device and to provide the spanning and
physical extent information.
For example:
[root@l82bi199 /]# cd /vmfs
[root@l82bi199 /]# vmhba1:0:0:1
vmhba1:2:11:1 vmhba1:2:13:1 vmhba1:2:3:1
[root@l82bi199 /]# vmkfstools --list vmhba1:2:13:1
Name: 1270data13
(public)
Capacity 2826683392 (2679 file blocks * 1048576), 1701838848 (1623 blocks) avail
LUN Spanning

2-35

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
Permission
rw-------

Uid
0

Gid
0

Attr
Bytes (Blocks) Last Modified Filename
disk 1107296256 ( 1056) Aug 26 06:28 w2k3_data4.dsk

[root@l82bi199 /]# vmkfstools -P vmhba1:2:13:1

vmhba1:2:13:1 is a VMFS-2.11 volume spanning 4 physical extents.
Volume label (if any): 1270data13
UUID (if any): 409bb858-be8577b8-ccf1-0002b3248a69
Physical Extents:
vmhba1:2:13:1
vmhba1:2:5:1
vmhba1:2:6:1
vmhba1:2:7:1

To view the same volume information from the MUI, go to the

Options tab and select Storage Management. Select the Disks and
LUNs category and scroll to view the VMFS-2 volume that was
spanned. Please see the reference to Disk vmhba1:2:13 in the figure
that follows for an example.

2-36

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Setting Up and Installing Virtual Machines

The devices to be used for the Virtual Machines with VMFS are
formatted in the Storage Management > Disks and LUNs dialog.
When the Create Volumes option is selected, you are prompted to
select the method with which the devices will be created.

The Typical option formats the entire device.

The Custom option allows you to manually create one or more

partitions according to the size required.

Select the Typical formatting option, as shown in Figure 2-5. If your

system has not been configured with a Core Dump Partition, then
you will be prompted to create one.

Figure 2-5

Storage Management > Disks and LUNs Tab

When booting from an internal hard drive, it is recommended that

the core dump partition be created on the hard disk internal to the
system, not a SAN attached device. If booting from a SAN device,
then the core dump partition must be created on a SAN device as
well.
An example of the dialog providing this prompt is in Figure 2-6 on
page 2-38.

Setting Up and Installing Virtual Machines

2-37

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Figure 2-6

Storage Management > Disks and LUNs > Core Dump Partition

Select Yes, and then select Finish.

After the partition has been created, it should be formatted with
VMFS. You also can provide it with a Volume Label, or alias, that is
listed as the VMFS Name.
For VMFS file systems, you can specify whether the file system is
Public (accessible by multiple physical servers) or Shared (accessible
by multiple physical servers, and can be used simultaneously by
virtual machines on these servers).
The accessibility should be set to the default which is Public in almost
all cases. The accessibility should be set to Shared if the VMFS will
hold disks to be used for MSCS clustering.
An example of the accessibility setting window is shown in
Figure 2-7.

2-38

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Figure 2-7

Access Mode

After the devices are formatted with the VMFS file system, the Virtual
Machines can be configured and installed.
In the main screen of the VMware ESX Server, go to the Status
Monitor tab and select Add Virtual Machine. The Wizard will guide
you through the Virtual Machine configuration.
An example of the Virtual Machine Configuration dialog for a
Windows 2000 virtual machine is in Figure 2-8 on page 2-40.

Setting Up and Installing Virtual Machines

2-39

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Figure 2-8

Add Virtual Machine Dialog

In this dialog, you can allocate a name for the virtual machine and a
location for the configuration file.
The name allocated in the Virtual Machines dialog will be displayed
in the Status Monitor dialog as shown in Figure 2-9 as well as listed
in the Remote Console listing:

Figure 2-9

2-40

Status Monitor > Virtual Machines

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

After the Virtual Machines are configured, the OS must be installed

on that Virtual Machine. To access a virtual machine, the VMware
Remote Console software needs to be installed on either a Windows
or Linux machine.
The Remote Console software is available to download from the
Status Monitor dialog, as shown in Figure 2-9 on page 2-40, which
provides a listing of the Virtual Machines configured on the VMware
Server. The Remote Console software is available as executable for
Windows NT/Windows 2000 and as an RPM and a tar archive for
Linux. Select and download the appropriate application for your
operating system.
After you have installed the VMware Remote Console software on
your operating system, start the application and log onto the
VMware ESX Server.
Figure 2-10 is an example of what the dialog will look like.

Figure 2-10

VMware Remote Console > Connect to Virtual Machines

Select a Virtual Machine, and select the Power On button. This will
start up the virtual machine.

Setting Up and Installing Virtual Machines

2-41

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
The Phoenix BIOS is used for the Virtual Machines. To initiate an
installation, press F2 to enter the Setup Menu. Go to the Boot menu
and move the entry for the ATAPI CD-ROM Drive to be the first
entry.
For example:

Press F10 to save the configuration and exit.

When the server reboots, it will now be ready for an OS installation.
The actual OS installation may be done with actual CDs inserted into
the VMware host or via ISO images. In order to select the device or
the ISO image, select Devices and from the drop-down menu, select
ide1:0 (Not connected)

2-42

-> Connect
Edit

If you are using an actual CD, select the Connect option.

If you are using an ISO image, select the Edit option.

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

The figure below is an example of what the console will look like
when using the Edit option.
Select the option to Use physical drive, and connect the CD-ROM
device if an actual CD is to be used for the virtual machine OS
installation.
If an ISO image is to be used for the OS installation on the virtual
machine, select Use ISO image, and browse to the location of the ISO
image. In Figure 2-11, an ISO image is selected for installation.

Figure 2-11

Configuration Editor, Hardware, Connection

After the installation has completed, the Virtual Machine must be

rebooted to be ready for use.

Setting Up and Installing Virtual Machines

2-43

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2
The following is an example of the Virtual Machine Overview dialog
with multiple Virtual Machines installed:

2-44

EMC Host Connectivity Guide for VMware ESX Server v2.x

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

Additional Notes

Currently, EMC software will function on neither the Service

Console nor the Virtual Machines as the currently-released
versions of the CLARiiON-based software applications do not
include support for VMware ESX Server.

Currently, VMware ESX Server v2.x does not support accessing

SnapView, MirrorView, or SAN Copy LUNs.

Again, please note that PowerPath is not available for the

VMkernel. The failover supported with VMware ESX Server v2.x
and EMC Storage Arrays is native to the VMware ESX Server
v2.x itself. The native failover functionality performs failover
only, not load balancing.

The VMware ESX Server host needs to be registered manually on

the CLARiiON array as the Naviagent is not supported for
VMware ESX Server.

Neither the Naviagent nor the NaviCLI are supported for

VMware ESX Server at this time.

It is recommended that Virtual Machines running on VMware

ESX Server v2.x boot from the array.

For the Virtual Machines, it is recommended, but not required,

that the boot LUNs be in a CLARiiON RAID group that is
separate from that RAID group for the data volumes.

Prior to VMware ESX Server v2.5, the VMware ESX Server itself
was not supported booting from the array. VMware ESX v2.5
adds support for booting from the SAN for the ESX Server itself.

When booting VMware ESX Server v2.5.0 from an EMC storage

array, it is recommended that the boot LUN be in a RAID group
separate from the data volumes.

When booting the VMware ESX Server from an internal drive, the
VMware Core Dump and Swap Files should be created on
internal drives rather than SAN attached devices.

It is highly recommended that the vmware-tools be installed on

each Virtual Machine. The use of the vmware-tools will save on
resources and provide better memory management. The
vmwaretools will also provide a VGA device driver and a
heartbeat mechanism to communicate with the VMkernel.

Additional Notes

2-45

Installation Information for VMware ESX Server v2.x with Symmetrix and CLARiiON Arrays

2-46

VMware ESX Server hosts can be attached to both CLARiiON and

Symmetrix Storage Arrays simultaneously.

The restriction stating that VMware ESX Server v2.1.x hosts must
be offline for CLARiiON-licensed (FLARE) upgrade and
Storage Processor replacement has been removed. The native
failover functionality within the VMware ESX Server itself has
been successfully qualified and provides the necessary path
failover. NDUs and SP replacements may be performed while the
VMware ESX Server is online.

EMC Host Connectivity Guide for VMware ESX Server v2.x

3
VMware ESX Server v2.x and
Symmetrix Storage Array
Fibre Channel Environment

This chapter lists Symmetrix Fibre Channel support information

specific to the VMware ESX Server environment.

VMware/Symmetrix Environment .................................................3-2

Booting the VMware ESX Server v2.5 from the Symmetrix ........3-6
Fabric Zoning......................................................................................3-8
Required Storage System Configuration ........................................3-8
Useful VMware ESX Server Utilities and Functions.....................3-9
Addressing Symmetrix Devices.....................................................3-10

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

3-1

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

VMware/Symmetrix Environment
This section lists some Symmetrix Fibre Channel support information
specific to the VMware ESX Server environment.

Host and Hardware Connectivity

Refer to the EMC Support Matrix or contact your EMC representative
for the latest information on qualified hosts, host bus adapters, and
connectivity equipment.
EMC does not support mixing HBAs from different vendors in the same host.

Symmetrix Connectivity
The Symmetrix system is configured by an EMC Customer Engineer
via the Symmetrix service processor.
The EMC Customer Engineer (CE) should contact the EMC
Configuration Specialist for updated online information. This
information is necessary to configure the Symmetrix system to
support the customers host environment.
When attaching a VMware ESX Server host to a Symmetrix, the Linux
default FA settings should be used. They are as follows:

SCP3

SCSI-3

PTOP:

Point to Point

VCM:

Volume Logix

C:

Common Serial Number

EAN:

Enable Auto Negotiation

Volume Logix must be enabled on the Symmetrix.

After the EMC CE has assigned target IDs and LUNs and brought the
Symmetrix channel and disk directors online, reboot the network
operating systems, and go into the configuration program.
All qualified HBAs are listed in the EMC Support Matrix.

3-2

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

Note that the VMware ESX Server installer will recognize LUNs
25MB or less as management LUNs. This includes any gatekeepers
assigned to the VMware host via Solutions Enabler.
It is recommended that each VMware ESX Server host have a
maximum of 8 paths to Symmetrix Storage Arrays. A path is defined
by one HBA port that is zoned to one SP port.
With a recommended maximum of 8 paths from a VMware ESX
Server to Symmetrix Storage Arrays, it is recommended that each
VMware ESX Server host be zoned to a maximum of 4 storage arrays.
A couple of possible configuration scenarios are described in Example
1 on page 3-4 and Example 2 on page 3-5.

VMware/Symmetrix Environment

3-3

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

3
Example 1

In this example as shown in Figure 3-1, one host with two HBAs is
attached to one Symmetrix array using two separate switches. Two
FA ports on each of two FAs within the array are being used. HBA1 is
zoned to 15D0 and to 16C1. HBA2 is zoned to 15D1 and to 16C0. The
zones should be composed of a single initiator and a single target so
they would be created with one HBA and on FA port. Two HBAs
with two paths each to the array totals four paths and if using single
initiator/single target zoning, there are 4 zones.
In this particular example, two switches are used. Using only one
switch is supported, but such a configuration would lack
redundancy. Preferably, a minimum of two switches should be used
to add another level of redundancy. Alternatively, for additional
redundancy, two separate fabrics could be utilized.
Server
H
B
A
0

d0 d1 c0

c1

FA
15D Symm 1
Mirrored

Figure 3-1

3-4

H
B
A
1

d0 d1 c0

c1

FA
16C Symm 1
Mirrored

One Host, Two Switches, and One Symmetrix Array

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

Example 2

In this example as shown in Figure 3-2, one host with two HBAs is
attached using a two-switch fabric to four Symmetrx arrays FA ports.
In this configuration, the zones are created with one HBA and one FA
port. That is,

HBA1 is zoned to one 15D0 port on each of the four arrays.

HBA2 is zoned to one 16C1 port on each of the four arrays.

S e rv e r
H
B
A
0

H
B
A
1

d0 d1 c0
d0 d1 c0

c1

FMA ir1 r5oDr ed S y m m 4

d0 d1 c0

d0 d1 c0

c1

c1

F AM 1ir6 rCo re Sd y m m 3

F AM 1ir6 ro
C re dS y m m 4

c1

FMA ir1 r5oDr ed S y m m 3

d0 d1 c0

d0 d1 c0
c1

F AM 1ir5 rDo re dS y m m 1

c1

d0 d1 c0

F AM 1ir5 rDo re dS y m m 2

d0 d1 c0

c1

c1

F AM 1ir6 rCo re Sd y m m 2

F AM 1ir6 rCo re Sd y m m 1

Figure 3-2

One Host, Two Switches, and Four Symmetrix Arrays

All qualified HBAs are listed in the EMC Support Matrix.

When assigning Symmetrix LUNs to a VMware ESX Server host, the

LUNs should be assigned to the host across both FAs since the
Symmetrix is an active/active array.

VMware/Symmetrix Environment

3-5

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

Booting the VMware ESX Server v2.5 from the Symmetrix

VMware ESX Server v2.5 hosts have been qualified for booting from
Symmetrix devices interfaced through Fibre Channel as specified in
the EMC Support Matrix.
EMC does not recommend booting VMware ESX Server from the
Symmetrix system unless the host is using VMwares native failover
functionality.
Note that if VMwares native failover functionality is not used and it is
necessary to use a Symmetrix device as a boot disk, you should shut down
the Virtual Machines and the ESX Server during any maintenance procedures
that might cause the boot disk to become unavailable to the host.

The Symmetrix device that is to contain the Master Boot Record

(MBR) for the host must have a lower logical unit number (LUN)
than any other device visible to the host. This device must be
mapped as /dev/sda by the Linux operating system for the boot to
succeed from the device.
Note that this includes the case of the Symmetrix Volume Logix
database device. The Volume Logix device is write-protected so the
installer will fail to write the MBR to this device.

To force the installer to avoid an attempt to write to this device,

EMC recommends masking the LUN. The administrative host
must first initialize the Volume Logix database. The LUN may
then be masked by modifying the active configuration file to
enable the fba flag2 to restrict access to the Volume Logix
database.

An alternative to masking the VCM DB, is to map the Volume

Logix database device so that it is the highest LUN presented to
the host However, please be aware that the LUN number should
not be higher than 254 (FE) if it is to be used by Solutions Enabler
or Control Center running on a Microsoft host.
The EMC-recommended method is to use LUN masking.

When attaching the VMware host to the Symmetrix storage array, use
the adapter in the lowest-numbered PCI slot in the server.
To keep the configuration and installation simple, it is recommended
that only that HBA be cabled to the array. Ensure that the boot BIOS

3-6

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

or firmware has been applied to your HBA and that the boot BIOS
has been enabled on the HBA to be used for boot.
Prior to the installation on a Symmetrix LUN, the Linux host HBA
must have successfully logged into the array. Using Solutions
Enabler from another host, at least one LUN must be assigned to the
host.
During the installation procedure, it is recommended, but not
required, that only one LUN be allocated to the host for ease of use.
After the installation has completed, additional LUNs can be
assigned to the host.
When performing the installation, you will need to manually type
bootfromsan or bootfromsan-text at the installation prompt.
By selecting this option, the HBAs will automatically be set to be
shared between the Service Console and the Virtual Machines.
Remember that this parameter should not be changed when creating
your VMware ESX Server configuration.
Cautions

CAUTION
If VMware loses all paths to the array for a long enough period, the
disks disappear from the system. A hard reboot is required to
bring the system back to a usable state.
Any of these events could crash a system booting from a Symmetrix
storage array:

Lost connection to the Symmetrix system (pulled or damaged

cable connection).

Symmetrix service and upgrade procedures, such as on-line

Symmetrix microcode upgrades and/or configuration changes.

Symmetrix director failures, including failed lasers.

Symmetrix system power failure.

Storage area network service/upgrade procedures such as

firmware upgrades or hardware replacements.

Booting the VMware ESX Server v2.5 from the Symmetrix

3-7

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

3
Restrictions

Emulex HBAs are not supported when booting from the fabric;
only QLogic HBAs are supported.

Raw disk mapping is not supported when booting from the

fabric.

Only FC-SW environments are supported when booting from the

SAN.

Fabric Zoning
When using VMware ESX Server hosts in a fabric environment, the
zoning must be set up as single initiator and single target zoning. A
single initiator/single target zone is composed of one HBA and one
Symmetrix storage array port.
Multi-initiator zones are not supported in a VMware ESX Server fabric
environment.

Required Storage System Configuration

Symmetrix system configuration is performed by an EMC Customer
Engineer through the Symmetrix service processor.
Using Volume Logix to mask LUNs is highly recommended.
To verify that the VMware ESX Server host can see all of the
Symmetrix target devices, configure the host as described in the
remainder of the chapter.

3-8

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

Useful VMware ESX Server Utilities and Functions

Utility/Function

Description

fdisk

Command used to create and manipulate partition tables.

grep

Command used to search through a file or files to find specific text.

vmkpcidivy

Command used to create a boot configuration for the VMkernel.

vmkfstools

Command used to create and manipulate files on LUNs owned by the VMware ESX Server host.

vmkload_mod

Command used to view, load, remove driver modules in the VMkernel.

vmkmultipath

Command used to display and set the configuration on a particular device and the paths attached
to that device.

vm-support

Command used to gather information about the VMware ESX Server itself and virtual machines to
assist in debugging issues or to obtain performance information for the virtual machines.

vmkdump

Command used to manage the VMkernels dump partition.

Useful VMware ESX Server Utilities and Functions

3-9

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

Addressing Symmetrix Devices

Fabric Addressing
Each port on a device attached to a fabric is assigned a unique 64-bit
identifier called a World Wide Port Name (WWPN). These names are
factory-set on the HBAs in the hosts, and are generated on the Fibre
Channel directors in the Symmetrix system.
For comparison to Ethernet terminology, an HBA is analogous to a NIC card,
and a WWPN to a MAC address.

The ANSI standard also defines a World Wide Node Name (WWNN), but
this name has not been consistently defined by the industry.

When an N_Port (host server or storage device) connects to the fabric,

a login process occurs between the N_Port and the F_Port on the
fabric switch. During this process, the devices agree on such
operating parameters as class of service, flow control rules, and fabric
addressing. The N_Ports fabric address is assigned by the switch
and sent to the N_Port. This value becomes the source ID (SID) on
the N_Port's outbound frames and the destination ID (DID) on the
N_Port's inbound frames.
The physical address is a pair of numbers that identify the switch and
port, in the format s,p, where s is a domain ID and p is a value
associated to a physical port in the domain. The physical address of
the N_Port can change when a link is moved from one switch port to
another switch port. The WWPN of the N_Port, however, does not
change. A Name Server in the switch maintains a table of all
logged-in devices, so an N_Port may adjust automatically to changes
in the fabric address by keying off the WWPN.
The highest level of login that occurs is the process login. This is used
to establish connectivity between the upper-level protocols on the
nodes. An example is the login process that occurs at the SCSI FCP
level between the HBA and the Symmetrix system.

3-10

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

SCSI-3 FCP Addressing

The Symmetrix director extracts the SCSI Command Descriptor
Blocks (CDB) from the frames received through the Fibre Channel
link. Standard SCSI-3 protocol is used to determine the addressing
mode and to address specific devices.
The Symmetrix supports three addressing methods based on a
single-layer hierarchy as defined by the SCSI-3 Controller Commands
(SCC):

Peripheral Device Addressing

Logical Unit Addressing

Volume Set Addressing

All three methods use the first two bytes (0 and 1) of the 8-byte LUN
addressing structure. The remaining six bytes are set to 0s.
For Logical Unit and Volume Set addressing, the Symmetrix port
identifies itself as an Array Controller in response to a hosts Inquiry
command sent to LUN 00. This identification is done by returning
the byte 0x0C in the Peripheral Device Type field of the returned
data for Inquiry. If the Symmetrix system returns the byte 0x00 in the
first byte of the returned data for Inquiry, the Symmetrix system is
identified as a direct access device.
Upon identifying the Symmetrix system as an array controller device,
the host should issue a SCSI-3 Report LUNS command (0xA0), in
order to discover the LUNs.

Addressing Symmetrix Devices

3-11

VMware ESX Server v2.x and Symmetrix Storage Array Fibre Channel Environment

3
The three addressing modes, contrasted in Table 3-1, Symmetrix
SCSI-3 Addressing Modes, differ in the addressing schema (Target ID,
LUN, and Virtual Bus) and number of addressable devices.
Table 3-1

Symmetrix SCSI-3 Addressing Modes

Possible
Addresses

Maximum
Logical
Devicesb

Access LUNs
directly

16,384

256

0x0C Array
Controller

Host issues
Report LUNS
command

2,048

128

0x0C Array
Controller

Host issues
Report LUNS
command

16,384

512

Addressing Mode

Codea

A
Bit

V
Bit

Response to
Inquiry

Peripheral Device

00

0x00 Direct
Access

Logical Unit

10

Volume Set

01

LUN
Discovery
Method

a. Bits 7-6 of byte 0 of the address.

b. The actual number of supported devices may be limited by the type host or host bus adapter used.

The addressing modes are provided to allow flexibility in interfacing with

various hosts. In all three cases the received address is converted to the
internal Symmetrix addressing structure. Volume Set addressing is the
default for Symmetrix systems. Select the addressing mode that is
appropriate to your host.

3-12

EMC Host Connectivity Guide for VMware ESX Server v2.x

Invisible Body Tag

VMware ESX Server v2.x and

CLARiiON Storage Array
Fibre Channel Environment

This chapter lists CLARiiON Fibre Channel support information

specific to the VMware ESX Server environment.
Refer to the E-Lab Navigator or contact your EMC representative for
the latest information on qualified hosts.

VMware/CLARiiON Environment ................................................4-2

Failover Functionality in VMware ESX Server v2.x for Attach to
CLARiiON Storage Arrays ...............................................................4-9
Manual Registration of VMware ESX Server v2.x for Attach to
CLARiiON Storage Arrays .............................................................4-13
Adding the VMware ESX Server Host to a Storage Group .......4-20
Manual Trespass on CLARiiON Arrays to Recover the Original
Path ....................................................................................................4-21

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4-1

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

VMware/CLARiiON Environment
This section lists CLARiiON Fibre Channel support information
specific to the VMware ESX Server environment.

Host and Hardware Connectivity

Refer to the EMC Support Matrix or contact your EMC representative
for the latest information on qualified hosts, host bus adapters, and
connectivity equipment.
EMC does not support mixing HBAs from different vendors in the same host.

CLARiiON Connectivity
Access Logix must be installed on the CLARiiON storage array to
which the VMware ESX Server is being attached.
Currently, the VMware ESX Server host must be manually registered
on the CLARiiON array. The currently-released version of the
Naviagent does not include support for VMware ESX Server.
VMware ESX Server owns the HBAs, not the operating systems
running in the virtual machines. As a result, the VMware ESX
Servers HBAs will be registered on the CLARiiON and assigned to a
Storage Group.
The virtual machines will be assigned LUNs through the VMware
ESX Server service console.
It is recommended that each VMware ESX Server host have a
maximum of 8 paths to CLARiiON Storage Arrays. A path is defined
by one HBA port that is zoned to one SP port.
With a recommended maximum of 8 paths from a VMware ESX
Server to CLARiiON Storage Arrays, it is recommended that each
VMware ESX Server host be zoned to a maximum of 4 storage arrays.
A couple of possible configuration scenarios are described in Example
1 on page 4-3 and Example 2 on page 4-4.

4-2

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

Example 1

In this example as shown in Figure 4-1, one host with two HBAs is
attached to one CLARiiON array using two separate switches. Two
SP ports on each SP within the array are being used. HBA1 is zoned
to SPA0 and to SPB1. HBA2 is zoned to SPA1 and to SPB0. The zones
should be composed of a single initiator and a single target so they
would be created with one HBA and on SP port. Two HBAs with two
paths each to the array totals four paths and if using single
initiator/single target zoning, there are 4 zones.
In this particular example, two switches are used. Using only one
switch is supported, but such a configuration would lack
redundancy. Preferably, a minimum of two switches should be used
as this adds another level of redundancy. Alternatively, for additional
redundancy, two separate fabrics can be utilized.

Server
H
B
A
0

SP A
rored

Figure 4-1

H
B
A
1

SP B
Mirrored

One Host, Two Switches, and One CLARiiON Array

VMware/CLARiiON Environment

4-3

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4
Example 2

In this example as shown in Figure 4-2, one host with two HBAs is
attached using a two-switch fabric to four CLARiiON array SPs. In
this configuration, the zones are created with one HBA and one SP
port. For instance:

HBA1 is zoned to one SPA port on each of the four arrays.

HBA2 is zoned to one SPB port on each of the four arrays.

S e rv e r
H
B
A
0

0
0

M ir r oSrePd B

P dB
M irrSo re
1

Pd A
M ir rSo re

Figure 4-2

Pd A
M irrSo re
0

M irr So rePd B

3M ir roS rPe d A

H
B
A
1

M ir roS rPe d A

M ir r oSrePd B

One Host, Two Switches, and Four CLARiiON Arrays

All qualified HBAs are listed in the EMC Support Matrix.

When assigning CLARiiON LUNs to a VMware ESX Server host, the

LUNs may be assigned to only one SP or their assignments may be
split between the two SPs. Either configuration is valid.

4-4

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

Booting the VMware ESX Server v2.5 from CLARiiON Storage Arrays
VMware ESX Server v2.5 hosts have been qualified for booting from
CLARiiON devices interfaced through Fibre Channel as specified in
the EMC Support Matrix.
EMC does not recommend booting VMware ESX Server from the
CLARiiON storage array unless the host is using VMwares native
failover functionality.
Note that if VMwares native failover functionality is not used and it is
necessary to use a CLARiiON device as a boot disk, you should shut down
the Virtual Machines and the ESX Server during any maintenance procedures
that might cause the boot disk to become unavailable to the host.

The CLARiiON device that is to contain the Master Boot Record

(MBR) for the host must have a lower logical unit number (LUN)
than any other device visible to the host. This device must be
mapped as /dev/sda by the VMware ESX Server operating system
for the boot to succeed from the device.
Note that this includes the case of the CLARiiON ghost LUN. If the
LUN 0 presented to a SCSI path is not owned by the SP attached to
that path, a disconnected LUN 0 (ghost LUN) will be presented. The
ghost LUN is not write-enabled so that the MBR cannot be written to
it. Always ensure that the boot LUN is owned by the correct SP and
does not trespass at any time; trespass of the boot LUN would result
in a system crash.
When attaching the VMware host to the CLARiiON storage array, use
the adapter in the lowest-numbered PCI slot in the server. To keep
the configuration and installation simple, it is recommended that
only that HBA be cabled to the array. Ensure that the boot BIOS or
firmware has been applied to your HBA.
Prior to the installation, the VMware ESX Server must have been
manually registered on the array and assigned to a Storage Group. It
is recommended that you attach the VMware ESX Server to the
lowest numbered SP with visible LUNs. At least one LUN must be
bound to the hosts Storage Group and owned by the SP connected to
the HBA being used for the fabric boot. The lowest-numbered path
to the boot LUN must be the active path.
Since the lowest numbered path to the boot LUN must be active,
attaching to the CLARiiON via direct connect or FC-AL is not

VMware/CLARiiON Environment

4-5

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4
supported. Only FC-SW environments are supported when booting
from the SAN.
It is required that the boot LUN be assigned Host LUN ID 0.
During the installation procedure, it is recommended, but not
required, that only one LUN be assigned to the Storage Group for
ease of use. After the installation has completed, additional LUNs
can be added to the Storage Group.
When performing the installation, you will need to manually type
bootfromsan or bootfromsan-text at the installation prompt.
By selecting this option, the HBAs will automatically be set to be
shared between the Service Console and the Virtual Machines.
Remember that this parameter should not be changed when creating
your VMware ESX Server configuration.
Cautions

CAUTION
If VMware loses all paths to the array for a long enough period, the
disks disappear from the system. A hard reboot is required to
bring the system back to a usable state.
Any of these events could crash a system booting from a CLARiiON
storage array:

Restrictions

4-6

Lost connection to the CLARiiON storage array (pulled or

damaged cable connection).

Symmetrix service and upgrade procedures, such as on-line

CLARiiON FLARE upgrades and/or configuration changes.

CLARiiON SP failures, including failed lasers.

CLARiiON storage array power failure.

Storage area network service/upgrade procedures such as

firmware upgrades or hardware replacements.

Trespass of a CLARiiON LUN from one SP to another.

Emulex HBAs are not supported when booting from the fabric;
only QLogic HBAs are supported.

Raw disk mapping is not supported when booting from the

fabric.

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

Only FC-SW environments are supported when booting from the

SAN.

Fabric Zoning
When using VMware ESX Server hosts in a fabric environment, the
zoning must be set up as single initiator and single target zoning. A
single initiator/single target zone is composed of one HBA and one
CLARiiON storage array port.
Multi-initiator zones are not supported in a VMware ESX Server
fabric environment.

Required Storage System Configuration

CLARiiON configuration is done by an EMC Customer Engineer
(CE) through Navisphere Manager. The CE will configure your
FC4700 or CXx00 series system settings for each Fibre Channel port.
The procedures in this document assume that you have already
installed any switches and storage systems used in this configuration,
and connected the CLARiiON storage system SPs (storage
processors) to the switch ports.5-3
To verify that the VMware ESX Server host can see all of the
CLARiiON devices, configure the host as described in the remainder
of the chapter.

Useful VMware ESX Server Utilities and Functions

Table 4-1

Useful VMware ESX Server Utilities and Functions

Utility/Function

Description

fdisk

Command used to create and manipulate partition tables.

grep

Command used to search through a file or files to find specific text.

vmkpcidivy

Command used to create a boot configuration for the VMkernel.

vmkfstools

Command used to create and manipulate files on LUNs owned by the VMware ESX Server host.

vmkload_mod

Command used to view, load, remove driver modules in the VMkernel.

VMware/CLARiiON Environment

4-7

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4
Table 4-1

4-8

Useful VMware ESX Server Utilities and Functions (continued)

Utility/Function

Description

vmkmultipath

Command used to display and set the configuration on a particular device and the paths attached to
that device.

vm-support

Command used to gather information about the VMware ESX Server itself and virtual machines to
assist in debugging issues or to obtain performance information for the virtual machines.

vmkdump

Command used to manage the VMkernels dump partition.

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

Failover Functionality in VMware ESX Server v2.x for Attach to

CLARiiON Storage Arrays
Native failover functionality has been incorporated into the
VMkernel beginning with VMware ESX Server v2.1.0. The VMkernel
itself will see the multiple paths configured to the storage for the host.
However, the Virtual Machines will be presented only a single SCSI
path to the devices that are allocated or assigned to each Virtual
Machine.
VMware ESX Server v2.x does support multiple paths to a
Symmetrix/ CLARiiON array, but load balancing is not currently
supported. However, there are two alternate path polices that may be
implemented. The path policies are MRU (Most Recently Used) and
Fixed.
PowerPath is not available for VMware ESX Server.

The default policy is Fixed for active/active disk arrays and MRU for
active/passive disk arrays.
The MRU policy is strongly recommended for CLARiiON arrays and
any other active/passive arrays. The reason for this recommendation
is the higher likelihood of path thrashing occurring on
active/passive types of arrays when using the Fixed failover policy.
Path thrashing occurs when two hosts are accessing the same LUN
via different storage processors on a disk array. In the case of the
CLARiiON, this may cause a loop where a LUN is trespassed from
one storage processor to the other and back again. Such thrashing
behavior will cause poor performance and may cause errors.
With the MRU policy, VMware ESX Server hosts will use one active
path to a storage processor. A trespass will be forced to the other
storage processor only when the initial path has failed and there is no
access to devices via that path. Therefore, VMware ESX Server hosts
using the MRU policy will quickly settle on accessing a LUN through
the storage processor that is accessible to all hosts.
The MRU policy will not automatically restore the I/O back to the
initial path. This limitation means that there is not an easy way to
restore the paths initially used by the VMware ESX Server hosts to
access the LUNs. The MRU policy will choose the first available path
that avoids performing a trespass.

Failover Functionality in VMware ESX Server v2.x for Attach to CLARiiON Storage Arrays

4-9

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4
The trespass back to the original path will need to be performed
manually using the Navisphere Manager.
The Fixed path policy allows you to manually configure the path to
be used. If the preferred path fails, the Fixed path policy will transfer
I/O to the live path until the time when the preferred path is restored.
When the preferred path is restored, all I/O will be transferred back
to the preferred path.
The paths currently being used can be viewed and changed using the
command vmkmultipath or using the MUI. When using the MUI,
select the Options tab and then selecting Storage Management and
last, Failover Paths.

4-10

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

Below is an example of the information reported with the Failover

Paths dialog:

The Failover Paths illustrates the SAN LUN information, the paths
being used, and the failover policy. As can be seen in the figure, the
LUNs attached to the VMware ESX Server host are individually listed
with the pertinent HBA, target, and LUN information. The current
HBA being used and the failover policy being used may be edited
here on a per LUN basis.
When a cable is pulled, I/O from the VMware ESX Server and the
Virtual Machines essentially will be frozen for approximately a
minute. The fibre channel HBA driver within VMware ESX Server
must identify that the link is down before a failover will occur.

Failover Functionality in VMware ESX Server v2.x for Attach to CLARiiON Storage Arrays

4-11

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4
During the time that the link is down, the Virtual Machines will not
be responsive, but they should resume to their normal behavior once
the failover has occurred. In the event that all paths to the storage
have failed, then I/O errors will be reported by the Virtual Machines
and access to the /vmfs directory will also fail.

4-12

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

Manual Registration of VMware ESX Server v2.x for Attach to

CLARiiON Storage Arrays
Currently, the VMware ESX Server host must be manually registered
on the CLARiiON array as the currently released version of the
Naviagent does not include support for VMware ESX.
In order to manually register the host, you must first identify the
WWNs of the HBAs. The WWNs are used when creating zones on
the switches and in registering the VMware ESX Server host onto the
CLARiiON arrays. Zoning by WWN is not required as zoning by
either WWN or by switch port is supported.
1. The recommended method to discover WWNs is to run the
wwpn.pl command for either QLogic or Emulex HBAs. For each
vmhba instance, the wwpn.pl will provide the corresponding
QLogic or Emulex WWPNs.
For example:
[root@l82bi199 /]# /usr/sbin/wwpn.pl
vmhba2: 210000e08b0910a7 (Qlogic) 6:1:0
vmhba3: 210100e08b2910a7 (Qlogic) 6:1:1

An alternate method to obtain Emulex HBAs initiator and target

information is to refer to /proc/scsi/lpfcdd/N (where N
indicates the file for each adapter in the system) when the driver
is loaded. By greping the file(s), the necessary information to
register to host will be reported.
grep the file to obtain the initiator and target information.
For example,
grep DID /proc/scsi/lpfcdd/1

produces output similar to the following for the first Emulex

HBA:
lpfc0t00 DID 060300 WWPN 50:06:01:61:10:60:12:5c WWNN 50:06:01:60:90:60:12:5c
lpfc0t01 DID 060400 WWPN 50:06:01:69:10:60:12:5c WWNN 50:06:01:60:90:60:12:5c

Also, when using QLogic HBAs, the same information is logged

in /proc/scsi/qla2x00/N (where N indicates the file for each
adapter in the system) when the driver is loaded.
grep the file to obtain the initiator and target information.

Manual Registration of VMware ESX Server v2.x for Attach to CLARiiON Storage Arrays

4-13

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4
For example, for a host with QLA23xx HBAs:
grep scsi-qla /proc/scsi/qla2300/0

produces output similar to the following for the first QLogic

HBA:
scsi-qla0-adapter-node=200000e08b0910a7;
scsi-qla0-adapter-port=210000e08b0910a7;
scsi-qla0-target-0=5006016810601270;
scsi-qla0-target-1=5006016010601270;
scsi-qla0-target-2=50060160082012bb;
scsi-qla0-target-3=50060169082012bb;

Now that the WWNs have been identified, the VMware ESX Server
host can now be registered to the CLARiiON.
The following section will describe the manual registration process.
In order to manually register the host on the CLARiiON array,
perform the following steps:
1. Start the Navisphere Manager in a web browser on a host to be
used for management purposes.
2. Select the Storage tab so that the arrays being managed by the
Navisphere Manager are displayed.
3. Right click on the appropriate array, and select the Connectivity
Status option.
4. The Connectivity Status dialog for that array will show the
Initiator WWNs for each host logged into the array.
An example of the Connectivity Status dialog can be seen Figure 4-3
on page 4-15.

4-14

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

Figure 4-3

Connectivity Status dialog

5. In order to manually register your hosts HBAs, select the WWN

of the HBA and click on Register. You will be prompted to add
the server information such as the server name and IP address.
Repeat for each HBA instance for this VMware ESX Server host.
This will register this HBA or HBAs for your VMware host.
6. Another dialog will appear called Register Initiator Record.
When attaching a VMware ESX Server to a CLARiiON, the
standard default Linux director bit settings should be used.
An example of a non-failover environment is a host with one
HBA zoned only to one SP port, such as SPA2.

Manual Registration of VMware ESX Server v2.x for Attach to CLARiiON Storage Arrays

4-15

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4
For a non-failover environment, the required settings are as
follows:
Initiator Type:

CLARiiON Open

ArrayCommPath:

Disabled

FailOverMode:

Unit Serial Number: Array

For a failover-enabled environment, the required settings are as
follows:
Initiator Type:

CLARiiON Open

ArrayCommPath:

Enabled

FailOverMode:

Unit Serial Number:

Array

Please note that the failover functionality referred to here is the native
failover functionality incorporated into the VMkernel, not
PowerPath. PowerPath is not available for the VMkernel.

4-16

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

For an example of registering for a failover-enabled environment, see

the screen shot below.
Note that the box for the ArrayCommPath parameter is checked and the
Failover Mode is set to 1.

Because no Naviagent is used on the VMware ESX Server, you will

receive a warning message when registering the host.

Manual Registration of VMware ESX Server v2.x for Attach to CLARiiON Storage Arrays

4-17

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4
See below for an example of the warning message.

Because the Naviagent is not being used, this warning is to be

expected and is acceptable.
1. Repeat steps 1 through 4 for each HBA in the VMware ESX Server
system.
2. To verify that your host has been properly registered, right click
on the Array and go to the Hosts tab.

4-18

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

The host will be reported as attached, but manually registered as in

the example of the system named l82bi199.lss.emc.com in the
following figure.

Manual Registration of VMware ESX Server v2.x for Attach to CLARiiON Storage Arrays

4-19

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

Adding the VMware ESX Server Host to a Storage Group

After the host is registered manually, the host can be added to a
Storage Group so that it can be allocated devices and can access those
devices.
1. Right click on the array, and select the option Create Storage
Group. Provide it with a name such as the name or IP address of
the host or a nickname.
2. After the Storage Group is created, then LUNs and the host may
be added to the Storage Group. Select the newly created Storage
Group from the list, and right click on it.
3. Select the Properties option and select LUNs to add to the group
and select the host to own the group.

4. The VMware host will be listed under the Hosts To be Connected

and the OS will report itself as Unknown. This is to be expected
since the host was manually registered on the array and is
acceptable behavior.

4-20

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

Manual Trespass on CLARiiON Arrays to Recover the Original

Path
In some cases when using the MRU policy, it may be desirable to
trespass the LUNs back to their original path. This may be performed
manually using the Navisphere Manager. The instructions for a
manual LUN trespass are below.
1. Open a web browser and type in the IP address of the CLARiiON
array your host is attached to.
2. Once the GUI is updated with the log-in window, type in the user
name and password for the array.
3. Click on the array to expand the listing of properties.
4. Click on the SP that holds the active path for the LUNs. The
LUNs allocated to that VMware ESX Server host should be listed
and should be associated with the hosts name and the vmhba
device numbers.
For example, refer to the LUNs 192, 193, 196, and 197 owned by
host l82bi199 in the figure below.

Manual Trespass on CLARiiON Arrays to Recover the Original Path

4-21

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4
5. 1.To view the Properties of the LUNs, right click on the LUN. A
window will provide the specific properties of the LUN.
For example, if you were to select LUN 196 that is allocated to the
host, then you would see the following:

As can be seen in the General tab above, LUN 196 is originally

owned by SP A. In this instance, LUN 196 has been trespassed to
SP B.
6. For the host in this example, both LUN 192 and 196 are owned by
SP A while LUNs 193 and 197 are owned by SP B. A trespass has
occurred and currently, all four LUNs are located on SP B.

4-22

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

In order to trespass LUNs 192 and 196 back to SP A, right click on

those LUNs. In the menu that is presented (see figure below for
an example), select the option to Trespass.

7. With the appropriate LUNs are highlighted, select the option to

Trespass. You will be prompted to confirm that the LUNs should
be trespassed as can be seen in the example below.
Select Yes to continue and to trespass the LUNs.

Manual Trespass on CLARiiON Arrays to Recover the Original Path

4-23

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4-24

EMC Host Connectivity Guide for VMware ESX Server v2.x

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

8. After the trespass has been completed, only LUNs will be

reported on SP B as is demonstrated in the figure below.

Manual Trespass on CLARiiON Arrays to Recover the Original Path

4-25

VMware ESX Server v2.x and CLARiiON Storage Array Fibre Channel Environment

4-26

EMC Host Connectivity Guide for VMware ESX Server v2.x

Index

addressing modes 3-12

GSX 1-3

binding, persistent 2-30

boot disk 3-6
bootfromsan 3-7

help xii

C
CLARiiON 4-2
connectivity 4-2
Command Descriptor Blocks (CDB) 3-11
Common Serial Number 3-2
connectivity
CLARiiON 4-2
Symmetrix 3-2
customer support xii

D
device mapping 1-3

E
Enable Auto Negotiation (EAN) 3-2

F
FA settings, Linux default 3-2
Fibre Channel
CLARiiON 4-2
Symmetrix 3-2

I
Intel-based servers 1-3

L
LUN
spanning 2-34
trespass, manual 4-21

M
Management User Interface (MUI) 2-3
manual LUN trespass 4-21
mapping, raw device 1-3
Master Boot Record (MBR) 3-6

O
Opteron-based servers 1-3
options, VMware ESX Server 2-4

P
parameters
VMkernel 2-12
pass-through technology 1-3
path policies
Fixed 4-9

EMC Host Connectivity Guide for VMware ESX Server v2.x

i-1

Index

Most Recently Used (MRU) 4-9

pbind.pl script 2-31
pbindings file 2-31
persistent binding 2-30

Z
zoning
CLARiiON 4-7
multi-initiator 3-8
single initiator/single target 3-8

R
raw device mapping 1-3
Remote Console software 2-41

S
SCSI pass-through technology 1-3
SCSI-3 Controller Commands (SCC) 3-11
servers
Intel-based 1-3
Opteron 1-3
service xii
Startup Profile 2-4
Symmetrix 3-2
addressing 3-10
booting the VMware ESX Server 3-6
connectivity 3-2
Linux default FA settings 3-2
Volume Logix device 3-6

T
technical support xii

V
Virtual Machines
install 2-37
set up 2-37
VMFS 2-37
VMFS-2 disks 2-34
VMkernel parameters 2-12
VMware
ESX Server 1-3, 2-2
ESX Server options 2-4
GSX 1-3
Volume Logix 3-6
Volume Logix (VCM) 3-2

W
World Wide Port Name (WWPN 3-10

i-2

EMC Host Connectivity Guide for VMware ESX Server v2.x