Dell EqualLogic Configuration Guide
Dell EqualLogic Configuration Guide
Dell EqualLogic Configuration Guide
EqualLogic
Configuration
Guide
A guide to building an
iSCSI based SAN solution
with Dell EqualLogic
PS Series Arrays
January 2009
THIS WHITE PAPER IS FOR INFORMATI/ONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL
INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.
Dell and EqualLogic are trademarks of Dell, Inc.
Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their
products. Dell disclaims proprietary interest in the marks and names of others.
2009 Dell Inc. All rights reserved. Reproduction in any manner whatsoever without the express
written permission of Dell, Inc. is strictly forbidden. For more information, contact Dell.
Page i
Table of Contents
Table of Contents..............................................................................................................ii
Introduction ..................................................................................................................... 1
EqualLogic Infrastructure Requirements .......................................................................... 1
Network Configuration and Design Considerations ......................................................... 1
Infrastructure Configuration ............................................................................................ 4
Array to Switch Infrastructure ...................................................................................... 4
Single Switch/Single Array Controller ....................................................................... 4
Single Switch/Dual Array Controller ......................................................................... 5
Dual Switch/Single Array Controller ......................................................................... 5
Dual Switch/Dual Array Controller ........................................................................... 6
Host to Switch Infrastructure ....................................................................................... 7
Single Initiator per Server......................................................................................... 7
Multiple Initiators per Host/Single Switch ................................................................ 7
Multiple Initiators per Host/Dual Switch .................................................................. 8
Switch to Switch Connections ...................................................................................... 9
Stackable Switches ................................................................................................... 9
Non-Stackable Switches ......................................................................................... 10
Putting It All Together ................................................................................................ 11
Fully Redundant SAN .............................................................................................. 12
Partially Redundant SAN Configurations ................................................................ 13
Appendix A: General Requirements for Switches When Used with EqualLogic PS Series
Arrays ............................................................................................................................ 16
Appendix B: PowerConnect 54xx Configuration ............................................................ 18
Disabling iSCSI Optimization Setting .......................................................................... 19
Page ii
Page iii
Introduction
The Dell EqualLogic Configuration Guide is provided as an aid to help storage
administrators determine how best to build an iSCSI infrastructure for use within an
EqualLogic SAN solution. This document will focus on configuration best practices,
connection rules, general switch configuration parameters, and other helpful
information. This document should not be considered a statement of support for any
specific configuration. Actual viability of any configuration will depend on the
capabilities of the individual components (switches, initiators, etc.) that make up the
SAN infrastructure. This document should be used strictly as a guide in planning an
EqualLogic SAN solution.
EqualLogic Infrastructure Requirements
Dell recommends Cisco and PowerConnect 5400 and 6200- series switches for use with
EqualLogic arrays. Configuration guidance for these switches is provided in the
Appendix of this document.
Dell will support any switch infrastructure component within an EqualLogic SAN solution
assuming it meets minimum standards (Defined in Appendix A) required to support
high-performance iSCSI traffic. Dell will provide full support to resolve customer issues
within the SAN solution. If an infrastructure component is identified as causing an issue,
the customer may be required to directly contact that component vendor for further
support.
Network Configuration and Design Considerations
Each control module has three network interface ports, labeled eth0, eth1, and eth2. A
dual control module array provides three pairs of network interfaces. For example, eth0
on Control Module 0 (CM0) and eth0 on Control Module 1(CM1) is a pair. Only one port
in a pair is active for I/O at one time. All ports are considered active on a single
controller at a time.
Page 1
Description
An array must have at least one functioning network interface connected to a
network (through a network switch, if possible). When you run the setup utility,
you will assign an IP address and subnet mask to this interface.
Connectivity to group IP
address
Each array must have at least one functioning network interface that is on the
same subnet as the group IP address.
Switch connectivity
Flow Control enabled on Enable Flow Control on each switch port that handles iSCSI traffic. If your server
is using a software iSCSI initiator and NIC combination to handle iSCSI traffic, you
switches and NICs
must also enable Flow Control on the NICs to obtain any performance benefit. PS
Series storage arrays will correctly respond to Flow Control.
Page 2
Recommendation
Redundant network
Paths
Description
Using a multi-pathing solution helps to ensure that no single point of failure exists
between hosts and arrays. MPIO implementations should be available for most
modern operating system environments.
For replication, a
reliable, adequately
sized network link
For effective and predictable replication, be sure that the network link between
the primary and secondary groups is reliable and provides sufficient bandwidth for
copying data.
No STP functionality on
switch ports that
connect end nodes
Do not use Spanning-Tree (STP) on switch ports that connect end nodes (iSCSI
initiators or storage array network interfaces). However, if you want to use STP or
Rapid STP (preferable to STP), you should enable the port settings available on
some switches that let the port immediately transition into STP forwarding state
upon link up. This functionality can reduce network interruptions that occur when
devices restart, and should only be enabled on switch ports that connect end
nodes.
Note: The use of Spanning-Tree when using multiple independent single-cable connection
between switches is encouraged.
Note: Using native Link Aggregation trunking functionality between non-stacking
switches rather than multiple, independent, single-cable connections is highly
encouraged.
Enable Jumbo Frames on each switch that handles iSCSI traffic. If your server is
using a software iSCSI initiator and NIC combination, you must also enable Jumbo
Frames on the NICs that handle iSCSI traffic to obtain any performance benefit
and ensure consistent behavior.
VLANs
Configure switches to use VLANs in order to separate iSCSI SAN traffic from other
network traffic.
Switched Gigabit
Ethernet network
Connect arrays and hosts to a switched network and ensure that all network
connections between hosts and arrays are Gigabit Ethernet. An array can operate
at 10 and 100 Mbps, but performance will be significantly degraded.
Multiple network
Connections
Disabled unicast storm control on on each switch that handles iSCSI traffic.
Page 3
Infrastructure Configuration
Dell recommends that all EqualLogic SAN solutions be configured for full redundancy.
The following table identifies all opportunities for redundancy:
Component
Hosts
SAN Switches
EqualLogic Array
Redundant Configuration
Two or more network interface ports attached to the SAN. Each
port should be connected to a different switch within the SAN.
MPIO software solution enabled for redundancy
Two or more switches configured for inter-switch communications
with standard ports for uplink or stacked using proprietary stacking
technology.
Each array should have at least one port from each controller
connected to each switch.
Single controller arrays can connect to one or more switches within the SAN
infrastructure. For the single array controller/single switch solution, one, two or more
array controller ports should be connected to the switch as illustrated in Figure 1.
These connections must not be aggregated in any way.
Note: This configuration does not provide a fully redundant storage infrastructure from
host to array. Additional components are required for full redundancy.
Page 4
CM0
Stackable Switch 0
EqualLogic
For dual controller arrays connecting to a single Ethernet switch, one, two or more ports
from controller CM0 should be connected to the switch. In addition, at least one port
from array controller CM1 should also be connected to the switch. All three ports from
CM1 can be connected to the switch to ensure that there is no performance decrease if
there is a failure in CM0. This will provide a redundant path to the array should an array
controller fail as shown in Figure 2.
Stackable Switch 0
CM0
CM1
Note: This configuration does not provide a fully redundant storage infrastructure from
host to array. Additional components are required for full redundancy.
EqualLogic
Page 5
switches must be connected together via stacking connections or uplinking ports that
are aggregated together to provide equivalent bandwidth based on the active CM0
ports connected to the SAN.
Note: This configuration does not provide a fully redundant storage infrastructure from
host to array. Additional components are required for full redundancy.
Stackable Switch 0
CM0
Stacking
Connection
Stackable Switch 1
EqualLogic
Stackable Switch 0
CM0
Stacking
Connection
Stackable Switch 1
CM1
Dual controller arrays should be configured such that two ports on Controller 0 attach to
one switch and the third port connects to a second switch. Two ports on Controller 1
should connect to the second switch and the third port should connect to the first
switch. The complete diagram in Figure 4 illustrates this connection requirement.
EqualLogic
Page 6
iSCSI
Server
Note: This configuration does not provide a fully redundant storage infrastructure from
host to array. Additional components are required for full redundancy.
NIC 0
NIC 1
Switch 0
Corporate
LAN
Redundant host solutions will incorporate two or more NICs or HBAs connected to one
or more SAN switches. For hosts connecting to a single switch SAN, each NIC or HBA
should be connected to the SAN switch as illustrated in Figure 6. Each NIC should have
Dell EqualLogic Configuration Guide
Page 7
Server
iSCSI
MPIO
the iSCSI software initiator enabled and have either the Dell EqualLogic MPIO driver
(located in the Host Integration Toolkit for Windows) or the native MPIO functionality
for the host OS installed. HBA connections should leverage the standard MPIO
functionality of the driver provided by the vendor to support failover and/or loadbalancing features.
NIC 2
NIC 3
NIC 1
NIC 0
Switch 0
Corporate
LAN
For a redundant host connection to more than two switches, two or more NICs or HBAs
will be configured such that one NIC/HBA is connected to one switch within the SAN
infrastructure and the other NIC/HBA is connected to a second switch within the SAN
infrastructure as show in Figure 7.
NOTE: See the Section discussing switch to switch connections elsewhere in this
document for details on configuring a redundant SAN switch infrastructure.
Page 8
iSCSI
NIC 2
NIC 3
NIC 1
NIC 0
Stacking
Connection
Server
MPIO
Stackable Switch 1
Stackable Switch 0
Corporate
LAN
Page 9
Stackable Switch 0
Stackable Switch 1
Stackable Switch 2
Stacking Port
Stacking Cable
Non-Stacking switches are a viable solution only for small EqualLogic SANs of 3 or fewer
arrays. Non-stacking switches should be connected by using standard Ethernet ports
configured into a Link Aggregation Group (LAG). Depending on the vendor, this will
either be a LACP-compliant link group or an EtherChannel link group. If the switches
are of different vendors, LACP will be the standards-based protocol to be used. Please
follow the directions provided by the switch vendor to implement the appropriate LAG
type.
All LAGs should consist of an equivalent number of links to the total number of active
ports on all arrays within the SAN. Each array can have up to 3x 1Gbps active ports, so
LAG should be configured for 3x 1Gbps links per array. Figure 9 illustrates a nonstackable switch infrastructure with 4 ports on each switch configured into a LAG.
Page 10
Non-Stacking Switch 1
Non-Stacking Switch 0
Ethernet Port
Ethernet Cable
LACP Aggregation/EtherChannel
Page 11
The only design that addresses all of the possible failure modes for an iSCSI SAN that utilizes an
EqualLogic array is illustrated in Figure 10. In this SAN design, each component of the SAN
infrastructure has a redundant connection or path.
NIC 3
Switch 0
CM0
iSCSI
NIC 2
NIC 0
Stack
or
Uplink
Server
MPIO
Switch 1
NIC 1
CM1
NOTE: For any production environment, this is the ONLY configuration that will completely
protect your access to data and is the ONLY configuration recommended by Dell.
EqualLogic
Corporate
LAN
Page 12
Though each of the SAN designs below will allow each host to access its data within the SAN,
these are not recommended for production deployment, and should not be interpreted as a
recommendation, as they do not provide a fully redundant SAN solution.
NIC 0
NIC 1
Switch 0
CM0
iSCSI
Server
EqualLogic
Corporate
LAN
iSCSI
NIC 2
NIC 3
NIC 1
NIC 0
Switch 0
CM0
Server
MPIO
EqualLogic
Corporate
LAN
Page 13
NIC 3
NIC 0
Switch 0
CM0
iSCSI
NIC 2
Stack
or
Uplink
Server
MPIO
Switch 1
NIC 1
EqualLogic
Corporate
LAN
NIC 0
NIC 1
Switch 0
CM0
iSCSI
Server
CM1
EqualLogic
Corporate
LAN
Page 14
NIC 0
NIC 1
Switch 0
CM1
CM0
Stack
or
Uplink
iSCSI
MPIO
Server
Switch 1
EqualLogic
Corporate
LAN
NIC 3
CM1
iSCSI
NIC 2
NIC 1
NIC 0
Switch 0
CM0
Server
MPIO
EqualLogic
Corporate
LAN
Page 15
Page 16
For SAN infrastructures consisting of more than 2 non-stacking switches, RSTP must be enabled on all ports used for ISLs. All non-ISL ports should be
marked as edge ports or set to portfast.
Support for Jumbo Frames
Not a requirement, but desirable. Many storage implementations can take
advantage of Jumbo Frames. Jumbo frames may not provide any
performance increases depending on the application and data
characteristics.
Ability to disable Unicast Storm Control
iSCSI in general, and Dell EqualLogic SANs in particular can send packets in a
very bursty profile that many switches mis-diagnose as a viral induced
packet storm. Since the SAN should be isolated from general Ethernet
traffic, the viral possibilities are non-existent. Switches need to always pass
Ethernet packets regardless of bandwidth utilization.
Adequate Buffer Space per switch port
The Dell EqualLogic SAN solution makes use of the SAN infrastructure to
support inter-array communication and data load balancing on top of
supporting data transfers between the hosts and the SAN. For this reason,
the more buffer space per port that a switch can provide the better.
Due to the multitude of buffer implementations used by switch vendors,
Dell cannot provide definitive guidelines as to how much is enough, but this
should not be an issue for most Enterprise-class switch vendors solutions.
Page 17
Note: You may be prompted for a password after submitting the enable
command
The PowerConnect 54xx Family of switches must be place into configuration
mode before any configuration steps can be performed. To enter configuration
mode, the following command must be entered:
console# configure
console(config)#
Page 18
Page 19
interface Ethernet g1
Page 20
console(config-if)# exit
console(config)#
exit
The following steps are an example of how to disable storm control on all ports of
a PowerConnect 5448 switch:
console# configure
console(config)# interface range ethernet all
console(config-if)# no port storm-control broadcast enable
console(config-if)# exit
console(config)# exit
console# copy running-config startup-config
console# exit
Page 21
Note: You may be prompted for a password after submitting the enable
command
The PowerConnect 62xx Family of switches must be place into configuration
mode before any configuration steps can be performed. To enter configuration
mode, the following command must be entered:
console# configure
console(config)#
Page 22
Page 23
console(config)# flowcontrol
console(config)# exit
console# copy running-config startup-config
console# exit
The following steps are an example of how to disable unicast storm control on all
ports in a PowerConnect 62xx switch stack:
console# configure
console(config)# interface range ethernet all
console(config-if)# no storm-control unicast
console(config-if)# exit
console(config)# exit
console# copy running-config startup-config
console# exit
Page 24
Page 25
Cisco IOS based switches must be place into configuration mode before any
configuration steps can be performed. To enter configuration mode, the
following command must be entered:
Switch# configure terminal
To configure a port on non-chassis based Cisco IOS switch, the interface type,
stack member number, module number, and switch port number must be
provided, and enter interface configuration mode.
Stack member numberIdentifies the switch within the stack. The switch
number range is 1 to 9 and is assigned the first time the switch initializes. All
standalone switches have stack member number equal to 1. When a switch is
added to an existing stack it will receive a new stack member number and it
keeps that number until another is assigned to it. Non-Stackable switches
have a stack member number of 1.
The switch port LEDs can be configured in Stack mode to identify the stack
member number of a switch.
Port numberReflects the actual port number on the switch. Port numbers
always begin at 1, starting with the far left port when facing the front of the
switch.
Page 26
To view or confirm Port Fast status on a port, use the following command.
Dell EqualLogic Configuration Guide
Page 27
Page 28
Switch(config)# exit
Switch# copy running-config startup-config
To view or confirm Flow Control status on a port, use the following command:
Switch# show flowcontrol interface gigabitethernet1/0/1
To view or confirm storm control status on a port, use the following command.
Switch# show storm-control gigabitethernet1/0/15 unicast
Page 29
The following example shows the output when you try to set Gigabit Ethernet
interfaces to an outof-range number:
Switch(config)# system mtu jumbo 25000
^
% Invalid input detected at '^' marker.
Once the switch reloads, the configuration can be verified by entering the
following commands:
Switch> enable
Switch# configure t
Switch(config)# show system mtu
Switch(config)# exit
Page 30
The commands shown next are used to enable an individual VLAN to use Jumbo
Frames. Note that VLAN1 cannot have Jumbo Frames enabled. VLAN 2 must be
used if Jumbo Frames are required.
Switch# vlan database
Switch(vlan)# vlan 2 mtu 9000
Switch(vlan)# exit
To view or confirm MTU size on port 7 of the switch, use the following command.
Switch# show interface gigabitethernet1/0/7
Page 31