Symmetrix SRDF Product Guide

EMC® Symmetrix® Remote
Data Facility (SRDF)
PRODUCT GUIDE
P/N 300-001-165
REV A05
EMC Corporation
Corporate Headquarters:
Hopkinton, MA 01748-9103
1-508-435-1000
www.EMC.com
Copyright © 2001-2007 EMC Corporation. All rights reserved.
Published February, 2007
EMC believes the information in this publication is accurate as of its publication date. The information is
subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO
REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS
PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR
FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.
All other trademarks used herein are the property of their respective owners.
2 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

Contents
Preface............................................................................................................................ 11
Chapter 1 Introducing SRDF

SRDF product overview................................................................... 16
Base SRDF family products...................................................... 16
SRDF family options ................................................................. 17
SRDF/Synchronous (SRDF/S) ................................................ 18
SRDF/Asynchronous................................................................ 18
SRDF/Data Mobility (SRDF/DM) .......................................... 19
SRDF/Automated Replication (SRDF/AR) .......................... 19
SRDF/Star................................................................................... 20
SRDF/Consistency Groups (SRDF/CG)................................ 20
SRDF/Cluster Enabler for MSCS or VCS............................... 21
SRDF functional overview............................................................... 22
Basic SRDF configuration ......................................................... 22
SRDF interfamily connectivity................................................. 23
SRDF supported data protection options............................... 24
Monitoring and controlling SRDF........................................... 24
SRDF director hardware .................................................................. 25
SRDF director functions............................................................ 25
SRDF configurations......................................................................... 27
SRDF configuration using switched Fibre Channel ............. 30
SRDF fully switched fabric connectivity ................................ 30
Switched and Concurrent SRDF.............................................. 32
SRDF with native GigE ............................................................. 33
SRDF and FarPoint .................................................................... 34
Chapter 2 SRDF Technical Concepts

SRDF volume types .......................................................................... 38
Primary (source) volumes ........................................................ 38
EMC Symmetrix Remote Data Facility (SRDF) Product Guide 3
Contents
Secondary (target) volumes ..................................................... 39

Local volumes ............................................................................ 39
Dynamic SRDF devices............................................................. 39
EMC Compatible Peer .............................................................. 41
SRDF groups...................................................................................... 42
Dynamic SRDF groups ............................................................. 43
SRDF volumes: Special considerations.......................................... 44
Dynamic Sparing with remotely mirrored pairs (SRDF)..... 44
Open systems metavolumes .................................................... 44
Symmetrix RAID 10 (mirrored striped mainframe
volumes) ..................................................................................... 45
PPRC command support.......................................................... 46
SRDF link configurations................................................................. 47
SRDF unidirectional link configuration ................................. 47
SRDF bidirectional link configuration.................................... 47
SRDF dual-directional link configuration.............................. 47
SRDF link and volume states .......................................................... 48
SRDF link states ......................................................................... 48
Configuration settings affecting device ready and
link states .................................................................................... 48
Logical volume states................................................................ 49
SRDF volume states (Symmetrix view).................................. 50
Host accessibility ....................................................................... 51
Primary modes of operation ........................................................... 53
Synchronous mode.................................................................... 53
Semi-synchronous mode .......................................................... 54
Secondary modes of operation ....................................................... 56
Adaptive copy modes ............................................................... 56
Additional SRDF modes and attributes ........................................ 58
Domino modes........................................................................... 58
Invalid tracks attribute ............................................................. 59
SRDF system-level attributes................................................... 59
Concurrent SRDF .............................................................................. 60
SRDF/Consistency Groups (SRDF/CG)....................................... 62
How a consistency group works ............................................. 62
Continuous processing ............................................................. 64
Technical considerations........................................................... 65
Chapter 3 SRDF Operations

Write operations................................................................................ 68
Write operations in a unidirectional or dual-directional
configuration .............................................................................. 68
Write operations in an ESCON bidirectional

Contents
configuration .............................................................................. 68
Read operations................................................................................. 69
Primary volume read operations............................................. 69
Secondary volume read operations......................................... 69
Recovery operations ......................................................................... 71
Failover to the secondary Symmetrix system ........................ 71
Failback to the primary Symmetrix system ........................... 71
Recovery for a large number of invalid tracks ...................... 72
Business continuance using SRDF.................................................. 73
Business continuance using SRDF and TimeFinder..................... 75
Using TimeFinder/Mirror BCVs with primary devices....... 75
Using a BCV as a primary (source) device ............................. 75
Using TimeFinder BCVs with secondary devices ................. 76
Using a BCV as a secondary (target) device........................... 76
SRDF remote command support ............................................. 77
SRDF Multi-Hop ........................................................................ 77
R1/R2 swap ....................................................................................... 79
R1/R2 swap procedure history................................................ 79
Dynamic R1/R2 swap ...................................................................... 80
Migrating data from R1 to a larger R2 device............................... 81
Chapter 4 SRDF/Asynchronous Operations

SRDF/A overview ............................................................................ 84
SRDF/A benefits ............................................................................... 85
Requirements and limitations ......................................................... 86
SRDF/A history................................................................................. 87
Enginuity 5670 SRDF/A single session .................................. 87
Enginuity 5670.50 SRDF/A Multi-Session
Consistency, MSC....................................................................... 87
Enginuity 5671 Multiple SRDF/A Single Session SRDF
groups per Symmetrix............................................................... 87
Enginuity 5671 SRDF/A Multi-Session Consistency, MSC . 87
Enginuity 5671 Concurrent SRDF support............................. 88
Enginuity 5671 Dynamic SRDF support................................. 88
Enginuity 5671 Tunable Cache utilization.............................. 89
SRDF/A Reserve Capacity ....................................................... 89
Tolerance mode.................................................................................. 90
Locality of reference.......................................................................... 91
SRDF/A single session mode.......................................................... 92
SRDF/A single session mode dependent-write consistency...... 93
SRDF/A single session mode states............................................... 95
Not Ready (NR) state (system startup)................................... 95

Contents
Inactive state............................................................................... 95
Active state ................................................................................. 96
SRDF/A single session mode delta set switching ....................... 97
SRDF/A single session mode state transitions .......................... 102
Switching to SRDF/A mode .................................................. 102
Transition from synchronous to asynchronous................... 102
Transition from adaptive copy write-pending mode to
asynchronous ........................................................................... 103
Transition from adaptive copy disk mode to SRDF/A...... 103
Switching to SRDF/S mode from SRDF/A single session
mode.......................................................................................... 104
Coming out of the SRDF/A active state .............................. 104
Dropping SRDF/A single session mode.............................. 105
Pend-dropping SRDF/A single session mode .................... 105
Deactivating SRDF/A single session mode......................... 106
SRDF/A single session cleanup process ..................................... 107
SRDF/A single session mode recovery scenarios...................... 108
Temporary link loss................................................................. 108
Permanent link loss ................................................................. 108
Primary Symmetrix global memory full condition ............ 109
Failback from secondary symmetrix devices ...................... 110
SRDF/A multi-session consistency (MSC) mode ....................... 111
SRDF/A MSC mode dependent-write consistency................... 112
Entering SRDF/A multi-session consistency ...................... 113
Performing a SRDF/A MSC consistent cycle switch ......... 114
SRDF/A MSC mode delta set switching..................................... 116
SRDF/A MSC session cleanup process ....................................... 122
Chapter 5 SRDF/Star Operations

SRDF/Star overview ...................................................................... 126
SRDF/Star benefits.................................................................. 128
Known requirements and limitations at this release.......... 128
How SRDF/Star works.................................................................. 129
SRDF/Star control for mainframe......................................... 130
SRDF/Star control for Open Systems................................... 130
SRDF/Star automation for mainframe................................. 131
SRDF/Star automation for open systems ............................ 132
Index .............................................................................................................................. 133

Figures
Title Page
1 Basic SRDF configuration............................................................................... 23

2 Two production sites and one recovery site................................................ 27
3 Data vaulting solution .................................................................................... 28
4 Sites containing both primary and secondary volumes ............................ 29
5 Switched SRDF with multiple primary and secondary devices............... 30
6 Switched SRDF over Fibre Channel with Enginuity 5x67......................... 31
7 Switched and Concurrent SRDF configuration example........................... 32
8 Switched GigE network configuration example......................................... 34
9 SRDF with and without FarPoint.................................................................. 35
10 Dynamic SRDF................................................................................................. 40
11 PPRC and GDPS support ............................................................................... 46
12 SRDF logical volume state.............................................................................. 49
13 Synchronous mode.......................................................................................... 54
14 Semi-synchronous mode ................................................................................ 55
15 Concurrent SRDF configuration.................................................................... 60
16 Primary and Secondary relationships .......................................................... 62
17 Failed link between Primary 2 and Target 1................................................ 63
18 Primaries 1, 2, and 3 in a consistency group ............................................... 64
19 Failed link between Primary 2 and Target 1................................................ 64
20 SRDF business continuance ........................................................................... 73
21 Primary-to-secondary resynchronization .................................................... 74
22 Secondary-to-primary resynchronization.................................................... 74
23 SRDF single-hop configuration (BCV functioning as a primary
SRDF device) .................................................................................................... 76
24 SRDF multi-hop configuration ...................................................................... 78
25 R1/R2 swap concept ....................................................................................... 79
26 Synchronous and asynchronous block transfer comparison .................... 91
27 SRDF/A delta sets and their relationships .................................................. 93
28 SRDF/A single session allowed state transitions ....................................... 95

Figures
29 Single session capture delta set collects application write I/O................ 97

30 SRDF/A single session transmit delta set empties .................................... 98
31 SRDF/A single session SRDF transfer is halted prior to Primary Symme-
trix cycle switch ............................................................................................... 98
32 SRDF/A single session Primary Symmetrix delta set switch................... 98
33 SRDF/A single session new capture delta available for host I/O........... 99
34 SRDF/A single session secondary Symmetrix wait for apply
delta set to be restored.................................................................................... 99
35 SRDF/A single session secondary Symmetrix delta set switch............. 100
36 SRDF/A single session secondary Symmetrix new receive
delta set is available for SRDF ..................................................................... 100
37 SRDF/A single session secondary Symmetrix begins restore
of apply delta set ........................................................................................... 101
38 SRDF/A single session primary Symmetrix begins SRDF transfer....... 101
39 SRDF/A single session transition path...................................................... 102
40 SRDF/A MSC delta sets and their relationships...................................... 112
41 SRDF/A MSC allowed state transitions .................................................... 114
42 SRDF/A MSC capture delta set collects application write I/O ............. 116
43 SRDF/A MSC Primary Symmetrix transmit delta set switch
is emptied ....................................................................................................... 117
44 SRDF/A MSC Primary Symmetrix halts the SRDF transfer .................. 117
45 SRDF/A MSC Secondary apply delta set restore complete ................... 118
46 SRDF/A MSC Primary Symmetrix cycle switch while
I/O is deferred............................................................................................... 119
47 SRDF/A MSC I/O is released and a new capture delta
set continue to accept Host I/O .................................................................. 119
48 SRDF/A MSC Secondary Symmetrix cycle switch.................................. 120
49 SRDF/A MSC Secondary new receive delta set is available .................. 120
50 SRDF/A MSC Primary Symmetrix systems begin SRDF transfer......... 121
51 SRDF/A MSC Secondary Symmetrix begins the apply
delta set restore process ............................................................................... 121
52 Concurrent SRDF configured for SRDF/Star support ............................ 127
53 SRDF/Star configuration reference............................................................ 131

Tables
Title Page
1 Symmetrix model numbers ........................................................................... 17

2 Data Protection options for SRDF disk devices ......................................... 24
3 Primary (R1) volume accessibility ................................................................ 52
4 Secondary (R2) volume accessibility ............................................................ 52

Tables

Preface
As part of its effort to continuously improve and enhance the performance

and capabilities of the EMC product line, EMC periodically releases new
versions of both the EMC Enginuity Operating Environment and
Symmetrix Remote Data Facility (SRDF). Therefore, some functions
described in this guide may not be supported by all versions of Enginuity
currently in use. For the most up-to-date information on product features,
refer to your product release notes. If an SRDF feature does not function
properly or does not function as described in this guide, please contact the
EMC Customer Support Center for assistance.
Audience This guide provides an overview of SRDF and SRDF family of

product offerings for users requiring a basic knowledge of SRDF
concepts and operation. Detailed SRDF installation and configuration
information is available in the documents listed in Related
documentation on page 11.
Related For additional information on SRDF and all Symmetrix

documentation systems-related publications, contact your EMC Sales Representative
or refer to the EMC Powerlink website at:
http://powerlink.EMC.com
For information on configuring SRDF, refer to:
◆ EMC Support Matrix at
http://www.EMC.com/interoperabilityindex.jsp
◆ EMC Networked Storage Topology Guide at
http://Powerlink.EMC.com
Follow these links: Support, Document Library, Solutions
Guides

Preface
◆ Symmetrix Remote Data Facility (SRDF) Product Guide at

◆ EMC SRDF Connectivity Guide at
Follow these links: Support, Document Library, Software, SRDF
◆ Symmetrix (Model or Series) Product Guide at
Follow these links: Support, Document Library, Symmetrix
For detailed information regarding solutions that involve other
qualified vendors' equipment, consult the appropriate product
specification manuals available from each vendor, or ask your site
representative to contact the appropriate EMC partner vendor or
EMC Technical Support/Engineering.
Conventions used in EMC uses the following conventions for notes and caution notices.
this document
Note: A note presents information that is important, but not hazard-related.
! CAUTION
A caution contains information essential to avoid data loss or
damage to the system or equipment. The caution may apply to
hardware or software.
! IMPORTANT
An important notice contains information essential to operation of
the software. The important notice applies only to software.

Preface
Typographical conventions
EMC uses the following type style conventions in this guide:
Normal font In running text:
• Interface elements (for example, button names, dialog box
names) outside of procedures
• Items that user selects outside of procedures
• Java classes and interface names
• Names of resources, attributes, pools, Boolean expressions,
buttons, DQL statements, keywords, clauses, environment
variables, filenames, functions, menu names, utilities
• Pathnames, URLs, filenames, directory names, computer
names, links, groups, service keys, file systems, environment
variables (for example, command line and text), notifications
Bold In procedures:
• Names of dialog boxes, buttons, icons, menus, fields
• Selections from the user interface, including menu items and
field entries
• Key names
• Window names
In running text:
• Command names, daemons, options, programs, processes,
notifications, system calls, man pages, services, applications,
utilities, kernels
Italic Used for:
• Full publications titles referenced in text
• Unique word usage in text
Bold Italic Anything requiring extra emphasis
Courier Used for:
• System output
• Filenames
• Complete paths
• Command-line entries
• URLs
Courier, bold Used for:
• User entry
• Options in command-line syntax
Courier, italic Used for:
• Arguments used in examples of command-line syntax
• Variables in examples of screen or file output
• Variables in path names
Courier, bold, Variables used in a command-line sample
italic

Preface
<> Angle brackets enclose parameter or variable values supplied by

the user
Where to get help EMC support, product, and licensing information can be obtained as
follows.
Product information — For documentation, release notes, software
updates, or for information about EMC products, licensing, and
service, go to the EMC Powerlink website (registration required) at:
Technical support — For technical support, go to EMC WebSupport

on Powerlink. To open a case on EMC WebSupport, you must be a
WebSupport customer. Information about your site configuration and
the circumstances under which the problem occurred is required.
Your Comments Your suggestions will help us continue to improve the accuracy,
organization, and overall quality of the user publications. Please send
your opinion of this guide to:
techpub_comments@EMC.com

Invisible Body Tag
1
Introducing SRDF
This chapter introduces SRDF and explains basic concepts including

SRDF volume types, director hardware, and example configurations.
This chapter contains the following topics:
◆ SRDF product overview.................................................................... 16
◆ SRDF functional overview................................................................ 22
◆ SRDF director hardware ................................................................... 25
◆ SRDF configurations.......................................................................... 27
Introducing SRDF 15
Introducing SRDF
SRDF product overview

The EMC® Symmetrix® Remote Data Facility (SRDF®) family of
replication software offers various levels of Symmetrix based
business continuance and disaster recovery solutions. The SRDF
products offer the capability to maintain multiple, host-independent,
mirrored copies of data. The Symmetrix systems can be in the same
room, in different buildings within the same campus, or hundreds to
thousands of kilometers apart.
By maintaining copies of data in different physical locations, SRDF
enables you to perform the following operations with minimal
impact on normal business processing:
◆ Disaster restart
◆ Disaster restart testing
◆ Recovery from planned outages
◆ Remote backup
◆ Data center migration
◆ Data replication and mobility
Base SRDF family The SRDF family consists of three base solutions:
products ◆ SRDF/Synchronous (SRDF/S, formerly SRDF) —
High-performance, host-independent, real-time synchronous
remote replication from one Symmetrix to one or more
Symmetrix systems.
◆ SRDF/Asynchronous (SRDF/A ) — High-performance extended
distance asynchronous replication using a Delta Set architecture
for optimal bandwidth utilization and minimal host performance
impact.
◆ SRDF/Data Mobility (SRDF/DM ) — Rapid transfer of data from
source volumes to remote volumes anywhere in the world,
permitting information to be shared and content to be distributed,
or information consolidated for parallel processing activities.

Introducing SRDF
SRDF family options There are a number of additional options and features that can be
added to the base solutions to solve specific service level
requirements. These options include:
◆ SRDF/Automated Replication (SRDF/AR) solutions for meeting
very specific, remote replication service-level requirements
◆ SRDF/Star for advanced multisite failover with continuous
protection
◆ SRDF/Consistency Groups (SRDF/CG) for data consistency
◆ SRDF/Cluster Enabler (SRDF/CE) for integration with
host-based clustering products such as Microsoft Cluster Server
(MSCS) and VERITAS Cluster Server (VCS).
Note: For simplicity, this document uses the term SRDF to represent all EMC
SRDF related products, including: SRDF/A, SRDF/AR (multi-hop and
single-hop), SRDF/DM, and SRFD/CG. Most of the document focuses on
SRDF/S, but generally it applies to any of the variations.
Note: The SRDF family of products can also be used with the EMC
TimeFinder® family of products, which includes TimeFinder/Mirror,
TimeFinder/Clone, and TimeFinder/Snap. For simplicity, this document
uses the term TimeFinder to represent all EMC TimeFinder related products.
This product guide refers to the Symmetrix models shown in Table 1

on page 17.
.
Table 1 Symmetrix model numbers
Cabinet description Symmetrix 3xxx/5xxx series Symmetrix 8000 series Symmetrix DMX series
1/2 Bay 3330/5330 and 3630/5630 8230 DMX800 (rackmount)
1 Bay 3430/5430, 3830/3832a /5830 8430, 8530 DMX1000
2 Bay DMX2000
3 Bay 3700/5700 and 3930/5930 8730, 8830 DMX3000
1 system bay and from DMX-3

2 to 8 storage bays
a. The Symmetrix 3832 is a 3830 model with a split backplane.
SRDF product overview 17

Introducing SRDF
SRDF/Synchronous Symmetrix Remote Data Facility/Synchronous (SRDF/S) is a

(SRDF/S) business continuance solution that maintains a real-time
(synchronous) copy of data at the logical volume level in Symmetrix 3
8xxx or Symmetrix DMX™ and Symmetrix DMX-3 systems in the
same or separate locations.
SRDF/S offers the following major features and benefits:
◆ High data availability
◆ High performance
◆ Flexible configurations
◆ Host and applications software transparency
◆ Automatic recovery from a component or link failure
◆ Significantly reduced recovery time after a disaster
◆ Reduced backup and recovery costs
◆ Reduced disaster recovery complexity, planning, and testing
The SRDF/S operation is transparent to the host operating system
and host applications. It does not require additional host software for
duplicating data on the participating Symmetrix units.
SRDF/S offers greater flexibility through additional modes of
operation, specifically:
◆ Semi-synchronous mode
◆ Adaptive copy write-pending mode
◆ Adaptive copy disk mode
Note: “Primary modes of operation” on page 53 and “Secondary modes of

operation” on page 56 provide more information on these SRDF/S modes of
operation.
SRDF/Asynchronous Beginning with Enginuity™ level 5670, the Symmetrix DMX system
supports the asynchronous replication product named
SRDF/Asynchronous (SRDF/A). SRDF/A is another mode of remote
replication that allows customers to asynchronously replicate data
while maintaining a dependent write consistent copy of the data on
the secondary (R2) device at all times. The dependant write
consistent point-in time copy of the data at the remote side is
typically only seconds behind the primary (R1) side. SRDF/A session
data is transferred to the secondary Symmetrix system in predefined
timed cycles or delta sets, eliminating the redundancy of multiple
same track changes being transferred over the link, potentially
reducing the required bandwidth.

Introducing SRDF
SRDF/A provides a long-distance replication solution with minimal

impact on performance. This level of protection is intended for
customers who require minimal host application impact while
maintaining a dependant write consistent, restartable image of their
data at the secondary site. In the event of a disaster at the R1 site or if
SRDF links are lost during data transfer, a partial delta set of data can
be discarded, preserving dependant write consistency on the
secondary site with a data loss of no more than two SRDF/A cycles.
Note: Chapter 4, ”SRDF/Asynchronous Operations,” provides more detailed

information on SRDF/A.
SRDF/Data Mobility SRDF/Data Mobility (SRDF/DM) is an SRDF product offering that

(SRDF/DM) permits operation in SRDF adaptive copy mode only and is designed
for data replication and/or migration between two or more
Symmetrix systems. SRDF/DM transfers data from primary volumes
to secondary volumes permitting information to be shared, content to
be distributed, and access to be local to additional processing
environments. Adaptive copy mode enables applications using that
volume to avoid propagation delays while data is transferred to the
remote site. SRDF/DM supports all Symmetrix systems and all
Enginuity levels that support SRDF, and can be used for local or
remote transfers.
Note: “Adaptive copy modes” on page 56 provide more information on

adaptive copy mode operation.
SRDF/Automated SRDF/Automated Replication (SRDF/AR) is an automation solution

Replication that uses both SRDF and TimeFinder to provide a periodic
(SRDF/AR) asynchronous replication of a restartable data image. Use a
single-hop SRDF/AR configuration that permits controlled data loss
(depending on the cycle time). However, if greater protection is
required, a multi-hop SRDF/AR configuration can provide long
distance disaster restart with zero data loss at a middle or "bunker"
site.
Note: “Business continuance using SRDF and TimeFinder” on page 75

provides more information on single-hop and multi-hop configurations.
Compared to traditional disaster recovery solutions with their long

recovery time and high data loss, disaster restart solutions using

Introducing SRDF
SRDF/AR provide remote restart with a short restart time and low
data loss.
SRDF/AR offers data protection with dependent write consistency
across a distance. This protection is accomplished by using
geographically separated replicas with hardware and software
products from EMC Corporation. The SRDF/AR process can be
implemented with TimeFinder/Mirror in a mainframe z/OS
environment, or through UNIX and Windows environments using
Solutions Enabler’s symreplicate command line interface.
SRDF/Star Available at Enginuity level 5x71, SRDF/Star provides advanced

multisite business continuity protection available for mainframe and
open systems environments. It enables concurrent SRDF/S with
consistency groups and SRDF/A with MSC operations from the same
source volumes with the ability to incrementally establish an
SRDF/A session between the two remote sites in the event of a
primary site outage—a capability only available through SRDF/Star
software.
This capability takes the promise of concurrent synchronous and
asynchronous operations (from the same source device) to its logical
conclusion. SRDF/Star allows you to quickly re-establish protection
between the two remote sites in the event of a primary site failure,
and then just as quickly restore the primary site when conditions
permit.
With SRDF/Star, enterprises can quickly resynchronize the SRDF/S
and SRDF/A copies by replicating only the differences between the
sessions—allowing for much faster resumption of protected services
after a source site failure.
SRDF/Consistency EMC SRDF/Consistency Groups (SRDF/CG) is an SRDF product

Groups (SRDF/CG) offering designed to ensure the dependent write consistency of the
data remotely mirrored by the SRDF operations in the event of a
rolling disaster. Most applications, and in particular database
management systems (DBMSs), have dependent write logic imbedded
in them to ensure data integrity if a failure occurs in:
◆ The host processor
◆ The software
◆ The storage subsystem

Introducing SRDF
A dependent write is a write is not issued by an application until a

prior, related write I/O operation is completed. An example of
dependent write is a database update.
For example, when updating a database, a DBMS takes the following
steps:
1. The DBMS writes to the disk containing the transaction log.
2. The DBMS writes the data to the actual database dataset.
3. The DBMS writes again to the log volume to indicate that the
database update was made.
In a remote disk copy environment, data consistency cannot be
ensured if one of these writes was remotely mirrored, but its
predecessor was not remotely mirrored. This could occur, for
example, in a rolling disaster where a communication loss occurs that
affects only a subset of the disk controllers performing the remote
copy function.
SRDF/CG prevents a rolling disaster from affecting the integrity of
the data at the remote site. When SRDF/CG detects any write I/O to
a volume that cannot communicate with its remote mirror, SRDF/CG
suspends the remote mirroring for all volumes defined to the
consistency group before completing the intercepted I/O and
returning control to the application. In this way, SRDF/CG prevents
dependent I/O from reaching its remote mirror in the case where a
predecessor I/O only gets as far as the local mirror.
Note: “SRDF/Consistency Groups (SRDF/CG)” on page 62 provides more

information on consistency group operations.
SRDF/Cluster SRDF/Cluster Enabler for MSCS or VCS provides high availability

Enabler for MSCS or and automated failover through storage-based replication and server
VCS clustering through SRDF/S and MSCS or VCS. For more information
consult the Microsoft Cluster Server (MSCS) and VERITAS Cluster
Server (VCS) documentation.

Introducing SRDF
SRDF functional overview

Symmetrix systems use local mirroring, or RAID 1, as one method of
protecting data by maintaining data on both a production volume
and a mirror volume within the same storage unit. SRDF uses a
method of data protection known as remote mirroring. Remote
mirroring is similar to local mirroring, except that the production
volume resides in one storage unit while its remote mirrors (up to
two remote mirrors are supported with Concurrent SRDF) reside in a
different storage unit.
When the main storage systems are down for a planned or
unplanned event, SRDF enables fast switchover from the primary
(source) data to the secondary (target) copy. The local SRDF device,
known as the primary (source, R1) device, is configured in a pairing
relationship with a secondary (target, R2) device, forming an SRDF
pair. While the R2 device is mirrored with the R1 device, the R2
device is either write disabled or not ready to its host. (Not ready
means disabled for both reads and writes.) After the R2 device
becomes synchronized with its R1 device, you can split the R2 device
from the R1 device at any time, making the R2 device fully accessible
again to its host. After the split, the secondary (target, R2) device
contains valid data and is available for performing business
continuance tasks through its original device address or restoring
(copying) data to the primary (source, R1) device if there is a loss of
data on that device.
Note: For more information on SRDF operations, refer to Chapter 3, ”SRDF

Operations.”
Basic SRDF A basic SRDF configuration consisting of a production site and a

configuration recovery site is illustrated in Figure 1 on page 23. At the production
site, a local host connects to Symmetrix A. The device containing the
production data to be remotely mirrored is called the primary
(source) or R1 volume. At the recovery site, a second (optional) host
connects to Symmetrix B with the secondary (target) or R2 volume
containing the remotely mirrored data. The Symmetrix systems
communicate through SRDF links.
Note: Additional SRDF configurations are explained in “SRDF

configurations” on page 27.

Introducing SRDF
Site A Site B
(Production) (Recovery)
Local Remote
Host Host
Active Host Recovery

Path Path
Symmetrix Symmetrix
A B
SRDF Links
R1 R2
Local Primary Local Secondary

Volume (Source) Volume Volume (Target) Volume
Figure 1 Basic SRDF configuration
SRDF interfamily SRDF supports connectivity between Symmetrix models running

connectivity various levels of Enginuity. Enginuity level 5x67 is the first
Symmetrix family of Enginuity that supports connectivity with
subsequent families of Enginuity (interfamily), allowing
nondisruptive Enginuity upgrades in SRDF environments. This
feature’s usefulness is realized with all Enginuity releases from level
5x68 and higher.
Interfamily SRDF enablesan upgrade of a network of Symmetrix
systems over a period of weeks or months while maintaining SRDF
protection. This connectivity support is described at:
https://elabnavigator.EMC.com/emcpubs/elab/esm/pdf/
srdf%20interfamily%20support.pdf.
SRDF functional overview 23

Introducing SRDF
SRDF supported The Symmetrix data protection options described in Table 2 on

data protection page 24 can be purchased separately and implemented into the
options Symmetrix SRDF operation.
Table 2 Data Protection options for SRDF disk devices a
Data Protection Option Description
Mirroring (RAID 1) Provides the highest level of performance and availability for all mission-critical and
business-critical applications by maintaining a duplicate copy of a volume on two disk devices.
Symmetrix RAID 1/0 Provides a combination of RAID 1 and RAID 0 for open systems environments. Data is striped
across mirrored pairs.
Symmetrix RAID 10 Provides a combination of RAID 1 and RAID 0 used for mainframe environments.
Parity RAID Provides:

• High performance dependent upon volume layout and external striping
• High availability - data from lost volume is regenerated from remaining members
A Parity RAID (3+1) group consists of three data volumes to one parity volume. A Parity RAID
(7+1) group consists of seven data volumes to one parity volume.
RAID 5 Provides:
• High performance with automatic striping across hypervolumes
• High availability - lost hypervolumes regenerated from remaining members
RAID 5 is configured in (3+1) and (7+1) groups. RAID 5 technology stripes data and distributes
parity blocks across all the disk drives in the RAID group.
Permanent Member Sparing Replaces a faulty drive automatically from a list of available spares residing in the Symmetrix
system without CE involvement on site. Release 5771 only.
Dynamic Sparing Increases data availability by copying the data on a failing volume to a spare volume until the
original device is replaced. Dynamic Sparing is used as additional protection for mirrored Parity
RAID and SRDF volumes.
a. When configuring multiple data protection options, consult your EMC Sales Representative for configuration rules. Not all data protection options
are available on all Symmetrix systems or Enginuity levels.
Monitoring and An EMC representative installs and initially configures SRDF at your
controlling SRDF site using the Symmetrix service processor.
After SDRF is up and running, you can monitor and control its
operation by purchasing the appropriate EMC ControlCenter®
software, Solutions Enabler for UNIX and Windows environments, or
the SRDF Host Component for z/OS. For more information on SRDF
related software, consult the EMC Powerlink website at:
or contact your EMC Sales Representative.

Introducing SRDF
SRDF director hardware

SRDF director hardware consists of a director and adapter board set.
This hardware provides the communications physical layer for SRDF
data and information exchanges between Symmetrix systems. SRDF
director hardware includes any of four types of director/adapter
board sets, depending on the protocol:
◆ Multiprotocol Channel Director (MPCD), supported with
Symmetrix DMX series, and configurable for SRDF over Gigabit
Ethernet (GigE).
◆ GigE remote adapter (RE)
◆ Fibre Channel remote adapter (RF).
◆ ESCON remote adapter (RA).
The SRDF network options listed above can be used for any host
environment. SRDF uses a storage protocol based on the Gigabit
Ethernet, Fibre Channel FC-4, or ESCON specifications to remotely
mirror data between Symmetrix systems. The host attachment, I/O
protocol, and disk data structures that each host requires are
independent of the SRDF operation between Symmetrix systems.
Note: Some restrictions apply in mixed environments with iSeries and other
host types. For more information, consult EMC Customer Support.
! CAUTION
EMC recommends that the Symmetrix system has at least two
director boards for SRDF protection; one processor on each board
for SRDF to provide redundant links in case a communications
link fails or in the unlikely event a director fails. Having two
director boards for SRDF avoids a potential single point of failure,
if one processor on the director is used for SRDF and the other
processor(s) is/are used to connect to a host.
SRDF director The SRDF director/adapter board sets described above provide the
functions link connections, fiber-optic protocol support, and communications
control between two Symmetrix systems in an SRDF configuration.
GigE remote directors (RE) or Fibre Channel remote directors (RF)
maintain a peer-to-peer relationship at the transport layer of
communications.
SRDF director hardware 25

Introducing SRDF
The ESCON remote director (RA) board set that normally sends data
across an SRDF link is known as an RA-1. An RA-1 functions like a
host channel interface. The ESCON RA board set that normally
receives data sent across an SRDF link is known as an RA-2. An RA-2
functions like a storage director interface. An RA-1 and its
corresponding RA-2 are known as an RA pair.
◆ With Symmetrix 3xxx or 5xxx models, there can be multiple RA
pairs in an SRDF configuration, up to a maximum of 16 pairs.
◆ With Symmetrix 8xxx models, an optional four-processor ESCON
board can be used for SRDF.
◆ With Symmetrix DMX1000, DMX2000, and DMX3000 models, an
optional four-processor ESCON board can be used for SRDF.
Note: Each Symmetrix Product Guide provides detailed descriptions of the

SRDF supported hardware functionality.

Introducing SRDF
SRDF configurations
This section provides examples of typical SRDF configurations.
In Figure 2 on page 27, two production sites (A and C with
Symmetrix DMX-3 system bay) send data across SRDF links to one
recovery site (B Symmetrix DMX).
Site A Site C
(Production) (Production)
Local Site B Local

Host (Recovery) Host
Remote
Active Host Host Active Host
Path Path
Symmetrix A Recovery Symmetrix C

Path
Symmetrix B
R1 SRDF Link SRDF Link R1
R2 R2
Local Volume R1 Source Volume R2 Target Volume
Figure 2 Two production sites and one recovery site
SRDF configurations 27
Introducing SRDF
In Figure 3 on page 28, one recovery site (G) provides a data vaulting
solution for six production sites (A through F).
Site A Site B Site C
Site G acts as a Data Vaulting

Site G site for Sites A, B, C, D, E, and F
Site D Site E Site F
Figure 3 Data vaulting solution

Introducing SRDF
Figure 4 on page 29 illustrates the versatility of SRDF; some sites have

either primary (R1) or secondary (R2) devices, while other sites have
both primary (R1) and secondary (R2) devices.
Symmetrix D
Symmetrix E Symmetrix C
R2 R2 R1 R2 R1 R2
R1 R1 R2 R1
Symmetrix F Symmetrix B
R1 = Primary Volumes
R2 R1
R2 = Secondary Volumes
Symmetrix G Symmetrix A
Figure 4 Sites containing both primary and secondary volumes
Introducing SRDF
SRDF configuration Figure 5 on page 30 shows a more flexible switched SRDF

using switched Fibre configuration using Fibre Channel switches connected through
Channel E_Ports. Note that in this configuration, multiple primary (source) R1
devices are remotely connected using a Storage Area Network (SAN)
to multiple secondary (target) R2 devices.
R2
R2
Switch Switch Switch

E_Port E_Port E_Port E_Port
R1
R1 R2
R1 = Primary (Source) Devices

R2 = Secondary (Target) Devices
Figure 5 Switched SRDF with multiple primary and secondary devices
SRDF fully switched Enginuity 5x67 and later supports fully switched open SRDF (fibre)
fabric connectivity connections. Switched SRDF allows all RFs in the Symmetrix systems
connected to the fabric to communicate with each other, providing
greater connectivity and configuration flexibility. Switched SRDF
involves non blocking switching devices that interconnect two or
more nodes. Switched SRDF also enables fewer RFs to be present on
the fabric, depending on system performance and redundancy
requirements.

Introducing SRDF
In Figure 6 on page 31, Symmetrix A is communicating with

Symmetrix B and Symmetrix C simultaneously through two RF
directors. When Symmetrix A is communicating with Symmetrix C,
either of its RF directors can communicate with either RF director in
Symmetrix C.
Symmetrix A Symmetrix B
DMX, 5669 Point-to-Point 3XXX/5XXX, 5267
FC
R2 Volumes Switch
RF RF
R1 Volumes RF RF R1 Volumes
R2 Volumes Symmetrix C
8000, 5568
Point-to- R2 Volumes
RF
Multipoint
RF
R1 Volumes
Figure 6 Switched SRDF over Fibre Channel with Enginuity 5x67
There are four logical paths from Symmetrix A to Symmetrix C (two

logical paths on each RF director to each RF director in Symmetrix C).
When Symmetrix A is communicating with Symmetrix B, there is a
single logical path from each RF director to one other RF director.
These are point-to-point Fibre Channel connections; the same as was
provided with earlier Enginuity levels. Also note that all three
Symmetrix systems have primary (source or R1) devices. The ability
to support point-to-multipoint connectivity is unrelated to primary
(source) or secondary (target) device relationships.
Introducing SRDF
Switched and In Figure 7 on page 32, Symmetrix A is communicating with

Concurrent SRDF Symmetrix B and Symmetrix C simultaneously through two RF
directors, as well as using Concurrent SRDF to mirror its primary
devices (R1s) to a second set of secondary devices (R2s) in
Symmetrix D. Each RF director in Symmetrix A is configured to
support two SRDF groups. When Symmetrix A is communicating
with Symmetrix C, either of its RF directors can communicate with
either RF director in Symmetrix C.
DMX, 5669 3XXX/5XXX, 5267
FC
Symmetrix Devices Switch
RF RF
00-0F R1 Volumes R1 Volumes RF RF R2 Volumes
10-1F R1 Volumes R1 Volumes RA Symmetrix C

RA 8000, 5568
Any Supported Link

RF
R2 Volumes
RF
RA
RA
Symmetrix Devices 00-1F
R2 Volumes (second remote copy of all
Symmetrix D
3XXX/5XXX or 8000, 5x67 SRDF devices in Symmetrix A)
Figure 7 Switched and Concurrent SRDF configuration example
There are four logical paths from Symmetrix A to Symmetrix C (two

logical paths on each RF director to each RF director in Symmetrix C).
When Symmetrix A is communicating with Symmetrix B, there is a
single logical path from each RF director to one other RF director.
These are point-to-point Fibre Channel connections, the same as was
provided with earlier Enginuity levels.
The ability to support point-to-multipoint connectivity is not related
to primary (source) or secondary (target) device relationships. A
separate SRDF group in Symmetrix A is configured with FarPoint™
on ESCON RAs for Concurrent SRDF to Symmetrix D, possibly using
Adaptive Copy Disk mode and/or FarPoint.

Introducing SRDF
For Concurrent SRDF, the primary SRDF mode is a synchronous

mode of Symmetrix As primary devices (R1s); however, the
secondary mode of adaptive copy can be enabled for either one or
both secondary R2s. In Figure 7, the RA links are extended via
channel extenders over a WAN using adaptive copy mode and/or
FarPoint for asynchronous replication to Symmetrix D.
Note: As of Symmetrix Enginuity level 5671, SRDF/A is supported on one of

the two groups using Concurrent SRDF.
With Enginuity level 5567 and the Symmetrix 8000 systems, the same
local primary volume can be remotely mirrored in two different
locations. This functionality is called Concurrent SRDF. The
Symmetrix 3xxx/5xxx systems can be configured with a secondary
volume that is one of two remote copies from a Symmetrix 8000
primary volume; however, the Symmetrix 3xxx/5xxx systems cannot
be configured to provide two remote copies of the same primary
volume.
Note: For more information on Concurrent SRDF, refer to “Concurrent

SRDF” on page 60.
SRDF with native Native IP support for any SRDF based product on Symmetrix
GigE systems is based on GigE technology that enables direct Symmetrix
system-to-IP-network attachment. This increases the options for
Symmetrix system to Symmetrix system connectivity, and allows a
Symmetrix system to connect to existing Ethernet infrastructure and
directly access high-speed data transmission conduits via IP.
Symmetrix DMX series systems provide native IP support through
the Multiprotocol Channel Director (MPCD). Symmetrix 8000 series
systems provide native IP support through the GigE remote director.
The MPCD and GigE remote directors provide comparable
functionality, with the exception of data compression, which is a
feature of the MPCD only. Unless otherwise specified, the following
information applies to both Symmetrix 8000 series and Symmetrix
DMX GigE remote director configurations.
Introducing SRDF
Figure 8 on page 34 illustrates GigE connectivity into an IP network

infrastructure. Each Symmetrix GigE source port will establish a
logical TCP connection to each GigE target port. Hence, this
configuration will have four logical TCP connections between the
Symmetrix pair.
Note: The EMC Network Storage Topology Guide and the EMC SRDF
Connectivity Guide contain further information.
GigE GigE
Switch/Router Switch/Router
RE RE
WAN
RE RE
IP Network
Figure 8 Switched GigE network configuration example
SRDF and FarPoint FarPoint is an SRDF feature used only with ESCON extended distance
solutions (and certain ESCON campus solutions) to optimize the
performance of the SRDF links. This feature works by allowing each
RA to transmit multiple I/Os, in series, over each SRDF link.
Standard SRDF without The standard ESCON SRDF protocol (without SRDF FarPoint) allows
FarPoint only one write I/O to occupy a communication link at a time. The
next write I/O occurs only after the remote Symmetrix system has
acknowledged receipt of the previous I/O from the local Symmetrix
system and an ending status is presented to the local Symmetrix
system as shown in the top half of Figure 9 on page 35.

Introducing SRDF
SRDF with FarPoint With SRDF FarPoint, a single remote adapter can serially transmit
more than one write I/O over the SRDF link. This method (referred to
as pipelining) enables the SRDF communication link to be more fully
utilized, depending on the distribution of application I/O write
activity across multiple logical volumes. The number of write I/Os
that can be placed on the link at one time is determined by the
capacity of the type of SRDF link being used. With Enginuity level
5x67 or later, multiple copy tasks can be placed on the SRDF link for a
single logical volume.
Symmetrix SRDF without FarPoint Symmetrix
Data
Response
Symmetrix SRDF with FarPoint Symmetrix
Data Data Data
Response Response Response
Figure 9 SRDF with and without FarPoint
Preserving From the point of view of the host, FarPoint does not change the
synchronization SRDF protocol. In synchronous mode, the Symmetrix system returns
a completion status to the host only after the write operation is
performed on the remote Symmetrix system. Without FarPoint, the
Symmetrix system waits for one write operation to complete before
sending the next one. With FarPoint, the Symmetrix system, while
waiting for the status of the first write operation, uses the free link
bandwidth to send another write operation from a different primary
Introducing SRDF
volume. Interaction with the host remains unchanged, so the

synchronous condition is fully preserved, and the data on the remote
SRDF device is 100 percent consistent from the host's point of view.
Servicing remote Remote read operations (mainly used for recovery purposes), as well
reads as other special I/Os, are serviced using the standard SRDF protocol
(without FarPoint). In this situation, the write pipeline is cleared
before any read operations are performed.
Performance impact An SRDF configuration using FarPoint can improve performance

of FarPoint substantially at the Symmetrix system level across a group of SRDF
devices depending on the length of the SRDF link. Even on a short
link, the FarPoint solution may provide some benefits.
However, write operations to a single device are still serialized by the
host, so in the case of a synchronous device, a new write operation
does not start until the previous write operation to that device
receives a status from the remote system. This means that the
maximum number of write operations per device is still low, and using
FarPoint in this situation does not improve system performance.
Note: The introduction of PAV/MA does increase I/O concurrency at the

volume level, potentially improving throughput in FarPoint configuration
operating in a mainframe environment.

Invisible Body Tag
2
SRDF Technical
Concepts
This chapter describes SRDF links, modes of operation, system-level

attributes, and consistency groups.
This chapter contains the following topics:
◆ SRDF volume types ........................................................................... 38
◆ SRDF groups ....................................................................................... 42
◆ SRDF volumes: Special considerations ........................................... 44
◆ SRDF link configurations.................................................................. 47
◆ SRDF link and volume states ........................................................... 48
◆ Primary modes of operation............................................................. 53
◆ Secondary modes of operation......................................................... 56
◆ Additional SRDF modes and attributes.......................................... 58
◆ Concurrent SRDF ............................................................................... 60
◆ SRDF/Consistency Groups (SRDF/CG) ........................................ 62
SRDF Technical Concepts 37

SRDF Technical Concepts
SRDF volume types

SRDF refers to Symmetrix volumes as:
◆ Primary (source) volumes (R1)
◆ Secondary (target) volumes (R2)
◆ Local volumes
◆ Dynamic SRDF devices
◆ EMC Compatible Peer (PPRC mode) devices
Note: In the context of this discussion, volumes are synonymous with devices.
Primary (source) Primary (source) volumes contain production data that is mirrored in
volumes a different Symmetrix system. Primary volumes are also referred to as
R1 volumes. Updates to a primary volume are automatically
mirrored to a secondary (target) volume in the other Symmetrix
system.
Primary volumes can be locally protected by:
◆ A dynamic spare “Dynamic Sparing with remotely mirrored pairs
(SRDF)” on page 44 provides more information)
◆ Permanent member sparing, supported on Enginuity level 5771
◆ RAID 1, conventional mirroring, (the primary volume is then
referred to as a mirrored pair)
◆ RAID 10, (the primary volume is a striped mirrored device)
◆ RAID 1/0, provides a combination of RAID 1 and RAID 0 for
open systems. Data is striped across mirrored pairs
◆ Symmetrix Parity RAID (3+1) and Parity RAID (7+1) protection
(the primary volume is a Parity RAID data volume)
◆ RAID 5 (3+1) and RAID 5 (7+1) protection (the primary volume is
a RAID 5 data volume)
Additionally, a primary volume can be paired with a Business
Continuance Volume (BCV) to provide an additional working copy of
the data at the same location.

Secondary (target) Secondary (target) volumes contain a mirrored copy of data from a
volumes primary volume. Secondary volumes are also referred to as R2
volumes. As with primary volumes, secondary volumes can be
locally protected by:
◆ A dynamic spare provides “Dynamic Sparing with remotely
mirrored pairs (SRDF)” on page 44 more information)
◆ Permanent member sparing, supported on Enginuity level 5771
◆ RAID 1, conventional mirroring (the primary volume is then
referred to as a mirrored pair)
◆ RAID 10, (the primary volume is a striped mirrored device)
◆ RAID 1/0, provides a combination of RAID 1 and RAID 0 for
open systems. Data is striped across mirrored pairs
◆ Symmetrix Parity RAID (3+1) and Parity RAID (7+1) protection
(the primary volume is a Parity RAID data volume)
◆ RAID 5 (3+1) and RAID 5 (7+1) protection (the primary volume is
a RAID 5 data volume)
A secondary volume also can be paired with a BCV to provide an
additional working copy of the data at the same location.
Local volumes Local volumes can reside on an SRDF enabled Symmetrix system, but
they do not participate in SRDF activity. Local volumes are typically
protected either by RAID 1, RAID 10, Parity RAID (3+1), Parity RAID
(7+1), RAID 5 (3+1), RAID 5 (7+1), or a dynamic permanent member
spare.
Dynamic SRDF Since Enginuity level 5567, devices can be configured to be Dynamic
devices SRDF capable devices. The only Dynamic SRDF functionality
delivered in Enginuity level 5567 is R1/R2 personality swap where
primary and secondary volume pair roles can be reversed. Further,
this capability is only allowed in non-FarPoint configurations. The
Dynamic SRDF functionality, delivered in Enginuity level 5568,
enables you to create, delete, and swap SRDF pairs, using EMC
host-based SRDF control software, while the Symmetrix system is in
operation. With Dynamic SRDF, you can create SRDF device pairs
from non-SRDF devices, and then synchronize and manage them in
the same way as configured SRDF pairs. Figure 10 on page 40 shows
Dynamic SRDF connections in a switched fabric environment.
SRDF volume types 39

Note: At Enginuity level 5x71, SRDF/A devices also can be configured as

Dynamic SRDF capable devices. Prior to Enginuity level 5x71 this function
was not available for SRDF/A.
Dynamic SRDF is supported over the following topologies:

◆ ESCON point-to-point connection (RA)
◆ Fibre Channel point-to-point connection (RF)
◆ Switched Fibre Channel fabric connection (RF)
◆ GigE connection (RE)
Symmetrix
R2
Symmetrix
Switch
Possibility
#1 Switch
API/ R1 Symmetrix
CLI/ Possibility
ECC/etc. #2 Switch
Switch
R2
Figure 10 Dynamic SRDF
Dynamic SRDF provides the following capabilities:

◆ Personality swap between primary and secondary volumes
◆ Terminate and reestablish a relationship with a new secondary
volume
◆ Create a new primary/secondary pair relationship from
non-SRDF devices
Note: The above functionality is not available for devices protected by Parity
RAID. Further, the personality swap function is unavailable for ESCON
Farpoint configuration.
These capabilities, when combined with Dynamic SRDF groups,

enable the user to have additional control over their SRDF
configuration. Using EMC host-based software, the user can create or
remove RA directors, SRDF groups, and SRDF devices. Consult the
appropriate host-based software product guide for specific
instructions on how to create or remove RAs.

EMC Compatible Enginuity level 5568 (or higher) enables the Symmetrix system to
Peer support native IBM Peer-to-Peer Remote Copy (PPRC) commands
through a Symmetrix feature called EMC Compatible Peer.
Compatible Peer mode requires Dynamic SRDF capable devices. A
Dynamic SRDF device becomes a PPRC mode device upon receipt of
a PPRC CESTPAIR command. From the time that the device enters
PPRC mode, no host-based EMC SRDF control software can operate
on the device; it is truly a PPRC device and can only be controlled by
PPRC commands received from the host.
Note: Control of these devices via out-of-band software is unsupported.
Beginning with Enginuity level 5568, Compatible Peer implements

PPRC version 1, architectural level 2. Enginuity level 5671 adds
support for PPRC version 1, architectural levels 3 and 4 Hyperswap
support, including failover/failback functionality.
Note: Advanced SRDF features such as Concurrent SRDF or

SRDF/Asynchronous are not supported on EMC Compatible Peer. Parity
RAID data protection is not supported on EMC Compatible Peer. All other
Symmetrix data protection mechanisms are supported. For more detailed
information about implementations, go to:
http://www.redbooks.ibm.com/abstracts/sg246374.html
SRDF volume types 41

SRDF groups
SRDF groups define relationships between Symmetrix systems. An
SRDF group is a set of SRDF director port connections configured to
communicate with another set of SRDF director ports in another
Symmetrix system. Logical volumes (devices) are assigned to SRDF
groups.
An SRDF group configured through Symmetrix configuration is
called a static SRDF group. With Enginuity levels up to and including
5x68, each Symmetrix system supported up to 16 static SRDF groups.
Prior to Enginuity level 5568, static SRDF groups were required to
have at least one static SRDF device assigned to the groups. Further
device assignments could be made via static (configurations changes)
or dynamic SRDF.
Enginuity level 5669 increases the maximum number of definable
SRDF groups from 16 to 64. This enhancement allows more flexibility
when working with the following SRDF features:
◆ EMC Compatible Peer
◆ Switched SRDF
◆ Concurrent SRDF
◆ Dynamic SRDF

Dynamic SRDF At Enginuity level 5669 or above, a user can dynamically create
groups empty SRDF groups and dynamically associate the groups with Fibre
Channel or GigE SRDF directors. Removing dynamic SRDF groups is
also possible. Both of these operations are accomplished using EMC
host-based SRDF control software. Dynamic SRDF groups created
through this method are persistent through Symmetrix power on or
IMPL.
Note: EMC Compatible Peer groups require static SRDF groups. Dynamic
groups are not supported for PPRC.
This feature, when combined with dynamic SRDF devices, allows the
user complete control over their SRDF configuration. Using EMC
host-based software, the user can create or remove RA directors,
SRDF groups, and SRDF devices. Consult the appropriate host-based
software product guide for specific instructions on how to create or
remove RAs.
SRDF groups 43
SRDF volumes: Special considerations

The following Symmetrix features require special considerations
when used in an SRDF environment:
◆ Dynamic Sparing with remotely mirrored pairs (SRDF)
◆ Open systems metavolumes
◆ mainframe RAID 10 volumes
◆ PPRC command support
Dynamic Sparing Typically, when a disk drive fails, it does not fail suddenly; it fails
with remotely over a period of time during which it gives clues that it is failing. The
mirrored pairs Symmetrix system monitors the operation of all its disk drives and
(SRDF) can detect when a disk is beginning to fail.
A Symmetrix system can be configured with physical disk devices
known as dynamic spares, or hot spares. As the name suggests, the
purpose of a dynamic spare is to take the place of a failed or failing
disk device. If a disk drive begins to fail, the Symmetrix system
automatically invokes the dynamic spare.
When the Dynamic Sparing option is invoked for a remotely
mirrored SRDF pair, the Symmetrix system automatically activates an
available spare in the Symmetrix system containing the failing device
and copies data from the failing device to the spare. The system
continues processing I/O requests with the spare functioning as one
volume of a mirrored pair while the failing device and its remote
mirror all operate without interruption. If the Symmetrix system
cannot copy all data from the failing device to the spare, it will
retrieve the unavailable data from the good member of the remote
pair.
Open systems When users have open systems hosts attached to Symmetrix systems,
metavolumes they can create a logical device that spans multiple Symmetrix logical
volumes. This logical device is a metavolume.
A metavolume is a logical volume set created from individual
Symmetrix logical volumes that can comprise one-to-multiple
physical disks. Metavolumes are functionally the same as logical
volume sets implemented with host volume manager software. Some
metavolumes are created to define volumes larger than the current
Symmetrix maximum hypervolume size of approximately 32 GB.

Physically, a metavolume is two or more Symmetrix hypervolumes

presented to a host as a single addressable device. The metavolume
consists of a head device, some number of member devices (optional),
and a tail device.
Symmetrix metavolumes can contain up to 255 logical volumes and
be up to 7.65 TB in size. Metavolumes can be composed of
nonsequential and nonadjacent volumes. In an SRDF configuration, if
you create metavolumes for the primary volumes, you must also
create them for the secondary volumes. The primary and secondary
meta members must be equal in number. Concatinated meta volumes
must have equal size members (primary and secondary).
Extra care must be observed with striped meta volumes. Contact
EMC Customer Service for support where an R2 is larger than an R1.
Note: When configuring a metavolume, each metavolume device is counted

as a single logical volume. Using metavolumes will reduce the maximum
number of host-visible devices. That is, each member of the metavolume
must be counted toward the maximum number of host-supported
Symmetrix logical volumes. Consult the specific Symmetrix model product
guide for information on supported Symmetrix logical volume limits.
Symmetrix RAID 10 To improve mainframe volume performance, Symmetrix RAID 10

(mirrored striped stripes data of a logical device across multiple Symmetrix logical
mainframe devices. Four Symmetrix devices (each one-fourth the size of the host
volumes) facing mainframe device) appear as one mainframe device to the
host. Any four Symmetrix logical devices can be chosen to define a
RAID 10 group provided they are the same type (for example, IBM
3390) and have the same mirror configuration. Striping occurs across
this group of four devices with a striping unit of one cylinder. Since
each member of the stripe group is mirrored, the entire set is
protected. RAID 10 uses four pairs of disks in its Symmetrix
implementation.
Note: Enginuity level 5568, in an SRDF configuration, RAID 10 striped

volumes can be remotely paired with non-RAID 10 volumes. The size of the
non-RAID 10 volume must equal the sum of all members in the RAID 10
volume. At Enginuity level 5669, the sum of the R2 volumes can be larger
than the sum of the R1 volumes.
SRDF volumes: Special considerations 45

PPRC command Enginuity level 5568 or higher enables the Symmetrix system to
support support native IBM Peer-to-Peer Remote Copy (PPRC) commands
through a Symmetrix feature called Compatible Peer. PPRC is the
remote copying solution available with IBM storage systems as
shown in Figure 11 on page 46.
Enginuity level 5568 supports PPRC version 1, architecture level 2
(CGROUP FREEZE/RUN functionality. Enginuity level 5771 adds
support for PPRC version 1, architectural levels 3 and 4 Hyper-Swap
support, including failover/failback functionality). As a result,
Symmetrix systems will support these capabilities in IBM’s
Geographically Dispersed Parallel Sysplex (GDPS) solution.
Compatible Peer is available on Symmetrix systems with connections
to ESCON and/or FICON hosts.
Note: Contact your local EMC representative for specific details regarding
your Symmetrix system's support for Compatible Peer.
Primary Site Recovery Site
Automated site
Automated failover
site via via
failover
compiled REXX“Scripts”
compiledREXX Scripts
Symmetrix Fully
Symmetrix Fully PPRC
PPRC
Command Compliant
Command Compliant
PPRCcommands
PPRC commands PPRCcommands
PPRC commands
converted
converted to SRDF
to SRDF converted toSRDF
converted to SRDF
R1 R1 SRDF
SRDF R2 R2
SYM-000092
Figure 11 PPRC and GDPS support
Note: For more information on PPRC, go to:

http://www.redbooks.ibm.com/abstracts/sg246374.html

SRDF link configurations

The links between a given pair of Symmetrix systems in an SRDF
configuration can use one of the following methods for transmitting
data:
◆ Unidirectional
◆ Bidirectional
◆ Dual-directional
SRDF unidirectional If all primary (source, R1) volumes reside in one Symmetrix system
link configuration and all secondary (target, R2) volumes reside in another Symmetrix
system, write operations move in one direction, from primary to
secondary. This is a unidirectional configuration, in which data moves
in the same direction over every link in the SRDF group.
SRDF bidirectional If an SRDF group contains both primary and secondary volumes,
link configuration write operations move data in both directions over the SRDF links for
that group. This is called an SRDF bidirectional configuration.
Note: For information about how write operations occur using an ESCON
SRDF bidirectional link configuration, go to “Write operations” on page 68.
SRDF For an ESCON-based extended-distance SRDF configuration (for

dual-directional link example using E1, E3, T1, T3, and ATM links and/or IP network
configuration connections) that require data to move in two directions, a
dual-directional link configuration may be required. For more
information, consult the EMC SRDF Connectivity Guide. With a
dual-directional configuration, multiple SRDF groups are used; some
groups send data in one direction, while other groups send data in
the opposite direction.
SRDF link configurations 47

SRDF link and volume states
SRDF link states SRDF links can be in two possible states: online or offline. The SRDF
link is online when the following occurs:
◆ The remote adapter is operational and enabled on both sides of
the SRDF configuration.
◆ The Symmetrix systems are configured properly on both sides of
the SRDF configuration.
◆ The external link infrastructure components are operational.
The SRDF link is offline if one or more of the following occurs:
◆ The remote adapter is offline — link disabled.
◆ The remote adapter is online but the link is offline (damaged or
disconnected cable or other damaged hardware) — link disabled.
Configuration If all SRDF links fail, the Symmetrix system retains the last known
settings affecting logical state for the affected devices. This behavior is a pure ready state.
device ready and In other words, if a device is in a ready state before a link failure, it is
link states restored to a ready state. If a device is in a not-ready state before a
link failure, it is restored in a not-ready state after the link is again
made ready. However, all devices in the group remain not ready on
the SRDF link after all links fail if the prevent automatic recovery
after all links fail setting is set to Yes in the Symmetrix configuration.
The configuration setting, Force RAs offline after power up, prevents
SRDF links from coming online following a Symmetrix power cycle.
Prior to Enginuity level 5669, both configuration settings described
above were applied at the Symmetrix system level. At Enginuity level
5669 and later, these configuration settings are applied at the SRDF
group level.
Note: The device level pure ready state behavior does not apply to SRDF/A
mode of operation. Consult Chapter 4, ”SRDF/Asynchronous Operations,”
for information on how SRDF/A works.

Logical volume This section describes the three device states that an SRDF logical
states volume (primary, source R1 or secondary, target R2) reflects to the
host connected to the Symmetrix unit on which the volumes are
located.
The device states can be:
◆ Not Ready
◆ Read Only
◆ Write Enabled
! CAUTION
Understanding the device states of an SRDF logical volume is
fundamental to SRDF operation. Ensure you fully comprehend this
section before attempting any SRDF operations.
Device states are determined by a combination of two substates: the

SRDF state and the device host interface state, as shown in Figure 12
on page 49.
Host Interface SRDF

State State
Host
Figure 12 SRDF logical volume state
In general, look at the two states as two layers:

◆ An internal layer (SRDF view )— The SRDF state
◆ An external layer (host view) — The host interface state
These two layers are configured by different sets of internal
Symmetrix parameters, which can be set by either EMC Customer
Service or by software. The state ultimately seen by the host is
determined by the combination of these two device states.
SRDF link and volume states 49

Table 3 on page 52 and Table 4 on page 52 summarize the device state

seen by the host as determined by the individual states or substates.
For example, only when a secondary (R2) device is write enabled for
both the host interface state and the SRDF state can the host write to
the device.
The following sections describe these two substates and how the
various device substate combinations determine the actual overall
device state presented to the host.
SRDF volume states This section lists the substates a primary (R1) or secondary (R2)
(Symmetrix view) volume can have for SRDF operations.
Primary (R1) volume A primary (R1) volume can have the following states listed for SRDF
states operations:
◆ Ready and Read/Write — In this state, the primary (R1) volume
is available for read/write operations. This is the default primary
(R1) volume state.
◆ Host Not Ready — In this state, the secondary (R2) volume
responds not ready to the host for all read and write operations to
that volume.
◆ Not Ready — SRDF volumes can be not ready locally or not ready
remotely on the SRDF link:
• Locally Not Ready — If the local (R1) volume fails, the host
continues to recognize that volume as available for read/write
operations as all reads and/or writes continue uninterrupted
with the secondary (R2) volume in that remotely mirrored
pair.
• Remotely Not Ready — If R1 volumes are remotely not ready,
write updates will not propagate to the secondary volumes.
Changes to the R1 volumes are marked invalid as owed to the
secondary volumes R2.
Secondary (R2) A secondary (R2) volume can have one of the three states listed for
volume states SRDF operations:
◆ Not Ready — In this state, the secondary (R2) volume responds
not ready to the host for all read and write operations to that
volume.
◆ Read Only — In this state, the secondary (R2) volume is available
for read-only operations. This is the default secondary (R2)
volume state.

◆ Read/Writ e — In this state, the secondary (R2) volume is

available for read/write operations.
Note: When a secondary volume is placed into a Read/Write state, the

corresponding primary volume is placed into a Remotely Not Ready state.
Symmetrix volume states (host view)
This section lists the states a primary (R1) or secondary (R2) volume
has for host operations. This represents the volume state as seen by
the host interface.
Primary (R1) volume A primary (R1) volume can have one of the following states. This
states state is seen by the host connected to the Symmetrix unit in which
that volume resides:
◆ Write Enabled — In this state, the primary (R1) volume is
available for read/write operations. This is the default primary
(R1) volume state.
◆ Read Only — In this state, the primary (R1) volume responds
device write protected to the host for all write operations to that
volume.
◆ SRDF Not Ready — In this state, the primary (R1) volume
responds not ready to the host for all host accesses to that volume.
Secondary (R2) A secondary (R2) volume can have one of the following states. This
volume states state is seen by the host connected to the Symmetrix unit in which
that volume resides:
◆ Write Enabled — In this state, the secondary (R2) volume is
available for read/write operations.
◆ Read Only — In this state, the secondary (R2) volume responds
device write protected to the host for all write operations to that
volume. This is the default secondary (R2) volume state.
◆ Not Ready — In this state, the secondary (R2) volume responds
not ready to the host for all host accesses to that volume.
Host accessibility The tables in this section describe the accessibility state of the
primary (R1) and secondary (R2) volumes to the host connected to
the Symmetrix system containing the primary (R1) volumes. The
accessibility of the host to a particular Symmetrix volume depends on
its state from both the host and SRDF view. Table 3 on page 52 lists
the accessibility for a primary (R1) volume. Table 4 on page 52 lists
SRDF link and volume states 51

the accessibility for a secondary (R2) volume. Consult the EMC

Solutions Enabler Symmetrix SRDF CLI Product Guide and the
Symmetrix SRDF Host Component for z/OS Product Guide for how this
information is presented in these operating environments.
Table 3 Primary (R1) volume accessibility
Host interface state SRDF state Accessibility
Write Enabled Read/Write Read/Write
Write Enabled Not Ready Depends on secondary (R2)

volume availability
Write Disabled Read/Write Read Only
Write Disabled Not Ready Depends on secondary (R2)

volume availability
Not Ready Read/Write Unavailable
Not Ready Not Ready Unavailable
Table 4 Secondary (R2) volume accessibility
Host interface state SRDF state Accessibility
Write Enabled Not Ready Unavailable
Write Enabled Read Only Read Only
Write Enabled Read/Write Read/Write
Write Disabled Not Ready Unavailable
Write Disabled Read Only Read Only
Write Disabled Read/Write Read Only
Not Ready Not Ready Unavailable
Not Ready Read Only Unavailable
Not Ready Read/Write Unavailable

Primary modes of operation

Select primary modes of operation for individual devices and
manage these modes at the logical volume level.
Note: In an SRDF/Asynchronous (SRDF/A) environment, select primary

modes of operation for individual devices and manage SRDF/A at the SRDF
group level.
Note: Semi-synchronous mode is not supported in Enginuity level 5771 or

higher, or if the host connection to the devices participating in SRDF is via a
mainframe FICON interface.
Synchronous mode Available with the SRDF/S product offering, synchronous mode
maintains a realtime mirror image of data between the primary and
secondary volumes. Data must be successfully stored in both the local
and remote Symmetrix systems before an acknowledgement is sent to
the primary site host.
Synchronous mode provides realtime mirroring of data between the
local Symmetrix system and the remote Symmetrix systems. Data is
written to global memory of both systems before the application I/O
is completed to the host, ensuring the highest possible data
availability. Synchronous mode processing is described in the
following steps and illustrated in Figure 13 on page 54.
1. The local Symmetrix system containing the primary (source)
volume receives a write operation from the host.
2. The write I/O operation is propagated to the remote Symmetrix
system (containing the secondary or target volume); the local
Symmetrix system does not accept additional write operations to
the primary volume.
3. The remote Symmetrix system sends an acknowledgement to the
local Symmetrix system.
4. The local Symmetrix system sends an I/O complete message to
the local host; the local Symmetrix system now accepts additional
host write operations to the primary volume.
Note: In the mainframe PAV environment, SRDF synchronous mode

supports multiple concurrent write I/Os at the volume level.
Primary modes of operation 53

Host A Host B
1 4
3
Global Memory Director Global Memory Director
Symmetrix containing Symmetrix containing

primary (R1) volume secondary (R2) volume
Figure 13 Synchronous mode
Semi-synchronous Used mainly for the extended distance solution, semi-synchronous

mode mode allows the primary and secondary volumes to be out of
synchronization by one write I/O operation. Data must be
successfully stored in the Symmetrix system containing the primary
volume before an acknowledgement is sent to the local host.
Semi-synchronous mode will not allow the next write operation to a
primary device until a positive acknowledgement is received from
the remote Symmetrix system that the first write operation was
received in the remote Symmetrix global memory. However, any
number of read operations can be performed to the primary device
while awaiting acknowledgement of the first write operation.
Semi-synchronous mode writes data to the primary device in the
local Symmetrix system, completes the I/O, and then synchronizes
the data with the secondary device in remote Symmetrix system as
shown in the following steps and illustrated in Figure 14 on page 55.
1. The local Symmetrix system containing the primary volume
receives a write operation from the user application.
2. The local Symmetrix system sends an I/O complete message to
the local host. The user application now considers the I/O to be
complete. Read operations continue normally. A second write
I/O is not accepted.
3. Data from the first write operation is propagated to the remote
Symmetrix system containing the secondary volume.

4. The remote Symmetrix system sends an acknowledgement from

the first write operation to the local Symmetrix system. When the
acknowledgement is received, another write I/O from the host
can be accepted, for this primary volume.
Note: Because the I/O is completed before synchronizing data with the
remote system, the semi-synchronous mode provides an added performance
advantage.
Note: In the mainframe environment, semi-synchronous mode is not

supported when the Symmetrix host attachment is through FICON
connection.
In the mainframe PAV environment, semi-synchronous mode is not

recommended because it provides no performance benefit. EMC
recommends that SRDF users seeking the increased parallelism offered by
PAVs run in SRDF in synchronous mode.
Global Memory Director Global Memory Director
Symmetrix containing Symmetrix containing

primary (R1) volume secondary (R2) volume
Figure 14 Semi-synchronous mode
Primary modes of operation 55

Secondary modes of operation

Note: SRDF/Data Mobility is not available in synchronous or
semi-synchronous modes, but is supported in adaptive copy mode only.
Refer to “Adaptive copy modes” on page 56 for more information.
With the SRDF/Synchronous product offering, secondary modes are

used in conjunction with the primary operational modes described in
“Primary modes of operation” on page 53. As with the primary
modes of operation, select secondary modes of operation for
individual devices and manage the following modes at the logical
volume level. With SRDF/Data Mobility, only these secondary modes
of operation are supported:
◆ Adaptive Copy Write-pending mode
◆ Adaptive Copy Disk mode
Adaptive copy Adaptive copy modes facilitate data sharing and migration. These
modes modes allow the primary and secondary volumes to be more than
one I/O out of synchronization. The maximum number of I/Os that
can be out of synchronization is known as the maximum skew value.
The default value is equal to the entire logical volume. The maximum
skew value for a volume can be set using the SRDF monitoring and
control software.
There are two adaptive copying modes: adaptive copy write-pending
(AW) mode and adaptive copy disk (AD) mode. Both modes allow
write tasks to accumulate on the local system before being sent to the
remote system.
Adaptive copy With adaptive copy write-pending mode, write tasks accumulate in
write-pending mode global memory. A background process moves, or destages, the
write-pending tasks to the primary volume and its corresponding
secondary volume on the other side of the SRDF link. When the
maximum skew value is reached, the primary volume reverts to its
primary mode of operation, either synchronous or semi-synchronous,
whichever is currently specified. The device remains in the primary
mode until the number of tracks to remotely copy becomes less than
the maximum skew value.

Note: Adaptive copy write-pending mode reverts to the currently specified

primary mode (synchronous or semi-synchronous) if 75 percent of the
write-pending limit for the Symmetrix system is reached, regardless of
whether the maximum skew value specified for each device is reached.
The advantage to this mode is that it is faster to read data from global
memory than from disk, thus improving overall system performance.
An additional advantage is that the unit of transfer across the SRDF
link is the updated blocks rather than an entire track, resulting in
more efficient use of SRDF link bandwidth.
The disadvantage is that global memory is temporarily consumed by
the data until it is transferred across the link.
Consequently, adaptive copy write pending mode should only be
used where detailed information about the host write workload is
fully understood.
Adaptive copy disk Adaptive copy disk mode is similar to adaptive copy write-pending
mode mode, except that write tasks accumulate on the primary volume
rather than in global memory. A background process destages the
write tasks to the corresponding secondary volume. When the skew
value is reached, the primary volume reverts to its primary mode of
operation, either synchronous or semi-synchronous, whichever is
currently specified.
The advantages and disadvantages of this mode are opposite from
those of the adaptive copy write-pending mode; that is, while less
global memory is consumed it is typically slower to read data from
disk than from global memory, additionally, more bandwidth is used
because the unit of transfer is the entire track. In addition, because it
is slower to read data from disk than global memory, device
resynchronization time will increase.
! CAUTION
Adaptive copy disk mode should not be used if the primary
volumes are not protected by RAID 1, RAID 10, Parity RAID (3+1),
Parity RAID (7+1), RAID 5(3+1), or RAID 5 (7+1).
Secondary modes of operation 57

Additional SRDF modes and attributes

There are several SRDF modes and attributes that affect SRDF
behavior under certain conditions. These attributes are described in
the following sections.
Domino modes Domino modes effectively stop all write operations to both primary
and secondary volumes if all mirrors of a primary or secondary
device fail, or if all SRDF links in a link group become unavailable.
While such a shutdown temporarily halts production processing,
domino modes can prevent data integrity exposure caused by rolling
disasters.
There are two types of domino modes:
◆ Device domino mode
◆ Link domino mode
Device domino mode You can set device domino mode at the device level on primary
volumes. If this mode is set to Yes on a primary volume, and the
secondary volume becomes unavailable to its primary volume for
any reason, the primary volume becomes unavailable to its host.
Link domino mode You can set link domino mode at the SRDF group level at either side
of the SRDF links. If this mode is set to Yes for an SRDF group, and
the last remaining link in the SRDF group fails, all primary (source)
volumes in the SRDF group become unavailable (not ready) to their
host.
Once the not ready condition is set, you must re-enable the volumes
using EMC host-based software.
! CAUTION
With either domino mode, the appropriate primary volumes are
made not ready and all related applications stop. This is an extreme
measure. A more moderate measure (if you are using SRDF in a
mainframe environment, or an open systems environment with
EMC PowerPath® software) is to implement consistency groups (go
to “SRDF/Consistency Groups (SRDF/CG)” on page 62 for more
information).

Invalid tracks The invalid tracks attribute can only be set on secondary SRDF
attribute volumes. When the invalid tracks attribute is enabled, SRDF makes
the device not ready to the host if you attempt to access data from
that secondary volume when it is not synchronized with its primary
volume.
The purpose of the invalid tracks attribute is to inform the user that
the data on the secondary devices may not be suitable for use in
disaster recovery situations. It is a user decision to proceed using data
from devices that were made not ready by the invalid tracks attribute.
In such cases another form of data recovery may be more
appropriate. If a user decides to use the existing data on the
secondary volume, and the not-ready condition has been set by this
attribute, the not-ready condition can be reset by host-based SRDF
control software.
SRDF system-level You can set system-level attributes to do the following:

attributes ◆ Force RAs offline after powerup — This attribute applies to
all SRDF director types and forces SRDF adapter ports offline
following a power outage. This attribute can be used to prevent a
data integrity exposure due to a rolling disaster following a
power outage.
◆ Prevent automatic link recovery after all links fail
— This attribute prevents SRDF device pairs from automatically
resuming their SRDF relationship following a failure of all SRDF
links in an SRDF group. This attribute can provide the
opportunity to preserve a consistent copy of data on the
secondary devices (using Business Continuance Volumes) by
ensuring that device level resynchronization does not occur
automatically following link restoration.
Note: Prior to Enginuity level 5568, the Prevent Automatic Recovery

attribute is a system level attribute. At Enginuity level 5568 and higher,
the Prevent Automatic Recovery attribute is an SRDF group-level
attribute.
Additional SRDF modes and attributes 59

Concurrent SRDF
Enginuity level 5567 and later supports the ability for a single
primary volume to be remotely mirrored to two secondary volumes
concurrently. This feature is called Concurrent SRDF and is supported
in ESCON, Fibre Channel, and Gigabit Ethernet SRDF configurations.
Concurrent SRDF requires that each remote mirror operate in the
same primary mode, either both synchronous or both
semi-synchronous, but allows either (or both) volumes to be placed
into one of the adaptive copy modes.
Figure 15 on page 60 shows a concurrent SRDF configuration in
which the primary volume is communicating with one secondary
volume in synchronous mode. Concurrently, the same primary
volume is communicating with its other secondary volume in one of
the adaptive copy modes (adaptive copy write-pending mode or
adaptive copy disk mode). Any combination of
synchronous/semi-synchronous and adaptive copy is allowed with
the exception of one volume operating in synchronous mode and the
other operating in semi-synchronous mode.
Symmetrix DMX
Secondary
Synchronous
Symmetrix DMX
Primary
Symmetrix 8000
Adaptive Copy
Secondary
Figure 15 Concurrent SRDF configuration

Normal operating rules for SRDF also apply to concurrent SRDF

configurations. When operating in synchronous mode, ending status
for an I/O is not presented until the remote Symmetrix system
acknowledges receipt of the I/O to the primary Symmetrix system. If
both secondary volumes are operating in synchronous mode, ending
status is not presented until both volumes acknowledge receipt of the
I/O. If one remote mirror is in synchronous mode and one remote
mirror is in adaptive copy mode, ending status is presented to the
host when the synchronous volume acknowledges receipt of the I/O.
Concurrent SRDF 61
SRDF/Consistency Groups (SRDF/CG)

SRDF/CG ensures the consistency of the data remotely copied by
SRDF/S operations in the event of a rolling disaster.
An SRDF consistency group is a group of Symmetrix devices
specially configured to act in unison to maintain the integrity of a
database distributed across multiple Symmetrix systems controlled
by a mainframe host software or open systems host software using
EMC PowerPath software.
Note: Another way to ensure the integrity of a remote database is to use

domino mode (go to “Domino modes” on page 58).
How a consistency Assume that you have an SRDF configuration in which three
group works Symmetrix systems contain primary (source) devices, and two
additional Symmetrix systems contain secondary (target) devices.
The units with primary devices send data to the units with secondary
devices as shown in Figure 16 on page 62.
Primary 1 Primary 2 Primary 3
Secondary 1 Secondary 2
Figure 16 Primary and Secondary relationships

Next, assume that the links between Primary 2 and Secondary 1 fail.
Without a consistency group, Primaries 1 and 3 continue to write data
to the Secondaries 1 and 2 while Primary 2 does not as shown in
Figure 17 on page 63.
Figure 17 Failed link between Primary 2 and Target 1
The result is that the copy of the data spread across Secondaries 1
and 2 becomes inconsistent.
However, if Primaries 1, 2, and 3, belong to a consistency group, as
shown in Figure 18 on page 64, and the link between Primary 2 and
Secondary 1 fails, the consistency group automatically stops
Primaries 1 and 3 from sending data to Secondaries 1 and 2, as shown
in Figure 19 on page 64.
Thus, the dependent-write consistency of the data (spanning
Secondaries 1 and 2) remains intact.
SRDF/Consistency Groups (SRDF/CG) 63

Consistency Group
Figure 18 Primaries 1, 2, and 3 in a consistency group
Consistency Group
Figure 19 Failed link between Primary 2 and Target 1
Continuous I/O to the primary devices in the consistency group can still occur
processing even when the devices are not ready on the SRDF links. Such updates
are not immediately sent to the remote side. However, they are
propagated after the affected links are again operational, and data
transfer from the primary devices to the secondary devices resumes.

Technical For more information about consistency groups and the consistency
considerations group utility, go to:
◆ EMC Consistency Group for z/OS Product Guide
◆ EMC SRDF Family Product Guide
SRDF/Consistency Groups (SRDF/CG) 65


Invisible Body Tag
3
SRDF Operations
This chapter describes basic SRDF operations and covers the

following topics:
◆ Write operations ................................................................................. 68
◆ Read operations.................................................................................. 69
◆ Recovery operations .......................................................................... 71
◆ Business continuance using SRDF................................................... 73
◆ Business continuance using SRDF and TimeFinder ..................... 75
◆ R1/R2 swap ........................................................................................ 79
◆ Dynamic R1/R2 swap ....................................................................... 80
◆ Migrating data from R1 to a larger R2 device................................ 81
SRDF Operations 67
SRDF Operations
Write operations
The write task is the most common SRDF/S operation. This section
describes the write operations for unidirectional, dual-directional,
and bidirectional configurations.
Write operations in a This process applies to write operations over links that use a
unidirectional or unidirectional or dual-directional protocol, in which data moves in
dual-directional only one direction over a given link.
configuration In synchronous or semi-synchronous mode, the local host sends a
write I/O. The I/O then moves across the SRDF links to the remote
Symmetrix system. When it receives the I/O, the remote Symmetrix
system returns an acknowledgement to the local Symmetrix system.
Write operations in In a bidirectional protocol, data flows in two directions over the same
an ESCON SRDF link. When an RA-2, normally the receiving end of the SRDF
bidirectional link, must send write data to its corresponding RA-1, normally the
configuration sending end of the SRDF link, the RA-2 cannot simply transmit the
data over the SRDF link to the RA-1. This is due to the nature of
ESCON protocol and the channel-to-control unit architecture of
ESCON-based SRDF. Instead, the RA-2 must ask the RA-1 to read the
data from a primary volume serviced by the RA-2. When the RA-1
reads the data from the RA-2 volume, the write operation from RA-2
to RA-1 is effectively accomplished.

SRDF Operations
Read operations
Primary volume Read operations from a primary volume behave as normal read
read operations operations and are not impacted in any way by SRDF.
Secondary volume Read operations from a secondary volume can be initiated in

read operations response to read I/Os issued by the host attached to the primary
Symmetrix system or by the host attached to the secondary
Symmetrix system.
Read operations from Read operations, in which the primary host reads data from a
the primary host secondary volume in the remote Symmetrix system, are performed
only to recover from a local data availability problem. Several events
can cause such a read operation to take place:
◆ If data is not in global memory and all of the primary (source)
devices are in a not-ready state.
◆ If a primary device is in a ready state but the requested track is
invalid on the local device.
◆ If a disk adapter has a problem accessing a primary device as in
the case of a drive timeout or cyclic redundancy check (CRC)
error. In these cases, the local Symmetrix system requests the data
from the remote Symmetrix system.
◆ If a track on a primary device is currently available only from a
RAID rebuild, the local Symmetrix system requests the track data
from the remote Symmetrix system. This method is faster than
accessing the data from the RAID rebuild. The Symmetrix system
always performs a full-track read for count-key-data (CKD) and
at least a contiguous-block read for fixed block architecture (FBA)
data.
In a read operation, the remote Symmetrix system reads data from
the secondary device and sends the data across the link to the local
Symmetrix system.
Note: If a data availability problem is caused by a CRC error, an invalid track,

or a RAID rebuild, data can be destaged (moved from global memory to disk)
within the local Symmetrix system if it is in global memory. This process can
eliminate the need to read the data from the remote Symmetrix system.
Read operations 69
SRDF Operations
Read operations from SRDF secondary volumes can optionally be made read-only while
the secondary host SRDF is in operation. A host attached to the secondary Symmetrix
system may read data from the secondary devices provided the
operating system has support for reading data from devices that are
hardware write-protected. Such support varies by operating system
type. Consult your server operating-system vendor.
Note: If the secondary host has read-only access to the secondary device, it
does not interfere with SRDF operations. SRDF ensures that the most current
image of data is made available to the secondary host.

SRDF Operations
Recovery operations
! CAUTION
This section explains recovery in the context of hardware failure.
For adequate protection against logical data corruption, EMC
recommends that you regularly back up your data.
If the local host or local Symmetrix system fails, and the primary
volumes are operating in synchronous mode, the remote Symmetrix
system can be ready for operations in minutes. When the failure
occurs, the primary and secondary volumes are synchronized within
one I/O and there are no tracks owed to either the primary or
secondary side.
Failover to the If a remote host is to be activated to continue production processing,

secondary you must initiate a failover process. You can use EMC software
Symmetrix system products such as EMC ControlCenter, Solutions Enabler, or the SRDF
Host Component for z/OS to control the failover process.
The failover process involves changing the states of the secondary
volumes to read/write and transferring production processing to the
secondary host. After the failover has been initiated, updates made to
the secondary volumes appear as invalid tracks owed to the primary
volumes. Processing continues on the remote host until the local host
and local Symmetrix system are operational again and a failback is
performed.
Failback to the After the primary host and the Symmetrix system containing the
primary Symmetrix primary volumes are again operational, production processing can
system resume on the primary host. The following steps are required to
transfer processing from the secondary host back to the primary host:
1. Halt processing on the secondary host and change the state of the
secondary devices to read-only or Host Not Ready.
2. If a power on or IML is required of the primary Symmetrix
system, make sure that the SRDF links are physically disabled to
prevent movement of invalid track metadata across the SRDF
links.
Recovery operations 71
SRDF Operations
3. Bring the Symmetrix system on which the primary volumes

reside to a ready state. If the SRDF links were physically disabled
in the previous step, re-enable the links.
4. Use EMC software products such as EMC ControlCenter,
Solutions Enabler, or the SRDF Host Component for z/OS to send
invalid track metadata information from the secondary volumes
to the primary volumes.
5. Bring the SRDF links online and restart the local host. The
primary volumes automatically receive the appropriate data from
the secondary volumes, effectively resynchronizing the primary
and secondary volumes.
Note: You can use EMC software products such as EMC ControlCenter,
Solutions Enabler, or the Stored Procedure Executive (SPE) of SRDF Host
Component for z/OS to automate or semi-automate this process.
Recovery for a large If the recovery site (the secondary host and the Symmetrix system
number of invalid containing the secondary volumes) has handled production
tracks processing for a long period of time, there might be a large number of
invalid tracks (for example, 500 GB) owed to the primary volumes. In
this case, you can resynchronize the primary and secondary volumes
while the secondary host continues production processing. Then,
when there is a relatively small number of invalid tracks on the
primary volumes (for example, 50 GB), you can shut down the
secondary host and restart the primary host.
Note: This capability is enabled by using the Solutions Enabler or the SRDF
Host Component.

SRDF Operations
Business continuance using SRDF

Business continuance refers to practices that enable you to achieve
nearly nonstop, 24x7 business operations.
SRDF enables business continuance by allowing you to suspend
remote mirroring and temporarily enable the secondary volumes for
read/write activity, effectively creating two separate host/storage
systems (Figure 20 on page 73).
Symmetrix 1 Symmetrix 2
Business
Production
Continuance
Host SRDF LInk
R1 R2 Host
Remote Mirroring Temporarily Suspended and

Target Volume (R2) Enabled for Read/Write Activities
Figure 20 SRDF business continuance
The following business continuance practices are possible with SRDF:

◆ Backups — Avoid taking system offline while nightly backups
run.
◆ Remote processing — For example, some systems must produce
daily billing statements. These statements can be produced at a
remote site without interrupting normal processing.
◆ Decision Support System (DSS) operations.
◆ Application testing.
Concurrent You can temporarily suspend the SRDF links so that you can read and
operations write data on both the primary and secondary volumes concurrently.
This enables you, for example, to run backups on the secondary
volumes while production processing continues on the primary
volumes (a business continuance practice).
You can then resume the links and copy data from the primary
volumes to the secondary volumes.
This last operation propagates any updates made to the primary
volumes while the links were suspended and overwrites any
changed data on the secondary volumes, bringing both volumes into
a synchronized state (Figure 21 on page 74).
Business continuance using SRDF 73

SRDF Operations
R1
Primary
Data
After completion of concurrent operations, the
links between primary and secondary are reestablished,
Updates and updates are propagated from the primary volumes
to the secondary volumes.
R2
Secondary
Data
Figure 21 Primary-to-secondary resynchronization
Alternatively, you can resynchronize data in the opposite direction,

from secondary to primary. This operation is useful if, for example,
you performed application testing on the secondary volumes,
production processing was halted on the primary volumes, the
testing was successful, and you want to keep the updates.
R1
Primary
Data
During an R2-to-R1 resynchronizing

Updates operation, data is copied from the secondary
volumes to the primary volumes.
R2
Secondary
Data
Figure 22 Secondary-to-primary resynchronization

SRDF Operations
Business continuance using SRDF and TimeFinder

The Symmetrix TimeFinder product offering uses one or more
software-controllable, dynamically assignable mirror volumes called
business continuance volumes (BCVs). SRDF, when used with EMC
TimeFinder software, allows you to perform business continuance
operations without temporarily suspending remote mirroring.
TimeFinder BCVs can be locally mirrored and/or remotely mirrored
via SRDF. The sections that follow describe how SRDF can be used
with TimeFinder for business continuance operations.
Note: The SRDF/Automated Replication solution (introduced on page 19)

combines host-based automation with TimeFinder and SRDF functionality.
This functionality, described below, provides an automated long distance
disaster restart solution.
Note: For information about EMC TimeFinder, go to the documentation

(available on the EMC website: http://Powerlink.EMC.com, then follow
these links: Support, Document Library, Software, TimeFinder, or contact
your EMC representative.
Using TimeFinder operations (establish, split, restore) are allowed between

TimeFinder/Mirror a BCV and a primary device. Restore operations to a primary device
BCVs with primary (R1) will result in the restored data being propagated across the SRDF
devices link if the remote device is not in a suspended state. If the remote
device is in a suspended state, restored data will be marked as invalid
(owed) to the secondary device.
Using a BCV as a A BCV device may also function as an SRDF primary device. Such a
primary (source) device is called a BCV R1 device. A BCV R1 device may optionally be
device locally mirrored.
When a BCV R1 is attached to a standard device, the corresponding
secondary device is suspended. When the BCV is split, the default
behavior is to propagate the changed data to the secondary device.
Optionally, data transmission can be suppressed and the BCV R1
device will mark the changed data as invalid (owed) to the secondary
device, pending the resumption of the BCV R1 to R2 device
relationship (Figure 23 on page 76).
Concurrent SRDF is not supported for BCV R1 devices because a BCV
R1 device is not allowed to have two remote mirrors.
Business continuance using SRDF and TimeFinder 75

SRDF Operations
Using TimeFinder TimeFinder operations (establish, split, restore) are allowed between
BCVs with a BCV and a secondary device. By default, restore operations will not
secondary devices result in data propagating back to the primary device. Changed data
will be marked as invalid (owed) to the primary device following a
restore operation. If data propagation back to the primary device is
desired, TimeFinder software options are available to enable this
function.
Standard
Device
SRDF
X Link
Host
1
2 3
R1 R2
BCV
Symmetrix Symmetrix
Site A Site B
1. The BCV device is split.
2. The R2 on the Symmetrix at Site B is resynchronized with the R1 BCV at Site A.
3. Data is mirrored to the R2 device on the Symmetrix at Site B.
Figure 23 SRDF single-hop configuration (BCV functioning as a primary

SRDF device)
Using a BCV as a A BCV device may also function as an SRDF secondary device. Such
secondary (target) a device is called a BCV R2 device. A BCV R2 device may optionally
device be locally mirrored.
When a BCV R2 is attached to a standard device, the corresponding
primary device is suspended. When the BCV R2 is split, it is again
available for SRDF operations.
Note: Host software may restrict functionality of BCV R2 devices. Please go

to the SRDF Host Component or Solution Enabler documentation for
information on using BCV R2 devices.

SRDF Operations
SRDF remote You can issue SRDF and BCV control commands from a host or server
command support connected to the local Symmetrix system and have those commands
transmitted across the SRDF link. A host or server does not have to be
located at the secondary location, allowing you for example, to create
a dependant write consistent point-in time copy of your data at the
remote site without needing a host computer or server at the remote
site.
SRDF Multi-Hop SRDF’s multi-hop capability creates the ability to provide disaster
recovery protection at great distances without any data loss. A middle
or bunker site transitions the synchronous replication solution by
using TimeFinder in conjunction with SRDF.
SRDF multi-hop uses a BCV R1 on the secondary side of an SRDF
synchronous configuration to perform an incremental
synchronization with a secondary volume on a third Symmetrix
system. When split, the BCV R1 copies only changed tracks to the
secondary volume located in the third Symmetrix system. This
process provides two benefits. It eliminates the additional
performance impact on synchronous operations across great
distances. In addition, the differential resynchronization dramatically
reduces bandwidth requirements.
Note: Although the local side of an SRDF multi-hop configuration requires

an SRDF license with the ability to do synchronous or semi-synchronous
operations, you can use SRDF - Data Mobility (in adaptive copy mode) on
Symmetrix systems connected to the SRDF extended distance configuration.
This requires an initial full synchronization to establish the relationship.
Figure 24 on page 78 illustrates SRDF multi-hop using ESCON

connectivity however, any type of SRDF connection is supported by
multi-hop.
Business continuance using SRDF and TimeFinder 77

SRDF Operations
Production Site Recovery Site
SRDF Extended Distance

Synchronous
Campus Fiber
RA-1 RA-2 RA-1 Any Supported RA-2

MAN or WAN
Production Data Mirrored

to Remote Symmetrix Differential Synchronization Copy
Production Production BCV
R2 Data Production Production
R1 Data Data Data
BCV/R1 R2
Re-Establish/Differential Split
Figure 24 SRDF multi-hop configuration

SRDF Operations
R1/R2 swap
Dynamic SRDF provides the capability of swapping the R1/R2
personality of an SRDF device pair. During a personality swap,
primary R1 devices become secondary R2 devices. The secondary R2
devices become primary R1 devices. Swapping personalities allows
the R2 side to take over operations while retaining a remote mirror on
the R1 side. Swapping is especially useful after failing over an
application from the R1 side to the R2 side.
Before performing an R1/R2 swap, the SRDF pair must be in a
suspended state.
After the swap operation is complete, the SRDF device pair remains
in the suspended state. Incremental or full resynchronization
capability is maintained and controlled through EMC SRDF host
based control software. For specific procedures for controlling the
R1/R2 swap operation, go to the SRDF Host Component for z/OS
Product Guide or the Solutions Enabler SRDF CLI Product Guide.
R1/R2 swap When introduced in Enginuity 5x65 and 5x66, the R1/R2 swap
procedure history feature used an online configuration change procedure to affect the
personality swap of the primary (source, R1) and secondary (target,
R2) devices. This process was often lengthy, taking several minutes to
complete. Beginning with Enginuity 5567, a new capability called
Dynamic SRDF was introduced, which dramatically improved the
performance of R1/R2 personality swaps (Figure 25 on page 79).
Initial View After R1/R2 Swap
R1 R2 R2 R1
R1 R2 R1 R2
SRDF Primary SRDF Secondary SRDF Primary SRDF Secondary

(Source) (Target) (Source) (Target)
Figure 25 R1/R2 swap concept
R1/R2 swap 79
SRDF Operations
Dynamic R1/R2 swap

Dynamic R1/R2 swap is available with Enginuity Version 5567 or
higher. Dynamic R1/R2 swap is faster than the static swap. Host
software can then be used to create Dynamic SRDF primary and
secondary device relationships. Host software can also be used to
swap the SRDF personalities of the SRDF devices. Primary (source)
R1 device(s) become secondary (target) R2 device(s) and secondary
(target) R2 device(s) become primary (source) R1 device(s).
Swaps using Dynamic SRDF can be accomplished faster but must
also be enabled in your Symmetrix system configuration to use this
feature.
Dynamic swap is not supported under the following conditions:
◆ If Enginuity Versions 5567 and 5568 are mixed between your local
and remote Symmetrix system
◆ In Enginuity Version 5669 or higher configurations where the R2
device is larger than the R1 device
◆ If the devices to be swapped are participating in an active
SRDF/A session
◆ When using ESCON RA directors configured with FarPoint
◆ For Parity RAID (3+1) and (7+1) protected devices
Note: For more detail regarding specific platform requirements and

functionality for dynamic swap, go to either the EMC Solutions Enabler
Symmetrix SRDF CLI Product Guide or the Symmetrix SRDF Host
Component for z/OS Product Guide.

SRDF Operations
Migrating data from R1 to a larger R2 device

Enginuity Version 5669 provides a temporary method for migrating
data from an R1 device to a larger R2 device. Data migration to a
larger R2 device is only allowed on Symmetrix DMX hardware
running Enginuity 5669 or higher.
Certain SRDF operations are blocked when the R2 device is larger
than the R1 device:
◆ Swap operations are not allowed.
◆ Data migrated to a larger R2 device, once the host has recognized
the additional capacity cannot be restored back to the R1 device.
◆ Striped metadevices are supported with restrictions.
◆ Mainframe meta devices are supported.
◆ Concatenated metadevices are not supported.
Migrating data from R1 to a larger R2 device 81

SRDF Operations

Invisible Body Tag
4
SRDF/Asynchronous
Operations
This chapter provides a technical overview of SRDF/Asynchronous

(SRDF/A). The following topics are covered:
◆ SRDF/A overview ............................................................................. 84
◆ SRDF/A benefits ................................................................................ 85
◆ Requirements and limitations .......................................................... 86
◆ SRDF/A history ................................................................................. 87
◆ Tolerance mode .................................................................................. 90
◆ Locality of reference........................................................................... 91
◆ SRDF/A single session mode........................................................... 92
◆ SRDF/A single session mode dependent-write consistency....... 93
◆ SRDF/A single session mode states................................................ 95
◆ SRDF/A single session mode delta set switching......................... 97
◆ SRDF/A single session mode state transitions............................ 102
◆ SRDF/A single session cleanup process....................................... 107
◆ SRDF/A single session mode recovery scenarios ....................... 108
◆ SRDF/A multi-session consistency (MSC) mode ........................ 111
◆ SRDF/A MSC mode dependent-write consistency .................... 112
◆ SRDF/A MSC mode delta set switching ...................................... 116
◆ SRDF/A MSC session cleanup process ........................................ 122
SRDF/Asynchronous Operations 83
SRDF/Asynchronous Operations
SRDF/A overview
SRDF/A provides a long distance disaster restart solution with
minimal impact on performance. This solution is intended for
customers requiring the ability to preserve dependent-write
consistency within and across the database and application
environment at an extended distance secondary site with minimal
host application impact. Data is transferred to the secondary
Symmetrix system in cycles, using delta sets.
This chapter explains how the dependent-write consistency is
maintained during the SRDF/A single session mode and SRDF/A
Multi Session Consistency (MSC) mode cycle switches.

SRDF/A benefits
SRDF/A provides the following features and benefits:
◆ Supports extended data replication with database and application
consistency
◆ Promotes efficient link utilization resulting in lower link
bandwidth requirements
◆ Maintains a dependent-write consistent point in time image on
the secondary (R2) devices
◆ Supports all current SRDF topologies (ESCON, FarPoint,
point-to-point and switched fabric Fibre Channel, and GigE)
◆ Requires no additional hardware, such as switches or routers
◆ Supports hosts listed in the EMC Support Matrix for CKD and
FBA data emulation types
Note: Note: For more information on supported hosts, go to the EMC

Support Matrix at http://EMC.com or http://Powerlink.EMC.com, or
contact you EMC Sales Representative
◆ Imposes minimal impact on the back-end disk directors

◆ Provides a performance response time equivalent to writing to
non-SRDF devices
◆ Allows failover and failback capability between the R1 and the R2
SRDF/A benefits 85
Requirements and limitations

Known requirements and limitations for this release of SRDF/A are
as follows:
◆ When using either TimeFinder/Clone™ BCVs or
TimeFinder/Snap™ BCVs, special considerations must be
followed. Consult the EMC Solution Enabler TimeFinder CLI
Product Guide or the SRDF Host Component for z/OS Product Guide.
◆ SRDF/A is supported for open systems through Solutions
Enabler. mainframe interface is supported through SRDF Host
Component V5.2 and higher.
◆ SRDF/AR (previously known as SAR) does not support SRDF/A
devices.
Note: For more information on specific restrictions, consult the EMC Solutions
Enabler Symmetrix SRDF CLI Product Guide and the Symmetrix SRDF Host
Component Product Guide.

SRDF/A history
The following sections describe the highlights of SRDF/A
development and their availability.
Enginuity 5670 Enginuity 5670 supported single session mode SRDF/A. This
SRDF/A single configuration allows a single SRDF group to participate in
session asynchronous mode within a single Symmetrix. This SRDF/A group
can not use Dynamic SRDF or participate in Concurrent SRDF
operations.
Enginuity 5670.50 Enginuity 5670.50 initiated the use of SRDF/A Multi-Session

SRDF/A Consistency, MSC, for mainframe. The initial support is limited to a
Multi-Session single SRDF/A group allowed per Symmetrix but multiple
Consistency, MSC Symmetrix systems can participate in a SRDF/A MSC configuration.
This feature provides for a special mode of SRDF/A operation where
cycle switching is controlled by the host application via a Symmetrix
system call interface, and could be used to provide
dependent-mainframe across several Symmetrix systems.
Enginuity 5671 Multiple SRDF/A groups (up to 64 depending on the configuration)

Multiple SRDF/A allowed per Symmetrix array. Implicit in this feature is the ability to
Single Session SRDF define a primary for one SRDF/A group and a secondary to another
groups per SRDF/A group in the same Symmetrix DMX. By doing this, you now
Symmetrix have the ability to use SRDF/A with bidirectional operations.
Note: Bidirectional operation within a single SRDF/A group is not

supported. All source-to-target data flow operations within a SRDF/A group
still remain unidirectional.
Enginuity 5671 SRDF/A Multi-Session Consistency, MSC, for mainframe and Open
SRDF/A Systems supports multiple Symmetrix with no restrictions to the
Multi-Session number of SRDF/A groups per Symmetrix operating in SRDF/A
Consistency, MSC MSC mode. This feature includes the addition of open systems
support and eases the existing restrictions implemented in Enginuity
5670.50 and higher for mainframe SRDF/A.
SRDF/A history 87
This feature may also include new functionality to facilitate interhost

(mainframe and open Systems) communications in a mixed,
multihost control environment.
Enginuity 5671 The feature provides the ability to replicate a group of devices in
Concurrent SRDF Synchronous mode to one secondary site and in Asynchronous mode
support to another secondary extended distance site. The same primary
device can be replicated synchronously using SRDF/S mode through
one SRDF link and asynchronously using SRDF/A mode through
another link. Performance is equivalent to a conventional Concurrent
SRDF/S configuration; the addition of the SRDF/A leg does not
impose additional performance restrictions.
This feature enables limited multisite protection capability. In the
event of a primary site failure, there would be no data loss since it is
synchronously replicated to a local/regional secondary site. The
limitation to the solution is that in the event of a loss of the primary
site, establishing remote replication between the remaining sites
would mean configuration changes and a full synchronization
process. To avoid the limitation, EMC has developed SRDF/Star (go
to the next section, SRDF/Star).
In the event of a regional disruption that affects the primary site and
the synchronous site, there would be controlled data loss. The data
has been simultaneously replicated asynchronously to a secondary
site much further away. The environment is still dependent-write
consistent and restartable but there may be some data loss. The data
loss is at most two times the SRDF/A cycle time.
Enginuity 5671 This feature allows adding and removing Dynamic SRDF groups that
Dynamic SRDF participate in SRDF/A, as well as adding and removing devices from
support SRDF/A groups. A concurrent mirror may be added dynamically to
a device that is in an SRDF/A group subject to the normal restrictions
for dynamic concurrent SRDF. However, only one concurrent mirror
per device is allowed to be in an SRDF/A group.
Adding or removing a device from an SRDF/A group requires the
use of the SRDF/A tolerance mode feature before actually adding
and/or removing the mirror, otherwise SRDF/A drops.
To remove a Dynamic SRDF/A group, the group must be empty of
devices and the SRDF/A session must be inactive and drained. To
remove a dynamic mirror from an SRDF/A session, the mirror must
be made not ready and all outstanding I/Os in that session for that
device must be drained or discarded.

Enginuity 5671 This host software configurable feature allows the individual
Tunable Cache SRDF/A processes, single session or MSC, to set the cache usage limit
utilization to a percentage of the system write pending limit. Previously,
SRDF/A cycles could grow until they reach the Symmetrix system
write pending limit, at which point you could choose to throttle the
host for a given amount of time or to drop SRDF/A immediately.
Without this feature, if system write pending limits are being
exceeded, performance suffers across the entire Symmetrix, not just
the SRDF/A application.
This feature also adds the ability for a user to define, if cache
resources are being stressed, which SRDF/A sessions to drop first. In
so doing, the user can effectively assign a priority to sessions keeping
SRDF/A active upon for as long as cache resources allows.
SRDF/A Reserve SRDF/A Reserve Capacity enhances SRDF/A's ability to maintain

Capacity operational state when encountering network resource shortfalls that
would have previously suspended SRDF/A operations. With
SRDF/A Reserve Capacity functions enabled, additional resource
allocation can be applied to address temporary workload peaks,
periods of network congestion, or even temporary network outages.
The two functions that implement SRDF/A Reserve Capacity are
Transmit Idle and Delta Set Extension (DSE), and they work together
to maximize availability of continuous remote replication operations
while minimizing operational overhead.
SRDF/A Transmit Idle enables asynchronous replication operations
to remain active in the event all links are lost temporarily due to
network outages. The time SRDF/A remains active will depend on
the system not reaching the system write pending limit or SRDF/A
max cache limit, which is user setable.
SRDF/A Delta Set Extension or DSE enables asynchronous
replication operations to remain active in the event system cache
resources are becoming in danger of reaching the system write
pending or SRDF/A max cache limit. This functionality is achieved
by offloading some or all of the active Delta Set data, that needs to be
transmitted to the target site, into preconfigured storage pools.
SRDF/A Transmit Idle and Delta Set Extension have the ability to
work together to improve the overall resiliency of SRDF/A during
workload and network resource imbalances.
SRDF/A history 89
Tolerance mode
Tolerance mode allows certain conditions to occur that would
normally drop SRDF/A. These conditions could include making the
secondary Symmetrix R2 devices Read/Write. When tolerance mode
is set to on, the dependent-write consistency is NOT guaranteed.
SRDF/A MSC still allows tolerance mode to be turned on, but note
that if it is set on for any SRDF/A group within the MSC
configuration, the result could be inconsistencies for the entire MSC
group.
The host software to implement tolerance mode is different for
mainframe and Open Systems. The mainframe software exports the
use of tolerance mode directly, where the Open Systems software
externalizes it through consistency enabling/disabling.
Note: For more information on Tolerance mode go to the Symmetrix SRDF

Host Component for z/OS Product Guide. For more information on
enabling/disabling consistency go to the EMC Solutions Enabler Symmetrix
SRDF CLI Product Guide.

Locality of reference
Locality of reference in SRDF/A environments improves the
efficiency of the SRDF network links. Even if there are multiple data
updates, (i.e. repeated writes) in the same cycle, the systems send the
data across the SRDF links only once.
This is a major advantage over competitive asynchronous replication
solutions. In competitive solutions of this type, every write is sent
across the link and the locality of reference is not utilized at all. These
asynchronous solutions consume as much bandwidth as an
synchronous solution, which must (by definition) send every I/O
across the links.
The advantage gained from the locality of reference on the SRDF link
is not necessarily the same as the advantage gained in cache memory.
The main difference has to do with the fact that I/Os sent on the link
are usually the same size as the host I/Os. The logic is similar to
SRDF Adaptive Copy Write Pending mode, as opposed to SRDF
Adaptive Copy Disk mode where the system always sends full
tracks. In such a case, the gain in bandwidth efficiency from the
locality of reference is mainly from rewriting to the same block and
not rewriting to the same track.
The rules for combining a number of small blocks to one larger I/O
are complex, and not discussed here. However, there are many cases
where the system combines the original I/Os and sends them as one
large I/O across the link. Even if this operation does not necessarily
decrease the bandwidth, it does decrease the number of IO/s the RA
handles, and thus reduces the processing overhead per host I/O.
Figure 26 on page 91 shows both the locality of reference and the
concatenation of small blocks to one larger I/O for transmission.
Applications tend to write data in proximity of time and place
Track 0 Track 1 Track 2
Synchronous mode: 10 I/Os, 10 blocks Asynchronous mode has:

Asynchronous mode: 3 I/Os, 7 blocks Less bandwidth (7 blocks vs. 10)
Less SRDF overhead (3 I/Os vs. 10)
SYM-001280
Figure 26 Synchronous and asynchronous block transfer comparison
Locality of reference 91
SRDF/A single session mode

Different from traditional ordered write asynchronous approaches,
Symmetrix systems implement asynchronous mode host writes from
the primary Symmetrix to the secondary Symmetrix using
dependent-write consistent delta sets transferred in cycles. Each delta
set contains groups of I/Os for processing, which are managed for
dependent-write consistency by the Enginuity operating
environment. SRDF/A transfers these sets of data using cycles of
operation, one cycle at a time, between the primary Symmetrix and
the secondary Symmetrix.
SRDF/A single session mode refers to the implementation of
SRDF/A between one (1) primary Symmetrix using one (1) SRDF
group to one (1) secondary Symmetrix using one (1) SRDF group.

SRDF/A single session mode dependent-write consistency

SRDF/A single session mode is the implementation of SRDF/A from
a single SRDF group on the primary Symmetrix to a single SRDF
group on the secondary Symmetrix. Enginuity controls the cycle
switching without any host software involvement. Multiple instances
of SRDF/A single session mode operation between Symmetrix
systems are available with Enginuity 5x71 or higher.
Dependent-write consistency is achieved through the processing of
ordered SRDF/A delta sets (cycles) between the primary Symmetrix
and the secondary Symmetrix.
1. The active cycle on the primary Symmetrix contains the current
host writes or N data version in the capture delta set.
2. The inactive cycle contains the N-1 data version that is transferred
via SRDF/A from the primary Symmetrix to the secondary
Symmetrix. The primary inactive cycle is the transmit delta set
and the secondary Symmetrix inactive cycle is the receive delta
set.
3. The active cycle on the secondary Symmetrix contains the N-2
data version in the apply delta set. This is the guaranteed
dependent-write consistent image in the event of a disaster or
failure.
Figure 27 on page 93 illustrates the delta sets and their relationships.
Primary Symmetrix Secondary Symmetrix
R1 Apply R2
Capture
N N-2
Transmit Receive
N-1 N-1
R1 R2
Capture Transmit Receive Apply

“Active” “Inactive” “Inactive” “Active”
cycle cycle cycle cycle
SYM-001276
Figure 27 SRDF/A delta sets and their relationships
SRDF/A single session mode dependent-write consistency 93

Dependent-write consistency is ensured within SRDF/A by the host

adapter obtaining the active cycle number from a single location in
global memory and assigning it to each I/O at the beginning of the
I/O and retaining that cycle number even if a cycle switch occurs
during the life of that I/O.
This results in the cycle switch process being atomic for
dependent-write sequences, even though it is not physically an
atomic event across a range of volumes. As a result, two I/Os with a
dependent relationship between them can either be in the same cycle,
or the dependent I/O can be in a subsequent cycle.

SRDF/A single session mode states

Figure 28 on page 95 shows the three logical states SRDF/A can be in:
◆ Not Ready (NR)
◆ Inactive
◆ Active
Active
NR Host commands Remote consistent or
(All devices are Not
or Enginuity inconsistent
Ready on the links)
(asynchronous)
Host commands Host commands
Inactive
(synchronous, semi-
synchronous, or
adaptive copy)
SYM-001277
Figure 28 SRDF/A single session allowed state transitions
Not Ready (NR) When the SRDF environment is configured, and the SRDF links come
state (system up, all SRDF volumes are in a not ready (NR) state by default. This
startup) means that all the remote devices on the primary Symmetrix are
not-ready on the SRDF link. These devices can be made ready on the
link by issuing the commands from the host software. By doing so,
the state would transition to the inactive SRDF/A state.
Inactive state In inactive state the devices are ready on the link, SRDF/A is inactive,
and all devices work in their assigned modes (synchronous,
semi-synchronous, or Adaptive Copy Write Pending/Disk Mode).
Various commands can transition SRDF/A from an active state to an
inactive state. Most of these commands maintain a dependent-write
consistent copy on the secondary Symmetrix and one of them
(deactivate) does not.
SRDF/A single session mode states 95

Active state The active state is the normal running state of SRDF/A. The
secondary Symmetrix is either consistent or inconsistent. The
consistent active state always represents a dependent-write
consistent image of the data. The inconsistent active state represents
previously owed tracks that have not yet been transferred to the
secondary Symmetrix. Dependent-write consistency is not
maintained for these owed tracks.
SRDF/A declares the secondary Symmetrix consistent once all of the
previously owed tracks from the primary Symmetrix have been
transferred to the secondary Symmetrix devices. Specifically, the last
cycle containing this data is fully copied to global memory and in the
N-2 cycle (apply delta set) on the secondary Symmetrix.

SRDF/A single session mode delta set switching

This section examines in detail, how the delta set switching works for
SRDF/A single session mode. Figures 29 through 38 assume that
SRDF/A has been activated and two cycle switches have occurred
previously. Before a primary Symmetrix cycle switch can occur two
things must be achieved:
1. Transmit delta set must have completed transferring the data to
the secondary Symmetrix system
2. Minimum cycle time must be reached
Figure 29 on page 97 displays the application write I/Os are being
collected in the capture delta set on the primary Symmetrix. The
previous cycles transmit delta set is completing the SRDF transfer to
the receive delta set, this is the N-1 copy. The secondary Symmetrix
apply delta set is being restored (data marked write pending to the
R2 devices), which is the N-2 copy.
R1 R2
N Capture Apply N-2 1. Capture delta set (DS) collects
N N-2
application write I/O
Transmit Receive
N-1 N-1
R1 R2
N N-2
Primary Symmetrix Secondary Symmetrix SYM-001265
Figure 29 Single session capture delta set collects application write I/O
Figure 30 on page 98 explains the primary Symmetrix waits for the

minimum cycle time to elapse and the transmit delta set to empty,
meaning all of the data has been transferred to the secondary
Symmetrix. Once these conditions are satisfied, the primary
Symmetrix sends a commit message to the secondary Symmetrix to
begin the secondary Symmetrix cycle switch in unison with the
primary Symmetrix cycle switch.
SRDF/A single session mode delta set switching 97

R1 R2
N N-2
Transmit Receive 2. Primary waits for the minimum cycle
N-1 time, and for the Transmit DS to empty
R1 2 R2
N a) Primary tells Secondary to commit the Receive
N-2 DS (begins Secondary step 3 in unison)
Figure 30 SRDF/A single session transmit delta set empties
The SRDF transfer is halted prior to the primary Symmetrix cycle

switch, as seen in Figure 31 on page 98.
R1 R2
N N-2
Transmit Receive 2. Primary waits for the minimum cycle
2b N-1 time, and for the Transmit DS to empty
R1 2 R2
N a) Primary tells Secondary to commit the Receive
N-2 DS (begins Secondary step 3 in unison)
b) SRDF transfer halted
Figure 31 SRDF/A single session SRDF transfer is halted prior to Primary Symmetrix
cycle switch
Figure 32 on page 98 displays the primary Symmetrix cycle switch

between the capture and transmit delta set. This is done
automatically through Enginuity because it is a single session (single
SRDF group) SRDF/A environment. Again, this is happening in
unison with the secondary cycle switch discussed in Figure 34 on
page 99.
1
1. Capture delta set (DS) collects
R1 R2 application write I/O
N Capture Apply N-2 2. Primary waits for the minimum cycle
N N-2
2c time, and for the Transmit DS to empty
Receive a) Primary tells Secondary to commit the Receive
Transmit
N 2b N-1 DS (begins Secondary step 3 in unison)
R1 2 R2
N b) SRDF transfer halted
N-2 c) Primary cycle switch occurs – Capture DS
becomes the Transmit DS
SYM-001268
Figure 32 SRDF/A single session Primary Symmetrix delta set switch

The new capture delta set is available to continue receiving new Host
I/O as seen in Figure 33 on page 99.
1
R1 R2
N Capture 2. Primary waits for the minimum cycle
Apply
N time, and for the Transmit DS to empty
2d 2c
Receive
a) Primary tells Secondary to commit the Receive
Transmit
N-1 2b N-2 DS (begins Secondary step 3 in unison)
R1 2 b) SRDF transfer halted
N R2
c) Primary cycle switch occurs – Capture DS
2d becomes the Transmit DS
Primary Symmetrix Secondary Symmetrix d) New Capture DS available for Host I/O
SYM-001269
Figure 33 SRDF/A single session new capture delta available for host I/O
Before a secondary Symmetrix cycle switch can occur two things

must be achieved:
1. Secondary Symmetrix received the commit message from the
primary Symmetrix (step 2a).
2. Apply delta set (N-2 copy) must complete its restore process
(marking the data write pending to the R2 devices).
Once the secondary Symmetrix receives the commit message from
the primary Symmetrix, the secondary Symmetrix verifies the apply
delta set has been restored (data has been marked write pending to
the R2 devices), Figure 34 on page 99. This occurs while the primary
Symmetrix is performing the cycle switch between the capture and
transmit delta sets.
1 3a
R1 R2
N Capture 2. Primary waits for the minimum cycle
Apply
N time, and for the Transmit DS to empty
2d 2c
Receive
Transmit
N R2
3. Secondary receives commit from Primary
a) Check if the data in Apply DS is restored (data
marked write pending to the R2 devices)
SYM-001270
Figure 34 SRDF/A single session secondary Symmetrix wait for apply

delta set to be restored

The next step is a delta set cycle switch on the secondary Symmetrix
between the receive (inactive) and apply (active) delta sets as shown
in Figure 35 on page 100. This preserves the dependent-write
consistent copy at the secondary Symmetrix prior to receiving the
next dependent-write consistent copy.
1 3a
R1 R2
N Capture Apply 2. Primary waits for the minimum cycle
N N-2 time, and for the Transmit DS to empty
2d 2c 3b
Receive
Transmit
N R2
b) Secondary cycle switch –
Receive DS becomes Apply DS
SYM-001271
Figure 35 SRDF/A single session secondary Symmetrix delta set switch
Figure 36 on page 100 shows a new receive delta set is available for
the SRDF transfer.
1 3a
R1 R2
2d 2c 3b a) Primary tells Secondary to commit the Receive
Transmit Receive
N-1 2b DS (begins Secondary step 3 in unison)
R1 2 3c b) SRDF transfer halted
N R2
c) New Receive DS available for SRDF transfer
SYM-001272
Figure 36 SRDF/A single session secondary Symmetrix new receive

delta set is available for SRDF
The secondary Symmetrix sends an acknowledgement to the primary

Symmetrix at this point. The data in the apply delta set begins the
restore process as shown in Figure 37 on page 101.

1 3a
3e application write I/O
R1 R2
N-2
2d 2c 3b a) Primary tells Secondary to commit the Receive
Transmit Receive
N-1 2b DS (begins Secondary step 3 in unison)
N R2
d) Secondary sends Primary
acknowledgement
e) Begin restore of Apply DS SYM-001273
Figure 37 SRDF/A single session secondary Symmetrix begins restore

of apply delta set
Figure 38 on page 101 depicts the SRDF transfer of the primary

Symmetrix transmit delta set to the secondary Symmetrix receive
delta set.
1 3a
3e application write I/O
R1 R2
N-2
2d 2c 3b
Receive
Transmit
N-1 2b, 4a N-1 DS (begins Secondary step 3 in unison)
N R2
d) Secondary sends Primary
acknowledgement
e) Begin restore of Apply DS
4. Primary receives acknowledgement of Secondary
cycle switch
a) SRDF transfer begins SYM-001274
Figure 38 SRDF/A single session primary Symmetrix begins SRDF transfer

SRDF/A single session mode state transitions

This section provides a general overview of how SRDF/A single
session mode moves between states discussed in SRDF/A Single
Session Mode States. Use Figure 39 on page 102 as a reference for
“Switching to SRDF/A mode” on page 102.
Synchronous SRDF-A
Adaptive copy
disk
Adaptive copy
Adaptive copy
pend off
WP
and SRDF-A
SYM-001278
Figure 39 SRDF/A single session transition path
Switching to SRDF/A The host software can be used to switch to asynchronous mode.
mode
Note: SRDF/A is an SRDF group-level feature, meaning that all devices
assigned to an SRDF group that is configured to operate in asynchronous
mode operate in asynchronous mode when the SRDF/A state is active.
SRDF verifies that all the devices are ready, and then moves the
system to active state (both primary and secondary Symmetrix). As a
result, the delta sets are established on the primary and secondary
Symmetrix and the SRDF/A mechanism is enabled.
This chapter assumes that tolerance mode is set to off.
Transition from When switching from SRDF synchronous mode, where all of the
synchronous to devices are in sync, the secondary Symmetrix shows a consistent state
asynchronous and the data is dependent-write consistent.
If there were previously owed tracks to be copied, there is no
dependent-write consistency at the secondary Symmetrix until the
last owed track has been sent to the secondary Symmetrix and is in
the N-2 cycle (apply delta set). This happens when the DA director
places the track owed to the secondary Symmetrix in the capture
delta set (s) and SRDF/A cycle switching occurs until that track is in

the apply delta set. Once this occurs, SRDF/A indicates the state is
consistent, which means the data is dependent-write consistent.
Note: It is recommended to capture a dependent-write consistent copy

(locally and/or remotely) on a set of BCVs or Clones prior to performing this
process.
Transition from When the mode is set to SRDF/A from adaptive copy write pending
adaptive copy mode all devices are moved into the adaptive copy pending off
write-pending mode. With SRDF/A active, when the DA scans a device in the
mode to pending off mode, rather than creating a separate SRDF queue
asynchronous record, it adds the slot to the active cycle (capture delta set). If there
are not any slots left write pending to the SRDF mirror that are not in
the SRDF/A cycle, the device can transition out of the pending off
mode. Once all devices transition out of pending off mode, two cycle
switches are required for the secondary Symmetrix to report a
consistent state and have the data be dependent-write consistent.
Transition from Transition into SRDF/A mode from adaptive copy disk mode is
adaptive copy disk immediate. Tracks owed to the secondary Symmetrix as a result of
mode to SRDF/A adaptive copy disk skew are scheduled as resynchronization
operations. These are copy I/Os scheduled by the disk adapter to be
serviced by SRDF/A. Each cycle switch (new capture delta set) limits
the copy I/Os to 30,000 tracks to avoid using all of the cache in the
primary Symmetrix. Host I/Os continue to be serviced in the current
SRDF/A cycles (capture delta set). The length of time to send the
tracks owed with asynchronous mode depends on the number of
outstanding tracks owed prior to switching to asynchronous mode.
For example, 90,000 tracks owed take a minimum of three SRDF/A
cycle switches to transmit the data. There are another two cycle
switches required to ensure the data is in the apply delta set, or the
N-2 copy of data. SRDF/A produces a consistent state on the
secondary Symmetrix and a dependent-write consistent copy of data
after all resync operations are complete and the two additional cycle
switches have occurred.
SRDF/A single session mode state transitions 103

Switching to SRDF/S It is possible to transition to a synchronous state from SRDF/A

mode from SRDF/A without losing dependent-write consistency for Enginuity 5x71 and
single session mode above. This is only allowed for SRDF/A single session mode. The
following caveats apply:
◆ The transition is not immediate. Once a transition is requested, it
may take some time for SRDF/A mode to be dropped and
replaced with synchronous.
◆ Some performance degradation occurs with synchronous mode
while the transition takes place.
◆ Requires both Symmetrix systems to be running Enginuity 5x71
or higher
Note: It is recommended to capture a dependent-write consistent copy

(locally and/or remotely) on a set of BCVs or Clones prior to performing this
process.
Coming out of the This section refers to the SRDF/A active state that is consistent,
SRDF/A active state meaning the data is in a dependent-write consistent state. This means
tolerance mode is off.
SRDF/A supports several methods of dropping out of the active state
into the Not Ready state. In order to maintain a dependent-write
consistent copy two options are discussed; DROP and PEND-DROP.
Note: It is recommended to capture the resulting dependent-write consistent

data with either a set of BCVs or Clones prior to any resynchronization.
During the resynchronization activity the dependent-write consistent image
at the secondary Symmetrix is compromised.
The third option is simply remove SRDF/A from the active state and
transition the SRDF mode to another state without preserving the
dependent-write consistency at the secondary Symmetrix. This is not
a recommended option.
Note: If you are in an active state with an inconsistent secondary Symmetrix,

meaning you are still transferring accumulated copy I/Os, you are not able to
create a dependent-write consistent image on the secondary Symmetrix with
either method of dropping SRDF/A.

Dropping SRDF/A This option puts the devices in a Not Ready state immediately and
single session mode the current cycle does not complete. This results in tracks being
converted to tracks owed on both the primary and secondary
Symmetrix systems of the SRDF relationship. Resuming SRDF would
then require resolving the tracks in the normal way. Dropping out of
asynchronous mode does not compromise the dependent-write
consistency of the data at the secondary Symmetrix.
Enginuity initiated Enginuity may also drop an SRDF/A single session mode for a
drop number of reasons:
◆ Primary Symmetrix system write pending limit reached
• Bandwidth not sized properly
• Global memory not sized properly
• Workload allocation for the specific implementation
• Secondary Symmetrix device write pending limit reached –
unbalanced configuration
• Any combination of the above
◆ Secondary Symmetrix devices made NR on the link
• By host command
• Due to excessive device or link errors
◆ All links lost
• Manually
• External network or network equipment issues
• Excessive link errors
These all result in a dependent-write consistent image being
preserved on the secondary Symmetrix.
Pend-dropping PEND-DROP puts the devices in a Not Ready state only at the end of
SRDF/A single the current in-process cycle. Write-pending tracks in the active cycle
session mode are converted to tracks owed on the primary Symmetrix only. By
dropping SRDF/A on the cycle boundary, PEND-DROP, there is not a
need to resolve owed tracks upon resuming SRDF/A.
SRDF/A single session mode state transitions 105

Deactivating SRDF/A mode offers the option of moving out of the active state
SRDF/A single immediately while leaving the SRDF devices ready on the link.
session mode Because the devices are left ready on the SRDF link, data continues to
flow and the dependent-write consistency of the data at the
secondary Symmetrix is compromised. The capture and transmit
delta sets data are marked as owed tracks to the secondary
Symmetrix similar to a resync operation. These tracks owed are not
dependent-write consistent.

SRDF/A single session cleanup process

Once SRDF/A single session mode is dropped, a cleanup process
occurs automatically within Enginuity. The primary Symmetrix
marks new incoming writes as being owed to the secondary
Symmetrix. The primary Symmetrix does a “cleanup” of the delta
sets; capture and transmit. The capture and transmit delta sets are
discarded, but the data is marked as being owed to the secondary
Symmetrix. All of these tracks owed are sent to the secondary
Symmetrix once SRDF is resumed if the copy direction desired is
primary to secondary.
The secondary Symmetrix marks and discards the receive delta set.
This data is marked as tracks owed to the primary Symmetrix. Once
SRDF is resumed, these tracks are scheduled to be sent from the
primary Symmetrix if the copy direction has not changed.
The secondary Symmetrix makes sure the apply (N-2) delta set is
safely applied to disk; this is the dependent-write consistent image.
It is very important to capture a “gold” copy of the dependent-write
consistent data on the secondary Symmetrix R2 devices prior to any
resynchronization. Any resync process compromises the
dependent-write consistent image. The “gold” copy can be captured
on a remote set of BCVs or Clones.
SRDF/A single session cleanup process 107

SRDF/A single session mode recovery scenarios

This section briefly discusses the different recovery scenarios
associated with SRDF/A single session mode.
Temporary link loss If SRDF/A suffers a temporary loss (<10 seconds by default) on all of
the SRDF links, the SRDF/A state remains active and data continues
to accumulate in global memory. This may result in an elongated
cycle, but the secondary Symmetrix dependent-write consistency is
not compromised and the primary and secondary Symmetrix device
relationship are not suspended. The amount of time SRDF waits until
it declares a link loss permanent is configurable.
! CAUTION
Customers switching to SRDF/S mode with the link loss amount
configured for more than 10 seconds could experience an
application, database, or host failure if SRDF is restarted in
Synchronous or Semi-Synchronous mode.
Permanent link loss If SRDF/A experiences a permanent link loss, it drops all of the
devices on the link to not ready state. This results in all data in the
active and inactive primary Symmetrix cycles (capture and transmit
delta sets) being changed from write pending for the remote mirror to
owed to the remote mirror. In addition, any new write I/Os on the
primary Symmetrix system result in tracks being marked owed to the
remote mirror. All of these tracks are owed to the secondary
Symmetrix once the links are restored.
The secondary Symmetrix inactive cycle (receive delta set) data is
marked owed to the remote mirror. These are owed to the primary
Symmetrix. The active cycle (apply delta set) data completes its
commit to the secondary Symmetrix devices.
When the links are restored, normal SRDF recovery procedures are
followed. The track tables are compared and merged based on
normal host recovery procedures used by EMC host software. The
data is then resynchronized by sending the owed tracks as part of the
SRDF/A cycles.

Note: The data on the secondary Symmetrix devices is always

dependent-write consistent in SRDF/A active/consistent state, even when
the SRDF links have failed. However, the act of starting a resynchronization
activity compromises the dependent-write consistency until the
resynchronization is fully complete and two cycle switches have occurred.
For this reason, it is recommended a “gold” copy of the dependent-write
consistent image be saved using either a set of BCVs or Clones on the
secondary Symmetrix.
Primary Symmetrix It is possible that an imbalance may occur with SRDF/A between the
global memory full incoming write I/O workload and the outgoing SRDF/A bandwidth,
condition or the inability to de-stage data quickly enough at the secondary
Symmetrix, such that the global memory in the primary Symmetrix
becomes full. The inactive and active cycles (capture and transmit
delta sets) on the primary Symmetrix consume all the available write
memory in the Symmetrix system.
In this situation, SRDF/A behaves based on customer-configurable
settings:
◆ The primary Symmetrix system can throttle the host at the speed
of the links, and keep SRDF/A running. In this case the host
performance is equivalent to synchronous mode.
◆ The primary Symmetrix system throttles the host for a
customer-defined specified period of time, and if the condition
has not resolved itself at the expiration of this time, then SRDF/A
is dropped. The default behavior is to drop SRDF/A immediately
when this condition occurs
Note: To avoid a memory full condition, the SRDF/A environment must be

properly designed and configured. Factors and variables that may cause an
imbalance in the SRDF/A environment may include bandwidth, global
memory, unbalanced primary and secondary Symmetrix configuration, and
workload allocation for specific implementations. Contact your EMC
Customer Service Representative to initiate a study of your environment to
avoid such an imbalance.
In an effort to assist customers to better manage global memory full

conditions, Enginuity 5x71 introduced Tunable Cache Utilization,
(section Enginuity 5671).
SRDF/A single session mode recovery scenarios 109

Failback from In the event that a disaster occurs on the primary Symmetrix, the data
secondary on the secondary Symmetrix devices represents a dependent-write
symmetrix devices consistent image of data that can be used to restart an environment
with minimal data loss. Once the primary Symmetrix has been
repaired, the process for returning to the primary Symmetrix uses
exactly the same methods as are used for synchronous SRDF failback
operations.
Once the workload has been transferred back to the primary
Symmetrix hosts, SRDF/A can be activated and normal
asynchronous mode protection can be resumed.
In the event of an extended failover event, the SRDF/A configuration
can be reversed using either Dynamic SRDF or a configuration
change. SRDF/A can continue to process until a planned reversal of
direction can be performed again in order to restore the original
SRDF/A primary/secondary relationship.

SRDF/A multi-session consistency (MSC) mode

Mainframe software and Enginuity 5670.50 supports SRDF/A control
for multiple Symmetrix provided there is a single SRDF group per
Symmetrix. Beginning with Enginuity 5x71 for mainframe and Open
Systems, SRDF/A is supported in configurations where there are
multiple primary Symmetrix and/or multiple primary Symmetrix
SRDF groups connected to multiple secondary Symmetrix and/or
multiple secondary Symmetrix SRDF groups. This is referred to as
SRDF/A Multi-Session Consistency or SRDF/A MSC. SRDF/A MSC
configurations can also support mixed open systems and mainframe
data controlled within the same SRDF/A MSC session.
Achieving data consistency across multiple SRDF/A groups simply
requires that the cycle switch process described earlier in this chapter
be coordinated among the participating Symmetrix SRDF group
and/or systems, and that the switch occur during a very brief time
period when no host writes are being serviced by the Symmetrix.
Achieving this requires a single coordination point to drive the cycle
switch process in all participating Symmetrix systems. This function
is provided by the SRDF control software running on the host.
SRDF/A multi-session consistency (MSC) mode 111

SRDF/A MSC mode dependent-write consistency

From a single Symmetrix perspective, I/O is processed exactly the
same way in SRDF/A MSC mode as in single session mode.
1. The active cycle on the primary Symmetrix contains the current
host writes or N data version in the capture delta set.
2. The inactive cycle contains the N-1 data version that is transferred
via SRDF/A from the primary Symmetrix to the secondary
Symmetrix. The primary inactive delta set is the transmit delta set
and the secondary Symmetrix inactive delta set is the receive
delta set.
3. The active cycle on the secondary Symmetrix contains the N-2
data version on the apply delta set. This is the guaranteed
dependent-write consistent image in the event of a disaster or
failure.
Figure 40 on page 112 illustrates the delta sets and their relationships.
R1 Apply R2
Capture
N N-2
Transmit Receive
N-1 N-1
R1 R2
Capture Apply
N N-2
R1 R2
Transmit Receive
N-1 N-1
Capture Apply
N N-2
R1 R2
Transmit Receive
N-1 N-1
Capture Transmit Receive Apply

“Active” “Inactive” “Inactive” “Active”
cycle cycle cycle cycle
SYM-001275
Figure 40 SRDF/A MSC delta sets and their relationships

Entering SRDF/A For the host to control the cycle switch process, the Symmetrix
multi-session systems must be aware that they are running in multi-session
consistency consistency mode. This is done via the SRDF control software
running on the host.
The host software:
◆ Coordinates the cycle switching for all SRDF/A sessions
comprising the SRDF/A MSC configuration
◆ Monitors for a failure to propagate data to the secondary
Symmetrix devices and drops all SRDF/A sessions together to
maintain dependent-write consistency
◆ Performs MSC cleanup if able
Note: Simply activating SRDF/A does not place a session in multi-session

mode, and conversely exiting multi-session mode does not drop or deactivate
SRDF/A, it merely places SRDF/A in single-session mode. However, if
SRDF/A is dropped or deactivated, then multi-session mode is necessarily
terminated and would need to be reentered once SRDF/A was made active
again.
At this point SRDF/A enters MSC mode. As part of the process to

enter MSC mode, and with each cycle switch issued thereafter,
Enginuity assigns a cycle tag to each capture cycle that is retained
throughout that cycle’s life. This cycle tag is a value that is common
across all participating SRDF/A sessions and eliminates the need to
synchronize the cycle numbers across them. The cycle tag is the
mechanism by which dependent-write consistency is assured.
Figure 41 on page 114 updates the SRDF/A state diagram from single
session mode to incorporate multi-session mode for SRDF/A.
SRDF/A MSC mode dependent-write consistency 113

Multi-session
consistency
Remote site consistent
Host command or inconsistent
or Enginuity Host command
Host command
Active
Not ready (NR) Host command Remote site consistent
All devices are Not
or Enginuity or
Ready on the links
inconsistent
Host command Host command
Inactive
Synchronous or
adaptive copy modes
SYM-001279
Figure 41 SRDF/A MSC allowed state transitions
Performing a SRDF/A MSC mode performs a coordinated cycle switch during a

SRDF/A MSC very short window of time when there are no host writes being
consistent cycle completed. This time period is referred to as an SRDF/A window.
switch When the host software discovers that all the SRDF groups and
Symmetrix systems are ready for a cycle switch, it issues a single
command to each SRDF group that performs a cycle switch and
opens the SRDF/A window. The SRDF/A window is implemented
as a bit in the SRDF/A state table in global memory where the cycle
number and tag are also stored.
The table is accessed by the host adapter to obtain the cycle number
at the start of each write in single session mode. In multi-session
mode, the host adapter also checks the SRDF/A window bit, and if it
is on (an open window); it disconnects upon receiving host write IO
and begins polling the bit to see if the host software has closed the
window. While the window is open, any write I/Os that start are
disconnected and as a result no dependent-write I/Os are issued by
any host to any devices in the SRDF/A MSC group.
The SRDF/A window remains open on each SRDF group and
Symmetrix system until the last SRDF group and Symmetrix system
in the multi-session group acknowledges to the host software that the

switch and open command has been processed. At this point the host
software issues a close command for each SRDF/A group under MSC
control. As a result, dependent-write consistency across the SRDF/A
MSC group is created.
Note: Enginuity does provide a fail-safe mechanism to ensure that the

window does not remain open permanently due to a host software failure.
Enginuity closes the window itself if the host software has not closed it
within 5 seconds.
As part of this switch and open operation, the host software assigns a
cycle tag value to the active cycle (capture delta set). (This cycle tag
value is separate from the cycle number assigned internally by
SRDF/A.) This cycle tag is carried by the SRDF/A process to the
secondary Symmetrix and is used by the host software at the
recovery site to ensure that only data from the same host cycle is
applied to the secondary Symmetrix devices in each SRDF group and
Symmetrix system in the event of a disaster.
During this window, read I/Os complete normally to any devices
that have not received a write. The SRDF/A window is an attribute of
the SRDF/A group and is checked at the start of each I/O, at no
additional overhead, because the host adapter is already obtaining
the cycle number from global memory as part of SRDF/As existing
overhead.
SRDF/A MSC mode dependent-write consistency 115

SRDF/A MSC mode delta set switching

This section describes how the delta set switching works for SRDF/A
MSC mode. This next series of figures represent three SRDF/A single
sessions combined together to create a single SRDF/A MSC group.
There are 2 primary Symmetrix, one with a single SRDF/A group
and the other with 2 SRDF/A groups. The secondary Symmetrix is
the same configuration as the primary, a balanced configuration.
Figures 42 through 51 assume SRDF/A MSC has been activated and
two (2) cycle switches have occurred previously. Before a primary
Symmetrix cycle switch can occur two things must be achieved:
1. The primary Symmetrix transmit delta set must be empty.
2. The secondary apply delta set must have completed marking the
R2 devices write pending for the N-2 data.
Figure 42 on page 116 displays the current host I/O being collected
by the capture delta set on the primary Symmetrix. The primary
Symmetrix transmit delta set is continuing to send the data to the
secondary Symmetrix receive delta set. The apply delta set is
continuing to restore, or mark the data write pending to the
secondary Symmetrix R2 devices.
R1 R2
N N-2
Transmit Receive
N-1 N-1
R1 R2
N N-2
1
Capture Apply
R1 N N-2 R2
N N-2
Transmit Receive
N-1 N-1
Capture Apply
N N-2
R1 R2
N Transmit Receive N-2
N-1 N-1
Figure 42 SRDF/A MSC capture delta set collects application write I/O

Figure 43 on page 117 shows that SRDF transfer between the primary
Symmetrix transmit delta set and the secondary Symmetrix receive
delta set is complete.
R1 R2 1. Capture DS collects application write I/O

N Capture Apply N-2
N N-2
2. Primary Transmit DS complete
Transmit Receive a) Primary sends Secondary ‘ Transmit
N-1 complete ’ message
R1 2
R2
N N-2
1
Capture Apply
R1 N N-2 R2
N N-2
Transmit Receive
N-1
2
Capture Apply
N N-2
R1 R2
N Receive N-2
Transmit
N-1
2
Figure 43 SRDF/A MSC Primary Symmetrix transmit delta set switch

is emptied
The primary Symmetrix halts the SRDF transfer and sends a ‘transmit
complete’ message to the secondary Symmetrix as shown in
Figure 44 on page 117. The secondary Symmetrix stores the
information, which is used during cleanup (if SRDF/A drops), and
sends an acknowledgement back to the primary Symmetrix.

N Capture Apply N-2
N N-2
Transmit Receive a) Primary sends Secondary ‘ Transmit
2c N-1 complete ’ message
R1 2
R2 b) Primary waits for acknowledgement
N N-2
from Secondary
c) SRDF transfer halted
1
Capture Apply
R1 N N-2 R2
N N-2
Transmit Receive
2c N-1
2
Capture Apply
N N-2
R1 R2
N Receive N-2
Transmit
2c N-1
2
Figure 44 SRDF/A MSC Primary Symmetrix halts the SRDF transfer

SRDF/A MSC mode delta set switching 117
The secondary Symmetrix apply delta set completes the restore

process by marking the data write pending to the R2 devices as
shown in Figure 45 on page 118. When finished, the secondary
Symmetrix sends a ‘restore complete’ message to the primary
Symmetrix.
1 3

N Capture Apply N-2
N 2. Primary Transmit DS complete
Transmit Receive a) Primary sends Secondary ‘Transmit
2c N-1 complete’ message
R1 2
R2 b) Primary waits for acknowledgement
N N-2
from Secondary
1 3
3. Secondary completes Apply DS (N - 2)
Capture Apply
R1 N R2 restore (data marked write pending to
N N-2 the R2 devices)
Transmit Receive
2c N-1 a) Secondary sends Primary “restore
2 complete” message
Capture Apply
R1 N
R2
N Receive N-2
Transmit
2c N-1
2
Figure 45 SRDF/A MSC Secondary apply delta set restore complete
Once the primary Symmetrix receives the restore complete message

from the secondary Symmetrix, the primary Symmetrix responds to
polls from the SRDF/A MSC host software with a “ready to switch”
condition. Again this is because the primary Symmetrix transmit
delta set is empty and the secondary Symmetrix apply delta set has
completed the restore process. The SRDF/A MSC host software
initiates a primary Symmetrix cycle switch once all of the
participating SRDF groups in the SRDF/A MSC configuration report
a “ready to switch” state.
Figure 46 on page 119 displays this primary Symmetrix cycle switch
between the capture and transmit delta set. The SRDF/A MSC host
software that is coordinating the cycle switch and the process is
explained in detail in “Performing a SRDF/A MSC consistent cycle
switch” on page 114. The write I/O is deferred long enough for the
host software to coordinate the cycle switch across all SRDF groups
and primary Symmetrix.

1 3 1. Capture DS collects application write I/O

R1 R2
N a) Primary sends Secondary ‘Transmit
Capture Apply N-2
N complete’ message
5a
Transmit Receive b) Primary waits for acknowledgement
N 2c N-1 from Secondary
R1 2
R2 c) SRDF transfer halted
N N-2
3. Secondary completes Apply DS (N - 2)
restore (data marked write pending to
1 3 the R2 devices)
Capture Apply a) Secondary sends Primary “restore
R1 N R2
N 5a N-2 complete” message
Transmit Receive
N N-1 4. At next host poll, Primary will respond
2c
2 “ready to switch” (Transmit complete
Capture Apply
N and Apply restore complete both true)
R1 5a R2
N 5. “Switch/Open ” receive from host
Transmit Receive N-2
N 2c N-1 a) Primary cycle switch occurs while I/O
2 deferred – Capture DS becomes the
Primary Symmetrix Secondary Symmetrix Transmit DS
SYM-001259
Figure 46 SRDF/A MSC Primary Symmetrix cycle switch while

I/O is deferred
The primary Symmetrix releases the deferred I/O and a new capture
delta set accepts the host I/O as shown in Figure 47 on page 119. The
transmit delta set contains the N-1 copy of dependent-write
consistent data.

R1 R2
N a) Primary sends Secondary ‘Transmit
Capture Apply
N complete’ message
5b 5a
N-1 2c N-2 from Secondary
R1 2
N R2
5b 3. Secondary completes Apply DS (N - 2)
1 3 the R2 devices)
Capture Apply a) Secondary sends Primary “restore
R1 N
N 5a R2 complete” message
Transmit Receive
5b N-1 N-2 4. At next host poll, Primary will respond
2c
Capture Apply
N and Apply restore complete both true)
R1 5a
N R2 5. “Switch/Open ” receive from host
Transmit Receive
5b N-1 2c N-2 a) Primary cycle switch occurs while I/O
b) I/O released - New Capture DS available
for Host I/O
SYM-001260
Figure 47 SRDF/A MSC I/O is released and a new capture delta

set continue to accept Host I/O

The primary Symmetrix sends a commit message to the secondary

Symmetrix once the primary Symmetrix systems cycle switch occurs.
After receiving the commit message the secondary Symmetrix
systems performs a cycle switch between the receive and apply delta
sets, as shown in Figure 48 on page 120.

R1 R2
N Apply a) Primary sends Secondary ‘Transmit
Capture
N N-2 complete’ message
5b 5a 6a
N-1 2c N-2 from Secondary
R1 2
N R2
1 3 the R2 devices)
Capture Apply
N-2
a) Secondary sends Primary “restore
R1 N
N 5a 6a R2 complete” message
Transmit Receive
2c
Capture Apply
N N-2 and Apply restore complete both true)
R1 5a 6a
Transmit Receive
5b N-1 2c N-2 a) Primary cycle switch occurs while I/O
for Host I/O
6. Secondary receives commit message
from Primary
a) Secondary cycle switch – Receive DS
becomes Apply DS
SYM-001261
Figure 48 SRDF/A MSC Secondary Symmetrix cycle switch
The secondary Symmetrix now has a new receive delta set available
as shown in Figure 49 on page 120.

R1 R2
Capture
5b 5a 6a
Transmit
b) Primary waits for acknowledgement
Receive
N-1 2c from Secondary
R1 2 6b c) SRDF transfer halted
N R2
1 3 the R2 devices)
Capture Apply
N-2
R1 N
Transmit Receive
5b N-1 4. At next host poll, Primary will respond
2c 6b
Capture Apply
R1 5a 6a
Transmit Receive
5b N-1 2c a) Primary cycle switch occurs while I/O
2 6b
deferred – Capture DS becomes the
for Host I/O
from Primary
becomes Apply DS
b) New Receive DS available for SRDF transfer
SYM-001262
Figure 49 SRDF/A MSC Secondary new receive delta set is available

In Figure 50 on page 121, the SRDF transfer process beginning from

the primary Symmetrix systems to the secondary Symmetrix systems.

R1 R2
Capture
5b 5a 6a
N-1 2c, 6c N-1 from Secondary
R1 2 6b c) SRDF transfer halted
N R2
1 3 the R2 devices)
Capture Apply
N-2
R1 N
Transmit Receive
2 2c, 6c 6b
“ready to switch” (Transmit complete
Capture Apply
R1 5a 6a
Transmit Receive
5b N-1 2c, 6c N-1 a) Primary cycle switch occurs while I/O
2 6b
for Host I/O
from Primary
becomes Apply DS
c) SRDF transfer begins
SYM-001263
Figure 50 SRDF/A MSC Primary Symmetrix systems begin SRDF transfer
The secondary Symmetrix systems also begin the apply delta set
restore process and the process begins again as shown in Figure 51 on
page 121.
1 6d 1. Capture DS collects application write I/O

R1 R2
Capture N-2
5b 5a 6a
N-1 2c, 6c N-1 from Secondary
R1 2 6b
R2 c) SRDF transfer halted
N N-2
1 6d
the R2 devices)
Capture Apply
N-2
R1 N R2
N 5a 6a complete” message
N-2
Transmit Receive
2 2c, 6c 6b
“ready to switch” (Transmit complete
Capture Apply
R1 5a 6a R2
N 5. “Switch/Open ” receive from host
Transmit Receive N-2
5b N-1 2c, 6c N-1 a) Primary cycle switch occurs while I/O
2 6b
for Host I/O
from Primary
becomes Apply DS
c) SRDF transfer begins
d) Begin Secondary Apply DS restore
SYM-001264
Figure 51 SRDF/A MSC Secondary Symmetrix begins the apply

delta set restore process

SRDF/A MSC session cleanup process

When SRDF/A is deactivated or dropped while in multi-session
consistency mode, each R1 Symmetrix system starts the same cleanup
process as in single session mode – it discards all I/O from both
transmit and capture delta sets and marks the corresponding tracks
owed to the secondary Symmetrix.
The host software does not need to perform any special recovery on
the primary Symmetrix.
Enginuity at the secondary Symmetrix completes the restore of its
apply delta set automatically. For each SRDF group, Enginuity
discards any receive delta sets that are not complete. If the receive
delta set is a complete delta set for each SRDF group Enginuity marks
it as “needing cleanup” in cache, awaiting a decision from the host
software.
This is where SRDF/A MSC uses the host cycle tags. The host
software must use its cycle tags during recovery of the receive delta
sets on the secondary Symmetrix. There are three different scenarios
to be considered when SRDF/A has been terminated with respect to
the receive delta sets on the secondary Symmetrix. This is determined
by the following rules:
In the first case, all receive delta sets on all secondary Symmetrix
systems and SRDF groups have the same tag and are marked as
“needing cleanup”. Remember “needing cleanup” is Enginuity
stating the receive delta set is complete. This is the result of the
secondary Symmetrix receiving and acknowledging the ‘transmit
complete” message in step 2 of the SRDF/A MSC cycle switch
process.
In this case, the host software may choose to either commit all of the
receive delta sets or discard all of the receive delta sets. The default
behavior commits all of the receive delta sets to deliver the most
current dependent-write consistent data to the secondary Symmetrix
devices.
In the second case, all receive delta sets on all Symmetrix systems
have the same tag number, but at least one Symmetrix systems or
SRDF/A group does not have a receive delta set marked “needing
cleanup”. This means Enginuity discarded an incomplete receive
delta set.
The host software must discard all receive delta sets for this tag
number in this case. The most current data is already on the

secondary Symmetrix devices via the apply delta set. The data that
was in the discarded receive delta sets are marked as tracks owed to
the primary Symmetrix devices.
The third case occurs where there are different cycle tags within the
apply and receive delta sets. In this case, the secondary Symmetrix
can be divided into two groups. The first group has Symmetrix
systems with apply delta set cycle tags that match the receive delta
set cycle tags of the Symmetrix systems from the second group. In
other words, Symmetrix systems from the first group have received
the commit message for a certain host cycle, while the Symmetrix
systems from the second group have not. In this case, the receive
cycles of the Symmetrix systems from the second group are
necessarily complete and the host software must force their restore.
At the same time, host software must discard the receive delta sets of
the Symmetrix systems from the first group regardless of their
completeness.
SRDF/A MSC session cleanup process 123


Invisible Body Tag
5
SRDF/Star Operations
This chapter provides a technical overview of SRDF/Star. The

following topics are covered:
◆ SRDF/Star overview ....................................................................... 126
◆ How SRDF/Star works ................................................................... 129
SRDF/Star Operations 125

SRDF/Star overview
Available at Enginuity level 5x71 for mainframe and open systems
environments, SRDF/Star is a solution that operates in a concurrent
SRDF configuration (A-to-B and A-to-C) where one remote mirror
operates in SRDF/S mode (A-to-B) and the other remote mirror
operates in SRDF/A mode (A-to-C).
SRDF/Star provides for rapid reestablishment of cross-site protection
in the event of primary site (A) failure. Rather than a full
resynchronization between sites B and C, SRDF/Star provides a
differential B to C synchronization, dramatically reducing the time it
takes to remotely protect the new production site. SRDF/Star also
provides a mechanism to determine which site (B or C) has the most
current data in the event of a rolling disaster that affects site A. In all
cases, you maintain the ability to choose both which site to operate
from and which site’s data to use when recovering from a primary
site failure. Figure 52 on page 127 shows a SRDF/Star configuration.
It is strongly recommended that all SRDF devices be locally protected
and that there is capacity allocated for one replica (BCV, Snap, or
clone) at both of the remote sites in the configuration.

Primary Site (A)

Production Site Local Site (B)
SRDF/Synchronous
R1 R2
SRDF/Asynchronous
Active
Inactive
Remote Site (C)
R2
BCV
Figure 52 Concurrent SRDF configured for SRDF/Star support
The concurrent configuration option of SRDF/A offers the ability to

restart your environment at long distances with minimal data loss,
while simultaneously providing a zero data loss restart capability at a
local site. Such a configuration provides protection for both a site
disaster and a regional disaster while minimizing performance
impact and loss of data.
In a concurrent SRDF/A configuration without the SRDF/Star
functionality, the loss of the primary A site would normally mean that
the long distance replication would stop and data would no longer
propagate to the C site. Data at C would continue to age as
production was resumed at site B. Resuming SRDF/A between sites
B and C would require a full resynchronization to renewable disaster
recovery protection. This is both a time and resource consuming
process.
SRDF/Star overview 127

SRDF/Star avoids this full resynchronization by allowing a B-to-C (or

C-to-B) resynchronization to be done differentially using host
software commands and procedures.
In addition, SRDF/Star allows the SRDF personalities of sites A and B
to be swapped and the SRDF/A relationship to be transferred to sites
B and C during planned outages with no data movement at all.
SRDF/Star benefits SRDF/Star provides the following features and benefits:

◆ The ability to maintain business continuance despite the loss of
any site in a three-site configuration
◆ The ability to resume asynchronous protection between the two
secondary sites, with minimal data movement, in the event of a
primary site failure
Known Known requirements and limitations for this release of SRDF/A are
requirements and as follows:
limitations at this ◆ All SRDF/Star SRDF device pairs must be the same geometry and
release size.
◆ Only Fibre or GigE SRDF links are supported.
◆ All SRDF Groups (even inactive groups) must be defined prior to
entering SRDF/Star mode.
◆ SRDF/Star is defined by the host software. Refer to the
appropriate host platform software documentation.
Note: The R2 site becomes an R1 as a function of site switchovers.
Note: For more information on specific restrictions, refer to the Solutions

Enabler Symmetrix SRDF CLI Product Guide and the Symmetrix SRDF Host
Component Product Guide.

How SRDF/Star works

The SRDF/Star solution applies to a three site configuration (A, B, C)
providing for differential resynchronization between sites B and C
and a restart at either site.
◆ Normal operation — This refers to function provided by the
multi-session consistency task running in the host. MSC manages
and manipulates Symmetrix data structures and SRDF features to
enable differentia B-to-C or C-to-B resynchronization in the
SRDF/Star configurations.
◆ Planned Switchovers — This refers to host based automation
executed at a remote site to effect reconfiguration of SRDF/Star in
support of planned site switchovers between the synchronous
SRDF sites.
◆ Unplanned Failovers — This refers to host based automation
executed at the remote site that achieves the reconfiguration.
Descriptions of states and events in this chapter are from the R1 point
of view unless otherwise noted. Refer to Figure 53 on page 131 for all
scenarios.
The initial configuration is site A has R1 devices in a SRDF/S
relationship with site B and in a concurrent SRDF/A relationship
with site C. SRDF Consistency Group is required between sites A and
B. SRDF/Star requires an active host at site A. A host is also required
at either site B or site C for recovery processing following a site A
failure. BCVs are required at site B, and site C. If planned switching
between site A and site B is going to be performed, supported BCVs
are also required at site A. The BCVs retain any consistency images
created either during a planned or unplanned event.
SRDF/Star provides processes and procedures to reconfigure
SRDF/S and SRDF/A, provide differential resynchronization
between sites B and site C, and manage planned and unplanned
failover from site A to site B or site A to site C. Restart may occur at
either site B or site C so resynchronization procedures are presented
for each case.
The solution provides a mechanism to determine whether site C or
site B has the most current data. Data resynchronization is possible
from either site B or site C to the other site, and is a decision that is
made by the user. SRDF/Star also provides a mechanism to
How SRDF/Star works 129

determine when current active R1 cycle (capture) contents reach the

active R2 cycle (commit).
SRDF/Star control for Normal operation of SRDF/Star is controlled by the host based
mainframe Multi-Session Consistency (MSC) task at the R1 site. MSC performs
the session management at the SRDF/S R2 site and when necessary
at the SRDF/A site C. MSC session management maintains the
information needed to perform differential synchronization between
site B and site C. Other host software including SRDF Host
Component, Consistency Groups, support utilities, automation
utilities, and documented procedures are used to accomplish
resynchronization and manage the reconfigurations. Automation for
some basic procedure operations is provided with SRDF/Star and is
supported by EMC if unaltered.
Mainframe customers wishing to script additional suggested
procedures and/or extend them could use any number of automation
products including native REXX or EMCSPE.
SRDF/Star control for Solutions Enabler Software along with the SRDF daemon control
Open Systems operations of SRDF/Star at the R1 site with the use of a user created
composite group. This will include the required session management
at the SRDF/S R2 site and when necessary at the SRDF/A R2 site.
The host software and Enginuity maintain the information needed to
perform the differential synchronization between the synchronous
and asynchronous secondary sites. The symstar command was
created to control, manage, and automate the SRDF/star processes in
an Open System environment.

Site A Site B
H1 H2
P1
P2
Sym1 L1(Sync) Sym2
H1 - Host 1
H2 - Host 2
H3 - Host 3
L3(Async)
L(
2 A
syn
c)
P1 - Host to Sym path 1
P2 - Host to Sym path 2
P3 - Host to Sym path 3 Site C
L1 SRDF link 1 in synchronous mode (SRDF/S) Sym3

L2 SRDF link 2 in asynchronous mode (SRDF/A)
L3 SRDF link 2 in asynchronous mode (SRDF/A)
P3
Sym1 - Symmetrix 1
Sym2 - Symmetrix 2
Sym3 - Symmetrix 3
Each Synchronous R2 has 2 SDDF sessions H3
Each Asynchronous R2 has 1 SDDF session
Figure 53 SRDF/Star configuration reference
SRDF/Star EMC provides host-based automation for both planned switchover

automation for actions and unplanned failover actions. This automation executes in a
mainframe z/OS host and is written in REXX, using the EMC Stored Procedure
Executive automation tool. Detailed descriptions and
implementation guidelines can be found in SRDF/Star for z/OS - An
Implementation and Usage Primer.
How SRDF/Star works 131

SRDF/Star EMC provides host-based automation, via the symstar command, for
automation for normal, transient fault, unplanned switch, and planned switch
open systems operations in the Open Systems environments. This automation is
delivered in Solutions Enabler with the SRDF/Star license. Detailed
descriptions and implementation guidelines can be found in EMC
Solutions Enabler Symmetrix SRDF Family CLI Product Guide. Other
licenses are also required to maintain the dependent-write
consistency.

Index
A data vaulting 28
adaptive copy database management systems 20
disk mode 18, 57 DBMS 21
write-pending mode 18, 56 dependent write operations 21
adaptive copy mode 32, 33, 56 device states 49
ATM links 47 domino mode 58
attributes, system-level 59 device domino mode 58
link domino mode 58
dual-directional configuration 47
B dual-directional link protocol 47
backups without a remote host 77 dynamic spare 38, 39
BCV 39 Dynamic Sparing 24, 44
as a primary (source) device 75, 76 Dynamic Sparing with mirrored pairs 44
concurrent operations 73 Dynamic Sparing with SRDF 44
performing remote backups 77 Dynamic SRDF 39, 40, 42, 80
SRDF multi-hop 77 dynamic SRDF devices 43
business continuance dynamic SRDF groups 43
operations 73 to 77
business continuance volumes (BCVs) 38, 75
E
E1, E3, T1, T3, and ATM links 47
C EMC compatible peer 42
channel interface states 51 EMC ControlCenter software 24
Concurrent SRDF 32, 33, 42, 60 EMC SRDF/Consistency Groups 20
configurations Enginuity 30, 32, 33, 39, 43, 48, 60
SRDF 27 to 36 ESCON 34, 40, 46, 47, 60
connectivity ESCON remote adapter (RA) 25
point-to-multipoint 31 ESCON remote director (RA) 26
point-to-point 31, 32 Ethernet infrastructure 33
D F
data protection options 24 fabric connectivity 30
data recovery 44 fully switched 30

Index
failover (secondary Symmetrix system takeover) primary volume 50, 51

71 read only 49
FarPoint 32, 34, 35, 36 secondary volumes 50, 51
performance impact 36 SRDF view 50 to 51
preserving synchronization 35 write enabled 49
Fibre Channel 60
Fibre Channel directors 43
M
Fibre Channel point-to-point connection 40
mirroring
Fibre Channel remote adapter (RF) 25
local vs. remote 22
Fibre Channel remote directors (RF) 25
modes of operation
FICON 46
primary ?? to 55
secondary 56 to 59
G Multi-hop 77
GDPS 46 multiple RA pairs 26
Gigabit Ethernet 60 multiple Symmetrix logical volumes. 44
GigE 33 Multiprotocol Channel Director (MPCD) 25, 33
GigE connection 40
GigE remote adapter (RE) 25
N
GigE remote directors (RE) 25
Native IP 33
GigE SRDF directors 43
not ready state
primary volumes 50, 51
H secondary volumes 50, 51
host accessibility 51, 52
host volume manager software 44
hypervolume 44
P
Parity RAID 24, 40
size 44
Parity RAID (3+1) 38, 39
Parity RAID (7+1) 38, 39
L PAV/MA 36
link protocols Peer-to-Peer Remote Copy (PPRC) 41, 46
dual-directional 47 Permanent Member Sparing 24
unidirectional 47 PowerPath 62
links PPRC mode 41, 44, 46
possible states 48 primary (source) volumes 38
local mirroring 22 primary (source, R1) volumes 47
local volumes 39 primary devices 29, 30, 31, 33
logical device 44, 45 primary volume 38
logical path 32 primary volumes 38, 47
logical paths 31 host accessibility 52
logical volume states 49 host view 51
host accessibility 51 not ready state 50, 51
host view 51 read/write state 50
not ready 49 read-only state 51

Index
states 50, 51 hardware 25

write-enabled state 51 Host Component for z/OS 24
link states 48
monitoring and controlling 24
R
Multi-hop 77
RAID 1 22, 38, 39
over Fibre Channel 30
RAID 10 38, 39, 44, 45
overview 16
RAID 5 24
Symmetrix interfamily connectivity 23
RAID 5 (3+1) 38, 39
unidirectional configurations 68
RAID 5 (7+1) 38, 39
volumes states 50
read operations 69 to 70
SRDF/A 16, 18, 33, 48
read/write state
SRDF/AR 17, 19
primary volumes 50
SRDF/Asynchronous 16, 18, 41
secondary volumes 51
SRDF/Automated Replication 17, 19
read-only state 51
SRDF/CG 17, 20
secondary volumes 50
SRDF/Cluster Enabler 17
recovery operations 71 to 72
SRDF/Consistency Groups 17, 62 to??
remote mirroring 22
SRDF/Data Mobility 16, 19
SRDF/DM 16, 19
S SRDF/S 16, 18, 62
SAN 30 SRDF/Star 17
secondary (target) volumes 47 SRDF/Synchronous 16
secondary devices 29, 30, 31 Storage Area Network (SAN) 30
secondary Symmetrix system takeover 71 swap SRDF devices 80
secondary volumes 39, 47 switched Fibre Channel configuration 30
host accessibility 52 switched Fibre Channel fabric connection 40
host view 51 Switched SRDF 30, 42
not ready state 50, 51 Symmetrix
read/write state 51 RAID 10 45
read-only state 50, 51 Symmetrix 3XXX/5XXX systems 33
states 50, 51 Symmetrix 8000 systems 33
semi-synchronous mode 18, 54 Symmetrix DMX systems 18, 33
Solutions Enabler software 24 Symmetrix DMX1000, DMX2000, and DMX3000
SRDF systems 26
basic configuration 22 Symmetrix model numbers 17
bidirectional configuration 47 Symmetrix Remote Data Facility
concurrent operations 73 Refer to SRDF
configurations 27 to 36 Symmetrix Remote Data Facility/Synchronous
control operations 18
swap 80 synchronous mode 33
devices 40 system-level attributes 59
director hardware 25
director/adapter board sets 25 T
dual-directional configurations 68
TCP connections 34
FarPoint 34
TimeFinder
FarPoint, performance impact 36
operations (establish, split, restore) 76
groups 40, 42

Index
U
unidirectional link protocol 47
UNIX 24
V
volume types 38 to 45
W
WAN 33
Windows 24
write operations 68
write-enabled state 51
Z
z/OS 24

Symmetrix SRDF Product Guide

Uploaded by

Copyright:

Available Formats

Symmetrix SRDF Product Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Symmetrix SRDF Product Guide

Uploaded by

Copyright:

Available Formats

EMC® Symmetrix® Remote

Data Facility (SRDF)

Published February, 2007

2 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

Chapter 1 Introducing SRDF

Chapter 2 SRDF Technical Concepts

Secondary (target) volumes ..................................................... 39

Chapter 3 SRDF Operations

4 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

Chapter 4 SRDF/Asynchronous Operations

EMC Symmetrix Remote Data Facility (SRDF) Product Guide 5

Chapter 5 SRDF/Star Operations

Index .............................................................................................................................. 133

6 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

1 Basic SRDF configuration............................................................................... 23

EMC Symmetrix Remote Data Facility (SRDF) Product Guide 7

29 Single session capture delta set collects application write I/O................ 97

8 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

1 Symmetrix model numbers ........................................................................... 17

EMC Symmetrix Remote Data Facility (SRDF) Product Guide 9

10 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

As part of its effort to continuously improve and enhance the performance

Audience This guide provides an overview of SRDF and SRDF family of

Related For additional information on SRDF and all Symmetrix

EMC Symmetrix Remote Data Facility (SRDF) Product Guide 11

◆ Symmetrix Remote Data Facility (SRDF) Product Guide at

12 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

EMC Symmetrix Remote Data Facility (SRDF) Product Guide 13

<> Angle brackets enclose parameter or variable values supplied by

Technical support — For technical support, go to EMC WebSupport

14 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

This chapter introduces SRDF and explains basic concepts including

SRDF product overview

16 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

This product guide refers to the Symmetrix models shown in Table 1

Table 1 Symmetrix model numbers

1/2 Bay 3330/5330 and 3630/5630 8230 DMX800 (rackmount)

1 Bay 3430/5430, 3830/3832a /5830 8430, 8530 DMX1000

3 Bay 3700/5700 and 3930/5930 8730, 8830 DMX3000

1 system bay and from DMX-3

SRDF product overview 17

SRDF/Synchronous Symmetrix Remote Data Facility/Synchronous (SRDF/S) is a

Note: “Primary modes of operation” on page 53 and “Secondary modes of

18 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

SRDF/A provides a long-distance replication solution with minimal

Note: Chapter 4, ”SRDF/Asynchronous Operations,” provides more detailed

SRDF/Data Mobility SRDF/Data Mobility (SRDF/DM) is an SRDF product offering that

Note: “Adaptive copy modes” on page 56 provide more information on

SRDF/Automated SRDF/Automated Replication (SRDF/AR) is an automation solution

Note: “Business continuance using SRDF and TimeFinder” on page 75

Compared to traditional disaster recovery solutions with their long

SRDF product overview 19

SRDF/Star Available at Enginuity level 5x71, SRDF/Star provides advanced

SRDF/Consistency EMC SRDF/Consistency Groups (SRDF/CG) is an SRDF product

20 EMC Symmetrix Remote Data Facility (SRDF) Product Guide

A dependent write is a write is not issued by an application until a

Note: “SRDF/Consistency Groups (SRDF/CG)” on page 62 provides more

SRDF/Cluster SRDF/Cluster Enabler for MSCS or VCS provides high availability

SRDF product overview 21

SRDF functional overview