Pennell Ipodwdm (IP Over DWDM

IP over DWDM
Tim Pennell WW SP Technical Operations
IPoDWDM
2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
Outline
The case for IP optimized transport Changes to the architecture of the network Cisco solution and direction Network Management Summary
IPoDWDM
Cisco Confidential
Overall Traffic Growth Is Straining All Known Network Architectures

Global IP TrafficBy Segment
Legacy applications moving to IP

Video SD, HD, Broadcast Cable Voice
New applications almost exclusively IP

Video - On Demand, DVRs, Switched Digital, Video conferencing Audio Streaming audio, Internet radio, Digital juke boxes,etc. For Perspective: 1 Exabyte = 5 X All the Worlds Printed Matter 5 Exabytes = All Words ever Spoken
IPoDWDM 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential
High-Speed Data, Internet Over-the-Top Content providersi.e., YouTube
Household Bandwidth Needs in 2010:

Applications: HDTV + SDTV + PVRs + HSD + VoIPPhones Twenty such homes would generate more traffic than traveled the entire Internet backbone in 1995
3
*Source: Cisco Estimates, Ovum, Gartner, IDC, Merrill Lynch, MRG, MPA, Public Company Data
IPoDWDM
Cisco Confidential
Where does this take us ?

Higher bandwidths are needed to address this growth:
10 Gig networks beginning to feel the strain Cannot rely on L2/L3 aggregation: Cannot rely on L1 aggregation: LAG 4 X 10G 40G DWDM ports are not unlimited
Increase wavelength capacity as soon as viable:

Move to higher data rates per lambda, i.e. 40G and 100G BUT must operate over existing infrastructure AND ideally with equivalent performance to 10G Requires advanced optical modulation schemes
Remove all unnecessary network layers leaving only:

Service layer (IP) Transport layer (DWDM)
Integrate DWDM technology on Router: IPoDWDM

Typical IP Network + Optical Network

Transponders converting short reach to
Core Router
Router blind to L0 issues
Electrical switching OEO conversions
P2P DWDM
Electrical XC Manual patching of connections
IPoDWDM
Cisco Confidential
Cisco IPoDWDM Architecture

Integrated transponders
Core Router
Router aware of L0
Optical control plane Exchanging data w routers
Photonic switching no OEO conversions
Mesh ROADM
ROADM
Cisco Confidential
IPoDWDM
Cisco IPoDWDM Strategy
Element Integration
Integrate transponder functionality onto routing platforms (10GE, 40G, G.709, EFEC) Integrating photonic switching into DWDM platforms (ROADM / WXC) CRS-1 12000 7600
Control Integration
DWDM aware GMPLS for provisioning of lambdas driven by IP control plane Coordination between layers (e.g. SRLG, NLAC)
SP Benefits
Scale & Performance Lower CAPEX/OPEX Increased Reliability Speed to Service
IP Layer
Element Integration
Control Integration
Management Integration
Management Integration
Separate or integrated management
Service Provider Core Network DWDM Layer
ONS 15454 MSTP ROADM & WXC

8
Immediate Benefits of IPoDWDM solution

Before WDM Transponders Integrated into Router Lower CapEx Elimination of OEOs Lower OpEx Space, power, management Enhanced resiliency Fewer active components Investment protection
Router Transponder ROADM Router
DWDM I/F
ROADM
40G and beyond, interoperability over existing 10G systems
IPoDWDM
Cisco Confidential
Replacing TDM layer functions

1. Easy operations (OAM&P)
G.709 overheads mimic SONET/SDH functions GMPLS allows optical layer visibility into hard to detect failures Integrated optics allows for low cost optical monitoring Control plane provides fast failure indications to optical switches Router based fast reroute (FRR) can be more economical and as fast/reliable as transport layer protection Not needed if router trunks can fill 10G wavelengths Manage bandwidth at the wavelength level using optical switches
2.
Fast protection

3.
Sub-wavelength grooming & aggregation

IPoDWDM
Cisco Confidential
10
MSTP: Leveraging the Intelligent WDM layer
Open WDM Architecture:

Transparent Transmission High-performance (EFEC, adv. mod.) Bit-rate Independent Alien-Wavelength (emerging std.) Operations Friendly G.709 OAMP, tunability, monitoring, GMPLS
ROADM Multidegree ROADM ROADM
IPoDWDM interoperability:
Network planning flexibility ROADM, Planning tools State-of-the-art performance over MSTP Field tested Alien-Wavelength over existing (3rd party) WDM Systems
11
IPoDWDM
Cisco Confidential
MSTP: Industry First Ethernet Aggregation Integrated into DWDM

Introducing Ethernet XPonders (pronounced Cross-Ponder) Single card for Ethernet Add-Drop, muxponder and transponder Innovations:
Layer-2 Ethernet aggregation of NxGE into 10GE Sub-wavelength add/drop and drop and continue 50 ms resiliency G.709 (WDMPHY): OAMP, EFEC
20xGbE and 2x10GE XPonder
4x10GE Ethernet XPonder
MEF 9 and 14 Certified
10 GE LAN PHY
10 GE G.709
20GE Ethernet XPonder
. . .
CRS-1 / GSR / 7600 IPoDWDM Interface
GE 1 LAN PHY GE 2 LAN PHY GE 3 LAN PHY GE 20 LAN PHY
4x10GbE XPonder
10 GE G.709
12
MSTP: An Open DWDM Layer Not as hard as it seems

Public references:
Cisco ONS1545 MSTP Nortel CPL (Comcast) Alcatel 1626LM (Embratel) Padtec MetroPad (Embratel) Lucent OLS400 (Dreamhack) Siemens SURPASS hiT7550 (indirectly via SL OEM) Tellabs TITAN 7100 (via published Stratalight interop) Fujitsu Flashwave (Comcast metro) Ciena CoreStream (Sprint) Huawei 80CH DWDM (China Telecom) Ericsson MHL-3000 (DT)
Other trials:
Not generally a technical barrier more a political one

13
Mama Network 40G IPoDWDM over Ciena LH DWDM

40Gig, Plug and Play Circuit Turn Up
The most difficult part of the whole project was installing Windows on Sigbritts PC,said Hafsteinn Jonsson of Karlstad Stadsnat.
Karlstad
Stockholm
405Km
IPoDWDM
Cisco Confidential
14
Recent Trends Towards IPoDWDM

Other Router Vendors Following Ciscos Lead
10GE WDMPHY demo from Juniper/Adva [OFC08 NME4]

15
Other vendors also believe this is the Future
IPoDWDM
Cisco Confidential
16
Cisco Commitment to IPoDWDM Innovation

Interface speed
Cisco 1st to productize 40G Leading 100G effort in IEEE (HSSG) 100G development work in progress 100Gbps Switch Fabric, MSC and PLIMs 10G, 40G, 100G interfaces Target for existing CRS-1 chassis with no power/cooling upgrades Reduction of power/G by a factor of 2.5 Increased line card scale & new feature development Prefixes, policers, queues, VLANs Ability to run in lower performance and/or bandwidth mode to save power - Green Mode 8% to15%
Pogo
Beluga
Tor
Seal
Crab
SuperStar
17

Interface speed
Cisco 1st to productize 40G Leading 100G effort in IEEE (HSSG) 100G development work in progress
Optical switching capability

Cisco is leading ROADM market Innovative, operationally friendly, mesh ROADMs Early to provide configurable add/drop switching Line Line Line
18
Add/drop
Passive mesh panel
IPoDWDM
Cisco Confidential

Interface speed
Cisco 1st to productize 40G Leading 100G effort in IEEE (HSSG) 100G development work in progress
Optical switching capability

Cisco is leading ROADM market Innovative, operationally friendly, mesh ROADMs Early to provide configurable add/drop switching
Control plane
Introduced partial control plane (OSPF, NLAC) Experimenting with DWDM impairment aware GMPLS
19
Careful DWDM layer design with the future in mind

Future proof grid
Must accommodate a variety of possible future modulation formats for 100Gbps and beyond Flex Spectrum instead of a fixed grid In the mean time sticking to 100GHz grid is safest
In-service upgradeable optical switching nodes

MSTP architecture easy to upgrade as more DWDM links added and more add/drop is needed
Add/drop Add/drop
Passive mesh panel
Line Line Line

20
IPoDWDM
Cisco Confidential

Future proof grid
Must accommodate a variety of possible future modulation formats for 100Gbps and beyond Flex Spectrum instead of a fixed grid In the mean time sticking to 100GHz grid is safer
In-service upgradeable optical switching nodes

MSTP architecture easy to upgrade as more DWDM links added and more add/drop is needed
Open DWDM layer

Full support for alien wavelengths in MSTP Working to extend GMPLS to support 3rd party Tx/Rx No need to restrict Tx/Rx innovation to Transport vendor roadmap
21
DWDM Layer Evolution must be Closely Tied to IP Layer Evolution
Proactive protection SRLG sharing
demonstrated, working toward product
working toward product working toward product
Coordinated maintenance Flexible bit rate investigating
trade-off bit rate for regeneration;
Optical auto bandwidth
investigating
IPoDWDM
Cisco Confidential
22
Proactive Protection
Working path SwitchProtect over = path lost data Working path Protect path
BER
FEC
FEC limit FEC limit
Transponder FEC
Corrected bits
Corrected bits
BER
SR port on router
LOF
Near-hitless switch
WDM port on router
Protection trigger
Optical impairments WDM WDM
Optical impairments
Todays protection
Proactive protection
23
Pre-FEC Preliminary results

Tested manual/fast cuts, slow OSNR and PMD degradations Tested MPLS FRR, IP FRR, ISIS convergence In all cases, achieve near zero outage for slow failures
Protection Type
Proactive Proactive Proactive Standard Standard Standard
Fault
Optical-switch (25ms) Noise-injection Fibre-pull Optical-switch (25ms) Noise-injection Fibre-pull
Packet Loss (ms)

Highest 11.48 0.12 14.97 11.61 2852 83.43 Lowest 10.99 0 0 11.16 2602 13.49 Average 11.24 0.05 4.97 11.32 2727 37.63
Noise injection 0.1 ~ 1dB / 1000ms

24
IPoDWDM
Cisco Confidential
25
IPoDWDM
Cisco Confidential
26
IPoDWDM Supports 2 network management models

1. Segmented Management:
Retain existing operational model for certain SPs Respect boundaries between IP/Transport groups End to end provisioning Better trouble shooting 1 mgmt system, 1 DB Unified look & feel Lower OPEX
IP Mgmt Transport Mgmt
WDM I/F WDM I/F
2. Integrated management:

Common EMS
WDM I/F WDM I/F
IPoDWDM
Cisco Confidential
27
The Virtual Transponder (VTXP) Concept

Virtual transponder protocol
Secure session between Router and WDM NE Supports full FCAPS for WDM i/f XML based
WDM i/fs WDM i/fs WDM i/fs Transponder WDM NE
The WDM NE converts VTXP info to its legacy information model to the EMS
Router reflected as a transponder shelf WDM i/f reflected as a transponder
IP manager
Transport manager
VTXP
Router WDM i/fs WDM i/fs WDM i/fs WDM i/fs WDM NE
No change to NMS/OSS
IPoDWDM
Cisco Confidential
28
Brief Comparison
Typical IP + DWDM Traffic addition L0-L3 coordination Availability Labour intensive Labour intensive: lots of processes Lower added OEO Lower all outages are traffic affecting Lower uncoordinated SRLGs Low unplanned maintenance is not coordinated Management Space and Power consumption
IPoDWDM
IPoDWDM Zero touch (mostly) Seamlessly achieved via control plane Higher minimum OEO Higher slow outages are hitless Higher SRLGs are in sync High even unplanned maintenance activity is coordinated Separate or integrated management, NLAC Optimal
29
Separate IP and DWDM management systems Sub-optimal

Cisco Confidential
Summary
Traffic growth requires more focus on IP routers and DWDM technology Other services must still be supported but the network is not optimized around them IPoDWDM provides the required innovation to save CAPEX, OPEX and reduce power for the network Todays features are only the beginning. We are committed to evolve the solution to a new level that is unachievable w/o integration Main challenges are non-technical. If you see the value, you can make it happen
30
Q and A
IPoDWDM
Cisco Confidential
31
IPoDWDM
Cisco Confidential
32
WDMPHY: Cost-Effective 10GE DWDM

10GE LANPHY payload over G.709 payload (over-clocked) OAM&P based G.709 Standard (SDH-like) FEC enabled transmission G.709 Standard modes, and Enhanced-FEC >1500 km 10GE LANPHY
km of Reach (in a typical WDM system)

1400 1200 1000 E-FEC
ITU standard references: G.709

Overclocking: G.sup43, sub-clause 7.1 Enhanced FEC: G.975.1 Appendix I.7
800 600 400 200 0 1 2 3 4 No FEC
FEC
5 6 7 8 9 10 11 12 13 14 15 Number of spans
33
IPoDWDM
Cisco Confidential
Network Architecture
First 40G IPoDWDM Network in the World
Seattle
The Comcast National Network is Currently:

Largest IPoDWDM Deployment in the World Largest 40G DWDM Deployment in the World
Portland Ogden
Chicago Cleveland Omaha Denver Philadelphia Indianapolis Pittsburgh DC
Boston
New York
Sacramento
San Jose
Nashville
Charlotte
To Atlanta
Los Angles
Atlanta
Dallas Santa Teresa To Dallas
Miami CRS1 Router w / 40G Integrated Optics
Additional 40G Express Links not Shown

34
IPoDWDM Interface Evolution

High Cost / complexity ? Next100G 40G Gen ?
DPSK+ integrated TDC (G+) Ext TDC 40G ODB (G) Ext PMDC
10G OOK+EFEC (Rodan) Low ~1000km 2000km 1000km 300km ~50km Old fiber (high PMD) Good fiber (post 1995)
IPoDWDM
Cisco Confidential
35
Where is 100Gig
IEEE Standard 2008 1st Draft 2009 Last change 2010 Std
Cisco is working closely with IEEE and ITU Cisco is also working in Parallel on a final product, not waiting for entire standard IEEE focused on 40Gig E and 100Gig E SR
Cisco will also implement WDMPHY
Target FCS 1HCY10
IPoDWDM
Cisco Confidential
36
Segmented Control Plane

L0 topology visibility (OSPF database) L3 IGP
RP/0/RP0/CPU0:MiniRex#sh ospf vrf optical database OSPF Router with ID (10.10.10.10) (Process ID 1)
L0 IGP
Summary ASB Link States (Area 0)
Router
10 DWDM nodes
Router
Control plane debugging (Ping)

RP/0/RP0/CPU0:MiniRex#ping vrf optical 10.85.85.200 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.85.85.200, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
Link ID ADV Router Age 10.85.85.200 10.85.85.229 182 10.85.85.200 10.85.85.238 177 10.85.85.201 10.85.85.229 182 10.85.85.201 10.85.85.238 177 10.85.85.230 10.85.85.229 182 10.85.85.230 10.85.85.238 177 10.85.85.235 10.85.85.229 182 10.85.85.235 10.85.85.238 177 10.85.85.236 10.85.85.229 182 10.85.85.236 10.85.85.238 177 10.85.85.237 10.85.85.229 182 10.85.85.237 10.85.85.238 177 10.85.85.239 10.85.85.229 182 10.85.85.239 10.85.85.238 177 10.85.85.244 10.85.85.229 182 10.85.85.244 10.85.85.238 177
Seq# Checksum 0x80000006 0x00027e 0x8000000b 0x002840 0x80000006 0x0049c7 0x8000000b 0x00cb0a 0x80000006 0x004f6d 0x8000000b 0x007f70 0x80000006 0x00b60b 0x8000000b 0x00b32d 0x80000006 0x00c163 0x8000000b 0x0095e5 0x80000006 0x00cbbc 0x8000000b 0x00779e 0x80000006 0x00658f 0x8000000b 0x00b4f0 0x80000006 0x00ae9b 0x8000000b 0x00083e
IPoDWDM
Cisco Confidential
37
Control Plane Futures

Sharing L0 SRLGs w L3 L3 IGP
1, 2, 3, 4, 5
4 1 2 3
L0 IGP 1. Provision L0 SRLGs into DWDM nodes & discover them via L0 IGP 2. Discover SRLGs per link in L0 network 3. Aggregate SRLG info & advertise SRLGs in the L3 IGP 4. Periodically check if SRLGs have changed 5. Use this info for applications such as FRR, alarms, and L3 planning
38

Planned L0 Maintenance Driving Graceful L3 Shutdown
Maintain
FMI FMI
FMI
FMI
1. User informs L0 network of a span/NE that needs to be maintained 2. Hitless Fwd Maint Indicator (FMI) sent to end points of lightpaths traversing optical NE requiring maintenance 3. FMI signal can be propagated to router, invoking proactive protection for all traffic using these lightpaths 4. Router can alarm user or send back Nack if it cannot handle this link going down at this time 5. Eliminates need to coordinate maint activity between L0 & L3
39

Optical Auto-Bandwidth Vary channel count as trunk load changes
Router measures utilization for L2 link bundles (LAG) If utilization is high request the set up of another wavelength from L0 and add it to the link bundle Original topology retained to avoid convergence issues channel add/remove within existing link bundles
40
Segmented Management via CTC

Managing the Transport Layer incl DWDM i/fs in MSTP R9.0 CTC shows CRS nodes w WDM i/fs as well as MSTP nodes CTC can set up A-Z connections (incl DWDM controller) CTC displays show DWDM controller results and alarms
OCH Trail Circuit
XML / CLI
CRS1_1 IP/MPLS
CTC
XML / CLI
CRS1_2 IP/MPLS
Corba
MSTP
MSTP WDM cloud LMP
LMP
41
Segmented Management via CTC

Show Router DWDM Controller: Port status
IPoDWDM
Cisco Confidential
42

Future proof grid
Must accommodate a variety of possible future modulation formats for 100Gbps and beyond Pushing for Flex Spectrum instead of a fixed grid In the mean time sticking to 100GHz grid is safer 50GHz grid (80 ch) is OK for 40G
Will it be optimized for 100G?
Best solution: Flex Spectrum
100GHz is a safe bet near term

43
LMP and alarm correlation details

EMS/ NMS
LMP Indicator
EMS/ NMS
LMP Indicator
Alarm
NonCritical
NonCritical
Alarm
Goal: ensure only one system must act upon a failure Value: Downstream indicator allows router alarm to be downgraded avoid L3 craft action Upstream indicator allows transport alarm to be downgraded avoid L0 craft action
44
IPoDWDM
Cisco Confidential
45

Pennell Ipodwdm (IP Over DWDM

Uploaded by

Copyright:

Available Formats

Pennell Ipodwdm (IP Over DWDM

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Pennell Ipodwdm (IP Over DWDM

Uploaded by

Copyright:

Available Formats

How is overall traffic growth straining network architectures?

How is overall traffic growth straining network architectures?

What changes are needed to network architectures to address traffic growth?

What changes are needed to network architectures to address traffic growth?

IP over DWDM

Tim Pennell WW SP Technical Operations

2008 Cisco Systems, Inc. All rights reserved.

2008 Cisco Systems, Inc. All rights reserved.

Overall Traffic Growth Is Straining All Known Network Architectures

Legacy applications moving to IP

New applications almost exclusively IP

High-Speed Data, Internet Over-the-Top Content providersi.e., YouTube

Household Bandwidth Needs in 2010:

2008 Cisco Systems, Inc. All rights reserved.

Where does this take us ?

Increase wavelength capacity as soon as viable:

Remove all unnecessary network layers leaving only:

Integrate DWDM technology on Router: IPoDWDM

Typical IP Network + Optical Network

Router blind to L0 issues

Electrical switching OEO conversions

2008 Cisco Systems, Inc. All rights reserved.

Cisco IPoDWDM Architecture

Optical control plane Exchanging data w routers

Photonic switching no OEO conversions

2008 Cisco Systems, Inc. All rights reserved.

Cisco IPoDWDM Strategy

Service Provider Core Network DWDM Layer

ONS 15454 MSTP ROADM & WXC

Immediate Benefits of IPoDWDM solution

40G and beyond, interoperability over existing 10G systems

2008 Cisco Systems, Inc. All rights reserved.

Replacing TDM layer functions

Sub-wavelength grooming & aggregation

2008 Cisco Systems, Inc. All rights reserved.

MSTP: Leveraging the Intelligent WDM layer

Open WDM Architecture:

2008 Cisco Systems, Inc. All rights reserved.

MSTP: Industry First Ethernet Aggregation Integrated into DWDM

MEF 9 and 14 Certified

CRS-1 / GSR / 7600 IPoDWDM Interface

GE 1 LAN PHY GE 2 LAN PHY GE 3 LAN PHY GE 20 LAN PHY

MSTP: An Open DWDM Layer Not as hard as it seems

Not generally a technical barrier more a political one

Mama Network 40G IPoDWDM over Ciena LH DWDM

2008 Cisco Systems, Inc. All rights reserved.

Recent Trends Towards IPoDWDM

10GE WDMPHY demo from Juniper/Adva [OFC08 NME4]

Other vendors also believe this is the Future

2008 Cisco Systems, Inc. All rights reserved.

Cisco Commitment to IPoDWDM Innovation

Cisco Commitment to IPoDWDM Innovation

Optical switching capability

Passive mesh panel

2008 Cisco Systems, Inc. All rights reserved.

Cisco Commitment to IPoDWDM Innovation

Optical switching capability

Careful DWDM layer design with the future in mind

In-service upgradeable optical switching nodes