R.

Brough Turner
NMS Communications
rbt@nmss.com
October 1, 2003
Acknowledgements

z This tutorial draws heavily on the work of

„ Chuck Byers, Lucent Technologies
„ Jim Kennedy, Intel
„ Mark Overgaard, Pigeon Point Systems
„ Michael Thompson, Pentair/Schroff
„ Henry Wong, Motorola
who jointly presented a more detailed half-day
AdvancedTCA seminar on May 21, 2003, see:
„ http://www.picmg.org/advancedTCA_Tutorial_0503.stm
z I am also indebted to Joe Pavlat, President of
PICMG, for diverse content and advice

AdvancedTCA
TM

October 1, 2003 Slide 2

Today’s Goal
z Describe the AdvancedTCA platform, and its
applications, in enough detail so that you can
make informed decisions

Outline
z AdvancedTCA background and market focus
z Technical overview
z Details, details, details
z Application examples
z Current market status
z Conclusions and Q&A

AdvancedTCA
TM

October 1, 2003 Slide 3

Evolving Telecom Market

z Much hyped voice-data convergence

„ Slower than once predicted, but happening
z Core network data rates increasing
„ Network equipment performance limited by I/O
z Enhanced service platforms leveraging
standard computers and operating systems
„ Frequently SPARC Solaris or Linux on Intel
z Tier 1 equipment providers have downsized
„ Seeking to outsource and leverage COTS technology

AdvancedTCA
TM

October 1, 2003 Slide 4

Shortfall of Existing Standards

z Board area, board spacing, power and cooling

inadequate for existing and emerging silicon
z Backplane capacity limited
z Diverse system management approaches
z Difficulty meeting availability objectives
„ Legacy bus (VME, PCI) limitations
„ Power distribution issues
z Poor match to telecom equipment practices
„ Chassis mechanical standards don’t match
equipment frames or support needed I/O
„ Systems fail to leverage telecom power architectures

AdvancedTCA
TM

October 1, 2003 Slide 5

AdvancedTCA Objectives

z Meet the evolving needs of communications

network infrastructure
„ Telecom focus
„ Edge, core and transport, as well as data center
„ Wireless, wireline and optical network elements
„ High levels of service availability (99.999% or more)
„ Scalable performance and capacity
z Reduce development time and total cost of
ownership
„ Open architecture, modular components sourced by
a dynamic, interoperable, multi-vendor market

AdvancedTCA
TM

October 1, 2003 Slide 6

AdvancedTCA Features
z Serial links & switch fabric technology
„ Redundant star or full mesh data transport, with
switch fabric alternatives, can scale to 2.5Tb/s
„ Dual redundant Ethernet control plane
z Large board area supports latest silicon
„ Power and cooling for up to 200 watts per slot
z 600 mm (~24”) ETSI frame equipment practice
„ Provision for 19” and 23” versions
z Sophisticated system management
„ Advanced software infrastructure for OAM&P
„ Health monitoring; module power control; electronic
keying; active cooling control

AdvancedTCA
TM

October 1, 2003 Slide 7

AdvancedTCA Features (cont.)

z Telecom centric I/O

„ Multi-protocol support for interfaces up to 40 Gb/s
„ Front-to-back clearances for simultaneous front and
rear I/O including fiber bend radii
„ Optional rear transition modules for rear I/O
z Availability scales to 99.999% or more
„ Single board failure domain
„ Hot swap capability for all boards & active modules
„ Redundant -48 VDC power feeds
„ Redundant control and data planes
„ Sophisticated system management

AdvancedTCA
TM

October 1, 2003 Slide 8

 Comparison With PICMG 2.X
Attribute PICMG2 / CPCI PICMG2.16 / CPSB PICMG3 / ATCA
Board Size 6U x 160mm x .8” 6U x 160mm x .8” 8U* x 280mm x 1.2”
57 sq in + 2 Mez 57 sq in + 2 Mez 140 sq in +4 Mez
Board Power 35-50W 35-50W 150-200W
Backplane Bandwidth ~4Gb/s ~38Gb/s ~2.4Tb/s
# Active Boards 21 19 16
Power System Central Converter Central Converter Distributed Converters
5, 12, 3.3V Backplane 5, 12, 3.3V Backplane Dual 48V Backplane
Management OK OK Advanced
I/O Limited OK Extensive
Clock, update, test bus No No Yes
Regulatory conformance Vendor specific Vendor specific In standard
Multi-vendor support Extensive Building Anticipated in 2003
Base cost of shelf Low Low - Moderate Moderate
Functional density of shelf Low Moderate High
Lifecycle cost per function High Moderate Low
Standard GA Schedule 1995 2001 2H2003
Source: Chuck Byers, Lucent Technologies
AdvancedTCA
TM

October 1, 2003 Slide 9

AdvancedTCA History

z Informal “Santa Barbara” group 7/2001

z PICMG 3 work group chartered 11/2001 with
over 100 companies participating
„ Subteams: Form Factor, Backplane/Fabric, RASM
„ Base spec (PICMG 3.0) for mechanical, power,
cooling, interconnect, and RASM properties
„ Sub-specs for data transport fabrics alternatives
z PICMG 3.0 spec ratified 12/30/2002
„ 430 pages; 11,000 person hours of meetings and
conference calls; Untold hours of individual work
z Real inter-operable products available today
„ Fifth inter-op workshop held September 2003
AdvancedTCA
TM

October 1, 2003 Slide 10

 Details, Details, Details...
z Mechanical
z Power & Thermal
z Management
z Data Transport
z Mezzanine Cards
z Regulatory

AdvancedTCA
TM

October 1, 2003 Slide 11

Mechanical Configuration
z 8U boards in 12U chassis
„ 16 slots in 600 mm frame
„ 14 slots in 19” cabinet
z 1.2” board pitch allows
heat sinks plus rear SMT
z Forced air cooling for up
to 200 watts per slot
z Front and rear fiber bend
area in 600 mm depth
z Simplified sheet metal
construction
z ETSI & NEBS vibration,
shock and serviceability

AdvancedTCA
TM

October 1, 2003 Slide 12

 Chassis Mechanics, Depth
z 600 mm cabinet depth
z 25 mm thick doors
z 100 mm front cable/air area
z 60 mm rear cable/air area
z 390 mm remains for chassis
z 388 faceplate to faceplate

Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 13

Sheet Metal Construction
z Lower cost in high volume
z Looser tolerances
z Earthquake performance
z 1.6 mm to 2.4 mm slots

ESD clip
Guide Rail funnel
Retention Screw receptacle
Alignment Pin receptacle

Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 14

 Board Mechanics
z Front board 8U x 280 mm
z Rear board 8U x 70 mm
„ Connects directly to front board
z Board width 6HP (1.2”)
z PCB thickness: 1.6 mm –
2.4 mm allowed
z Simplified telecom packaging
z Provisions for 4 PMCs
z Alignment/key pins

Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 15

 Face Plate
z Sheet metal solution
z EMC gasket on left side
„ Less prone to damage
z M3 retaining screw
„ No tool required
z Handle actuated micro
switch for Hot Swap
z PCB offset 0.1” for all
boards
z Alignment/ground pin on
front panel

Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 16

Rear Transition Module (RTM)
z 8U x 70 mm form factor
z Same mechanics as front
board (mirror image)
z Alignment and keying
„ To backplane
„ To front board
z Front board to RTM
connector not specified
„ Application-specific
vendor choice

Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 17

 Allowable Component Heights
z 21.33 mm maximum
component height on side 1
z Maximum component
height on side 2 depends on
the board design
z ~2.41 mm without side 2
cover and 1.6 mm PCB
z ~1.61 mm without side 2
cover and 2.4 mm PCB
z ~2.53 mm with 1 mm side 2
cover and 1.6 mm PCB
z ~1.73 mm with 1 mm side 2
cover and 2.4 mm PCB
Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 18

ESD Discharge

ESD discharge strips on bottom edge (front board and RTM)

z Segment 1: 10 Mohms to face plate
„ Discharges installer
z Segment 2: 10 Mohms to logic ground
„ Discharges board
z Segment 3: 0 ohms to face plate
„ Solid ground even if the board is not mated with the backplane
z Keep out area for isolation

Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 19

Alignment & Keying
z Four stages of alignment
„ Subrack guide rails
„ Backplane alignment/key pins
„ Front panel alignment/key pins
„ Connectors
z Front board key (A1)
z RTM key (rA1)
z Front board to RTM key (A2)
z Keys and receptacles have a
flat side
z Eight key and receptacle
positions
Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 20

 AdvancedTCA Backplane

z Zone 3 for front board to

RTM interconnection
„ Connector not defined
by ATCA; could be fiber,
coax, signal, etc.
„ Separate keying
z Zone 2 for base interface
and fabric
z Zone 1 for power and
system management
Source: PICMG

AdvancedTCA
TM

October 1, 2003 Slide 21

 Backplane
Zone 3 z Connector population
options depend upon
fabric architecture
Zone 2, J20
z Full mesh backplane
Zone 2, J21 shown here

Zone 2, J22

Zone 2, J23

Zone 2, J24 Node slots

Base Interface
Zone 1, Power Hub slots
&
Management
Source: Michael Thompson, Pentair/Schroff
AdvancedTCA
TM

October 1, 2003 Slide 22

 Details, Details, Details...
z Mechanical
z Power & Thermal
z Management
z Data Transport
z Mezzanine Cards
z Regulatory

AdvancedTCA
TM

October 1, 2003 Slide 23

 Power Requirements

z -48/-60 VDC power input

„ -40.5 to -57 VDC, -50 to -72 VDC
z Redundant power inputs
z Distribution of ringing voltages
z Capacity of over 3,200 Watts per shelf
z Local power conversion
„ DC-DC converters on each board

Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 24

 Power Distribution

Source: Michael Thompson, Pentair/Schroff

z Redundant power inputs

z Positronic power connector with sequenced pins
z Other power distribution architectures are allowed

AdvancedTCA
TM

October 1, 2003 Slide 25

Zone 1 Connector

Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 26

Board Power, Simple

Source: Michael Thompson, Pentair/Schroff

z Single DC-DC converter fed through four diodes from

both DC inputs
„ Option for (two) separate DC-DC converters fed from both
DC inputs (with or without diodes)
z Inrush current limiting and fusing

AdvancedTCA
TM

October 1, 2003 Slide 27

 Board Power, Complete

z A board may only consume 10 W until more power is

negotiated with the Shelf Manager
Source: Michael Thompson, Pentair/Schroff

AdvancedTCA
TM

October 1, 2003 Slide 28

Thermal Principles

z Central office environment

„ NEBS/ETSI; N+1 Cooling; 72 hr A/C failure means
high ambient temperatures
z 200 W per board with forced air cooling
„ 16 boards in a 12U shelf, ~ 3200 watts/shelf
„ 3 shelves per frame, ~ 10 kW frame
z Provide framework for interoperability
„ Designer, manufacturer, system integrator
„ Define thermal interfaces
z Provide design guidelines

AdvancedTCA
TM

October 1, 2003 Slide 29

Extensive Simulations Run
Computational Fluid Dynamics
z Determine “up front” if
200 W per slot was
possible
z Followed with flow
chamber measurements
z Final design supports
„ Up to 200 W per slot
„ Less than 10 °C
temperature rise, with
„ 55 °C intake air and
one failed fan

Source: PICMG

AdvancedTCA
TM

October 1, 2003 Slide 30

 Shelf Cooling
z Front to rear airflow “should”
z Side to side airflow “may”
AIR
z Zone 3 airflow seal
OUTLET
z NEBS air filter
TOP PLENUM z Cooling arrangement not
FANS
specified
z Flow curve requirements
„ Pressure drop vs. flow rate

FRONT RTM
BOARD Slot Fan Empty Slot
Flow Impedance
∆P
AIR
INLET

BOTTOM PLENUM

Source: Henry Wong, Motorola

CFM
AdvancedTCA
TM

October 1, 2003 Slide 31

 Front Board
z 200 Watts per slot
z Airflow direction
„ Bottom to top
„ Right to left (front view)
z Cooling requirements
„ CFM = f(power) @ 10 °C
„ Pressure drop vs. flow rate
Front Board

∆P

Board
Impedance
Even inlet airflow distribution
CFM
Source: Henry Wong, Motorola

AdvancedTCA
TM

October 1, 2003 Slide 32

 How Much Flow Per Slot?
z System integrator incorporates data from shelf and board vendors
z Simple pass/fail test based on flow (not temperature)

Slot + Board Impedance

Slot Fan Flow Operating
(shelf vendor) Point
∆P
(system integrator)
Empty Slot Impedance
(shelf vendor)

Board Impedance
(board vendor)

Source: Henry Wong, Motorola

CFM
AdvancedTCA
TM

October 1, 2003 Slide 33

RTM Cooling
z Minimum support 5 W RTMs
„ 5 W for natural convection
„ Do not burden shelves which do not
require active cooling
„ Zone 3 Airflow Seal
z Air Flow Direction
„ Bottom to top
„ Right to left (front view)
z If active cooling is provided, cooling
requirements
„ CFM = f(power) @ 10 C
„ Pressure drop vs. CFM

Source: Henry Wong, Motorola

AdvancedTCA
TM

October 1, 2003 Slide 34

 Thermal Management
1. Discovery z Temperature sensors
„ Shelf inlet “shall”
„ Shelf outlet “should”
2. Normal Operating „ Field replaceable units
(FRUs)
4. Repair
z Thermal alarming
3. Abnormal „ Critical, major, minor,
normal
High/Low Ambient
Fan Failure Temperature
„ Thresholds, hysteresis
Board or
Component z Rules
Over Temp
„ FRU, ShMM, System
etc... Manager
Clogged Filter

Source: Henry Wong, Motorola

AdvancedTCA
TM

October 1, 2003 Slide 35

 Details, Details, Details...
z Mechanical
z Power & Thermal
z Management
z Data Transport
z Mezzanine Cards
z Regulatory

AdvancedTCA
TM

October 1, 2003 Slide 36

ATCA™ Shelf Management
z Monitor & control low-level aspects of ATCA boards
and other field replaceable units (FRUs) within a shelf
z Watch over basic health of the shelf, report anomalies,
take corrective action when needed
z Retrieve inventory information & sensor readings
z Receive event reports and failure notifications from
boards and other intelligent FRUs
z Manage power, cooling & interconnect resources in the
shelf
z Enable visibility into a shelf for a logical System
Manager — some mix of software + “swivel chair folk”

Source: Mark Overgaard, Pigeon Point Systems

AdvancedTCA
TM

October 1, 2003 Slide 37

ATCA™ Management Approach

z Focus on low-level hardware management

„ Required on all boards and shelves
„ Monitor/control of FRUs in shelf
z Adopt Intelligent Platform Management
Interface (IPMI) 1.5 Revision 1.1 as foundation
„ IPMI widely used in PC and Server industry
z Facilitate supplementary higher level services
„ Shelves must provide IP-compatible transport
„ “In-band” application management expected, but not
specified
Source: Mark Overgaard, Pigeon Point Systems

AdvancedTCA
TM

October 1, 2003 Slide 38

ATCA Shelf w/ Dedicated Shelf
Management Controllers
IPM Controller (IPMC)

Shelf-External System Manager Shelf Management Controller (ShMC)

AdvancedTCA Board
Shelf Manager w/ Dedicated ShMC
Other Field Replaceable Unit (FRU)

Shelf Shelf
Fan
Power
Entry
Power
Entry
Key
Manager Manager Tray
(Active) (Backup) Module Module

Implementation
Dependent
Connection
ShMC ShMC IPMC

2x Redundant, Bussed or Radial, IPMB-0

IPMC IPMC IPMC IPMC IPMC IPMC IPMC IPMC

ATCA ATCA ATCA ATCA ATCA ATCA ATCA ATCA

Board Board Board Board Board Board Board Board

2x Redundant Radial Internet-Protocol-Capable Transport

Source: PICMG

AdvancedTCA
TM

October 1, 2003 Slide 39

Shelf Manager Services
z Access inventory information for all FRUs
z Manage power consumption
„ Using FRU information (from non-volatile store on FRU)
z Simplified framework for cooling management in R1.0
„ Using FRU-configured temperature threshold events
z Manage distributed sensors
„ Based on Sensor Data Records (SDRs)
z Manage data transport interconnect compatibility
„ Based on FRU information
z Collect IPMI events in persistent store and, optionally,
perform configurable actions in response
„ Platform Event Filtering (PEF) supports configurable
actions on events (e.g. an SNMP trap)

AdvancedTCA
TM

October 1, 2003 Slide 40

Implementation Options

AdvancedTCA
Source: PICMG TM

October 1, 2003 Slide 41

IPM Controller Extensions
Beyond IPMI 1.5 Commands and FRU Information

z Redundant connections to shelf manager

z Hot swap state management for FRUs
(including represented FRUs)
„ Compatibly extends the CompactPCI operator
interface including blue LED and handle switch
z Electronic keying
„ Commands + FRU information
z LED management, including color, lamp test
z Fan control for interoperable fan trays
z Payload power control & negotiation w/ shelf

AdvancedTCA
TM

October 1, 2003 Slide 42

 Board-Specific
Management
Key ATCA
Infrastructure
Board &
IPMC
Interfaces

Source: Mark Overgaard, Pigeon Point Systems

ATCA Management Summary

z Significant PICMG effort defines interoperable

extensions to IPMI
„ 30% of PICMG 3.0 specification devoted to
Shelf Management
z Sophisticated system management to support
telecom reliability, availability and
serviceability
„ PICMG 3.0 specification ensures boards, backplanes
and chassis from different vendors can work
together as a communications system

AdvancedTCA
TM

October 1, 2003 Slide 44

 Details, Details, Details...
z Mechanical
z Power & Thermal
z Management
z Data Transport
z Mezzanine Cards
z Regulatory

AdvancedTCA
TM

October 1, 2003 Slide 45

Backplane Performance Needs

z Different network elements require different

raw backplane capacities, for example:
„ POTS switch (20k lines) 2.5Gb/s
„ Voice gateway (50k ports) 10Gb/s
„ DSLAM (4k lines @ 10Mb/s) 40Gb/s
„ Metro optical mux (1k ports @OC-3) 160Gb/s
„ Fiber to the Home (4k ports @100Mb/s) 500Gb/s
„ Core router (256 ports @ OC-192) 2.5Tb/s

Source: Chuck Byers, Lucent Technologies

AdvancedTCA
TM

October 1, 2003 Slide 46

AdvancedTCA Backplane Goals
z Improve upon existing Modular Open Systems
„ Independent rack-mount servers (pizza boxes)
„ Integrated systems of modular elements (CompactPCI)
z Performance headroom
„ Differential signaling capable of 10 Gbps (XAUI), per fabric
channel, today
z Flexibility & scalability to address many markets
„ Single backplane configuration may support different
fabric technologies and topologies
„ Cost reduced configurations
z Carrier-grade availability
„ No single point of failure for any system interconnect

AdvancedTCA
TM

October 1, 2003 Slide 47

 AdvancedTCA and Fabrics
PICMG
PICMG3.0
3.0Specification
Specification
z Power Distribution z Connector Zones and type
z Mechanical Elements z Fabric Topology
z System Management z Thermal Management Guidelines
z Regulatory Guidelines

PICMG
PICMG3.1
3.1 PICMG
PICMG3.2
3.2 PICMG
PICMG3.3
3.3 PICMG
PICMG3.4
3.4 PICMG
PICMG3.5
3.5
Specification
Specification Specification
Specification Specification
Specification Specification
Specification Specification
Specification

Ethernet
Ethernet&& InfiniBand
InfiniBand StarFabric
StarFabric PCI-Express
PCI-Express RapidIO
RapidIO&&
Fibre
Fibre &&Advanced
Advanced Advanced
Advanced
Channel
Channel Switching
Switching Fabric
Fabric
Interface**
Interface**

z Backplanes are fully defined in the PICMG 3.0 specification

z Interoperable Boards are defined in Subsidiary specifications

Source: Jim Kennedy, Intel Corporation

AdvancedTCA
TM

October 1, 2003 Slide 48

 PICMG 3.0 Backplane (Logical View)

Timing Clocks
(6 x 1 diff. pair each)
Update Ports
(10 diff. pairs each)

Fabric Interface
Zone 2 Full Mesh shown
(8 diff. pairs each)

Base Interface
Ethernet Star
(4 diff. pairs each)

IPMB*
(2 traces each)
Zone 1
8 Ring/Test Lines
-48VDC Power
(2 feeds of –48V, RTN)
Node Slots Hub Slots Node Slots

Source: Jim Kennedy, Intel Corporation Mesh Slots

AdvancedTCA
TM

October 1, 2003 Slide 49

 ATCA Board Configurations
Rear I/O Hub Board
(Zone 3) z Supplies switching resources between all
Hub other boards in the shelf
z Base only (PICMG 3.0), fabric only, or base
Board + fabric configurations possible
z Requires many Zone 2 connectors
z Installed in designated Hub slots

Mesh Enabled Board

z Supports a direct channel connection
Mesh between all other boards in the shelf
Base, Fabric
Enabled Connectors
z Fabric only, or fabric + base configurations
possible
Board (Zone 2)
z Requires many Zone 2 connectors
z Installed into any slot

Node Board
z Single, dual, many star connections
possible
Node z Base only (PICMG 3.0), fabric only, or base
Board Power
+ fabric configurations possible
Connector z Single Zone 2 connector
(Zone 1) z Installed into any node slot
Source: Jim Kennedy, Intel Corporation
AdvancedTCA
TM

October 1, 2003 Slide 50

 Fabric Topologies — Star

Nodes support
a point-to-point
link to the
switch hub

Number of routed
traces remains
Dedicated relatively low
system slot for keeping
switching backplane costs
resources low
(Hub)
Application Target
z Non-carrier-grade applications with little latency sensitive data traffic
z Unified approach reduces cost, complexity

Source: Jim Kennedy, Intel Corporation

AdvancedTCA
TM

October 1, 2003 Slide 51

 Fabric Topologies — Dual Star

Redundant switches
increase system Nodes support
availability, eliminates redundant
single point of failure links, one to
each switch

Number of routed
traces remains
Link between relatively low
switches keeping
facilitates Two dedicated
system slots backplane costs
coordination, fail- low
over for switching
resources

Application Target
z Carrier-grade applications with non-latency-sensitive data requirements
(e.g., modular server)
z Unified approach reduces cost, complexity
Source: Jim Kennedy, Intel Corporation
AdvancedTCA
TM

October 1, 2003 Slide 52

 Fabric Topologies: Dual-Dual Star
Each Node
supports 4 links,
Separate, one to each switch
dedicated fabrics resource
for Control and
Data Plane

Link between each

function specific
redundant switch

Application Target
z Carrier grade applications with latency sensitive streaming data requirements
and significant control & mgmt (TCP/IP-based) workload.
z Separate data plane fabric allows optimized data throughput

Source: Jim Kennedy, Intel Corporation

AdvancedTCA
TM

October 1, 2003 Slide 53

 Fabric Topologies — Full Mesh
All system slots are
Switching identical. Direct connections
services and to all other slots, no
management are
dedicated slots
distributed across
all system slots

Data throughput Fabric links are

capacity scales inherently
with each added redundant, highly
Node board, large available
capacities possible

Application Target
z Carrier-grade applications with large data throughput requirements (routers)
z Intelligent nodes provide simple layer 2 switching and higher level services
z Highly redundant and scalable

Source: Jim Kennedy, Intel Corporation

AdvancedTCA
TM

October 1, 2003 Slide 54

ATCA Fabric Alternatives
z PICMG 3.1 — Ethernet, Fiber Channel
„ SERDES based Ethernet (GbE) and/or Fiber Channel
z 1, 2 or 4 links per Fabric Channel

„ XAUI 10Gb Ethernet — 1 link per Fabric Channel

z PICMG 3.2 — InfiniBand
„ 1x or 4x links defined
z PICMG 3.3 — StarFabric
„ Up to 2.5 Gb links
z PICMG 3.4 — PCI Express
„ 1x, 2x, or 4x lanes per Fabric Channel
„ Supports PCI Express Base or Advanced Switching
z PICMG 3.5 — Advanced Fabric Interface / S-RapidIO
„ 1x or 4x links
Source: Jim Kennedy, Intel Corporation

AdvancedTCA
TM

October 1, 2003 Slide 55

 Details, Details, Details...
z Mechanical
z Power & Thermal
z Management
z Data Transport
z Mezzanine Cards
z Regulatory

AdvancedTCA
TM

October 1, 2003 Slide 56

Mezzanine Cards on ATCA

z Large ATCA front panel supports up to four

PMC modules with front panel access
„ Works with today’s numerous PMC modules
z New requirements need new mezzanine card
„ ATCA supports higher speed interfaces
„ New chips require power and space beyond PMC
„ Deeper ATCA board allows longer mezzanine cards
„ Need to minimize equipment impacted by single
failure drive desire for hot-swapable mezzanines
z AdvancedMC work group chartered
„ AMC specification work-in-progress…

AdvancedTCA
TM

October 1, 2003 Slide 57

 ATCA Carrier Card Reference Design
Backplane Backplane
Interface

Local Routing 16 X 16
Power Control
Table RAM Switched Hub

MAC / Host Adapter / Buffer

Network Processor RAM
Board
Level
Processor
Controller
I/O & Processing
Core
Peripherals RAM

MEZ #1 MEZ #2 MEZ #3 ... MEZ #N

Processor DSP NP I/O

Source: Chuck Byers, Lucent Technologies

AdvancedTCA
TM

October 1, 2003 Slide 58

 Details, Details, Details...
z Mechanical
z Power & Thermal
z Management
z Data Transport
z Mezzanine Cards
z Regulatory

AdvancedTCA
TM

October 1, 2003 Slide 59

ATCA Regulatory Guidelines

z Telecom central offices have unique

regulatory requirements
„ which vary by region and sometimes by network
z PICMG 3.0 spec includes extensive references
„ Safety: UL, CSA, LVD, EN
„ Electromagnetic compatibility: FCC, CSA, EN

„ Reliability: Telecordia

Other:
„ Heat and temperature, fire, earthquakes, acoustics
(noise), lightning, grounding and bonding
„ Ergonomic, environmental, ecology

AdvancedTCA
TM

October 1, 2003 Slide 60

 Applications

AdvancedTCA
TM

October 1, 2003 Slide 61

 Network Architecture Today
Soft- Video Web
Switch Server
Data Center Elements Server
Firewall
Gateway E-Commerce
PSTN Cache Server Filter Internet

FTTH Terminal ATM Switch ATM Switch FTTH Terminal

WDM Terminal WDM Terminal

Fiber

CMTS CMTS
Cable IP Router Transport Elements IP Router

MSC MSC
DSLAM DSLAM
Core Elements
Copper
Loop

Wireless Wireless
Base Access / Edge Elements Base

Radio

Customer Premises Equipment

AdvancedTCA
Source: Chuck Byers, Lucent Technologies TM

October 1, 2003 Slide 62

 Network Architecture Tomorrow
Data Center Elements
Soft- Video E-Commerce Web
Gateway Cache Server Filter Firewall
Switch Server Server
PSTN PICMG 3 PICMG 3 PICMG 3 PICMG 3 PICMG 3 PICMG 3 PICMG 3 PICMG 3 Internet

FTTH Terminal ATM Switch ATM Switch FTTH Terminal

Fiber PICMG 3 PICMG 3

WDM Terminal WDM Terminal PICMG 3 PICMG 3

PICMG 3 PICMG 3

CMTS CMTS
Cable PICMG 3 IP Router Transport Elements IP Router PICMG 3

PICMG 3 PICMG 3
MSC MSC
DSLAM DSLAM
Copper
PICMG 3 Core Elements PICMG 3
PICMG 3 PICMG 3
Loop

Wireless PICMG 3 Wireless

Base Access / Edge Elements PICMG 3
Base

Radio

Customer Premises Equipment

AdvancedTCA
Source: Chuck Byers, Lucent Technologies TM

October 1, 2003 Slide 63

 Network — Day After Tomorrow
Data Center Elements
Universal Server Element
PSTN Internet
PICMG 3

Universal Universal
Core Core
Fiber Element Element
Universal Universal
PICMG 3 PICMG 3
Access Access
Element Element
Cable PICMG 3 PICMG 3

Core Elements
Copper
Loop

Wireless PICMG 3 Wireless

Base Access / Edge Elements PICMG 3
Base

Radio

Customer Premises Equipment

AdvancedTCA
Source: Chuck Byers, Lucent Technologies TM

October 1, 2003 Slide 64

NextGen Media Server
PSTN Other
Interfaces I/O Interfaces

Other
PSTN Audio/Video System
IP Telephony Computing
GATEWAY Conferencing Controller
Media Blades or I/O
BLADE(s) Blade(s) (1+1)
Resources

Switch Fabric
Links
(Base Interface
Application
Links not shown)
Processors
IP Media
SWITCH Links
FABRIC
Management
(1+1)
IP Links

AdvancedTCA Media Server Chassis

AdvancedTCA
TM

October 1, 2003 Slide 65

APG
APG

TM

AdvancedTCA
* Media Engines
* Scripting
GP CPUs

etc
Media Server — 10k Ports

Media Engine
NMS

Media Engine
NMS

Media Engine
NMS

Media Engine
MGWs

NMS

Trunking Gateway
NMS

Slide 66
Trunking Gateway
NMS

Trunking Gateway
NMS

Trunking Gateway
NMS

Signaling Gateway
NMS
SFCs MSEs

Signaling Gateway
NMS

Media Service Controller

NMS

Media Service Controller

NMS

Switch and Flow Controler

NMS

Switch and Flow Controller

NMS

Plugin

Plugin

Plugin

Plugin

October 1, 2003
connectivity
Network
 APG
APG

Voice Portal with Full Speech Support

NMS HearSay 1.1 NMS HearSay on ATCA
2,016 ports 2,016 ports
HENDRY HENDRY HENDRY HENDRY HENDRY

50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50
ALARM ALARM ALARM ALARM ALARM ALARM ALARM ALARM ALARM ALARM

1 5 10 HENDRY 1 5 10 1 5 10 HENDRY 1 5 10 1 5 10 HENDRY 1 5 10 1 5 10 HENDRY 1 5 10 1 5 10 HENDRY 1 5 10

A B A B A B A B A B
A POWE
R
P OWE
R
B A POWE
R
P OWE
R
B A POWE
R
P OWE
R
B A POWE
R
P OWE
R
B A POWE
R
P OWE
R
B
FAIL FAIL FAIL FAIL FAIL
1 10 ALARM 1 10 1 10 ALARM 1 10 1 10 ALARM 1 10 1 10 ALARM 1 10 1 10 ALARM 1 10

1 5 10 HENDRY 1 5 10 1 5 10 HENDRY 1 5 10 1 5 10 HENDRY 1 5 10 1 5 10 HENDRY 1 5 10 1 5 10 HENDRY 1 5 10

A B A B A B A B A B

A POWE
R
P OWE
R B A POWE
R
P OWE
R B A POWE
R
P OWE
R B A POWE
R
P OWE
R B A POWE
R
P OWE
R B
FAIL FAIL FAIL FAIL FAIL
1 10 ALARM 1 10 1 10 ALARM 1 10 1 10 ALARM 1 10 1 10 ALARM 1 10 1 10 ALARM 1 10

SYSTEM
CONSOL
ETHERNET ALAR M
E AC - 48V INPUT RESET ABORT
FAU LT/ CRITICAL LED RELAY POWE
Q UIET
MAJOR O R
O OS TEST TEST FUSE
MINOR A B
A B

SYSTEM SYSTEM SYSTEM SYSTEM

CONSOL CONSOL CONSOL CONSOL
ETHERNET ALAR M ETHERNET ALAR M ETHERNET ALAR M ETHERNET ALAR M
E - 48V INPUT RESET ABORT E - 48V INPUT RESET ABORT E - 48V INPUT RESET ABORT E - 48V INPUT RESET ABORT
FAU LT/ CRITICAL LED RELAY AC POWE FAU LT/ CRITICAL LED RELAY AC POWE FAU LT/ CRITICAL LED RELAY AC POWE FAU LT/ CRITICAL LED RELAY AC POWE
Q UIET R
Q UIET R
Q UIET R
Q UIET R
O OS MAJOR TEST TEST O FUSE O OS MAJOR TEST TEST O FUSE O OS MAJOR TEST TEST O FUSE O OS MAJOR TEST TEST O FUSE
MINOR A B MINOR A B MINOR A B MINOR A B PO WER RPS ACTIVITY
A B A B A B A B
CompactP CI CompactP CI

TX

RX

RX
TX
1

11

13

15

17

19

21

23

25
FAULT

MG M T PO RT
CONSOL E

R T
E L
W U
O A
P F

R T
E L
W U
O A
P F

R T
E L
W U
O A
P F

R T
E L
W U
O A
P F
CPC440 0

SW ITCH

PORT 25

PORT 26

ACT/ LI NK
24 +2

SPEED
SYSTEM

ACT/ LI NK
SPEED
2

10

12

14

16

18

20

22

24

26

CompactP CI

CompactP CI

CompactP CI
TX

RX

RX

TX

RX

RX

TX

RX

RX

TX

RX

RX
TX

TX

TX

TX
1

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25
SYSTEM FAULT

MG M T PO RT

SYSTEM FAULT

MG M T PO RT

SYSTEM FAULT

MG M T PO RT

SYSTEM FAULT

MG M T PO RT
CONSOL E

CONSOL E

CONSOL E

CONSOL E
CPC440 0

CPC440 0

CPC440 0

CPC440 0
SW ITCH

PORT 2 5

PORT 2 6

SW ITCH

PORT 2 5

PORT 2 6

SW ITCH

PORT 2 5

PORT 2 6

SW ITCH

PORT 2 5

PORT 2 6
ACT/ LI NK ACT/ LI NK ACT/ LI NK ACT/ LI NK

200W Power Supply 200W Power Supply 200W Power Supply 200W Power Supply
2 4+2

2 4+2

2 4+2

2 4+2
SPEED SPEED SPEED SPEED
ACT/ LI NK ACT/ LI NK ACT/ LI NK ACT/ LI NK
SPEED SPEED SPEED SPEED
2

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26
R
E
W
O
P

R
E
W
O
P

R
E
W
O
P

R
E
W
O
P
T
L
U
A
F

T
L
U
A
F

T
L
U
A
F

T
L
U
A
F
CompactP CI

CompactP CI

CompactP CI

CompactP CI
TX

RX

RX

TX

RX

RX

TX

RX

RX

TX

RX

RX
TX

TX

TX

TX
1

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25
SYSTEM FAULT

MG M T PO RT

SYSTEM FAULT

MG M T PO RT

SYSTEM FAULT

MG M T PO RT

SYSTEM FAULT

MG M T PO RT
CONSOL E

CONSOL E

CONSOL E

CONSOL E
CPC440 0

CPC440 0

CPC440 0

CPC440 0
SW ITCH

SW ITCH

SW ITCH

SW ITCH
PORT 2 5

PORT 2 6

PORT 2 5

PORT 2 6

PORT 2 5

PORT 2 6

PORT 2 5

PORT 2 6
ACT/ LI NK ACT/ LI NK ACT/ LI NK ACT/ LI NK
2 4+2

2 4+2

2 4+2

2 4+2
SPEED SPEED SPEED SPEED

SPEED
ACT/ LI NK
200W Power Supply SPEED
ACT/ LI NK
200W Power Supply SPEED
ACT/ LI NK
200W Power Supply SPEED
ACT/ LI NK
200W Power Supply
2

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26
NM NM NM NM NM NM NM NM NM NM NM NM NM NM
CompactP CI CompactPCI

S S S S S S S S S S S S S S
TX

RX

RX
TX
1

11

13

15

17

19

21

23

25

R T
E L
W U
O A
P F

R T
E L
W U
O A
P F

R T
E L
W U
O A
P F

R T
E L
W U
O A
P F
FAULT

M G MT PO RT
CONSOLE

CPC4 400

SWI TCH

PORT 25

PORT 26

ACT/ L I NK
24 +2

SPEED
SYSTEM

ACT/ L I NK
SPEED
2

10

12

14

16

18

20

22

24

26

Media Engine
Switch and Flow
Switch and Flow
Media Service Controller
Media Service Controller
Signaling Gateway
Signaling Gateway
Trunking Gateway
Trunking Gateway
Trunking Gateway

Media Engine
Media Engine
Media Engine
Media Engine
CompactP CI

CompactP CI

CompactP CI
TX

RX

RX

TX

RX

RX

TX

RX

RX

TX

RX

RX
TX

TX

TX

TX
1

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25
FAUL T

M G MT PO RT

FAUL T

M G MT PO RT

FAUL T

M G MT PO RT

FAUL T

M G MT PO RT
CONSOLE

CONSOLE

CONSOLE

CONSOLE
CPC4 400

CPC4 400

CPC4 400

CPC4 400
SWI TCH

SWI TCH

SWI TCH

SWI TCH
PORT 25

PORT 26

PORT 25

PORT 26

PORT 25

PORT 26

PORT 25

PORT 26
ACT/ L I NK ACT/ LI NK ACT/ LI NK ACT/ LI NK
24 +2

24 +2

24 +2

24 +2
SPEED
200W Power Supply SPEED
200W Power Supply SPEED
200W Power Supply SPEED
200W Power Supply
SYSTEM

SYSTEM

SYSTEM

SYSTEM
ACT/ L I NK ACT/ LI NK ACT/ LI NK ACT/ LI NK
SPEED SPEED SPEED SPEED
2

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26
R T
E L
W U
O A
P F

R T
E L
W U
O A
P F

R T
E L
W U
O A
P F

R T
E L
W U
O A
P F
CompactP CI

CompactP CI

CompactP CI

CompactP CI
TX

RX

RX

TX

RX

RX

TX

RX

RX

TX

RX

RX
TX

TX

TX

TX
1

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25

11

13

15

17

19

21

23

25
FAUL T

M G MT PO RT

FAUL T

M G MT PO RT

FAUL T

M G MT PO RT

FAUL T

M G MT PO RT
CONSOLE

CONSOLE

CONSOLE

CONSOLE
CPC4 400

CPC4 400

CPC4 400

CPC4 400
SWI TCH

SWI TCH

SWI TCH

SWI TCH
PORT 25

PORT 26

PORT 25

PORT 26

PORT 25

PORT 26

PORT 25

PORT 26
ACT/ L I NK ACT/ LI NK ACT/ LI NK ACT/ LI NK
24 +2

24 +2

24 +2

24 +2
SPEED SPEED SPEED SPEED

200W Power Supply 200W Power Supply 200W Power Supply 200W Power Supply
SYSTEM

SYSTEM

SYSTEM

SYSTEM
ACT/ L I NK ACT/ LI NK ACT/ LI NK ACT/ LI NK
SPEED SPEED SPEED SPEED
2

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26

10

12

14

16

18

20

22

24

26
PO WER RPS ACTIVITY PO WER RPS ACTIVITY PO WER RPS ACTIVITY PO WER RPS ACTIVITY

Netra 120

Sun
microsystems

Netra 120

Sun
microsystems

Netra 120

Sun
microsystems

Netra 120

Sun
microsystems

Netra 120

Sun
microsystems

Netra 120 Netra 120 Netra 120 Netra 120

Sun
microsystems
Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

Netra 120 Netra 120 Netra 120 Netra 120

Sun
microsystems
Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

Netra 120 Netra 120 Netra 120

Netra 120

Sun
microsystems
Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

Netra 120 Netra 120 Netra 120

Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

Netra 120 Netra 120 Netra 120 NM NM NM NM NM NM

Netra T4 S S S S S S
Blank Blank Blank Blank Blank Blank Blank Blank
Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

Media Engine
Media Engine
Media Engine
Media Engine
Switch and Flow
Switch and Flow
SYSTEM

Netra 120 Netra 120 Netra 120

ALARM 1

ALARM 2 Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

DC-A

DC-B Sun
microsyst ems
Netra 120 Netra 120 Netra 120

Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

Netra 120 Netra 120 Netra 120

Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

Netra 120 Netra 120 Netra 120

Netra T4
Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

SYSTEM

Netra 120 Netra 120 Netra 120

ALARM 1

ALARM 2 Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

DC-A

DC-B Sun
microsyst ems
Netra 120 Netra 120 Netra 120

Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

Netra 120 Netra 120 Netra 120

Sun
m icr osystems
Sun
m icr osystems
Sun
m icr osystems

Netra T4 Netra T4 Netra T4

Netra T4

SYSTEM SYSTEM SYSTEM

SYSTEM

ALARM 1 ALARM 1 ALARM 1

ALARM 1
ALARM 2 ALARM 2 ALARM 2
ALARM 2

DC-A DC-A DC-A

DC-A

DC-B Sun DC-B Sun

microsystems
DC-B Sun
microsystems
DC-B Sun
microsystems
microsyst ems

2U/3U Storage Server

Netra T4 Netra T4 Netra T4 Netra T4

SYSTEM SYSTEM SYSTEM SYSTEM

ALARM 1 ALARM 1 ALARM 1 ALARM 1

ALARM 2 ALARM 2 ALARM 2 ALARM 2

DC-A DC-A DC-A DC-A

DC-B Sun
microsyst ems
DC-B Sun
microsystems
DC-B Sun
microsystems
DC-B Sun
microsystems

Netra T4 Netra T4 Netra T4

SYSTEM SYSTEM SYSTEM

ALARM 1 ALARM 1 ALARM 1

ALARM 2 ALARM 2 ALARM 2

DC-A DC-A DC-A

DC-B Sun
microsystems
DC-B Sun
microsystems
DC-B Sun
microsystems

Netra T4 Netra T4 Netra T4

SYSTEM SYSTEM SYSTEM

ALARM 1 ALARM 1 ALARM 1

ALARM 2 ALARM 2 ALARM 2

DC-A DC-A DC-A

DC-B Sun
microsystems
DC-B Sun
microsystems
DC-B Sun
microsystems

Sun
microsystems Netra st
A1000

AdvancedTCA
TM

October 1, 2003 Slide 67

AdvancedTCA
Universal Telecom Platform
z High backplane capacity: up to 2.5 Tb/s
z High functional density
„ up to 200 W per slot, with 13” x 11” modules 1.2” thick
„ up to 16 slots per shelf, up to 3 shelves per frame
z Flexible and scalable
„ Packaging, partitioning, cooling and cabling options
z Reliable
„ Robust system management infrastructure
„ Full redundancy support, including update buses
z Telecom-specific features
„ Distributed 48 V power
„ Telecom clocks, ringing voltage buses, metallic test bus
„ Regulatory conformance: NEBS, UL, CSA, EU, etc.

AdvancedTCA
TM

October 1, 2003 Slide 68

AdvancedTCA Status

z Specifications complete and stable

„ System management interoperability demonstrated
in spring 2003 thus validating one of the most
complex and important parts of AdvancedTCA
z Interoperability workshops every two months
„ 5th workshop took place in September 2003
z Dozens of vendors have announced products
„ many products shipping
z Several major telecom equipment providers
basing next-gen systems on AdvancedTCA

AdvancedTCA
TM

October 1, 2003 Slide 69

NEC’s SGSN/GGSN Packet Node
Based on AdvancedTCA
z High performance packet
platform for 3G wireless
core networks
„ 6.2 Gbps throughput
z Enables mobile Internet
services for 2.5G & 3G
mobile wireless networks
z Announced 9-2003
„ http://www.nec.co.jp/press/
en/0309/1001.html
z Entering trials with NTT
DoCoMo in 4Q03

AdvancedTCA
TM

October 1, 2003 Slide 70

AdvancedTCA Outlook

z Telecom equipment market over US$200B/year

„ $215B in 2002 per RHK, Inc.
z Platforms represent more than 10% of these $
„ Today’s platforms are custom designed
„ VME & CompactPCI represent a small fraction of this
potential business
z AdvancedTCA ideal match to this opportunity
„ Specification driven by telecom supplier inputs
„ Major suppliers adopting AdvancedTCA platform
z ATCA poised to overtake and far exceed both
VME and cPCI (by 2006?)

AdvancedTCA
TM

October 1, 2003 Slide 71

Conclusions

z AdvancedTCA ready today!

z Ideal platform for a wide variety of telecom
and datacom applications
„ improves upon the shortcomings of previous
standards (including power, board area, cooling,
reliability, RASM, bandwidth and cost).
z Multiple vendors already producing
interoperable hardware and software
z For more information, see:
http://www.advancedtca.org

AdvancedTCA
TM

October 1, 2003 Slide 72

 Thank You

R. Brough Turner
Sr. VP & CTO
NMS Communications
rbt@nmss.com