Documenting and Managing

Infrastructure Connectivity
David Cuthbertson
Square Mile Systems Ltd

david.cuthbertson@squaremilesystems.com
www.squaremilesystems.com
 Square Mile Background
• Develop toolsets, training and
techniques for operational
management of complex IT
infrastructure
• Focus areas
– Data center management
– Connectivity management
– System change impact analysis
– Documentation techniques
– Infrastructure visualisation
• All technologies!
Data Center Infrastructure
Business Processes
Departmental, Company
Services
End user, infrastructure, supplier
Applications
PC, server, mainframe, SOA
Virtual Infrastructure
Network, Servers, Storage, DBMS
Hardware Infrastructure
Network, Servers, UPS, Storage, Other
Fixed Infrastructure
(Cabling, Power, Cabinets, Buildings)
 Session overview
• Communicate practices and techniques to
colleagues and customers that will aid
better management of infrastructure
connectivity
• Recognise good and bad methods of
labeling devices and cabling
• Take into account operational
management needs when designing or
installing cabling infrastructures
 Is it a Problem?
• Standards already cover cabling design,
installation and labelling
– TIA568A, ISO 11801, EN50173, r al l
TIA942
e f o
e s
th administration
• Standards already cover o w n s ?
fo l l ati o
– ANSI/TIA/EIA-606,
a n y n t
EN50174-1,
e
w m
ISO/IEC14763-1, p le m
TIA942, BS6701:2004
H o g im
li n
cab
• Connectivity is more than just about data
cabling!
– Power is becoming just as complex
 Changing Requirements
BEFORE AFTER
No. of Servers per cabinet 3-6 30-40
Power Dissipated per cab. 300-2000W 3kW - 25kW
Current service to cabinet 16A 2x32 A or 3 phase
Types of Equipment Servers Blade Servers
Monitor Power Distribution Units
KVMs MidSpan Boxes
Power Strips Disk Arrays (Storage)
UPS Smart Power Strips
Regular Power Strips
Network types 100M 1G, 10G, SAN
No. of Cables Power 1 or 2 2 to 6
(per server) Network 1 or 2 5 to 10
Cabinet Total 20-30 300 - 400
 Where Do We Focus?
• Earthing and bonding
• Containment
• Firestopping
• Spaces (rooms, racks etc.)
• Vertical wiring
• Horizontal wiring
• Power
• IT equipment
• Other equipment
Which is Easiest to Document?
 Standards Recommendations
1. Class or Hierarchy Structure
Class 1 – Single equipment room
Class 2 – Multiple rooms
Class 3 – Campus
Class 4 – Multiple sites
Example TIA606

Naming conventions and approach will differ

with administration systems.
 Standards Recommendations
2. Naming conventions and examples

Country, site, floor, room, rack, unit, sub-unit, port

1A-AC01/A-01

Floor Equipment Rack Patch Port

1 Room AC01 panel 01
A A
 Standards Recommendations
3. Standardised Naming & Coding
Glossary of terms and abbreviations
Symbols for drawing
Termination point colour coding
Patch cable coding
 Standards Recommendations
4. Recommended data sets to maintain
Port type - RJ45
Cable type - Cat6A
Cable length - 65m
User name - Daves PC
Test results - U:\Cabletest\1Atest.xls
Drawings - Floor, room drawings
Work flow - Work orders and changes
Why Does It Happen?
1. Standards have been
referenced for the design,
implementation and testing
of infrastructure

2. Good components have

been chosen - which rarely
go wrong

3. Is it only a people issue?

 How Do We Manage Today?
• Informal / formal processes
• Site survey, pre-installation checks, audits
• Ownership is often on a local basis
• Create knowledge sets as individuals or
within teams – Excel, Visio, Word, Notes,
Sharepoint, Access
• Or give the problem to someone else
– Outsource, out task.
 Different Teams, Different Focus
Business Processes
Departmental, Company

Services
Systems End user, infrastructure, supplier
Applications
Service
Applications
PC, server, mainframe, SOA
Management
Virtual Infrastructure
Mid-range PCs, Network, Servers, Storage, DBMS
Servers
Networks
Hardware Infrastructure
LAN/SAN PCs, Network, Servers, UPS, Storage, Other

Desktops Data
Fixed Infrastructure
IMAC Centre (Cabling, Power, Racks, Rooms, Buildings)
 Adding a New Server?
Structured cabling only KVM
LAN diagrams Architecture
Inventory list
Storage diagrams Patching
spreadsheets
KVM WAN diagrams
Asset list Point to Point Cabling
Building wiring
Rack Backbone switches
diagrams
Diagrams
IIS Architecture Power architecture Power distribution

Blade switches
Computer room Edge switches
layout PDUs SAN
Circuit breakers Labelling standards Architecture
Power strip
PABX port mapping Legacy systems
connections
LAN Architecture
 LINK 10/100

LAN
FEATURE

SERIAL
Different Views LINK 10/100

LAN

CURRENT
FEATURE

SERIAL

CURRENT

ON = I OFF = U
ON = I OFF = U BLINK = REMOTE
BLINK = REMOTE

OUTLET #
OUTLET #
Copper
I /U TOGGLE
I /U TOGGLE

Fibre
Firewall Power

Managing change at
equipment level
Server requires different
views of connectivity!
100-
240V
~

50~
60Hz
RESERVED
STATUS 9 10 11 12 13 14 15 16

Storage
1.2A 1 2 3 4 5 6 7 8

KVM
Or monitoring results
from power, network
or storage can’t be
interpreted

Switch
 Is It Just Too Difficult?
For many organisations, the internal culture does
not enable more coordinated team practices

Why is it difficult?
Creating a start point or baseline is costly
Defining update processes
Will involve multiple technical teams
Disinterest by project teams
 Connectivity Costs!
It is easy to use technical teams and suppliers inefficiently!
– Travel to check on existing connectivity or site survey
– Meetings to communicate local cabling architecture
– Reconciling differences in systems documentation
– Creating build and patch instructions
– Drawing diagrams which become out of date and uncontrolled
– Audit connectivity to check on resilient paths / SPOF
– Tracing connectivity to help with fault diagnosis
– Manually produce reports on numbers of changes / installs
– Switch port tracing to identify free and pre-patched ports
– Producing risk and test plans as part of projects
– Risk and recovery tasks creating additional knowledge sets
 What’s the upside?
1. Reduced operational expenditure
– Optimise existing infrastructure Power

2. Reduced cost of changes

– Planning, implementing, reporting
Time

3. Faster implementation of changes Project

SLAs
– Reducing planning and communication
Year
– Forward planning and capacity control
4. Reduced risk of disruption
– Impact analysis is easier, less mistakes
5. Less effort to keep accurate documentation
 What Types of Documentation?
Specific path connectivity
– End Points and paths

Definitions
– Types of ports, addressing, naming, labelling

Diagrams
– Views of physcial locations, physcial and logical paths

Capacity reports
– Fixed infrastructure (patch panels, power strips, PDUs)
– Active infrastructure devices (network switches, SAN, PBX)

Management & Workflow

– Reservation, build instructions, audit trail, changes
 Example - Cable Labeling
A C
B D E

Patch Patch Patch Patch

Panel Panel Panel Panel
SW01 01 02 13 14 UX01
F

Patch Patch Patch

Panel Panel Panel
SW02
G UX02
03 23 24

SW03 UX03
 Cable Labeling Examples
Possible Options - Some Benefits

1. No Label
No administration requirements

2. Port number at each end

Easy to ensure cable is in right port
(SW01/P01)
3. Local end points of cable
Easy to trace ends of patch cable
(SW01/P01 – PP01/01)
4. Device end points of cable
Easy to know impact of disconnect
(SW01/P01-UX01/ETH0)
5. Unique label on each cable
Easy for software tools to reference
(04567489)
6. Common path label on cable
Easy for workflow and end path tracing
(1123238)
 Recommended Labeling
• Our recommendation is;
– Unique labels at both ends of a patch cable
• Why
– No re-labelling if devices change names
– Cables can be re-used, pre-patched
– Easy to audit recent changes
– Needed by software tools as a reference
• But you need to look for a software tool
that accepts (and can create) cable labels
 Reducing The Amount of Data
Before - uncoordinated data After – Less data sources
Consistent views & reports
VisioWord
Excel Capacity & audit trails
Word
Word Visio Workflow and reservation
Visio Visio Excel Word
Excel Word
Excel

Visio Excel
Visio
Word
Word
Word Excel
Excel
Visio
Excel
Word

Word Reporting
Visio

Visio
Excel Excel
Excel Excel
Define the Level of Detail
1. Local patch 2. End to End path

Patch Patch Patch

Panel Panel Panel

Patch
Panel
3. All devices
connected to the
switch
Patch
Panel
 Example – Do This Yourself
Assets/Inventory Network G10-3

SW-BHAM-
CORE2
G10-4

SFP1 SFP1

Excel
11
SW-BHAM-05 SW-BHAM-04 23 SW-BHAM-02
27 15
9 1

ETH 0 ETH 2 ETH 0

Visio ETH 0
ETH 0
ETH 2

UK_BIRM_UX06 UK_BIRM_UX07 UK_BIRM_UX04 UK_BIRM_UX08 UK_BIRM_UX10

Port Connections
Power
Excel

1. Draw diagram using Excel data

2. Refresh Visio for updates
 More Sophisticated
1. Paths Physical
Specialist DC Logical
Management Toolset Device

2. Diagrams Physical
Logical
Multi-technology
Outputs
3. Capacity Fixed infrastructure
Active components
Power

Data Feeds 4. Workflow Reserve, design

Monitoring, Discovery, Test Build instructions
Results, Project Plans, Other
It is wise to develop your own
requirements before choosing
any specialist toolset!
 Steps to Success - Build
• Ensure hand over documentation reflects the
built environment
• Insist that format and content are consistent with
the standards or systems adopted
• Why not get suppliers to deliver the operational
processes as well as infrastructure data?
– They do it already for HVAC and power systems
• Any further works should result in updates to
existing documentation sets
 Steps to Success - Operate
1. Create an inventory
- All components involved in connectivity
2. Document the fixed infrastructure
- Backbone, power, SAN
3. Record the connectivity
- Paths, ports, labels
4. Create reports and diagrams to suit the need
- Capacity, topology diagrams
5. Embed in project workflow
 Are We Working Too Hard?
• Reverse engineering existing systems
• Producing different views of connectivity
• Travelling and meeting unnecessarily
• Coping with inconsistent information

• Be smart – work more as a team!

– Save on cost, time and effort
– Increase the level of control
Thank you for your attention

Questions or feedback?

David Cuthbertson
Square Mile Systems Ltd

www.squaremilesystems.com
www.assetgen.com