IT communication services in

accelerators and experiments

F. Chapron
IT/CS

15th of November 2012

CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it
 Agenda

Fixed Services
The technical network
VDSL (Wi-Fi) network
Public Address and Intercom
Fixed telephony services

Wireless Services
Wi-Fi
Mobile services
Radio services

CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it
 Fixed Services

CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it
 The Technical Network (TN)
Service definition:
Aim at providing high speed and reliable fixed network connectivity for the accelerator control
Protected from external access
Provide 10Mbps / 100Mbps / 1Gbps connection
Availability
Limited in PS ring,
At the top of all SPS sites (BAs), down in BA4 (CNGS)
LHC in Experimental caverns and Alcoves where there are UTP plugs
All Experiments
And a LOT of technical buildings (EN/EL, EN/CV, Access control, etc.)
Situation :
The switches (500 units) are obsolete and must be replaced to avoid jeopardizing beam stability
The new switch type is tested: Some of them are operational (startpoint in bld. 212, CCC)
The starpoint cabling will be re-done (takes a bit more time) while replacing switches
Constraints
Time to perform the replacement
Service interruptions may be up to 1 day
Network extensions (e.g. New network plugs or new starpoints) takes several weeks to be
implemented
Fibres may be missing to upgrade network uplinks to 1 Gbps

Contact name in IT/CS: Maryse Da Costa

Advantages/Drawbacks:
+Operated by CERN - Replacement feasible only during LS1
+Fully operational - Network plugs are 90 meters max from starpoints
CERN IT Department
CH-1211 Genve 23
+High data transfer
Switzerland +Low latency
www.cern.ch/it +Permanent and reliable solution
 VDSL & Wi-Fi (Only in LHC)
Service definition:
VDSL : Very high bit rate Digital Subscriber Line.
Similar to your ADSL line at Home
Plugs available every 100m in LHC tunnel connected to GPN(*)
MAX 50Mbps full-duplex up to 500m from DSLAM switch
Transmission capacity degrades with distance
Allow to connect a VDSL2 modem with Wi-Fi capability
but must be removed during LHC operation

Situation :
The VDSL switches in LHC alcoves are obsolete and must be replaced.
The new switch VDSL switches were selected and bought with a simplified all-in-one modem
VDSL2 with Wi-Fi Modems will be deployed by CS all around LHC tunnel for LS1
The existing Wi-Fi infrastructure in other LHC areas wont be upgraded
Schedule:
The VDSL infrastructure will be upgraded in LHC5 to LHC6 during TS4 (could be extended if we
have time).
The VDSL sockets in LHC5 to LHC6 will be tested
From mid-February:
Deployment of VDSL + Wi-Fi modems in LHC5 to LHC6 (2 days of work)
Upgrade of VDSL switches + Deployment of VDSL + Wi-Fi modems in all LHC tunnel (6 weeks)
Once the VDSL infrastructure is upgraded in an area the old VDSL kits do work any longer
Contact name in IT/CS: Sebastien Ceuterickx
Advantages/Drawbacks:
+Operated by CERN - Limited reliability (for now) GPN = General purpose network
CERN IT Department
CH-1211 Genve 23
+Fully operational - Only in LHC
Switzerland +Medium data transfer
www.cern.ch/it +Low latency
 VDSL & Wi-Fi (Only in LHC)

CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it
 Public Address (PA) and Intercom
Service definition:
PA: Broadcasting of voice messages (dynamic or pre-recorded) in specific areas
Intercom: Point to point communication between specific locations
In PS complex :
PA infrastructure is very very old !
Not monitored
Few spare parts
Low technical expertise at CERN (low technical documentation)
No redundancy
Intercom : CCC, PS, LINAC, BOR RF, ISOLDE
Around 20 stations connected over IP : No redundancy, low reliability (sensitive to power
cuts)
at Fire Brigade:
Intercom and PA Infrastructure to be replaced in the context of the TETRA project
General IT strategy
No modification accepted on intercom and PA networks
Decommission this public address networks ! (done 501, 504, 513)
Replace intercom stations by equivalent PABX system (Done in Medical Service)

Site surveillance and access control to SPS/PS based on intercom

Responsibility was transferred to GS/ASE in 2008
CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it Contact name in IT/CS: Frdric Chapron
 Fixed telephony services

Service definition
PABX network comprising 10 000 users
Targeted for voice communications
Fully redundant
4 hours of power autonomy with diesel back-up
Cover all CERN sites
Interconnected with 5 telecom operators (least cost routing)
Emergency phones (Red Phones) in LHC but not in PS and SPS
Intercom devices
Telephone Switchboard
Redundant call centers (with call prioritization)
Service Desk
IT Helpdesk
CCC
Fire Brigade
Voice recording
Voicemail
Soon in electronic mailbox
Audio conference services
Traditional services on PABX
Main PABX build 58
IP based MyTeamWork (will be replaced)
IP telephony services
IP Phones / Soft phones
Fax over IP integrated in electronic mailbox
CERN IT Department
CH-1211 Genve 23
Switzerland Contact name in IT/CS: Rodrigo Sierra
www.cern.ch/it
 Wireless Services

CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it
 Wi-Fi Services
Service definition:
Access to CERN General purpose network only!
Previous generation: 802.11g/a MAX 22Mbps Half-duplex Shared
Current generation: 802.11n from 120Mbps to 260Mbps Half-duplex Shared
Recommended max 25 stations per cell
To deal with multi-path environment such as underground, Antennas with spatial diversity, leaky
feeder or MIMO system are required
Situation
In LHC
35 Access Points in the UA/UJ/UL
only 802.11abg
Old equipment and technology to be replaced
Limited coverage in these areas
In SPS : only BA4 (CNGS) is partially covered
In underground areas: extensions are limited as network outlets not available everywhere
On surface: all is feasible but budget ?
http://service-wireless.web.cern.ch/service-wireless/service_evolution.html
Contact name in IT/CS: Sebastien Ceuterickx

Advantages/Drawbacks:
+Operated by CERN - Infrastructure costs covered by the
+Fully operational (where available) requesting service
+No need for special terminals - Expensive installation
CERN IT Department
CH-1211 Genve 23 +High data transfer - Infrastructure must be replaced
Switzerland - Limited coverage
+Low latency
www.cern.ch/it
 Mobile Services:
GSM/GPRS/EDGE technologies
Service definition:
Available everywhere at CERN out-door and indoor
Voice traffic:
16 simultaneous calls max between two consecutive LHC REs or RE to adjacent LHC Pit or between 2
BAs
8 simultaneous calls in PS, Booster, Linac II/III/IV, ISOLDE, CTF3, DIRAC, AD, ISR, FTN-TOF
Voice is handled with a higher priority than messaging and data traffic
Messaging:
SMS (Short Message Service) : 160 characters maximum
MMS (Multimedia Messaging Service): allow pictures and voice message attachments
Data transfer allows access to Internet OR CERN GPN through a mobile device (phone or USB
dongle):
No extra cost on Sunrise network
GPRS = General Packet Radio Services 2.5G
Maximum theoretical Data transfer rate : 170 kbits/s (practical: not more than 115kb/s)
Latency : several seconds
EDGE = Enhanced Data Rates for GSM Evolution 2.75G
Maximum Theoretical Data transfer rate : 473 kbits/s (practical: not more than 200kb/s)
Latency : ~800ms
Situation:
Fully operational
If subcontractors => promote CERN subscriptions to avoid paying when calling them!

Contact name in IT/CS: Aurlie Pascal

Advantages/Drawbacks:
+Infrastructure costs covered by CERN - Operated by Sunrise
+No extra costs when used in Switzerland - Poor latency
CERN IT Department
CH-1211 Genve 23 +Fully operational - Limited data transfer rate
Switzerland
+No need for special terminals March 2011
www.cern.ch/it A. Pascal Slide 11
 Mobile Services
UMTS 900 MHz (HSPA) in LHC
Service definition:
Similar to GSM/GPRS/EDGE services
UMTS = Universal Mobile Telecommunications System -> 3G
HSPA = High-Speed Packet Access-> 3.5G

Data transfer (theoretical): Basic tests in SD8:

21 Mbps for the downlink One emitter in LHCb and one in RE12
5.8 Mbps for the uplink Transmission not yet upgraded (limited to 5.4Mbps only)
Latency: < 100ms The maximum throughputs reached during the tests:
Real time Video 4.7 Mbps download and latency between 100-400 ms
1.3 Mbps upload and latency between 70-200 ms
4.2 Mbps dl / 0.5 Mbps ul and latency around 300-500 ms
Situation
Deployment in the LHC complex and experiments is on going (finished in
December 2012)
not yet on surface!
UMTS 900 MHz:
All new 3G devices compatible but not old mobile phones
Fibres may be missing to connect all repeaters in LHC Alcoves

Contact name in IT/CS: Rodrigo Sierra

Advantages/Drawbacks:
+Infrastructure costs covered by CERN - Operated by Sunrise
+No extra cost when used in Switzerland - Not yet operational
CERN IT Department +Reasonable latency (to be checked) - Need compatible devices
CH-1211 Genve 23
Switzerland +Reasonable data transfer
www.cern.ch/it +Available everywhere in LHC and Exp.
 Deployment in 2012/2013
UMTS 900 MHz (HSPA)

CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it
 Fire Brigade VHF network

The current VHF fire brigade network is at this end of life:

The infrastructure is difficult to maintain

recurrent problems due to old equipment (installed in 1994)

The infrastructure cannot be monitored

The network features do not meet any longer the new

operational needs of the fire brigade
No direct Inter-sites communications

The trust of the Fire Brigade in this VHF network is getting

lower and lower because of the recurrent problems

CERN IT Department This is putting safety at risk!

CH-1211 Genve 23
Switzerland
www.cern.ch/it
 New radio services :TETRA
Service definition
TETRA (Terrestrial Trunked Radio) is a secured digital radio network technology
First dedicated to Firebrigade operational needs
Then open to any other services which need group communications and
features of TETRA technology
Dimensioned for 500 terminals (is scalable to support several thousands)
In parallel, allow French and Swiss fire brigades to intervene in CERN accelerator
complex with their own radio devices
Key features:
Full outdoor/indoor coverage
High level of reliability and availability (all active equipment are on UPS + Diesel)
ATEX terminals (allow to enter potentially explosive areas without risk)
SDS messages (like SMS) 140 characters maximum/message
Packet Data Services (on CERN GPN):
Max Data transfer rate : 19.2kbits/s Latency : 250 ms
Lone worker protection!
Outdoor and Indoor geo-localization (to be confirmed for indoor)
Situation:
Core network being commissioned at the moment
CERN guards will start using it in November (thanks for playing guinea pigs!)
Contact name in IT/CS: Aurlie Pascal
Advantages/Drawbacks:
+Fully operated by CERN - Very Poor performances for data transfers
+Designed for safety - Specific and expensive terminals
CERN IT Department
+Infrastructure costs covered by CERN - Available only on CERN region
CH-1211 Genve 23
Switzerland +Group communications
www.cern.ch/it A. Pascal
+Outdoor & indoor geo-localization March 2011 Slide 15
 Expected surface radio coverage

Base Station

Repeater

Software: ProMan
from AWE

CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it 16
 Lone Worker Protection & Indoor geo-
localization (to be confirmed)
DISPATCHER / FB

2. When catching the ID

from the beacon, the
terminal forwards its
location to the Dispatcher
Workstation at the Fire
Brigade

2
3

1
3. When incident : man
down* detected or
emergency button
pressed emergency 1. The Beacon sends
SDS sent to the Fire its ID through a
Brigade Dispatcher remote antenna to
TETRA terminal

CERN IT Department
CH-1211 Genve 23 * Lone Worker Protection/Man down = lose of verticality and/or emergency call
Switzerland
www.cern.ch/it
 Summary of wireless data services in
technical areas

TN TETRA GPRS WiFi WiFi + UMTS

EDGE VDSL
Safety No Yes No No No No

Reliability Excellent Excellent Fair good Fair Fair

Availability Excellent Excellent Excellent Limited Limited Excellent

(for LHC)

Data rate Excellent Very Low Low Excellent Good Good

Latency Excellent Good ? Bad Excellent Excellent Excellent

(is real time ok?)

Costs Expensive Terminals Low Very Kits Low

expensive

CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it
 END

CERN IT Department
CH-1211 Genve 23
Switzerland
www.cern.ch/it