Professional Documents
Culture Documents
Internship Report: University of Applied Sciences Wildau
Internship Report: University of Applied Sciences Wildau
INTERNSHIP REPORT
supervising professor:
Prof. H. Helstrup
Contents
List of Figures
List of Tables
Glossary
1 Abstract
2 Preamble
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Description of the task . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
7
7
3 ALICE
3.1 General description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Time projection chamber . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
8
10
4 Detector electronics
4.1 Detector-Control-System . . .
4.2 Front-End-Electronics Server .
4.3 Readout-Control-Unit . . . .
4.4 Reconfiguration of the FPGA
.
.
.
.
11
11
11
12
13
.
.
.
.
.
14
14
15
15
16
18
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6 Conclusion
19
Bibliography
20
List of Figures
3.1
3.2
8
10
4.1
. . . . . . . . . . . . . . . . . . . . . . . .
12
5.1
5.2
14
18
List of Tables
5.1
5.2
5.3
Initial-Configuration-Block . . . . . . . . . . . . . . . . . . . . . . . . . .
Scrubbing-Block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Frame-by-Frame-Readback-Block . . . . . . . . . . . . . . . . . . . . . .
16
17
17
Glossary
ALICE
APR
CERN
Co2
Carbon dioxide
DCS
FEC
FEE
FPGA
LHC
N2
Nitrogen
Ne
Neon
QGP
RCU
TeV
TPC
1 Abstract
Radiation effects may influence the behaviour of logic devices. So the FPGAs that are used
in the ALICE Detector-Control-System (DCS) may expect these errors. To correct them the
data has to be prepared. In this report a solution to generate data for reconfiguration of
FPGAs is described.
Zusammenfassung
Strahlung beeinflusst das Verhalten von elektronischen Bauteilen. Die FPGAs, welche im
Detector-Control-System (DCS) des ALICE Experiments zum Einsatz kommen, arbeiten in
einer solchen Umgebung. Um den Auswirkungen der Strahlung zu begegnen wird der Speicher der FPGAs neu beschrieben. Die dafur benotigten Daten mussen im Vorfeld aufbereitet
werden. In diesem Praktikumsbericht wird ein Werkzeug vorgestellt, mit dessen Hilfe man
diese Daten generieren kann.
2 Preamble
2.1 Introduction
In these days at CERN1 near Geneva in Switzerland the Large-Hadron-Collidor (LHC) is
commissioned for its first run on September 9th. When this machine is completed it will be
the biggest one ever built. Physicists hope to investigate with this machine the origin of our
todays matter. Therefore several experiments are running. One of them is ALICE2 , where the
origination of today known particles like protons and neutrons is investigated.
1
2
Conseil Europeen pour la Recherche Nucleaire (European Council for Nuclear Research)
A Large Ion Collidor Experiment
3 ALICE
3.1 General description
ALICE is part of the LHC which is currently commissioned at CERN in Geneva. LHC consists
of a 27 km circumference tunnel where hadrons (protons and neutrons from atomic nuclei)
are accelerated up to nearly speed of light by two beams of opposite directions. The hadrons
collide on four points where the rings intersect. At these points detectors are installed to
grab any kind of data concerning the collision. Figure 3.1 shows the general layout of ALICE.
The acceleration to nearly speed of light gives the protons an energy of 7TeV, so that
when two protons collide the energy doubles up to 14 TeV. To imagine the energy, it can be
assumed, that the motion energy of a flying mosquito is about 1TeV, but the space in the
accelerator is many times smaller than a mosquito.
The goal of ALICE is to investigate the situation immediately after the big-bang. This
state is also known as QGP (Quark-Gluon-Plasma) and has last for only 10 microseconds
after the big bang. In this state, the particles and structures as we know them today, have
their origin. They were established when the QGP cooled down and collapsed to protons and
neutrons.
To reconstruct this state high energy is necessary, so that the particles can melt and squeeze.
This enormous energy is reached by acceleration to nearly speed of light. For more information
about ALICE see [1].
The FECs of each sector are connected to a Readout-Control-Unit (RCU), which grabs
the data from the FECs and sends them to the Data-Acquisition-System where either long
time saving for later offline reconstruction is made or online High-Level-Triggering. During
the experiment data rates up 80 MByte per event occur. When ALICE is running the events
are sampled by a rate of 100 kHz, so that the data rate increases up to hundreds of GByte.
10
4 Detector electronics
4.1 Detector-Control-System
The Detector-Control-System (DCS) used in many different detectors of ALICE. In the TPC
it is mounted on the RCU card and provides remote access to their functions. This board
contains an ALTERA FPGA which comprises itself an ARM9 controller. In addition to this
there are an 8 MB Flash ROM (radiation tolerant) and 32 MB SDRAM, an ethernet interface
and some other auxiliary devices mounted. This basic equipment allows running a light-weight
version of Linux, so that the devices can be accessed from remote.
11
4.3 Readout-Control-Unit
The TPC Readout-Control-Unit (RCU) was developed to readout, control and monitor any
kind of FECs in the TPC. The physical layout consists of a main FPGA which controls,
monitors, configures and reads out the FECs. The functional design can be distinguished in
four blocks:
ALTRO3 -Bus controller is used to connect the FECs to the RCU [3].
monitoring and safety module, that monitors power state, voltages, currents and temperature state of the FECs
interface to Detector Data Link Source Interface Unit (DDL SIU), which provides a
protocol to send data to the Data Acquisition (DAQ) or receives configuration data
for the FECs
interface to the DCS / Trigger board, this receives trigger data or configuration data
for the FECs or transmits them to the DCS
In the following figure you can see the functional sketch of the RCU (Figure 4.1).
12
13
This tool enables the user one the one hand to split a configuration file up to several frame
files and on the other hand to generate binary command blocks. To increase the usability the
program provides an additional SSH part where the user can either load the generated files
up to a dedicated server or program the devices on the RCU directly.
Since this tool should be able to handle different versions of Virtex FPGAs an external
configuration file is used. This configuration file is XML based and provides information for
all types of FPGAs. Furthermore all information concerning the binary command blocks are
included.
14
15
This algorithm is used to generate raw frame files, read frame files, write frame files and
the binary command for frame-by-frame readback (see section 5.2.2). The difference between
raw frames and read/write frames is that these frames contain header and footer fields where
information about the addresses the data is stored to and some commands is provided. Since
these values are device specific they are also available in the XML configuration file.
5.2.2 Binary-Command-Blocks
Besides generating frame files for APR the program is also able to generate blocks of binary
commands. These command blocks are used in the FeeServer (introduced briefly in section
4.2). As explained in [7] the FeeServers ControlEngine (CE) contains classes that represent
the hardware devices the DCS and RCU cards contain. One of these devices is the Actel
FPGA which is responsible for the reconfiguration of the Xilinx device. To handle this reconfiguration procedure the Actel needs instructions and data. This both is contained in the
binary command blocks ([7]) generated by the tool developed during this internship.
There are three kinds of command blocks being generated.
1. Initial-Configuration-Block
Header
0xF4060000
Initial addresses
...
Data
initial bitfile
Command tailer
0xDD330000
When programming a new firmware version to the Xilinx also the copy in the flash
memory has to be updated. To do this the Initial-Configuration-Block as seen above is
used. The header field contains the command what to do with the following data. The
fields below are addresses in the flash memory and the configuration file for the Xilinx
device. Closing this command block at the end the command tailer can be found. This
field consists of a command end marker and the command version.
16
2. Scrubbing-Block
Header
0xF4070000
Scrubbing addresses
...
Data
scrubbing bitfile
Command tailer
0xDD330000
In case of scrubbing, the block contains another bitfile which is surrounded by commands and addresses sent to the Actel. Similar to the Initial-Configuration-Block this
block also contains a command which tells the Actel what to do with the data, some
addresses for the flash memory and the data itself.
3. Frame-by-Frame-Readback-Block
Header
0xF40303A8
Frame
dresses
...
ad-
Data
control
information
concerning the
frames
Readframe
Data
readframes
Writeframe
Data
writeframes
Command tailer
0xDD330000
17
In this dialogue the user can choose either a file or a folder to load up to a dedicated
location on the server the user has write access to. To ensure a safe communication
the SFTP subsystem is used. Establishing a SFTP session a valid SSH session must
exist. File operations on this system are quite similar to normal POSIX5 file operations.
destFile = sftp open(sftpSession, destPath, O CREAT | O TRUNC | O WRONLY, NULL);
//destFile can be compared to a normal POSIX file descriptor
The only difference is that the SFTP session has to be added. Specifying the path where the file has to be opened and the opening flags (creating the file, truncating the content if
the file exists and opening the file for write access only) are similar to the POSIX standard.
4
5
http://0xbadc0de.be/wiki/libssh:libssh
Portable Operating System Interface, http://standards.ieee.org/regauth/posix
18
6 Conclusion
According to the introduction, the tool developed during this internship enables the user to
generate reconfiguration data from remote. This makes the work easier and the flexible design
ensures future extension and association to other FPGAs. Furthermore the ability to load the
generated files directly to system there are needed makes the work more efficient. However
it must be mentioned, that for a complete remote configuration the ability of programming
the Xilinx device from remote is not yet implemented completely. If this is working no direct
access to the DCS cards will be necessary.
19
Literaturverzeichnis
[1] The ALICE Collaboration, ALICE Technical Proposal for A Large Ion Collidor Experiment at
CERN LHC, Technical Report, CERN, 1995
[2] Gonzales Gutierez, C. et al., The ALICE TPC Readout Control Unit, in Nuclear Science Symposium Conference Record, vol. 1, pp. 575 - 579, 2005
[3] Esteve Bosch, R. et al., The ALTRO Chip: A 16 channel A/D and Digital Processor for Gas
Detectors, in IEEE Transactions on Nuclear Science, vol. 50 no. 6, pp. 2460 - 2469, 2003
[4] Richter, M. et al., The control system for the Front-End Electronics of the ALICE Time Projection Chamber, in IEEE Transactions on Nuclear Science, vol. 53 no.3, pp.980 - 985, 2006
[5] Carmichael, C. et al., Correcting Single-Event Upsets through Virtex Partial Reconfiguration,
Xilinx Application Note XAPP216, v1.0, 2000
[6] Bablok, S., Development and implementation of a safe and efficient communication software
in a heterogeneous system environment of a major research project, Diploma Thesis, University
of Applied Sciences Worms, 2004
[7] Fehlker, D., Development and commissioning of a software environment for controlling and reconfiguration of Xilinx Virtex FPGAs, Diploma Thesis, University of Applied Sciences Mittweida,
2007
[8] http://web.ift.uib.no/ kjeks/wiki/index.php?title=The Actel software device in the
FeeServer CE#Composition of the Binary command blocks, 2008-08-15
20
ERKLARUNG
Hiermit versichere ich, dass ich die vorliegende Arbeit selbststandig und ohne Benutzung
Stefan Kramer