www.thalesgroup.

com

Development and certification of

Avionics Platforms on Multi-Core
processors
CTIC CONFERENCE – MAY 2013

Marc GATTI – August 29th, 2013

/ Context

 This presentation is based on the final report

that concludes the MULCORS project contracted
with EASA.

 The reports provides the main outputs,

recommendations and conclusions per EASA
Specifications attached to the Invitation to
Tender EASA.2011.OP.30.
  
 Access to MULCORS report
 https://www.easa.europa.eu/safety-and-research/researc
h-projects/large-aeroplanes.php
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
/ AGENDA

 Multi-core:
 Introduction
 Problems to Solve
 Regarding certification
 Software Aspects
 Failure Mitigation Means & COTS Relative
Features
 Conclusion
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
/

Introduction

MULTI-CORE
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
/ Multi-Core: Introduction

 Multi-Core processor
Architecture: Unified
Memory Access

 Multi-Core processor
Architecture: Distributed
Architecture

Multi-Core processor
CTIC CONFERENCE – MAY 2013


Architecture: Single
Address space, Distributed
Memory
This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
/ Multi-Core: Introduction
Airb. SW Airb. SW Airb. SW
 Intended Function
Drivers Drivers Drivers  HW adaptation Layer (BSP)
O.S. O.S. O.S.  Hypervisor layer (when required)
 Operating System
Hypervisor
 Drivers
BSP BSP BSP  Airborne Software
Core Core Core Core Core Core

Cache Cache Cache Cache Cache Cache

External Network
External Bus
Register BUS Register Register BUS Register

EXT MEMORY EXT MEMORY

CTIC CONFERENCE – MAY 2013

Register Register Register Register

INTERCONNECT

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
/

Problems to Solve

MULTI-CORE
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
/ Multi-Core: Introduction
 What’s a multicore processor?
 Multicore processor characterized by N (N ≥ 2) processing cores + a set of
shared resources (Memories, PCIe, Ethernet, Cache, Registers, etc.)
CTIC CONFERENCE – MAY 2013

 Two main types of processors

 The first one where interconnect between cores is based on an arbitrated bus
 The second one where interconnect between cores is based on a network
 Multicore management in certified embedded platform can be summarize to
This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
/ Multi-Core: Introduction
 Access conflits
 Interconnect between cores
 If InterConnect = bus  Accesses arbitration is done at this level
 If InterConnect = network  Accesses arbitration depend of numbers of authorized
parallel routes (Memories accesses, Bus accesses, Networks accesses, etc.)

Conflicts
Si
SiInterConnect
InterConnect= =Réseau
Management BUS

Conflicts Conflicts
Conflicts Management Management
Conflicts
Management
Management
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
0 / Multi-Core: Introduction

 DETERMINISM IN EMBEDDED AIRCRAFT SYSTEMS

 Abstract notion partially described in DO-297
 Definition based on
 Execution Integrity
 WCET analysis
 Platform Usage Domain
 Robust Partitioning (not only for IMA system)

 Multicore COTS Processors

 Conflicts Management
 Spatial Management: how to manage accesses to be sure that one
core can’t access to a space reserved for another core.
 Temporal Management:
 For Memory Accesses
CTIC CONFERENCE – MAY 2013

 Operating System
 Architecture Choice regarding Industry needs (AMP or SMP)

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
1 /

Regarding Certification

MULTI-CORE
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
2 / Processor Selection

 Manufacturer Selection criteria

 Experience in Avionic domain
 Experience with the certification process
 Publication
 Life expectancy
 Long term support
 Design information on COTS processor
 Robustness tests like SEE (Single Event Effect) or SER

 Processor Architecture Focus

 Virtual Memory service
 MMU components
CTIC CONFERENCE – MAY 2013

 Use of hierarchical memory to improve Software

performances
This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
3 / Multi-Core Processor features

 INTERCONNECT
 The first shared resource between cores.
 Interleaves concurrent transactions sent by the cores to the
shared resources
 Architecture and impact on determinism
 Architecture and partitioning insurance
 Interconnect services to be managed
 Arbitration of incoming requests
 Arbitration rules
 Arbiter internal logic
 Network topology
 Allocation of the physical destination devices
 Allocation of a path to the destination.
 Support for atomic operations,
 Hardware locking mechanisms
 Snooping mechanisms for cache coherency
CTIC CONFERENCE – MAY 2013

 Inter Processors Interruptions (IPI) for inter-core communications

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
4 / Multi-Core Processor features

 SHARED CACHE
 Shared cache in Embedded Aircraft Systems requires a solution to the
following problems:
 Shared cache content prediction.
 Cache content integrity. .
 Concurrent accesses impact.

 Cache organizations
 Fully associative
 N-way set associative cache
 Direct mapped cache
 Replacement policies

 CACHE COHERENCY MECHANISM

 Required in architecture that integrates several storage devices
CTIC CONFERENCE – MAY 2013

hosting same data.

 Coherency protocols:
 Invalidate protocols
 Update protocols
This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
5 / Multi-Core Processor features

 SHARED SERVICES
 Providing Shared Services among the cores.
 Interruptsgeneration and routing to cores
 Core and processor clock configurations
 Timer configurations
 Watchdog configurations
 Power supply and reset
 Support for atomic operations

 CORES
 Support execution of multiple software instances in parallel.
 Use of inter-core interrupts.
 Memory mapping defined in the Memory Management Unit.
CTIC CONFERENCE – MAY 2013

 Warning: A non-coherent configuration may weaken Robust Partitioning.

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
6 / Multi-Core Processor features

 PERIPHERALS: MAIN MEMORY AND I/O’S

 Sharing main memory  sharing physical storage resources and
memory controllers.
 Space partitioning: Storage resource can be partitioned when necessary.
 Sharing accesses to the memory have to be well managed.

 Shared I/O features similar to shared services configuration:

 Access simultaneously read and/or write buffers.
 Classic rules of time and space partitioning can be applied
 Initiate specific protocols operations: uninterrupted access is required during the
protocol execution to be able to fulfill correctly the concerned protocol.
 Concurrent accesses to shared I/O may occur simultaneously from different
CTIC CONFERENCE – MAY 2013

cores.
 Some I/O are accessed according to a protocol, others are accessed from a read
and/or write buffer  Atomic access patterns have to be ensured.

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
7 /

Software Aspects

MULTI-CORE
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
8 /
Partitioned system features
 Components evolution to take benefit of multi-core platforms

 The most “flexible” component is the

integration software layer. Possible designs:
 A single OS instance shared among all the cores
 A private OS instance per core
 A virtualization layer hosting several operating systems
in dedicated virtual machines.

 Partition Deployment
 One partition is activated on all cores and has an exclusive access to platform
resources
 Symmetrical Multi-processing (SMP).
CTIC CONFERENCE – MAY 2013

 Each partition are activated on one core with true parallelism between partitions
 Asymmetrical Multi-processing (AMP).

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
9 / Operating System global view

From Single Core to Multi-Core in AMP (Asymmetric multi-processing)

APP1 APP2 APP3

T1 T1 T1

T2 T2
T2

T3 T3 T3

T4
T4

T5

Space & Time Partitionning Space & Time Partitionning Space & Time Partitionning
Operating System Operating System Operating System

CORE CORE CORE

BRIDGE INTERCONNECT
Solve
Memory
Memory I/O
I/O
BUS
BUS // Memory
Memory I/O
I/O
Conflict BUS
BUS // Memory
Network Network Memory
Controller
Controller Controller
Controller Network Controller Controller Network Controller
Interface Controller Controller Interface Controller
Interface Interface
CTIC CONFERENCE – MAY 2013

Example of two cores processor and two memory controllers.

For more than two cores (or less than two Memory Controller) conflicts to the
Memory Controller have to be managed

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
0 / Operating System global view

From Single Core to Multi-Core in SMP (Symmetric multi-processing)

APP1 APP2 APP3 APP1

T1 T1 T1 T1

T2 T2 T2

T2
T3 T3 T3

T4 T3
T4

T5 T4

Space & Time Partitionning Space & Time Partitionning

Operating System Operating System

CORE CORE CORE

BRIDGE INTERCONNECT
Solve
Memory
Memory I/O
I/O
BUS
BUS // Memory
Memory I/O
I/O
Conflict BUS
BUS // Memory
Network Network Memory
Controller
Controller Controller
Controller Network Controller Controller Network Controller
Interface Controller Controller Interface Controller
Interface Interface
CTIC CONFERENCE – MAY 2013

Example of two cores processor and two memory controllers.

For more than two cores (or less than two Memory Controller) conflicts to the
Memory Controller have to be managed

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
1 / Current mono-core concept

APP1 APP2 APP3

T1 T1 T1

T2 T2 T2

T3 T3 T3

T4
T4

T5

Space & Time Partitionning

Operating System

CORE

BRIDGE

Memory I/O BUS

BUS //
Memory I/O Network
Controller
Controller Controller
Controller Network
Interface
Interface

Thread /
Process
T5
T4 T4 T4 Appli. 1 T
Core
CTIC CONFERENCE – MAY 2013

OS
T3 T3 T3 T3 Appli. 2 T
T2 T2 T2 T2
Appli. 3 T
T1 T1 T1 T1 T1 T1 T1
idle
time
Partition 1 Partition 2 Partition 3 Partition 4

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
2 /
APP4 APP5 APP5 AMP
APP1 APP2 APP3 T1
T1 T1
T1 T1 T2
T1 T2 T2
T2 T2
T2 T3
T3 T3
T4 T3
T3 T3 T4
T4
T5

Space & Time Partitionning Space & Time Partitionning

Operating System Operating System

CORE CORE

INTERCONNECT

Memory I/O BUS

BUS //
Memory I/O Memory
Memory
Controller Controller Network
Network
Controller Controller Controller
Controller
Interface
Interface
Core 1

T5 T4 Thread /
T4 Process

OS 2
T3 T3 T3 T3 Appli. 1 T
T2 T2 T2 T2 T2 Appli.2 T
T1 T1 T1 T1 T1 T1
Appli 3 T
Partition 1.1 Partition 2.2 Partition 2.3 Partition 2.4 Appli 4 T
Core 2
CTIC CONFERENCE – MAY 2013

T4

OS 1
Appli 5 T
T3 T3 T3 T3 T3 T3
Appli 6 T
T2 T2 T2 T2
Appli 7 T
T1 T1 T1 T1 T1 T1 T1
idle
time
Partition 1.1 Partition 1.2 Partition 1.3 Partition 1.4
This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
3 / SMP
APP1 APP2 APP3
T2
T1 T1
T3 T3

In SMP mode, Processes,

T2 T4
T2
T1 T4 T3
T5

Threads or Tasks should be

Space & Time Partitionning

Operating System
allocated to cores statically to
achieve determinism
CORE CORE

INTERCONNECT

Memory I/O BUS

BUS //
Memory I/O Memory
Memory
Controller Controller Network
Network
Controller Controller Controller
Controller
Interface
Interface
Core 1 Core 2

Thread /
T2 T2 T2 T2 Process
T

OS
Appli. 1
T5 Appli. 2 T
CTIC CONFERENCE – MAY 2013

T4 T4 T4 Appli. 3 T
T3 T3 T3 idle

T1 T1 T1 T3 T1 T1 T1 T1

Partition 1 Partition 2 Partition 3

time
Partition 4
This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
4 /

Failure Mitigation Means & COTS Relative Features

MULTI-CORE
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
5 / Multi-Core: Failure Mitigation

 FMEA and/or FFPA for a single or a multi-core processor is

not achievable at processor level
 Mitigation has to be provided, by the equipment provider, at board level
where this processor is used
 Software Error Rate  SEE (Single Event Effect)
 Measurements on SER are usually performed by the manufacturers on
their own
 Deep Sub Micronics
 DSM has impact of long term reliability
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
6 /

CONCLUSION
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
7 / CONCLUSIONS

 Complexity of Multi-Core Processors

 Has increased over the past few years,
 Level of demonstration for design assurance remains at least the same as
or better than for COTS without such increment in complexity.

 A COTS component remains a COTS component

 Features proprietary data from the COTS manufacturer

 Approaches:
 Access to additional data under agreements with the COTS manufacturer
 And/or mitigation of potential COTS faults or errors at board or equipment
level,  
CTIC CONFERENCE – MAY 2013

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.
8 / CONCLUSIONS

 MULCORS put emphasis on specific Multi-Core features

linked to Shared Resource Accesses like Memory, Bus,
Network, Internal Registers, Clock Management, etc.

 Features that are the main differences between single-core

and multi-core devices that have to be managed
 Airborne Software Level
 Airborne Software behavior
 Airborne Software applications allocation to cores can demonstrate the non-interaction
between cores.
 Interconnect behavior
 Shall be well known and well managed

 Hypervisor level
CTIC CONFERENCE – MAY 2013

 Hypervisor can be used to constraint the behavior of the interconnect.

 Constraints reduce performances but offer determinism

This document is the property of Thales Group and may not be copied or communicated without written consent of Thales S.A.
Ce document est la propriété de Thales Group et il ne peut être reproduit ou communiqué sans autorisation écrite de Thales S.A.