SASSI13 Proceedings

Security Assessment for
Systems, Services and

Infrastructures (SASSI)
Proceedings
Jrgen Gromann (ed.)
SASSI13
Security Assessment for Systems, Services and Infrastructures
September 19 and 20, 2013 at the Technical University (TU) in Berlin
Security failures and data breaches are impacting not only enterprises but also critical
infrastructures and public services. Solely in Germany successful attacks on IT systems in cause
damage by 4.8 million euros a year. At the same time, we are experiencing how the current IT
landscape is changing rapidly. Just a few years ago, the Internet was dedicated to interconnect
stationary end user devices. Nowadays, the tendency towards an Internet of things makes the
situation more complex. Mobile devices, home automation, smart grids and even vehicles are
connected via the Internet and becoming theoretical accessible and thus vulnerable to hacker
attacks. However, we are more than ever dependent on a secure and mature ICT infrastructure.
One of the keys to get and maintain such a secure and dependable infrastructure is a mature,
systematic and capable security risk analysis and testing program. This workshop will provide a
forum to discuss innovative security testing approaches and their combination with security risk
analysis. At the same time, the workshop tries to draw a line to the industrial requirements and
the challenges that arise when security testing meets the demands of cost efficiency and
scalability. Experts from industry and academia will present and discuss their solutions to the key
issues security risk analysis, vulnerability testing, model based security testing, and
standardization. The contributions are complemented by industry grade research results from four
large European research projects.
Content:
Keynotes
Ralf Bker, BSI: Implementing Germany's Cyber Security Strategy
Stig Torsbakken, Nets: Risk analysis and testing - perspectives from the frontline
Session 1: Security risk assessment and testing
Jan Stijohann, SIEMENS: Towards Systematic and Traceable Security Assessment
Ketil Stlen, SINTEF: Test-based risk assessment
Samson Yoseph Esayas, Universitetet i Oslo: Legal Risk Management: a Method for Proactive Management of Legal Risks
Session 2: Standardization & Certification
Gerard Gaudin, G2C: A full set of new standards in Cyber Defence addressing the full scope of security event detection issues
Jrgen Gromann, Fraunhofer FOKUS: Security Testing Improvment Profile (STIP)
Luca Vigano, Universit di Verona: The SPaCIoS Tool - property-driven and vulnerability-driven security testing
Luca Compagna: Formal Validation and Testing of Security Standards at SAP: from research to industry
Session 3: Active security testing
Bruno Legeard, FEMTO-ST/UFC & Smartesting: Model-based vulnerability testing from patterns and behavioral model
Martn Ochoa, Siemens/TU Mnchen: Model-based vulnerability testing
Prof. Dr. Sachar Paulus, Kuppinger Cole: Trustworthy software development
Ari Takanen, Codenomicon: Traffic Capture Fuzzer: Effective method for model based fuzzing
Session 4: Active and passive security testing
Riccardo Scandariato, KU Leuven: Security vulnerability prediction
Graham Steel, Cryptosense: Security analysis of APIs, including the W3C Crypto API
Ana Cavalli, Institut Mines-Telecom: Application of passive testing techniques to secure interoperability testing
Wissam Mallouli, Montimage: Passive testing for security checking using MMT
Keynote
Ralf Bker
(Bundesamt fr Sicherheit in der Informationstechnik)
Implementing Germany's
Cyber Security Strategy
Ralf Bker
Implementing Germany's Cyber Security Strategy
Vita:
Ralf Bker is a cyber security consultant at the Federal Office for Security in IT
(BSI).
Ralf Bker
Implementing Germany's Cyber Security Strategy
Abstract:
Germany sees itself exposed to attack by cyber-activists in ever-increasing
extent. The attacks are not predictable and thus can not be planned for
thechallenged company. In recent years the attackers are becoming more
professional and effective with respect to their resources and tools. Thus, a
once achieved level of protection need always be regarded as provisional. The
Alliance for Cyber Security is an initiative of the German Federal Office for
Information Security (BSI), which has been founded in cooperation with the
German Federal Association for Information Technology, Telecommunications
and New Media (BITKOM). As a coalition of all the major players in the field of
cyber security in Germany, the Alliance aims to increase the cyber security in
Germany and to strengthen the resilience of Germany against cyber attacks.
Implementing
Germany's Cyber Security Strategy
Ralf Bker
Federal Office for Information Security (BSI)
Germany
19 September 2013
Contents
About BSI
Example: IT-Grundschutz
Germany's National Cyber Security Strategy

Evolution of cyber attacks
National Cyber Response Centre
National IT Crisis Reaction Centre
Alliance for Cyber Security

Objectives and roles
Implementation
Information pool and BSI publications on cyber security
Example: IT-Grundschutz
De-facto standard for information

security in Germany and other countries
Normal security requirements; basis for
high level security requirements
Applied by industry and administration
as best practice
IT-Grundschutz
IT-Grundschutz
Catalogues
Catalogues BSI
BSI Standards
Standards (100-1
(100-1 to
to
100-4)
100-4)
Web
Web Based
Based Training
Training
GS-TOOL
GS-TOOL (Software)
(Software)
IT-Grundschutz
IT-Grundschutz
Profiles
Profiles
IT
IT Security
Security Guidelines
Guidelines
ISO
ISO 27001
27001 Certificate
Certificate
About BSI
Prevention
Prevention
Founded in 1991
Cyber
Cyber Staff: ~ 600 employees
Security
Security Budget: 50 million
Crypto
Crypto
Innovation
Innovation
CI
CI Security
Security
Secure
Secure eIdentities
eIdentities
Certification
Certification
Awareness
Awareness Raising
Raising
Consultancy
Consultancy &
& Support
Support for
for Federal
Federal
Gov.
Gov.
Reaction
Reaction Sustainability
Sustainability
Target Groups
Public
Public Sector
Sector General
General
Public
Public
Research
Research Private
Private
Community
Community Sector
Sector
National
Cyber Response Centre
Information hub among agencies in charge

Directly participating agencies
Federal Office for Information Security (BSI, coordination)
Federal Office for the Protection of the Constitution (BfV)
Federal Office of Civil Protection and Disaster Assistance (BBK)
Staff: 10 (6 BSI, 2 BfV, 2 BBK)
Launch: 1-April-2011
Associated agencies
Federal Criminal Police Office (BKA)
Federal Police (BPol)
Customs Criminological Office (ZKA)
Federal foreign intelligence service (BND)
Federal Armed Forces (BW)
Liaison officers
Launch: 16-June-2011
BSI's Situation Room
National
IT Crisis Reaction Centre
Ensure rapid response to serious incidents

Enable timely countermeasures
Avoid more widespread damage
Standard Operating Procedures (SOPs)
24x7 availability, crisis-situation-related staff organization
Tested through exercises (also internationally)
Actual CS Situation
Evolution of cyber attacks
Cyber-Sicherheit -
Headlines
Phishing-Mails im Namen von Paypal an Eset-Kunden

(03/07/2013, Quelle: heise.de/-1910448)
Ubisoft: Unbekannte kopieren Namen, E-Mailadressen und Passwrter
von Kunden
(03/07/2013, Quelle: heise.de/-1910203)
Schwachstelle in Single-Sign-On-Software Atlassian Crowd
(03/07/2013, Quelle: heise.de/-1909611)
Attacken auf SCADA-Systeme nehmen zu
(03/07/2013, Quelle: heise.de/-1910037)
Angriff auf zap-hosting.com:
"Bekennerschreiben Hacker: einfache Universal-Passwrter,
nirgendwo SSH-Port gendert
Betreiber: Exploit auf SolusVM ausgenutzt"
(03/07/2013, Quelle: heise.de/-1909531)
Cyber-Sicherheit -
Headlines
Telekom: Router warnt bei Bot-Befall

(08.09.2013, Quelle: heise.de/-1952121)
Login-Gesten unter Windows 8 sind berechenbar
(06.09.2013, Quelle: heise.de/-1951483)
Signierte Java-Applets unter falscher Flagge
(05.09.2013, Quelle: heise.de/-1950043)
Sham G20 Summit Email Carries Split Backdoor
(05.09.2013, Quelle:
blog.trendmicro.com/trendlabs-security-intelligence/sham-g20-summit-email-carries-split-b
ackdoor/)
New online banking Trojan empties users' wallets, videos privates
(05.09.2013, Quelle:
http://www.theregister.co.uk/2013/09/05/hesperbot_online_banking_trojan/)
"Once installed, Hesperbot can silently snoop on passwords by logging a user's
keystrokes, take screenshots, record from a video camera if one is connected, intercept
network traffic, and pipe all this snaffled data to the crooks' command server. The Trojan
can also set up a hidden VNC service, allowing miscreants to remotely log in and take
control of the computer."
Large botnet cause of recent Tor network overload
Cause of Cyber-Problem
vulnerability of Standard Products
5257 new exploids in 2012 = 100 / Week
Critical Exploids 2012
Adobe Reader: 52
Adobe Flashplayer: 82
Apple Safari: 65
Google Chrome: 141
Linux Kernel: 28
Microsoft Windows: 64
Mozilla Firefox: 129
Oracle Java JDK: 141
Oracle Java JRE: 143
Quelle: BSI Schwachstellenampel

unpatched PCs (citisen / KMU)
41% angreifbar mit Microsoft XML Core Services ( 3 CVEs)

36% angreifbar mit Sun Java JRE 1.6.x/6.x (58 CVEs)
26% angreifbar mit VLC Media Player 2.x (23
CVEs)
17% angreifbar mit Oracle Java JRE SE 1.7.x/7.x (64 CVEs)
17% angreifbar mit Adobe Flash Player 11.x (70 CVEs)
15% angreifbar mit OpenOffice 3.x
( 5 CVEs)
14% angreifbar mit Adobe Reader X 10.x (69
CVEs)
14% angreifbar mit Apple QuickTime 7.x (26 CVEs)
CVE - Common Vulnerabilities and Exposures (CVE)

Quelle: Secunia
Ursachen fr
Cyber-Probleme
Ready to use Exploit Kits
Quelle: Contagio
Automatic production of Malware
At least 36.955.387 new Malware in 2012
Total: ber 100.000.000
Malware Construction Kits (example)
Winlock Builder (Ramsomware) ~80$
SpyEye ~150$
Carberp ~5000$
Polymorphe scrambling aditional ~1000$
BlackShades (Remote Administration Tool) ~75$
Beta Bot ~460$
Citadel (Banking Trojaner) ~3000$
Zeus/Zbot (Banking Trojaner) ~8000$
Quelle: Symantec, BSI/FKIE

Generierung von Schadprogrammen
- mit Komfort und Support -
Price list
Crimepack:
$400
Phoenix Exploits Kit:
$400
Adrenaline:
$3.500
(inkl. 24x7-Support)
Eleonore Exploits Pack
$700
YES Exploit System
Quelle: Pandalabs $800
Lagebild der
Cyber-Security Mobile Devices
Cause:
Exploids in Mobile Devices
Old OS Versionen: 36,5% Android V 2.3
Apple support only IPhone 4 and newer
Recklessness from User
Progression der mobilen Schdlinge

4000000
400000 35000
40000 30000
25000
4000
20000
400 15000
40 10000
4 5000
0
0,4 01.07.2012 01.09.2012 01.11.2012
klassische Malware Android andere Smartphone
Quellen: Gdata, BFK, Symantec, BSI

Allianz
Services of the
Cooperation platform for

government,
industry,
vendors, and
academia
Central information pool Exchange of

experiences
Information pool
Institutions of special public interest attack vectors,

(critical infrastructure, risk of espionage, attack analyses,
serious consequences) vulnerabilities,
services
Voluntarily registered participants

(German companies) situation,
warnings,
newsletter
Participants
recommendations,
analyses,
reporting service,
awareness
Alliance for Cyber Security in Germany
Main objectives
Assess risks of cyber space for Germany, design and
implement adequate security controls.
Strengthen national capabilities to protect cyber space, to
fend off cyber attacks, and to manage cyber crises.
On an international level, play a leadership role in the field of
cyber security.
Engagement with industry
Main goals of Cyber Alliance:

Assessment of risks for Germany in cyber space, develop and deploy
appropriate security measures.
Strengthen national capabilities for protection in cyber space, for
defence against cyber attacks and to mitigate cyber crises.
Assume a leading role internationally regarding cybersecurity.
BSI Publications on
Cyber Security
Categories
Immediate measures
Situation in cyber space
Attack methods
Tools
Best practices
Analyses
Business Continuity Management
Awareness
Background information
Stakeholder
in the Alliance for Cyber Security
Situational
Awareness Solutions
Cyber Exchange of
Security experiences
measures
Exchange of Experience
early warnings
distribution of experiences in
incident handling
dialog between solution
providers and customers
4. Alliance for Cyber Security: Information Flows
Numbers and Facts
(Stand September 2013)
Partner: 75 Multiplikatoren: 21
Homepage
Homepage
Homepage
Cyber-Security Situation Overview
Statutory Provision
At the moment there is no central and dedicated situation

overview in the field of Cyber-Security
But:
Corresponding BSIG 4 and the actual draft of the
Sicherheitsgesetzes it is a real task of BSI to provide a
situation overview for public authorities and critical
infrastructures.
To fulfill this requirement the BSI is creating a situation

overview. In a close partnership with science and
economy the BSI develops a draft version and provides
the information to all members of the Allianz fr
Objective
Why we create a overview?

Improve Cyber-Security situation in Germany sustainably
Provide risk and reasoning why investing resources in
Cyber-Security is necessary (awareness)
Create a proven and reliable source of the
Cyber-Security situation
Influence and support decision making of CISOs,
administrators and providers
Provide solutions and advice
What is not a Aim?
Deliver real Numbers
Individual risk assessment
Features
strategic operational
situation overview
up-to-date dynamic national focus online
neutral
proven continuous objective central

Target group
Who is the Target group?
Administration
German economic, CIIP Companies
Controlling institution
Research
But also institutions that have direct influence on

Cyber-Security, e.g.:
Provider
Cyber-Security service providers
Cyber-Security vendors
Development
Development and implementation will be step by step

Preparation of a first draft by BSI's experts
Open discussion with partners and target groups
Reach out to external experts who may provide input
data, e.g. providers, consultants and Cyber-Security
vendors
Implementation of the Proof-of-Concept
initial phase until start of the operational service
(iterative development)
Sources
Web-Page
BSI Allianz fr Cyber-Sicherheit
Situation Overview:
Statistics
Partner Allianz Indicators
fr Cyber-Sicherheit Numbers
News
BSI contractors
Open source
Information
Presentation
Presentation on multiple abstractionon levels
Target group Information

All
Indikator of the Day
(Start)
Management Situation Overview
(1. Level) Numbers and facts
CISO/SiBe
(2. Level) Background Information
Administrator
solutions and advice
Link (3. Level)
What's Next?
Areas of future development

Differentiation by targets groups: Administration, CIIP,
industry sectors ...
Presentation of information in greater detail
Outlook and Trends
Information and solutions for specific target groups
Content
First draft of BSI's Cyber Security Situation Overview will

cover:
Vulnerabilities
Malware
Botnets
Drive-by-Attacks
DDoS-Attacks
Spam
Exploit-Kits
Hacktivism
Contact
Federal Office for

Information Security (BSI)
Ralf Bker
Godesberger Allee 185-189
53175 Bonn
Germany
Phone: +49 228 99

9582-5368
Fax: +49 228 99 10
9582-5977
Ralf.Bker@bsi.bund.de
https://www.bsi.bund.de/EN/
Keynote
Stig Torsbakken
(Nets)
Risk analysis and testing - perspectives from the
frontline
Stig Torsbakken
Risk analysis and testing - perspectives from the frontline
Vita:
Stig Torsbakken is leading the Security Response Team (SIRT) in Nets and has
six years of experience from fighting the frontline and handling critical security
incidents. With prior risk analysis experience, he has genuine interest in
combining risk analysis and security testing with real-life incident handling
experience.
Stig Torsbakken
Risk analysis and testing - perspectives from the frontline
Abstract:
Nets - a northern European leader in payment solutions, information services
and digital security solutions - feels the heat of the frontline an a daily basis.
As part of Nordic, critical infrastructure protecting digital values, Nets is an
attractive target for cyber criminals equipped with ever more sophisticated
weapons. How can we use risk analysis and security testing to protect against
tomorrow's threatscape?
Nets Information Security
Risik analysis and testing: a frontline perspective
Stig Torsbakken Nets SIRT

C
l
i
c
k
t
o
e
d
i
Agenda
t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
Nets
Nets threat landscape
Nets security testing
Seurity testing and security incidents
2
C
l
i
c
k
t
o
e
d
i
Nets
t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
3
Three lines of defence
C
l
i
c
k
t
o
e
d
i
t
M
a
s
t
e
r
t
e
x
t
s
t
Risk management in Nets is organized around the 3 lines of defence model

y
l
e
s
(Ref. ECIIA & FERMA Guidance on the 8th EU Company Law Directive)
1 2 3
1st Line of Defence 2nd Line of Defence 3rd Line of Defence
Operational Control, policy and Audit

management framework
Board of Directors
Executive Management
Group Management (GM)

Reporting Reporting
Reporting
1 2
3
Information Security, Risk management Internal External
BUs/GUs & (Risk, Security, Compliance, Systems Audit
Management Control Quality, Continuity) Audit
Activities
Reporting Coordinating
Requirements & advising Requirements efforts
Reporting Reporting
Customers
4
NETS Information Security Organization
C
l
i
c
k
t
o
e
d
i
t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
5
C
l
i
c
k
t
o
e
d
i
Nets SIRT security incidents 2013

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
Critical 2
High 29 Category Exposed to
malicious code
Medium 115
Malicious code
Low 60 infection
206 Vulnerability
DDoS
Target Client
Reconnaissance
Perimeter
NemID Suspicious user

activity
User
Access
LAN
Mobile device
Policy violation
Network
Guest Exercise or
network network defense
Linux testing
Phishing
6
C
l
i
c
k
t
o
e
d
i
Non-target specific malware

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
We know that malware is part of our internal network

43 (21%) verified infections so far in 2013
73 (35%) exposures (i.e. malware downloads, unverified infections)
Nets impact
Ransomware
Banking Trojans
7
C
l
i
c
k
t
o
e
d
i
Nets SIRT security incidents 2013

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
8
C
l
i
c
k
t
o
e
d
i
DDoS attack April 10

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
Gruppe tager ansvar

NemID down for 7 hours
for NemID-angreb:
Vil forsge at sl til igen
En gruppe, der via Twitter har taget

ansvaret for torsdagens angreb mod
NemID, siger, at de vil forsge at f
login-tjenesten til at g ned igen.
9
C
l
i
c
k
t
o
e
d
i
t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
C
l
i
c
k
t
o
e
d
i
Norway target of APT1

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
From the Mandiant APT1 report
11
C
l
i
c
k
t
o
e
d
i
Security testing in Nets

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
In numbers Nets Norway
Vulnerability scanning
External: 4
Internal: 4+
Scanningless scanning: 365
Pentesting
New applications/systems: approx. 1 every week (!)
Yearly pentests: 20
Risk analysis
Yearly risk analysis of all infrastructure
Risk analysis of new services
How is this possible?

1 FTE (to be 3-4) dedicated to pentesting
25-30 security officers in Nets in total
Heavy use of consultants and 3rd party vendors
Narrowing scope
12
non-PCI VLAN X VLAN Z PCI
VLAN Y VLAN
768
C
l
i
c
k
Scanner
t
o
e
d
i

t
Virtual Scanners/HIABS
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
VLAN
769
FW_DH_PCI Scanner
Vulnerability scanning VLAN

758
FW_DH_CIS Management Scanner
Operator
VLAN DH_CIS_ADM
Quarterly external scan

FW_DH_Teller
VLAN
759
Scanner
PCI
Quarterly internal scan VLAN 157 VLAN 2
Scanner
Scanner
Scanningless scanning
Physical Scanners/HIABS
13
C
l
i
c
k
t
o
e
d
i

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
Pentesting
Typical findings
Input validation (XSS, CSRF, script injections etc.)
Software vulnerabilities, non-updated software
Default passwords
Vulnerable/unsecure protocols
Missing authentication/source verification
Design flaws
Information leakage (verbose error messages etc.)
70% of findings through manual tests

Narrow scope
Narrow scope
Narrow scope
14
C
l
i
c
k
t
o
e
d
i

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
Pentesting
How to narrow scope?

Tool-support
Nessus
Nmap
AppScan
Metasploit
soapUI
Burp Suite
ZAP (Zed Attack Proxy)
15
C
l
i
c
k
t
o
e
d
i

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
Pentesting
How to narrow scope?
Risk analysis involving pentest team

Business side priorities
Structured walk-through of functionality
Thorough system presentation
Pentest team involved in software development phase

Pentest team involved in security arcitechture design
16
C
l
i
c
k
t
o
e
d
i
Security testing and security incidents

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
Vulnerability
Threat scans
intelligence Pentests
Risk
Vulnerability Security
analysis
management incident
Patch
level Inventory
control
17
C
l
i
c
k
t
o
e
d
i

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
Nets SIRT vulnerability management procedure
Nets pentest/vulnerability scan findings Priority:

affect production systems Major
remote exploitable 1&2
would cause a Major 1 or 2 incident if exploited
do not affect production systems 3
not remotely exploitable (only internally)
would cause a Major 1 or 2 incident if exploited after
system is released
do not require immediate actions 4
18
C
l
i
c
k
t
o
e
d
i

t
M
a
s
t
e
r
t
e
x
t
s
t
y
l
e
s
Threat Vulnerability
intelligence scans
Pentests
Security Risk Vulnerability

incident analysis management
Inventory Patch
control level
19
Session 1
Security risk assessment and testing
Jan Stijohann
(Siemens)
Towards Systematic and Traceable Security Assessment
Ketil Stlen
(SINTEF)
Test-based risk assessment
Samson Yoseph Esayas
(Universitetet i Oslo)
Legal Risk Management: a Method for Proactive Management of Legal Risks
Session 1 Talk 1
Jan Stijohann
(Siemens)
Towards Systematic and Traceable Security

Assessments
Jan Stijohann
Vita:
Jan Stijohann works as a security researcher and consultant for Siemens. As
part of the CERT Security Assessment team (CSA), he performs practical
software security assessments but is also engaged in several internal and
external research projects. His research interests cover static and dynamic
binary analysis techniques to improve black box security testing as well as
general security assessment approaches. He graduated in 2011 with a Masters
degree from the Grenoble Institute of Technology (France) and the Karlsruhe
Institute of Technology (Germany).
Jan Stijohann
Abstract:
Todays security assessments are often not systematic much less standardized. In particular, there
are no clearly defined criteria for choosing certain assessment activities or test approaches. Thus
different analysts come to different results and sound quality assurance is hardly possible.
Literature suggests basing the choice and prioritization of tests on risk considerations but lacks a
systematic approach for a traceable transition from abstract and business-oriented risk analysis
into the concrete and technical security testing world. We aim at bridging this gap in two steps:
The first one bridges between high-level and non-technical business worst case scenarios and
less abstract technical threat scenarios using a technical description of the system and a
systematic STRIDE-based elicitation approach. The second is a rule-based step that maps a
technical thread scenario to test types, that is, to classes of tests that need to be adapted to the
particular system under validation. Our method provides traceability for the (risk-based) choice of
assessment activities and can be used to introduce a standardized minimum quality assurance
level. The talk goes through the above process, discusses first practical experiences, and outlines
future work to better adapt the process to the use in industrial environments.
SASSI 13, Berlin RERAT: Reverse Engineering Methods and Risk Analysis for Testing
Towards Systematic and

Traceable Security Assessments
Jan Stijohann, Jorge Cuellar
Siemens AG, CT RTC ITS
2013-09-19
Restricted Siemens AG 2013. All rights reserved
Outline
1. Challenges of black box security assessments / Motivation
2. Experiences from practical application
3. Selected details of our current research
Siemens AG 2013. All rights reserved

Page 2 June 2013 CERT Security Assessments | Jan Stijohann Restricted
Information Limited to System under Validation (SUV)
Black box testing
Limited access to source code, design documents, developers / architects
Security assessments of COTS products

Does a product meet a security standard?
Internal assessments of company products

Strict intellectual property or confidentiality restrictions,
Simulation of attackers
Test of legacy systems
Independent security researcher analyzes a product of public interest

without the manufacturers knowledge or consent.

Planning a Pen-Test
Where do I start?
What to do (next)?
When can I stop?
How to prioritize?

Frequent Test Situation:
Neither systematic nor standardized
Impossible to test all known vulnerabilities

Effort would be too huge
Would not correspond to the risk
How to decide which vulnerabilities should be selected for testing?

Buffer overflow
SQL injection
Format string vulnerabilities
Double frees
Null pointers
depending on the unique, specific characteristics of the SUV

Frequent Test Situation
Choice: vulnerabilities that are detectable with little knowledge

Fuzzing, automated scanning for known vulnerabilities (Nessus),
Strong dependence on individual know-how, experience, intuition, luck

Non-transparent
Bad coverage
Distorts possible risk-driven approaches

Reverse Engineering (RE) Methods
Static and dynamic binary analysis

The mean to obtain additional information,
But it is time-consuming, requires special and scarce expertise
Industry acceptance
Automation
Information gain must outweigh cost of preliminary analysis
Preliminary analysis must be manageable by security assessment people
no binary RE experts
(Note: we do not *do* reverse Engineering, but we use some RE methods)

Ideal:
From a Technical Analysis to Assessment Activities
Vulnerability indicators
Map to
activities
Assessment
activity
suggestions
Some Details
Static / Dynamic
binary analysis
SUV - Characteristics:
Pattern
Vulnerability indicators matching
Activities Map to
library activities
Assessment
activity
suggestions
Rules Siemens AG 2013. All rights reserved
Page 9 June 2013 ssments | Jan Stijohann Restricted
Challenges
Risk analysis often high-level

How to connect high-level results with practical assessment world?
How to consider relevant technical properties?
Static / dynamic binary analysis take time & require rare expertise
How to automate analysis?
How to leverage the required expert knowledge?

Security Relevant SUT properties
At design and architectural level
Interfaces, data sources and sinks, security controls

Data and control flows
Design of security controls
SWhite or black list filtering
Encryption initialization vectors
Sec controls on client- vs server-side,
Custom solutions vs standard libraries
Trust and local boundaries
Technological choices
Programming languages, protocols, data formats

Security Relevant SUT properties
At implementation level
Unsafe functions such as strcpy, printf,

The number of stack allocated buffers of size > x
The version of dynamically and statically included external libraries
Data and control flow on basic block-level

Risk-driven security testing
Business Worst Case Scenarios

informal
Personal damage of a patient
abstract
non-technical
Technical Threat Scenarios

based on SUT model
(inform. disclosure, data X) visualized as DFD
(DoS, process Y) still abstract, but
technical
(spoofing, data flow Z),
Assessment Activities
mutation-based or block-based fuzzing, Needs adaptation to SUT
tool-supported source code analysis, Collected in library
RE with custom debugging scripts, OS tools, etc.

Our black-box testing approach at a glance
Our black-box testing approach at a glance

Selected lessons learned from practical application
1. Significantly improved test coverage crucial

Well-stocked assessment library
2. Need of more detailed, more precise activity suggestions

More low-level model incl. basic blocks, functions, shared libraries,
3. Need more precise technical risk estimation, reduce false positives

More precise patterns that indicate vulnerabilities (not only DFDs)
DFDs
useful for high-level overview
detecting certain design flaws
keep DFD as additional high-level view

Patterns and SUV - Characteristics
Note: the exact pattern format is work in progress!
Pattern: Hard-coded password

(data flows to interface) and (data originates from binary) and (during an
authentication/authorization request)
RE activities to find relevant SUV - Characteristics:

Spot interfaces
(details missing)
Determine data origin
Spot interfaces
Identify data reads from binary modules
Recognize authentication/authorization routine
(details missing)
Track data flow
Differential debugging

RERAT: black-box testing approach at a glance
Non-technical / Static / Dynamic

Non-technical business-level binary analysis
risk factors risk analysis
Risk estimation SUV - Characteristics:

modules, functions, basic
Risk estimation blocks, API calls,
results Technical risk factors / control & data flow, etc.
Vulnerability indicators / Pattern

Other relevant Patterns matching
for security assessments
Assessment
Map to
activity
activities
library Assessment
activity
Mapping suggestions
Page 18 June 2013
rules ssments | Jan Stijohann Restricted
Reverse Engineering analysis
Automated IDA Pro analysis

System binaries
(IDB generation)
Static graph-based model

Running system
(raw data)
Static analysis (SA) Dynamic analysis (DA)

SA of binary with custom IDA Python Custom DA with IDA Python
scripts debugging scripts
SA of environment (folders, files, registry, tracing / differential debugging
user privileges, etc.) at OS-level DA of environment at OS-level
DFD generator /
DFD-relevant SA results DFD-relevant DA results
analyzer
Manual adaptations
High-level, annotated DFD
and extensions ights reserved
SUV - Characteristics: Code snippet examples
Static analysis
Static analysis

Static analysis

Dynamic analysis

Restricted
Dynamic analysis
Dynamic analysis

IDA Pro automated analysis and (raw) model
generation

IDA Pro scripting
References
[1] D. Doan, Commercial off The Shelf (COTS) security issues and
approaches, Monterey, California. Naval Postgraduate School, 2006.
[2] Stijohann, Jan, and Jorge Cuellar, Towards a Systematic Identification of

Security Tests Based on Security Risk Analysis. ESSoS Doctoral
Symposium 2013,
http://ceur-ws.org/Vol-965/mainproceeding.pdf#page=25, 2013
[3] Kwon, Taeho, and Zhendong Su. Automatic Detection of Unsafe

Component Loadings. Proceedings of the 19th International Symposium on
Software Testing and Analysis, 2010
[4] Eagle, Chris. The IDA Pro Book, 2nd Edition. No Starch Press, 2011.

Session 1 Talk 2
Ketil Stlen
(SINTEF)

Ketil Stlen
Vita:
Stlen is currently Chief Scientist at SINTEF and Professor at the University of Oslo. Stlen has
broad experience from basic research as well as applied research. He did his PhD at Manchester
University on formal reasoning about concurrent programs. At Technische Universitt Mnchen
his research focused on the theory of refinement and rules for compositional and modular system
development - in particular, together with Manfred Broy he designed the Focus method as
documented in the Focus-book published in 2001. At the OECD Halden Reactor Project he led
several research activities concerned with the modeling and dependability-analysis of safety-
critical systems. At SINTEF he led the development of the CORAS method since the very beginning.
He was the technical manager of the EU-project (the CORAS project) ending in 2003 in which the
first version of the CORAS method was developed. He has since then led several research projects
funded by the Research Council of Norway which have considerably refined and extended the
original CORAS approach. A book on the CORAS method supported by a free tool was published in
2011. Stlen is currently managing several major Norwegian research projects focusing on issues
related to modeling, security, risk analysis, and trust.
Ketil Stlen
Abstract:
Security risk analysis is a process that is carried out in order to identify and
assess security specific risks. Traditional risk analysis often rely heavily on
expert judgment for the identification of risks and their causes and the
estimation of their likelihood and consequence. The outcome of these kinds of
risk analysis is therefore dependent on the participant's background,
experience, and knowledge, which in turn reflects an uncertainty in the
correctness of the results. In order to validate the correctness of the risk
analysis results, the risk analysis process can be complemented by other ways
of gathering information of relevance. One possibility is to employ security
testing. In this talk we present experiences from combining the CORAS method
for security risk analysis with security testing.
Berlin 19.9 2013

Ketil Stlen
Technology for a better society 1

Outline
Test-based risk assessment versus Risk-based testing
Motivation for test-based risk assessment
Challenges with test-based risk assessment
Testing a CORAS model
Conclusions

Risk-based testing
Use of risk assessment to make the testing more effective and/or efficient
Identify and prioritize test-cases

Select test-cases
Decide on test execution issues
Structure and present test-execution results

Use of testing to support risk analysis
Identify potential vulnerabilities

Estimate frequencies and consequences
Validate the correctness of the risk picture

Motivation for test-based risk assessment
The "crap-in-crap-out" argument
Risk assessment is an empirical research method
A good empirical scientist triangulates

Challenges with test-based risk assessment
What do we mean by testing?
What can we really test?
Can we test a qualitative risk model?
What about frequencies, probabilities and consequences?
Where does the concept of uncertainty fit into this picture?

Testing a CORAS model

Annotated CORAS model
Ann

Conclusions
Risk assessment requires triangulation
Testing risk models is one way to triangulate
Practical experiences:
Risk models developed based on structured brainstorming in the segment 150-

250 man-hours are robust
Testing is nevertheless helpful
Testing of risk models gives more confidence

Funded by the following projects:
RASEN: http://www.rasen-project.eu/Home/tabid/439/language/en-US/Default.aspx
NESSOS: http://www.nessos-project.eu/
Diamonds: http://heim.ifi.uio.no/~ketils/diamonds/diamonds.htm

ketil.stolen@sintef.no

Session 1 Talk 3

(University of Oslo)
Legal Risk Management: a method for Proactive

Management of Legal Risks
Legal Risk Management
Vita:
Samson Yoseph Esayas (LLB, LLM, Researcher at Norwegian Research Center
for Computers and Law, University Of Oslo).
Abstract
It is commonplace that legal services are often sought reactively i.e. when a legal
problem has already occurred. Such an approach has not always been viewed as
satisfactory because disputes and litigation consumes time and resources which could
otherwise be used more productively. In the book The Future of Law, Richard Susskind
predicts a paradigm shift in the approach to a legal problem: from problem solving to
problem prevention: where understanding legal problems and identifying associated
risks and controlling them before any question of escalation becomes a priority. This
raises the questions of what kind of methods a lawyer can employ to ensure legal risk
management. One possibility is to supplement the conventional legal method of
identifying which law applies to a given case with methods for risk analysis developed
in other disciplines, such as IT Security. In such disciplines, the risks can be identified,
analyzed and addressed in a structured way. The question remains: to what extent, and
in which way, such methods for risk management may be applied within the legal
domain.
Legal Risk Management: Proactive
Management of Legal Risks
Samson Yoseph Esayas (Researcher)
Norwegian Research Center for Computers and Law (NRCCL)

The Faculty of Law
University of Oslo
Agenda
Introduction
Risk management
Legal risk management
A real-life example
Graphical modeling of legal

risk and ongoing RASEN
works
Introduction
Conventional legal problem solving: reactive

In Richard Susskinds book The Future of Law (98):
predicts a paradigm shift from problem solving to
problem prevention where
emphasis will shift towards legal risk management supported
by proactive facilities
What kind of methods a lawyer can employ to ensure
legal risk management?
Risk analysis methods from system analysis or
security management together with the conventional
legal methods
Risk management
What is risk management?
ISO 31000:2009
Establishing the context

Communication and consultation
Risk assessment
Monitoring and review
Risk identification
Risk analysis
Risk evaluation
Risk treatment
Risk matrix
Who does risk management in an
enterprise?
Board of directors/CEO: enterprise risk
Chief risk officer (CRO)/chief finance officer (CFO): financial
risk
Product developers (e.g. engineers): product risk
Safety officer: risk related to health/environment/safety
Project managers: project risk
IT system engineers: IT security risk
Compliance officer: compliance risk
General Counsel (Legal): Legal risk?

Risk and uncertainty
Risk is the effect of uncertainty on objectives

ISO 2009
Uncertainty is deficiency of information related to or

understanding or knowledge of an
event,
its consequence or
likelihood.
Uncertainty about
Facts
Legal norms
Legal risk
A legal risk is a risk that has a legal issue as its

source
Legal issue Risk
A legal issue is a set of facts that are assessed

under a set of legal norms.
Legal risk management
Legal risk management focuses on

the management of legal risk and
the legal management of risk
Topics in legal risk management
LEGAL RISK MANAGEMENT
Risk management methods

Structural risk management
Legal methods
Compliance risk management
Contractual risk management
Litigation risk management

Legal risk management process
The legal and factual context

between client, lawyers and non-lawyers
of legal issues, disputes and cases

Legal risk assessment
Monitoring and review

Identification of risks,
Communication
particularly legal risks
Risk analysis:
legal & factual uncertainty
Risk evaluation:
quantitative & qualitative
Risk treatment:
legal & factual risk controls
A real-life example
of identified legal risk
Source: SAP, IFRS FINANCIAL REPORTS

(2007), p. 56
Uncertainty
regarding facts
Product risk
Risk of actual or alleged failures of our software products
and services.
Description of facts/events
New products and product enhancements may still contain
undetected errors when they are first released.
As a result, it is feasible that certain customers may bring
claims in certain cases for cash refunds, damages,
replacement software, or other concessions.
SAP software products are chiefly used by customers in
business-critical applications and processes.
Uncertainty regarding the law
Description of the law/contract

Our contractual agreements generally
contain provisions designed to limit
SAPs exposure to warranty-related
risks.
However, these provisions may not
cover every eventuality or be entirely
effective under applicable law.
16
Risk estimation
Effect on SAPs objectives

Such claims could adversely affect our assets,
finances, income, and reputation.
Risk estimation
We believe it is
unlikely that our planned results will be
significantly impaired by product defect
claims from SAP customers.
17
Risk treatment
We counter these risks through

project management,
project monitoring,
rigid and regular quality assurance measures
certified according to ISO 9001, and
program risk assessments during product
development.
AND CONTRACTUAL MEASURES
18
Monitoring
The generally high quality of our

products is confirmed by
our low customer escalation handling
expenses
the low rate of litigation arising against us out
of our regular operations
and our constantly high customer satisfaction
ratings as measured by regular customer
surveys.
19
Graphical modelling of legal risk
Facts lead to risk
Legal norms contribute to risk
This depends on whether the
software seller knew or should
have known about the defect
Is this contractual provision

entirely valid under
applicable law and does it
cover all cases?
RASEN: Security risk assessment
and security testing
Focus on legal norms of relevance to
security
Checking compliance
through testing (audit testing)
Security risk analysis as

basis for legal risk analysis
Audit testing
Technical testing (conformance testing)- to gain
confidence in correct operation technical controls.
E.g. the evaluation of the correct implementation of
the technical measures in place that protect
information, such as measures that guard against
unauthorized access
it gives the organization the assurance that it is in
compliance with its legal obligations.
Non-technical testing- involves evaluating and
testing the effectiveness in the implementation of
plan policies, procedures and business processes
implemented to adhere to the legal norms.
Risk-based audit testing
Legal implications of security
risks
Con.
Concluding remarks
Risk management can be usefully applied in a legal

context, as a complement to existing methods.
It is possible that existing tools could be extended to
support legal risk analysis
In case of information security related legal norms,
considering legal and technical risks jointly is
essential, e.g.
Using testing for checking compliance and identifying further legal
risks (RASEN)
Session 2
Standardization and Certification
Gerard Gaudin
(GC)
A full set of new standards in Cyber Defence addressing the full scope of security event
detection issues
Jrgen Gromann
(Fraunhofer FOKUS)
Security Testing Improvment Profile (STIP)
Luca Vigano
(Universit di Verona)
The SPaCIoS Tool - property-driven and vulnerability-driven security testing
Luca Compagna
(SAP)
Formal Validation and Testing of Security Standards at SAP: from research to industry
Session 2 Talk 1
Gerard Gaudin
(G2C)
A full set of new standards in Cyber Defence

addressing the full scope of security event
detection issues
Gerard Gaudin
Vita:
Gerard Gaudin was graduated from Supelec School (US equiv. MSEE) in 1979.
After a career beginning in two IT multinational companies, he held at CS
senior executive positions managing large departments. Since 2003, he has
been leading as an independent consultant (GC) IT security activities
specializing in Cyber Defence and SIEM. In this field, he created by the end of
2008 the French Club R2GS not-for-profit security community, whose he is
the Chairman. Today, this association gathers more than 40 big companies
and organizations from various industry sectors, and is expanding across
Europe (started in the UK and Germany, other countries soon). Moreover, he
initiated in 2011 within ETSI a standardisation unit (called ISG ISI), whose goal
is to address all security incident detection matters (event classification model,
indicators, event detection testing ...), and whose he is the Chair. These
activities are carried out in relation with ISO SC27 and ITU-T.
Gerard Gaudin
Abstract:
Standards for IT security indicators and associated event classification model are missing (or are
still very poor), and are hindering IT security measures benchmarking. The ETSI ISG ISI initiative
(launched during fall 2011), which is based on 4-year experience and frameworks of the European
network of Club R2GS grassroots associations in Cyber Defence and SIEM (France, UK and
Germany today), fills this gap while being strictly compliant to ISO 2700x IT security standards. It
addresses the full scope of security event detection issues through 5 Working Items:
ISI-001(Indicators), a powerful way to assess security measures level of effectiveness,
ISI-002 (Event Model), a comprehensive security event classification model,
ISI-003 (Maturity), to assess the maturity level regarding overall event detection,
ISI-004 (Detection Implementation), to demonstrate how to produce indicators and how to
detect the related events,
ISI-005 (Event Testing), to produce fake events and to test the effectiveness of detection means.
ISG ISI (Information Security Indicators)
ETSI ISG ISI Standardization

(SASSI13 Workshop Security Assessment
for Systems, Services and Infrastructures)
Gerard Gaudin (GC)

Berlin 19 September 2013 Chairman of ETSI ISG ISI
ISG ISI positioning against Risk

Management and ISMS fields
Risk Management Dispatch and put into hierarchy Controls and ISMS
the133 ISMS control points
(ISO 27005) depending on IS components (ISO 27002/1 and Cobit)
(Plan et Do)
Implement key Deepen some ISMS

measures (Do) Remedy to secu- controls (Do)
- Security event detection and rity gaps (Act) - Security event detection
processing (workflow) and processing (workflow)
- Legal validity of evidence
Check continuously risk (forensics)
evaluation results (Check) Check continuously
- Checking of field situation ISMS relevancy (Check)
regarding residual risks
Event-model
- Through operational indicators
- Security event criticity centric vision (process, human, technical)
evaluation controls relevancy
Cyber Defence and SIEM

(ISO 27035 and new ISO, ITU-T
and ETSI standards to come) Club R2GS
Gerard Gaudin (ISG ISI chairman) SASSI13 Workshop in Berlin 19 September 2013 2
Address the scope of main missing security
event detection standardization issues
5 closely linked Work Items
ISI Indicators (ISI-001-1 and Guide ISI-001-2) = A powerful way to assess
security controls level of enforcement and effectiveness (+ benchmarking)
ISI Event Model (ISI-002) = A comprehensive security event classification
model (taxonomy + representation)
ISI Maturity (ISI-003) = Necessary to assess the maturity level regarding
overall SIEM capabilities (technology/people/process) and to weigh event
detection results. Methodology complemented by ISI-005 (which is a more
detailed and case by case approach)
ISI Event Detection (ISI-004) = Demonstrate through examples how to
produce indicators and how to detect the related events with various means
and methods (with classification of use cases/symptoms)
ISI Event Testing (ISI-005) = Propose a way to produce security events
and to test the effectiveness of existing detection means (for major types of
events)
ISI Work Items Positioning
Event
reaction
measures
Fake events
(Simulation)
Security Event
Real Detected
prevention detection
measures events measures events
Residual risk
(event model-
centric vision)
ISI Work Items positioned against other standards

Whole specifications Continuous assurance specifications
4
Global
frameworks ISO 27002 or NIST 800-53 ISO 27004 or NIST 800-55
3 ITU-T E.409 ISO 27035 or NIST 800-61

Implementation ISO 27003 or NIST 800-37
frameworks ITU-T X.1205 IETF RFC 2350 US CAG
NIST 800-92 NIST 800-137
2
Security Table Security policy Act Action Plans Indicators
frameworks
reference
Specific
Protect. Prof. Risk Analysis BCP Reaction Plans Event Model MITRE CAPEC
NIST 800-86 MITRE CEE
Projects Contracts Phys. Sec. Forensics Glossary
1
Base (or technical) IETF RFC 4765/ NIST 800-126 ITU-T
frameworks 5070/6045/5424 (SCAP) X.152X
ISI-001 specifications
Position the proposed operational indicators against
ISO 27002 controls and ISO 27006 technical controls =
provide more assurance to governance and auditors
ISO ISO 27006 Vulnerability
27002 technical Incident type (behavioural, software,
Comments
control control indicators configuration, general
areas areas security) type indicators
A5 Non-continuous checking
A6 Purely organisational issues
A7 IWH_UNA.1 VTC_NRG.1 Information classification + asset

VOR_PRT.1 management
A8 x IMF_LOM.1 VBH_PRC.1 to 6 Focus on deviant internal
IDB_UID.1 VBH_IAC.1 to 2 behaviours
IDB_RGH.1 to 7 VBH_FTR.1 to 3
IDB_IDB.1 VBH_WTI. 1 to 6
IDB_MIS.1 VBH_PSW.1 to 3
IDB_IAC.1 VBH_RGH.1
IDB_LOG.1 VBH_HUW.1 to 2
A9 x IEX_PHY.1 VTC_PHY.1 Marginal topic for a SIEM
approach
... ... ... ... ...
A15 XX IMF_TRF.2 to 3 VBH_IAC.2 Focus on configuration
VBH_WTI.2 vulnerabilities or non-
VBH_WTI.6 conformities
VBH_RGH.1
VCF_DIS.1
VCF_TRF.1
VCF_FWR.1
VCF_ARN.1
VCF_UAC.1 to 3
VTC_IDS.1
ISI-002 specifications (1)
The diversified uses of the event model
Easier link between SIEM and

risk assessment methods
Security event testing
(EBIOS, OCTAVE, CRAMM, )
(Detection effectiveness)
2
1 Possible design of a risk data base
Global and complete 3 on top of a security event data base
framework of reaction
plans (ITIL compatible)
Event model
9 Easier link between SIEM
and security policy and rules
SIEM 4
9 uses (ISO 27002) + Link with
Continuous Auditing (US CAG)
More readable reports 8 Comparison with public

(based on a common 5 reference statistical figures
framework of indicators) (by industry sector)
7 6
Easier analysis of malicious activity Support for insurance
and deviant behaviors (Link with offerings in cyber-risks
Counter Competitive Intelligence)
Consistency of the uses between each other

thanks to the event model pivotal role
ISI-002 specifications (2)
ISI-001 and ISI-002 against the ISO 27004
standard measurement model
Counting of some
events (ISI-001-1)
Event classifi-
cation model
(ISI-002)
The mandatory taking into account of the
organizations SIEM maturity level
A good security event detection level (still often very low today)
requires many conditions (tools appropriately configured, advan-
ced processes especially for use case creation, seasoned experts)
This overall maturity level can be assessed accurately through 10
KPIs (with a clear correspondence with the 20 US CAG Critical
Controls)
Provision (with these KPIs) of a reckoning formula to assess its
detection levels with major kinds of security events (and to weigh
the results of its own measurements)
This methodology may be complemented by a more dedicated and
case by case one based on the production of security events and
testing of the effectiveness of existing detection means (for major
types of events)
Guidelines to stimulate security events are missing
and are required (same motivations as ISI-003)
Objective of testing of detection means and tools during

development and deployment phases (lab and in-operation
situations), and of measurement of their effectiveness
Stimulate existing detection means by relevant events
(see ISI-002)
Try/perform fake incidents (to be identified/count)
Introduce vulnerabilities (to be identified/count)
Will rest on existing test patterns (Cf. DIAMONDS project), with
provision of catalogs (methods, configurations, scenarios)
Could also be used for penetration testing
More technical than conceptual specifications
ISG ISI schedule

Several standards already available
ISG ISI started in Autumn 2011 = Members of the Unit and of the 5
Work Items are European and US experts
ISI Indicators (ISI-001-1 and ISI-001-2 Companion Guide) and ISI

Event Model (ISI-002) have been published last April
ISI Maturity (ISI-003) will be available by the end of 2013
ISI Event Detection (ISI-004) will be available by the end of 2013
ISI Event Testing (ISI-005) started at the beginning of 2013
The Club R2GS network and standardization
ISO JTC1 SC27
Grard Gaudin
ITU-T SG17 Q4
Whole European MITRE (US)
Coordination ETSI ISG ISI
European
Commission
Local Local Local Local Local Local

Chapter Chapter Chapter Chapter Chapter Chapter
France UK Germany Italy Luxembourg Country X
- Grard Gaudin - Andrea Hazeldine/ - Jan deMeer/ - Romano Luigi - Christophe Bianco
Fortune Barnard Axel Rennoch
- Main works: - Main works: - Main works: - Main works: - Main works:
. 7 WGs (0 to 6) . Link with Cabinet . Marketing to be launched to be launched
. WG 4 (all added Office (Project collaterals in fall 2013 in fall 2013
values & especially Auburn) . Relations between
How to detect) . Establishment of a Risk Analysis and
. WG 6 (Security 1st set of UK state- ISI indicators and
Incident Manage- of-the-art figures event model (contri-
ment) (related to ISI indi- bution to ETSI and
. Matinales cators) ISO)
. Assises Cyber . ISO SC27/ETSI ISG
Dfense et SIEM ISI Liaison officer Link: http://en.wikipedia.org/wiki/Information_security_indicators
Grard Gaudin 29 August 2013
Session 2 Talk 2
Jrgen Gromann
(Fraunhofer FOKUS)
Security Testing Improvement Profile (STIP)

Jrgen Gromann
Vita:
As a member of the Competence Center Modeling and Testing for System and Service
Solutions (MOTION) he is involved / responsible for validation, verification and testing
projects on next generation networks and software technologies for embedded
systems. Jrgen Gromann is an expert on model-based development, model driven
testing as well as in security engineering and security testing. Furthermore Jrgen
Gromann has experiences in testing and modeling automotive software systems and
applications, especially ITS systems. He has extensive experience in developing test
benches and test laboratories for conformance and interoperability testing in this area.
Jrgen Gromann has experiences in numerous standardization activities for various
standardization bodies, including OMG, ETSI, ASAM and AUTOSAR. He is technical
expert for test specification in ETSI working groups, rapporteur for the ETSI MTS
Security SIG and involved in technical discussion in the C2CCC. He builds the bridge to
the developers of the advanced V2X test beds and playgrounds at FOKUS as well as to
the security engineers working for the BSI, ETSI etc.
Jrgen Gromann
Abstract:
Model-based Security Testing (MBST) is a relatively new field and especially focuses on a
systematic and efficient specification, generation, or documentation of test objectives, security
test cases, and test suites. The ITEA project DIAMONDS (www.itea2-diamonds.org) has developed
efficient and automated MBST techniques, tools and methods for highly secure systems in
multiple industrial domains (e.g. automotive, telecommunication, smart cards, banking etc.).
Amongst others, the project has had a special focus on Risk-Based Security Testing (RBST) and its
combination with test generation techniques, passive testing, and Fuzz testing techniques. To
analyse the effectiveness of the model-based security testing techniques, tools and methods we
have developed a profiling and assessment scheme called STIP (Security Testing Improvements
Profiling) Scheme, which allows an objective, detailed analysis and evaluation of the DIAMONDS
research & development results. The scheme allows an assessment of model-based security
testing processes and shows how security testing techniques, tools and methods fit together.
Finally STIP may be used to provide recommendations for other on how to pragmatically integrate
the DIAMONDS results to improve security-testing processes on hand. We have applied STIP to 8
of our case studies and we are sure, the approach can be used to effectively assess model-based
security software testing processes within an organization and develop process improvements by
leveraging the maturity of such a process in certain key areas.
An evaluation scheme for security testing
SASSI13 Security Assessment for Systems, Services and Infrastructures
September 2013 at the Technical University (TU) in Berlin
Jrgen Gromann, Fraunhofer FOKUS,

juergen.grossmann@fokus.frauhofer.de
Motivation
Technical Guide to Information Security Testing and Assessment NIST Special Publication 800-115
TMMi, TPI and TPI NEXT
Exemplary TPI spread sheet
TPI, TPI Next are registered

trademarks of Sogeti
TMMi is based on CMM, and
developed by the Illinois Institute of
Technology
The TMMi Model see http://www.tmmi.org/
TPI and TPI NEXT
Analysis with respect of the key areas

Levels are used to assign a degree of maturity to each key area
Checkpoints are defined to determine the level for each key area
Each higher level is better than its prior level in terms of time (faster), money
(cheaper) and/or quality (better).
Maturity Scale Key Areas

Staged representation:
Initial Maturity Levels
Controlled
Efficient
Optimizing
Improvement
Continuous representation Checkpoints
Sugges7ons
A M or 1 -13
Evaluation of the DIAMONDS Case Studies
Security Testing Improvement Profiles (STIP) enables an objective,

detailed analysis and evaluation of security testing processes
First introduced to evaluate the case studies of the DIAMONDS

project
Provide a detailed analysis and evaluation of our research &
development
Show how tools & techniques have evolved
Provide a template for other on how to pragmatically integrate the
DIAMONDS results to improve security testing processes on hand.
Analysis with respect of the key areas

Levels are used to assign a degree of progress to each key
area
Each higher level is considered better than its prior level in
terms of quality (e.g. exactness of the outcome) or
effectiveness (e.g. automation of activities).
STIP key areas
Incep7on and Elabora7on and Ar7fact consistency

Test Techniques
target analysis execu7on and tool support
Informa7on Security
Test depth
gathering func7onal tes7ng
Security test tool
Security risk Genera7on of integra7on
assessment security test Fuzzing
technique models
Security passive
Security risk Security test
tes7ng/security
assessment scope genera7on
monitoring Traceability &
Security test Security test Sta7c security test coverage
execu7on
iden7ca7on
automa7on tes7ng
STIP level definition
Key area: Security risk assessment technique
At this level, the security risk assessment is conducted in an

A: Informal
unstructured manner without a specic nota7on/language for
security risk
document risk assessment results or a clearly dened process for
assessment
conduc7ng the security risk assessment.
B: Model-based At this level, the security risk assessment is conducted with a

security risk language for documen7ng assessment results and a clearly dened
assessment process for conduc7ng the assessment.
C: Model and
test-based At this level, the model-based security risk assessment uses tes7ng
security risk for verifying the correctness of the risk assessment results.
assessment
STIP level definitions
Key area: Security test identification
A: Iden:ca:on Test iden7ca7on can be based on the analysis of the func7onal security
based on requirements (SFR) and their coverage through tes7ng. OQen these
requirements requirements have priority numbers that addi7onally provide guidance on the
importance of a requirement and the related test purpose.
analysis
B: Iden:ca:on Security threat/vulnerability models addi7onally allow for the iden7ca7on of
penetra7on tests that are based on es7ma7ons on poten7al threats and
based on threat/ poten7al vulnerabili7es. This allows tes7ng for unwanted incidents that are not
vulnerability models covered by the security func7onal requirements.
C: Iden:ca:on The combina7on of threat/vulnerability models and test paRern addi7onally
provides best prac7ces for the iden7ca7on and selec7on of tes7ng means
based on threat/
dedicated to well-known classes of threats or vulnerabili7es. This approach
vulnerability models provides extensive guidance to iden7fy adequate test purposes and to apply
and test paAern approved security tes7ng methods, techniques and tools.
Risk-based security test iden7ca7on and priori7za7on combines the
D: Risk-based advantages of Level 3 with a priori7za7on of the test purposes by considering
security test probabili7es of the unwanted incident and es7ma7ons on their consequences
iden:ca:on + (quan7ed security risks). The integra7on of test iden7ca7on with security
risk assessment allows for a problem and business specic priori7za7on of the
priori:za:on
iden7ed tests purposes and tes7ng approaches.
Analysis and improvement suggestions
A security testing matrix defines the current state of a process (blue

background).
Profiles define optimal and well aligned security testing levels (red line).
Improvements suggestions are to be defined on basis of dependencies
between key areas and their levels (red background)
e.g. Security test identification B requires Security risk assessment
technique B (green arrow)
Application of STIP
Evaluation of the DIAMONDS case studies
Security tes7ng solu7ons for six industrial domains in 8 case studies

Banking
Automo7ve
Radio protocols
Smart cards
Telecommunica7on
Industrial automa7on
STIP results for the international case studies
Progress in all case studies
Banknote processing machine case study
Summary & Conclusion
STIP is an evaluation and improvement scheme for security testing processes

First introduced to evaluate the case studies of the DIAMONDS project
Provide a detailed analysis and evaluation of security testing processes on hand
Provide a template to pragmatically improve security testing processes on hand
First version is available at http://www.itea2-diamonds.org/evaluation/stip/index.html
Can be used in addition to TMMi or TPI to emphasize security testing aspects.
FOKUS plans to offer consultancy and certification options on basis of STIP in the near
future
Contact:
Jrgen Gromann
Fraunhofer Institute for Open Communication Systems FOKUS
MOTION Modeling and Testing for System and Service Solutions
Kaiserin-Augusta-Allee 31, 10589 Berlin, Germany
E-Mail: juergen.grossmann@fokus.fraunhofer.de
Session 2 Talk 3
Luca Vigan
(Universit di Verona)
The SPaCIoS Tool property-driven and

vulnerability-driven security testing
Luca Vigano
The SPaCIoS Tool
Vita:
Prof. Dr. Luca Vigan received his Ph.D. in Computer Science from the
University of Saarbrcken, Germany, in 1997, and his Habilitation in Computer
Science from the University of Freiburg, Germany, in 2003. He held a senior
research scientist position at ETH Zurich, Switzerland, from January 2003 to
October 2006. Since October 2006, he is an Associate Professor of Computer
Science at the University of Verona, where he is the head of the Research
Group in Information Security (regis.scienze.univr.it). His research focuses on
formal methods and tools for the specification, verification, and construction
of secure systems, and on the theory and applications of non-classical and
security logics. He has participated in the administration and research activity
of a number of national and international projects on information security. In
particular, he was the coordinator of the FP7 project AVANTSSAR and currently
is the coordinator of the FP7 project SPaCIoS.
Luca Vigano
The SPaCIoS Tool
Abstract:
In this talk, we present how the SPaCIoS Tool supports security analysts and
developers in the security assessment of a system under testing. In particular,
we describe the main workflows and components that have been
implemented as part of the SPaCIoS Tool and that rely on a combination of
model-checking, model-based security testing, mutation testing, and
penetration testing techniques to detect vulnerabilities and to evaluate the
security implications of specific design and deployment decisions. We also
report on a number of experiments we have been carrying out. In particular,
we have been applying the tool as a proof of concept on a set of security
testing problem cases drawn from industrial and open-source web-based
application scenarios. We have also been executing collaboration projects with
business units at industry as a stepping stone towards the industry migration
of the SPaCIoS Tool.
The SPaCIoS Tool
property-driven & vulnerability-driven security testing for the Internet of Services
Luca Vigan
(Universit di Verona, Italy)
STREP Project number: 257876

Objective ICT-2009.1.4 c: Technology and Tools for Trustworthy ICT
01.10.10 31.01.14
www.spacios.eu
One example protocol/service: SAML SSO
Company enriching its products with security standards (SAML SSO, OAuth2, ..)
security standards are highly configurable which options and recommendations?
companys internal requirements some deviations w.r.t. standard?
security impact?
Identity Service
Client (C) Provider (IdP) Providers (SP)
Gmail resource?
Your credentials?
SP asks for my credentials
if then
Here they are your credentials!
My credentials
You can access to Gmail

From AVANTSSAR to SPaCIoS
(the story of SAML SSO)
Property
Property Model
Model
Model
Checker
SUV
Input
Output
Specification of the available services (new) Service specified
BPMN + Annotations
CONN HLPSL++
CONN AnB
CONN ASLan++
CONNECTOR
ASLan ASLan
The AVANTSSAR Validation Platform
Services Secured service/policy
Policy
P CP
CS
Composed service/policy
CP TS Wrapper
CS
orchestration/ validation
problem
TS ORCHESTRATOR composition TS VALI DATOR secure
insecure
TS Wrapper
Vulnerability
feedback : Tool input/output

P : Policy
S : Service
CP : Composed Policy
CS : Composed Service
TS : Trust and Security
Deploying secure SAML-based services is not an easy task
Ugarte: Look, Rick. Know what this is? Something that even you have never
seen. Letters of transit signed by General de Gaulle. [Marshal
Weygand] Cannot be rescinded. Not even questioned.
Deploying secure SAML-based services is not an easy task
1. SAML-based SSO for Google Apps (May 2008)

by Google
..a few but critical fields neglected in the IdP SSO service provisioned by Google..
..so that any SP can access to the Googles resources of IdPs members!
3 dcembre 1627.
C'est par mon ordre et pour le bien de l'Etat que le porteur du prsent a fait ce qu'il a fait.
Richelieu
Property
Property Model
Model
1. Step_C_1()
2. Step_SP_1()
3. Step_C_2() Model
Checker Can we automate this task?
Attack
trace
SUV
Abstract flaw automatically detected via

automated reasoning (AVANTSSAR)
Input Consequent XSS discovered via human
Output
inspection (manual testing)
(the story of SAML SSO) Security
impact?
Property
Property Model
Model
1. Step_C_1()
2. Step_SP_1()
3. Step_C_2() Model
Checker
Attack
trace GET http://
HTTP/1.1 200 OK
GET http://
HTTP/1.1 302
SUV data Concretization

Test execution
Test case
engine
SUV
Input
Output
Research prototype
Security
model checking
security testing
Analyst
penetration testing

The SPaCI oS Tool
User I nterface
Complements state-of-the-art
SUV Fault Security Model of User
source location goals the attacker guidance
Model code
of the
SUV
Targets industrially-relevant Source Trace-
Test Results
driven fault Libraries
based
Security Protocols & Web Apps M odel
inference localization
Property-driven
inference and and vulnerability-driven
adjustment Model of test case generation
the SUV
Broad security range
Abstract Test case Vulnerabilities
execution trace
logic-flaws, injections, AC, Attack Patterns
Security Goals
good coverage of OWASP top 10
Test Execution Engine Attacker Models
Promising results
SAML SSO, OAuth2, ..
WebGoat, Shopping Cart, ..
SUV
On-going transfers to SAP and

SIEMENS
(System Under Validation)
Research prototype
model checking
security testing
penetration testing

Rigorous, formal
Automated (at least most of it)
Targets industrially-relevant Synergic combination of independent components

Security Protocols & Web Apps
Logic workflow of the SUV

Discovering vulnerabilities that others do not find
logic-flaws, injections, AC,
Promising results

SIEMENS
Research prototype
model checking
security testing
penetration testing

Targets industrially-relevant
Security Protocols & Web Apps

logic-flaws, injections, AC,
Promising results

SIEMENS
The SPaCIoS tool
and what you can do with it
The SPaCIoS Tool Security
Analyst
The SPaCI oS Tool
User I nterface
Model code
of the
SUV
Source Trace-
Test Results

based
M odel Property-driven
adjustment Model of test case generation
the SUV
execution trace
Attack Patterns
Security Goals
SUV
Workflow
Workflow
Use case 1
property-driven security
testing
One example
companys internal requirements some deviations w.r.t. standard?
security impact?
Security
impact?
Property
Property Model
Model
1. Step_C_1()
2. Step_SP_1()
3. Step_C_2() Model
Checker
Attack
trace GET http://
HTTP/1.1 200 OK
GET http://
HTTP/1.1 302

Test execution
Test case
engine
SUV
Input
Output
Demo
later
Use case 2
model-inference
Black-box model inference
Models?
Property
Property Model
Model
Model
Checker
Attack Black-box model-

trace inference
Test execution
Test case
engine
SUV
Input
Output
White-box model inference Models?
Property
Property Model
Model
Model
Checker
Attack White-box model-

trace inference
SUV data Concretization source code

of system
Test execution
Test case
engine
SUV
Input
Output
Sequence diagrams
Property
Property Model
Model
Model
Checker
Attack
trace translator
SUV data Concretization Sequence

diagrams
Test execution
Test case
engine
SUV
Input
Output
Sequence diagrams
Network traces
Property
Property Model
Model
Model
Checker
Attack Network trace

trace translator
collector
SUV data Concretization Network

traces
Test execution
Test case
engine
SUV
Input
Output
Use case 3
mutation-based testing
No attack
traces?
Property
Property Model
Model
Model
Checker
Mutated Mutation Mutation

Attack Model engine operators
trace
Test execution
Test case
engine
SUV
Input
Output
Details
tomorrow
Use case 4
vulnerability-driven testing
Well-known
vulnerabilities?
Property
Property Model
Model
Model
Checker
Attack
trace
Test execution
Test case
engine
SUV
Input
Output
Well-known
vulnerabilities?
Attack
Instantiation
pattern
files
models
Test execution
Test case
engine
SUV
Input
Output
Attack Pattern + Instantiation file + SUV data
Details
tomorrow
Use case 5
Evolutionary fuzzing
for filtered type-1 and 2 XSS
Use case 6
Testing based on
Business logic patterns
Promising results
OWASP Top 10 The SPaCIoS Tool
A1 Injection WebGoat lesson: String SQL Injection
WebGoat lesson: Numeric SQL Injection
SIEMENS InfoBase and eHealth
A2 Broken Authentication & SAML, OpenID, OAuth: e.g., authentication logic-flaws
Session Management Password brute-forcing on SIEMENS InfoBase and eHealth
A3 Cross-Site Scripting WebGoat lesson: Stored XSS
WebGoat lesson: Reflected XSS
SIEMENS InfoCase and eHealth
A4 Insecure Direct Object SIEMENS InfoBase and eHealth: File Enumeration and Path Traversal
References
A5 Security Misconfiguration WebGoat lesson: Forced Browsing (File Enumeration)
A6 Sensitive Data Exposure SAML, OpenID, OAuth: data confidentiality logic flaws
A7 Missing Function Level WebGoat lesson: Bypass Business Layer Access Control,
Access Control WebGoat lesson: Bypass Data Layer Access Control
WebGoat lesson: Role Based Access Control
SIEMENS eHealth
A8 CSRF SIEMENS InfoBase and eHealth
A9 Using Components with

Known Vulnerabilities
A10 Unvalidated Redirects and
Forwards
Some highlights
SAML SSO authentication flaw and SAML ERRATA corrige
Filtered type-1 and type-2 XSS that other scan tools were not able to find
Shopping for free on several shopping cart web sites (to be published)
Transfers to SAP and SIEMENS

Vulnerability-driven security testing approach applied on Web Apps at SIEMENS
Property-driven and vulnerability-driven approaches applied at SAP: on development of
security standards and security core mechanisms
Research prototype
model checking
security testing
penetration testing

Thank you!
Security
Analyst
Targets industrially-relevant
Security Protocols & Web Apps The SPaCI oS Tool
User I nterface
Model code
of the
SUV
Broad security range Source Trace-
Test Results
based
M odel Property-driven
logic-flaws, injections, AC, adjustment Model of

the SUV
test case generation

execution trace
good coverage of OWASP top 10 Attack Patterns
Security Goals
Promising results
SUV
The SPaCIoS Tool will be available for public

On-going transfers to SAP and download end of January 2014
SIEMENS http://www.spacios.eu
Session 2 Talk 4
Luca Compagna
(SAP)
Formal Validation and Testing of Security

Standards at SAP: from research to industry
Luca Compagna
Formal Validation and Testing of Security Standards at SAP
Vita:
Dr. Luca Compagna is contributing to the Product Security Research at SAP
where is responsible for various internally- and externally-funded research
projects. He received his MSc in Informatics Eng. from the U. of Genova and
his Ph.D. in Computer Science jointly from the U. of Genova and Edinburgh. His
area of interests include security engineering, automated reasoning, and their
application to the modelling and analysis of industrial relevant scenarios. He
contributed to various projects on information security and he has published
various scientific publications in his area of interest.
Luca Compagna
Formal Validation and Testing of Security Standards at SAP
Abstract:
Security standards such as OASIS SAML Single Sign-On, OpenID, and OAuth are key enablers for
the challenging collaborative business scenarios that are envisaged for the cloud and the web
overall, both within corporate boundaries and beyond. Major software companies, SAP among
them, adopt and develop some of these security standards within their core products. Security
standards are highly configurable so to offer interoperability and to be applicable in a multitude
of environments. A risk emerges from this openness and freedom as the chosen configuration
options could have an impact on security. Companies must be extremely careful in properly
implementing their security standard solutions in a way that interoperability is achieved without
endangering the security of the overall targeted scenario. The picture is even more complex
considering that internal requirements of the company itself could demand for small deviations
from the standard and make the overall security assessment more difficult. All in all, evaluating
the consequences of a certain decision, regardless if that is a legitimate security standard option
or a reasonable small deviation, is not an easy task for human-being and would benefit a lot from
tool support. In this talk we will introduce all these important challenges through real industrial
examples and we will describe how SAP is leveraging on off-the-shelf research methodology and
tools achieved in the SPaCIoS EU-funded research project to cope with these challenges.
Validation and Testing of Security Standards at SAP:
from research to industry
Luca Compagna, Product Security Research (SAP AG)
Take away message
Good research + tangible outcomes + relevant industrial issue
Seed for successful research-industry collaborations
Cost/benefit: positive wrt consultancy model
Open challenge
- Development team using the prototype on their own?
- Costs/benefits: Added value perceived, but still some obstacle
- E.g., model generation from small amount of information is a must
2013 SAP AG. All rights reserved. 2

Agenda
The SAML2 SSO story: seed for internal transfer at SAP dev. units
Business problem and challenges
Instrumentation-based Testing with Demo
Conclusion

SAP 2008 / Page 3
SAML SSO: our achievements
Google vulnerability: weak
SAML assertion
Google patch and ack., US-
2 CERT
0 Press, scientific community,
0 experts,
8
Vulnerability on the SAML
SSO prototypical use case
(two different SSL
channels) ?
B 2
U 0 Exploitation on Google: XSS
S 0 Google patch
I
9
N
E
S Consolidation: it is a
S vulnerability, paper
2 Other products affected and
U
N
0 acks!
IT 1
Discussion with OASIS
S 0 members
Errata for SAML

Security Considerations for

SAML
SAML SSO: episode 1
SAML assertion
2 CERT
0 experts,
8
(two different SSL
channels) ?
B 2
S 0 Google patch
I
9
N
E
U
N
0 acks!
I 1
T 0 members
S
Errata for SAML

SAML
SAML SSO: episode 2
SAML assertion
2 CERT
0 experts,
8
(two different SSL
channels) ?
B 2
S 0 Google patch
I
9 SAP NetWeaver SAML2 NG SSO
N
E
U
N
0 acks!
I 1
T 0 members
S
Errata for SAML

SAML
SAML SSO: episode 3
SAML assertion
2 CERT
0 experts,
8
(two different SSL
channels) ?
B 2
S 0 Google patch
I
9
N
E
U
N
0 acks!
I 1
T 0 members
S
Errata for SAML

SAML
SAML SSO: episode 4
SAML assertion
2 CERT
0 experts,
8
(two different SSL
channels) ?
B 2
S 0 Google patch OpenID
I
9
N Our core results applies to this standard as well
E
U
N
0 acks! SAML 2.0 SSO for SAP NetWeaver
I 1
T 0 members
Results of the SAML 2.0 SSO work fed into SAML SSO solution
S
Errata for SAML
for SAP NetWeaver
Security Considerations for At the best of our knowledge no other SAML2 SSO producer
SAML
performed such a rigorous analysis
Agenda
Conclusion
SAP 2008 / Page 9

companys internal requirements some deviations wrt standard?
security impact?
SAML SSO SP-initiated profile

TLS/SSL everywhere or only in certain places?
Shall IdP require signed SAML requests?
Is SP checking ID really necessary?

security impact?

TLS/SSL everywhere or only in certain places?
Shall IdP require signed SAML requests?

security impact?
SAML SSO SP-initiated profile OAuth2 2-legged scenario

TLS/SSL everywhere or only in certain places? What if STS fails in authenticating the OAuth2 client?
Shall IdP require signed SAML requests? Would not be better to have Client_ID within the SAML Assertion?

WHAT-IF:
STS fails in
authenticating Client?
Options:
Client_ID
Property: resource
shall be confidential
Agenda
Conclusion

SAP 2008 / Page 14
Security
impact?
Property
Property Model
Model
1. Step_C_1()
2. Step_SP_1()
3. Step_C_2()
Model Checker
Abstract
trace
SUV
Input
Output
Security
impact?
Property
Property Model
Model
1. Step_C_1()
2. Step_SP_1()
3. Step_C_2()
Model Checker
Abstract
GET http://
trace
HTTP/1.1 200 OK
GET http://
SUV data Concretization HTTP/1.1 302

Test execution
Test case
engine SUV
Input
Output
Reports

SP checking ID does not seem necessary
use TLS/SSL everywhere
(1) = (2) unrealistic, so authc flaw unless:
client speaking SAML
SP setting a cookie

OAuth2 2-legged scenario

STS failing in authc C has low impact
Using Client_ID in assertion can prevent
unauthorized access to resource even when
AS fails in authenticating C

Design verification: abstract test case
Launching pad for XSS: combination of our authentication flaw with missing input validation

Security Testing
From abstract test-cases to real message exchanges (e.g., HTTP request/response)
GET uri_i
SUV
GET uri
HTTP 302, <SAML req.>

Tester
Tester
SUV
HTTP 302, <SAML req.>

Demo
Agenda
Conclusion

SAP 2008 / Page 21
Conclusion and next steps
Devised an approach to
detect logic flaws via model checking and test them on real systems
evaluate the security consequences of design/development decisions
Experimented on established security protocols (SAML SSO, OAuth2, OpenID, )
End users
Standardization bodies working on a draft protocol (design verification)
Development team implementing protocols in the companys premises (e.g., SAP)
Open challenges
Performances: offline analysis together with virtualization
Usability: where the models come from? E.g., sequence diagrams, network traces,

Thank You!
Contact information:
Luca Compagna
luca.compagna@sap.com
Session 3
Active security testing
Bruno Legeard
(FEMTO-ST/UFC & Smartesting)
Model-based vulnerability testing from patterns and behavioral model
Martn Ochoa
(Siemens/Technische Universitt Mnchen)
Model-based vulnerability testing
Prof. Dr. Sachar Paulus
(Kuppinger Cole)
Trustworthy software development
Ari Takanen
(Codenomicon)
Traffic Capture Fuzzer: Effective method for model based fuzzing
Session 3 Talk 1
Bruno Legeard
(FEMTO-ST/UFC & Smartesting)
Model-based vulnerability testing from patterns

and behavioural model
Bruno Legeard
Vita:
Professor at the University of Franche-Comt/Femto-st Institute, Bruno
Legeard is co-founder and senior scientist at Smartesting. He is internationally
recognized as an expert and a well-known speaker in the model-based testing
field. He is strongly experienced in deploying model-based testing solutions
both in enterprise information systems area and in the embedded systems
field. In 2007, Bruno Legeard authored with Dr. Mark Utting the first industry-
oriented book on model-based testing, "Practical Model-Based Testing: A Tools
Approach", Morgan & Kaufmann Publisher.
Bruno Legeard
Abstract:
Our experience with commercially available or open-source web application
vulnerability scanners show that these tools provide lots of false negative in
presence of complex vulnerabilities such as multistep XSS or logical
vulnerabilities. In this talk, we present a novel approach for vulnerability
detection based on test patterns and behavioral & environmental modeling.
These artifacts strongly help to be much more precise and pertinent in order to
reveal known vulnerabilities in the system. To minimize deployment efforts of
such Model-Based Vulnerability Testing technologies, we show how test
patterns may be generic and reusable, and how we can split the behavioral &
environmental model into generic and reusable modeling elements, and
specific and ad-hoc modeling elements to be developed for each test project.
Model-based and Pattern-driven
Vulnerability Testing
Efficient detection of Multistep XSS
Bruno Legeard
1
Agenda
Vulnerability testing State of the practice

The example of Multistep XSS
An approach for Model-based and Pattern-driven Vulnerability
Testing
Discussion and Future work
2
Model-Based Testing tool vendor
MBT for enterprise

application software
Functional and end-to-
end test
Large IT systems
Behavioral models with
BPMN and decision
tables
3
Model-Based Testing tool vendor
MBT for functional

security testing
Based on security
properties
Models in UML/OCL
Examples : Cryptographic
components, Smartcard
4
Agenda

Testing
5
Vulnerability testing state of the practice
Static Techniques Dynamic Techniques
Intrusive
Manual proxies
Manual Code
Penetration (Burp suite,
Techniques review
Testing Webscarab
.)
Static Dynamic Web Scanners,

Automated Application Application Fuzzing tools
Techniques Security Security
Testing Testing ..
(SAST) (DAST)
6
SAST vs DAST
Top five vulnerabilities discovered with static analysis in 2012

7
via HP Fortify on Demand (800 web apps) Source : HP 2012 Cyber Risk Report
SAST vs DAST
Top five vulnerabilities discovered with dynamic analysis in

8
2012 via HP Fortify on Demand (200 web apps) Source : HP 2012 Cyber Risk Report
SAST vs DAST Top 25 / 2011
SAST
DAST
9
DAST Main techniques
Commercial and open-source tools

Web application vulnerability scanner
Fuzzing
Research level
Behavioral fuzzing
10
DAST Main techniques
Commercial and open-source offers

Web application vulnerability scanner
Fuzzing
Research level
Behavioral fuzzing
11
Web application vulnerability scanners
www.sectoolmarket.com
An up-to-date
Benchmark of
Web
application
vulnerability
scanners
(July 2012)
12
Web application vulnerability scanners -
Architecture
Automated Crawler
Injection phase Source Qualys 2012
13
Example IBM Rational AppScan - Reporting
14
Benchmark #1
- vs1 HP WebInspect
- vs2 IBM Appscan
- vs3 - Acunetix
[Antunes10] N. Antunes and M.

Vieira, Benchmarking
Vulnerability Detection Tools for
Web Services, IEEE Eighth
International Conference on Web
Services (ICWS 2010),
Miami, Florida, USA: 2010, pp.
203-210.
15
SQL Injection Vulnerability Detection
Benchmark #2 Real e-learning web app
Autocomplete Web
Internal IP
DOM HTML Application
Unencrypted Directory Disclosure Server
Vulnerability Based Attribute Not Source Code
Login Request Listing Pattern Configuration Scanner
XSS Disabled for Disclosure
Found
Password Field Pattern Found
CWE ID 79 523 548 522 540 200 526

IBM
111 1 4 1 14 19 0
AppScan
e-learning
(Teacher login) 0 0 17 0 0 0 1 NTOSpider
5 N/A N/A N/A N/A N/A N/A Acunetix

IBM
70 1 4 1 14 19 0
AppScan
e-learning
(Learner login) 0 0 15 0 0 0 1 NTOSpider
5 N/A N/A N/A N/A N/A N/A Acunetix
Three days work to configure and analyze reported vulnerabilities

None of the detected vulnerabilities was considered by the
16
development team as possibly exploitable or relevant
Agenda

Testing
17
Prevalence of XSS attacks
Source : Microsoft Security Intelligence Report

Volume 13 - January through June, 2012
18
Cross-Site Scripting (XSS)
Occurs any time

Raw data from attacker is sent to an innocent user
Raw data
Stored in database
Reflected from web input (form field, hidden field, url, etc)
Sent directly into rich JavaScript client
Virtually every web application has this problem

Try this in your browser javascript:alert(document.cookie)
19
Cross-Site Scripting Illustrated
1 Attacker sets the trap update my profile
Application with
stored XSS
Attacker enters a malicious vulnerability
script into a web page that
stores the data on the server
Communication
Bus. Functions
Administration
Transactions
E-Commerce
Knowledge
Accounts
Finance
Mgmt
2 Victim views page sees attacker profile
Custom Code
Script runs inside victims

browser with full access to
the DOM and cookies
3 Script silently sends attacker Victims session cookie 20

XSS Types
Reflected
Link in other website / e-mail link
Stored
Multistep XSS
e.g. bulletin board, forum
DOM-Based
21
Multistep XSS WackoPicko Example
22
Challenge: Multi-step XSS Discovery
Multi-step XSS is a challenging vulnerability for automated tools.

It requires knowledge from the targeted application.
Model-based and Pattern-driven Vulnerability Testing

approach deals with this class of vulnerability, by applying a
Def/Use approach (All-def criterion).
Experiments are conducted on large web application (eLearning,

PubliWeb, )
23
Model-based and Pattern-based vulnerability
testing Research objectives
Research objectives: To improve the accuracy and precision of

vulnerability testing by means of models and test patterns, still
keeping a high level of automation.
Accuracy Precision
Capability to focus on the relevant Capability to avoid both false
part of the software (e.g. from a risk positive and false negative.
assessment point of view) depending
on the targeted vulnerability types. 24
Agenda

An approach for Model-based and Pattern-driven
Vulnerability Testing
Future work
25
MBVT Overall Process
Generic
Generic
Specific
This approach is composed of four Activities:
1. Test Purpose Definition
2. Model Design
3. Test Generation
4. Concretization, Test Execution and Verdict Assignment 26
Test Purpose for Multi-step XSS
Name Multi-step XSS
Description
Objective(s) Detect if an input can embed malicious datum enabling a Multi-step XSS attack.
Prerequisites N/A
Translation Procedure Identify a sensible user input, inject the malicious datum
<script>alert(rxss)</script>.
from vTP Oracle Find the page where the input is rendered, and check if a message box rxss
to test appears.
purposes Variant(s)
Known Issue(s)
Affiliated vTP Reflected XSS
Reference(s)
Generic
Def/use
method
27
1 - Test Purpose Definition
Vulnerability Test Patterns (vTP) are the entry-point :

Based on a study from the ITEA2 DIAMONDS project
Express testing needs and procedure to highlight a breach
The goal is to translate vTP into a machine-readable language.
Smartesting Test Purpose Language:

Designed for security means
Textual language based on regular expressions
Reasons in term of states to be reach and operations to be called
J Botella et. al., MBT of Cryptographic Components, Lessons Learned

28
from Experience, ICST 2013.
2 Model Design
Behavioral modeling notation is based on UML4MBT. Generic
A generic class diagram depicts the

structure of the SUT: pages, actions,
and in/out data (following the
def/use concept).
Class Diagrams (static)

Object Diagrams (initial state)
State-machine diagrams
(dynamic)
29
F Bouquet et. al., A subset of precise UML for model-based testing, 2007.
Modeling: Wackopicko Example
Class Diagram
Specific
State-Machine
Specific
30
3 - Test Generation
Test cases are generated using CertifyIt.
Test generation process is driven by test purposes and models:

Designed for security means
Textual language based on regular expressions
Reasons in term of states to be reach and operations to be called
Result: A suite of abstract vulnerability test cases.

31
4 - Execution and Verdict Assignment
Human Intervention during concretization:

List of malicious vectors (xml file)
Body of the SUTs operations (HTTP level, Browser level)
Observation Technique for XSS: crawl the source page to see if the
injected vector has been sanitized.
Test terminology dedicated to Vulnerability Testing:
List of malicious vectors (xml file)
Body of the SUTs operations (HTTP level, Browser level)
32
Experiments on WackoPicko
Experiment on Multi-Step XSS testing:

1 test purpose
2 abstract test cases:
Login input
Comment input
210 test executions:
105 variants retrieved from OWASPs XSS Cheat Sheet
Results:
All failed for Login input
85 Successes / 20 fails for comment input
Concordant with our manual experiments.

33
0 false positive, 0 false negative
Experiments on Stud-E (eLearning)
Application Under Test presentation:
Virtual Learning Environment
Highly used by french learning academies (> 90%)
More than 15000 users
Experiment on Multi-Step XSS testing:

1 test purpose
11 abstract test cases
1155 test executions:
105 variants retrieved from OWASPs XSS Cheat Sheet
16 steps per test case / 8 steps between injection and observation
1 detected defect (that was not detected previously)

34
No false positive
Agenda

Testing
35
Discussion
Model-based and Pattern-driven vulnerability testing

appears as an accurate and precise technique.
It also has its limitations, inherited from MBT:
Needed effort to provide Models, but a generic part
Needed effort to design Adaptation
Potential solutions:
Use of a behavioral crawler to infer most parts of models
Use of User traces to complement the results of the crawler
Identify the reusability capacity of each artifact
36
Discussion
The approach does not suit every vulnerability type.
MBVT Scope based on OWASP TOP 10 2013:
A1 - Injection
A2 - Broken Authentication and Session Management
A3 - Cross-Site Scripting (XSS)
Legend: A4 - Insecure Direct Object References
Done A5 - Security Misconfiguration
Doable A6 - Sensitive Data Exposure
Out of scope A7 - Missing Function Level Access Control
Under study A8 - Cross-Site Request Forgery (CSRF)
A9 - Using Components with Known Vulnerabilities
A10 - Unvalidated Redirects and Forwards
37
Conclusion and Future Works
Goal First Approach

To improve the precision vTP into Test purposes
and accuracy of Modeling of the SUT
vulnerability testing. Test cases fed with a vectors battery
Limitations Work in Progress

Needed effort to provide Model inference, User traces,
models and design generic artefacts, test purposes
extensions, real-life applications
adaptation layer.
experiments.
38
Thank you for your attention
39
Session 3 Talk 2
Martn Ochoa
(Siemens/TUM)

Martn Ochoa
Vita:
Martn Ochoa is a post-doctoral research in software engineering at the
Technical University of Munich. His research interests lie in the intersection of
security and applied formal methods. He obtained a M.Sc. in mathematics
from the Ludwig Maximillians University of Munich and a Ph.D. in Computer
Science from the Technical University of Dortmund. Prior to his current
affiliation at the TUM, he has worked as a security consultant and researcher
for Siemens.
Martn Ochoa
Abstract:
In this talk we present two methodologies developed within the context of the
SPaCIoS project for testing systems against common security vulnerabilities
based on models. SPaCiTE allows vulnerability testers to generate concrete
attack traces from models of the system behavior based on abstract formal
traces obtained with the help of a model-checker and mutation operators. On
the other hand, VERA allows vulnerability testers to generate concrete attack
traces from explicit models of the attackers. We show applications of these
technologies to WebGoat, an insecure application developed for didactic
purposes.
Model-based Vulnerability Testing
Attacker vs. System models
Martn Ochoa
TUM / Siemens
Together with
J.Oudinet, M.Bchler (TUM) A. Blome (SIEMENS)
M.Torabi-Dashti (ETH) M. Peroli (UNIVR) Keqin Li
(SAP)
SASSI 13
September 20 2013
Sunday, October 13, 13
Two complementary approaches
Goal
Find vulnerabilities
in the SUT

Approach I (VERA): We model

an attacker with no knowledge of the Goal
SUTs logic
in the SUT

Approach I (VERA): We model

an attacker with no knowledge of the Goal
SUTs logic
in the SUT
Approach II (SPaCiTE): We model

the SUTs logic

VERA: Modeling the attacker
Motivation
An abundant number of tools is available for testing

low-level aspects of system security (i.e. Burp, Nessus,
Nikto etc.) in a black-box fashion
URL

Motivation

XSS?
URL

Motivation

XSS?
URL
SQL-i?

Motivation

XSS?
URL
SQL-i?
Path-Traversal attack?
Motivation

XSS?
URL
SQL-i? Old software versions?
Motivation

XSS? Unprotected admin interfaces?

URL
SQL-i? Old software versions?
Challenges
What about particular system congurations?

What about new attack patterns?
What about rigorous documentation of the tests
performed?

Proposed solution: High-level modeling language

for attack patterns!
Example:
1. Attacker identies potentially vulnerable URL (i.e. Web)

II. Attacker analyzes response (i.e. page contains a form)
III. Attacker injects a malicious payload in one parameter (i.e. to
identify SQL-i vulnerabilities)
IV. Attacker analyzes response (Success OR Fail)


Visit URL


Visit URL Send crafted request


Visit URL Send crafted request Success!


Visit URL Send crafted request Success!
Fail

Proposed solution: high-level modeling language for

attack patterns! For instance an injection:
rcv(m)/ [E(m)] rcv(m)/

/snd(URL) snd(URL)
[!E(m)] rcv(m)/

Proposed solution: high-level modeling language for

attack patterns! For instance an injection:
rcv(m)/ [E(m)] rcv(m)/

/snd(URL) snd(URL)
[!E(m)] rcv(m)/
This basic model works for reected XSS, SQL-i,

Command injection, etc.
Cong
Attacker model
Payloads

XSS injection payloads
<script>alert(42)</script>
SQL injection payloads
Login/passwords
l:admin, p:admin

We have modeled the following attacks with VERA:

Various injections (XSS, SQL-i, Command)
Brute-force password attacks
Path traversal attacks
CSRF token strength tests
XML Signature attacks
And tested them on WebGoat and internal Siemens and

SAP applications successfully

Advantages
Models provide an intuitive view of attack patterns

Models are easier to maintain than low-level scripts
Models allow testers to dene/modify attack
patterns quickly

Proof of concept
Python back-end to execute models

Eclipse GUI to draw models
Demo

SPaCiTE: Modeling the system
Motivation: Tools for black-box testing are very

good at nding low level vulnerabilities, but have
several limitations:
Since system logic is unknown, they are not

exhaustive (what are all possible interesting
URLs/States?)
There are vulnerabilities that are not covered

because they do depend on internal system
logic

SPaCITE: Modeling the system
General approach:
I. Start with a secure model of the system
II. Mutate (modify) the model, so that it is not

secure anymore
III. Test the implementation as if it corresponds

to the insecure model

SPaCiTE: Modeling the system
Property:
To see a condential resource one must be authenticated and

authorized to do so
Model:
After authentication, User A is authorized to see his prole
Mutations possible:
User A can see prole A and B

Anybody can see any prole

Vulnerabilities considered so far:
Reected XSS
Persistent XSS
SQL-i
RBAC violations
and tested against WebGoat and SAP applications

Advantages of the mutation based approach:
Systematic coverage of potential vulnerabilities

Coverage of high-level and logical
vulnerabilities
Take advantage of model-specic information

such as data ows (i.e. persistent XSS)
Demo

More info
VERA:
VERA: A flexible model-based vulnerability testing tool.
Blome et al. ICST 2013
SPaCiTE:
Semi-automatic Security Testing of Web Applications from a Secure Model. Bchler
et al. SERE 2012

Session 3 Talk 3
Sachar Paulus
(Kuppinger Cole)

Sachar Paulus
Vita:
Prof. Dr. Sachar Paulus: Senior Analyst at Kuppinger Cole, CEO of a
management consultancy for security (paulus.consult), and Professor for
Information Systems and Security Management. Sachar was member of a
number of advisory boards (e.g., RISEPTIS, the Advisory Board for Research
and Innovation on Security, Privacy and Trust in the Information Society).
Sachar Paulus
Abstract:
In the last years, many attempts have been made to overcome the issue of
insecure and untrusted software. A number of terms have been used to catch
the expectation on how solid a piece of software should be: secure, safe,
dependable and trusted. Yet, we were as an industry not yet able to fulfill this
goal - both on the theoretical as well on the practical side lots of innovations
have appeared, but it seems that overall the situation has become worse
instead of better. This talk aims at giving an overview on current best practices
and corresponding research activities to help software developers creating
trustworthy software, and, among others, the OPTET project.
Trustworth So+ware Development
Prof. Dr. Sachar Paulus
2013 The OPTET consor?um. h/p://www.optet.eu/

Your Presenter please validate the credibility!
Higher Educa?on / Research (1988 1997)

Computer Science / Cryptography / Ph. D. in Number Theory / PostDoc
Industry Experience (1997 2007)

SMEs in the security / cryptography area: consultant
SAP: Director Security Product Management, Chief Security Ocer,
SVP Product Security
Back to Academia (2007 today)

Professor for Informa?on Systems and Security Management
FP7 Project: OPTET
2
This talk is about...
The OPTET Project
Trustworthiness vs. Security
Review of Development Models
Common Criteria and Trustworthiness
Measuring Trustworthiness A/ributes

3
OPTET = Opera?onally Trustworthiness Enabling Technologies
Main data
15 partners
6 Universities
3 SMEs
793,5 Person Months
Total Funding 7,1 M

Total budget 10,7 M
Start: 1st November 2012

Duration: 3 years
The OPTET Project - Outcome
A unied model of trust and OPTET will target generic

trustworthiness Trustworthiness enablers, aiming at:
OPTET trustworthy-by-design
Extend exis9ng pla:orms and primarily
methodology
Trustworthy Applica9on Factory - FI-WARE (FI-PPP Core placorm)
Trustworthy Service Marketplace - UNIVERSAAL
Trustworthiness Maintenance Improve trust and trustworthiness in a
Demonstrated through two Use holis?c manner
Cases: - addressing legal, socio-economic and
Cyber Crisis Management technological aspects.
Ambient Assisted Living
5
Use Case: Ambient Assistent Living
6
Trustworthiness vs. Security
Security Dependability Trustworthiness Trust
The
trustworthiness
A system's of a component
The state of availability,
is dened by
being free from reliability, and Reliance on
how well it
danger or its another en?ty
secures a set of
threat maintenance
func?onal and
support
non-func?onal
proper?es
7
Socio-technical Model
8
Trustworthiness and Trust in Socio-Technical Systems
Trustworthiness Trust
Likelihood that the Belief that engaging
system will in the system for
successfully meet all some purpose will
of the trustors produce an
requirements acceptable outcome
9
Trustworthiness a/ributes
Analysis of sonware a/ributes and proper?es, based on S-Cube and ISO 9126
Extensive literature review whether the a/ribute/property contributes to
trustworthiness (or to trust) (*)
Iden?ed ~40 a/ributes, of various relevance depending on the use case

A/ributes iden?ed aner Literature review

10
Development Models and Trustworthiness
Iden?ed 16 methodologies and

development approaches
None of the models fullls the needs for
trustworthy sonware development (yet)
Two approaches in parallel:
Use metrics for the trustworthiness a/ributes,
development prac?ces...
Development of metrics

Build on cer?able / cer?ed proper?es /
components / systems ...
Use of Digital Trustworthiness Cer9cates
Use / evolu?on of Common Criteria
11
Secure Development Prac?ces / Methodologies
Standard Standardized Best Prac?ces for

Development Security Secure Sonware
Models Approaches Development
Plan-driven Common Criteria BSIMM /
ISO 15408 OpenSAMM
Incremental
Microson SDL
SSE CMM
Reuse-oriented
ISO 21827
OWASP CLASP
Model-driven
ISO 27002 TOGAF
Test-driven
12
Common Criteria and Trustworthiness
Mapping of CC
FSRs to TW
a/ributes
Iden?fying
addi?onal reqs
for a CC+
Work in progress
13
Trustworthiness metrics
For every trustworthiness a/ribute (property), iden?fy (at least) one

metric
Literature research, usage of standards

Usage of the GQM methodology
Varia?on of metrics for the three contexts: engineering, marketplace and
run-?me
Planned ac?vi?es
Development of a computa?onal model for ONE metric for trustworthiness
Process metrics
14
Metrics - Example
15
Metric Tool as a Generic Enabler
Metrics Database for all

trustworthiness a/ributes
Library for metrics computa?on,
usage of tools if possible
Eclipse plugin for developer
context
Work in progress
16
Summary
Trustworthiness is more than security

We need both product and process aspects
Common criteria is not sucient
Developing good metrics needs considerable eort
OPTET develops Generic Enablers for the Future Internet PPP
17
References
"Towards Trustworthiness Assurance in the Cloud", Francesco Di Cerbo, Pascal
Bisson, Alan Hartman, Sebas?en Keller, Per Hakon Meland, Micha Moe, Nazila Gol
Mohammadi, Sachar Paulus and Stuart Short. In: Cyber Security and Privacy EU Forum
2013. To appear in Springer Communica?ons in Computer and Informa?on Science,
Heidelberg.
"Trustworthy So+ware Development", Sachar Paulus, Nazila Gol Mohammadi,

Thorsten Weyer. In: B. De Decker et al. (Eds.): CMS 2013, LNCS 8099, pp. 233--247.
IFIP Interna?onal Federa?on for Informa?on Processing (2013).
"An Analysis of So+ware Quality AFributes and their Contribu9on to

Trustworthiness", Nazila Gol Mohammadi, Sachar Paulus, Mohamed Bishr, Andreas
Metzger, Holger Koennecke, Sandro Hartenstein and Klaus Pohl. In: Proceedings of
the 3rd Interna?onal Conference on Cloud Compu?ng and Services Science (CLOSER
2013); Aachen, Germany, May 8-10, 2013.
18
Session 3 Talk 4
Ari Takanen
(Codenomicon)
Traffic Capture Fuzzer: Effective method for

model based fuzzing
Ari Takanen
Traffic Capture Fuzzer
Vita:
Ari Takanen, founder and CTO of Codenomicon, has been active in the field of
software security testing research since 1998 focusing on information security
issues in next-generation networks and security critical environments. He has
worked with numerous software development projects across the industry,
both from software quality aspect but lately more on security test automation
perspective. Ari is the author of several papers on software security, and is a
frequent speaker at security and testing conferences, leading universities and
international corporations. He is also the author of two books on VoIP security
and security testing.
Ari Takanen
Traffic Capture Fuzzer
Abstract:
Codenomicon's Traffic Capture Fuzzer (TCF) finds vulnerabilities in proprietary
protocols where there is no ready model based fuzzer available. TCF uses
captures of actual network traffic and creates a behavioral model based on
the captures. Codenomicon's powerful anomalisation engine creates the test
cases from this inferred model. The fuzzing framework can create and execute
fuzz tests for new or proprietary protocols easily, even during development.
Defensics Software Development Kit (SDK) allows users to enhance the TCF
operation. The SDK increases the focus and impact of the capture based
testing. Compared to the other fuzzing platforms, it requires much less
expertise, time and effort from the user to create the test cases: most of the
work is automatic.
Traffic Capture Fuzzer:
Effective method for
model based fuzzing
1
Background
Fuzzing and security testing since 1996
Model-based since 1998
Commercial spin-off Codenomicon in 2001
100 ppl around the world
200+ customers include Cisco, NSN, Microsoft, Adobe,
Verizon, T-Systems, BT, NTT, government agencies
New in Diamonds project:

Automated test generation through
deducted models from templates
General purpose fuzzing tools for
network protocols, files, and web services
1.11.2013 Copyright Codenomicon 2012 | Confidential 2

How is Security Compromised?
Availability:
A zero day attack is used to
compromise a specific computer
program, which often crashes as a
result Hacker can spawn new
processes
Integrity:
Hacker controlled processes can now change anything in
the system
Confidentiality
Hacker controlled processes can now eavesdrop on all data
and communications 4
Security Vulnerability
= Just A Software Bug
Industry is Slowly Waking Up to
the Unknown Threats
All software has All our zero-day
undetected exploitable vulnerabilities were found You would be a fool
vulnerabilities with Fuzzing. not to Fuzz.
- Security Vendor 2009 Software Vendor 2010 Analyst 2011
6
What is Fuzzing?
A testing technique where purposefully

unexpected and/or invalid input data is fed to
tested system in hope to find robustness and
security problems
7
Fuzzing techniques
Taming infinity
The set of malformed inputs is infinite
Need meaningful testing in finite time
A good fuzzer is skillful at picking values that
are most likely to find bugs
Boundary conditions
Illegal values
Bad checksums
Tough strings
Fuzzing techniques:
Random
Pump random bits at a port
No protocol knowledge
Dumb
Occasionally effective
Shallow code coverage
Something like waiting for the monkeys to

write Shakespeare
9
Advanced Fuzzing Techniques
Mutation/Template-Based Fuzzing
Quality of tests is based on the used template
(seed) and mutation technique
Slow to execute, least bugs found
Generational/Specification-Based Fuzzing
Full test coverage, as the model
is built from specification
Fast to execute, most bugs found
10
Example: Traffic
Capture Fuzzing
Step 1: Load Template
Traffic Capture Fuzzer is
special from other
model-based test
generators
It does not use internal
model
Test generation is based
on recorded traffic
capture

Step 2: Automatic Modeling
Behavioral model is
automatically created
from the network traffic
Tests can be executed in
both directions:
Server
Client such as browser

Model can be edited with
internal or external editors

Step 3: Check interoperability
Dynamic data
is added to the
model, and
interop with
test target is
automatically
verified

Step 4: Select tests and
coverage
Default test coverage is
optimized for speed and
efficiency
User can select a subset
of the protocol to
anomalize, or different
ranges of automated
test coverage

Step 5: Execute tests
Tests are dynamically
created during
execution, and each test
case is documented
Test execution statistics,
logs and test case
documentation can be
browsed during test
execution, or analyzed
after tests are complete
Step 6: Analyze failures
Failures (crashes, slow
behavior, wrong
responses) are
automatically detected
and documented
Test configuration and
failure reproduction can
be performed on the
same or different
location (for bug
reporting)
Fuzzer Smartness vs Coverage
Dumbest fuzzer is doing random mutations

to a template (file, PCAP)
Smart fuzzers use interface models and
carefully selected test generation algorithms
Fuzzer Efficiency Case Study
Most important efficiency metric for fuzzers:

How many bugs does it find
How much time does it take to find them
Smart model-based Generation fuzzer
generational fuzzer executed for 17 hours
found 10 unique bugs
Mutation fuzzer took

Both found 2 same bugs 118 hours (5 days) to
execute, after which no
Mutation fuzzer found more new bugs were
4 unique bugs found 20
Academics?
Ask for academic licenses!!
Contact:
Ari Takanen, CTO, art@codenomicon.com
Dr. Volker Baier, volker@codenomicon.com
Schedule a meeting with our Munich team!

THANK YOU QUESTIONS?
Thrill to the excitement of the chase!

Stalk bugs with care, methodology,
and reason. Build traps for them.
....
Testers!
Break that software (as you must) and
drive it to the ultimate
- but dont enjoy the programmers
pain.
[from Boris Beizer]
PROACTIVE SECURITY AND ROBUSTNESS SOLUTIONS

Session 4
Active and passive security testing
Riccardo Scandariato
(Katholieke Universiteit Leuven)
Security vulnerability prediction
Graham Steel
(Cryptosense)
Security analysis of APIs, including the W3C Crypto API
Ana Cavalli
(Institut Mines-Telecom)
Application of passive testing techniques to secure interoperability testing
Wissam Mallouli
(Montimage)
Passive testing for security checking using MMT
Session 4 Talk 1
(KU Leuven)

Vita:
Dr. Riccardo Scandariato received his PhD in Computer Science from Politecnico di
Torino, Italy, in 2004. During year 2003, he was a visiting research associate at the
University of Virginia, USA with the Dependability Research Group. In 2004-2005, he
was a post-doctoral researcher at Politecnico di Torino, with the Software Engineering
Research Group. Since January 2006, he joined the Distributed Systems and Computer
Networks Research Group (DistriNet) at KU Leuven, Belgium. In June 2009 he became a
permanent member of the staff and he currently leads a team of researchers in the
area of secure software. Dr. Scandariato's main research activities are in the area of
secure software engineering, with a particular focus on empirical methods for security
and security in software architectures. He has published about 50 papers in the area of
security and software engineering. He is an Associate Editor of the International
Journal of Secure Software Engineering (IJSSE).
Abstract:
Identifying security vulnerabilities in a code base is like finding the proverbial
needle in a haystack, as the occurrence of a vulnerability is a relatively rare
event. In this respect, prediction models are useful to guide the quality
assurance activities, e.g., to identify the code locations that deserve special
attention. We present an approach to predict which components of a software
system contain security vulnerabilities. The approach is based on text mining
the source code and leverages on machine learning techniques. The evaluation
is performed in the context of applications written in the Java and C++. The
results show that the technique has excellent prediction power and performs
better than state-of-the-art approaches.

!!!"#$
!%
&!'$
$(

)
*+,!
(
+-
)$ !+

.(!
/
!0
1$! (!
3
(

(
!$%

)

$(
!
! !
$

$

!
2

$!!+.+!0

5
(3!
!

!
"
#

4
!
$!!+.+!0

#

-
'+
$
($!.+!0
5$
# $ '.'30
3
!$
! .'3'07

!
#$

6
5!$/ !
9
* (.(0$!!+
($!
*$

!

9$
!
*
!+:+ !;
8

*$

!
9

=(.0*( .$0

*
$!!$=
$
!
<
*!
3
(
*!
?3)&,@ ,!
?!
!
,
$+!.
$$

!0
9!
>
:+ !
9

!$(!%
C(
(!(
+!$
,$
+
/ (! !(

1$$

!$$C(* .+ !0
D(! !(
$

E!$(
3/
$ + (!
AB
?$

3 ( $
7
! *!++ !! !(

("!

1/ C!!
(
!

!
5
(3!
1
!*

."!
0
AA
'($

+
( ( ( &
( (
' &' '

&
&
)
* %
!+' !,' !(&'
#

#

AF

$%G
! !
-
./3
(0'+!! !. "0
&'+++.#"0
1

23 (0
)
$+<<H
(+
!$
+A8H
A2
%
)+ $

!$$C(:
+ !
&($

($
*
$

!
3+:
'+

5$$()$
( A4
!$
(
Session 4 Talk 2
Graham Steel
(Cryptosense)
Security analysis of APIs, including the W3C

Crypto API
Graham Steel
Security analysis of APIs, incl. The W3C Crypto API
Vita:
Graham Steel holds a masters in mathematics from the University of
Cambridge and a Ph.D. in informatics from the University of Edinburgh. He has
been a researcher at INRIA, the French national agency for computer science
research, where he was part of the Prosecco project team based in central
Paris. He recently cofounded the Spin-Off company Cryptosense, which
provides solutions to an international clientele in particular in the financial,
industrial and government sectors. Steel's main research interests are in
formal analysis of information security and applied cryptography. His current
work on cryptographic API verification involves using formal techniques to
construct and analyze abstract models of cryptographic device interfaces. In
addition to international conference and journal publications, his recent
results have featured in Wired magazine and the New York Times.
Graham Steel
Security analysis of APIs, incl. The W3C Crypto API
Abstract:
Using a unique combination of reverse-engineering and model-checking, Cryptosense
technologies permit automated security analysis of cryptographic APIs (www.cryptosense.com). In
the domain of smartcards and Hardware Security Modules (HSMs), Cryptosense Analyzer has
already been used to find and help to fix several previously unknown vulnerabilities and to assist
with secure device configuration. It is now part of standard security testing and monitoring in two
national security agencies as well as a large international bank.
Currently, the commercially available version of Cryptosense Analyzer only deals with interfaces
conforming to the industry standard PKCS#11. In our development programme we are working on
the Cryptosense Generator that, given an annotated specification of an interface in e.g. a header
file, can produce a specific version of the Cryptosense Analyzer customized for that interface. We
are experimenting with MS and Java APIs as well as some proprietary interfaces. More recently,
we have been investigating the W3C Crypto API that lets developers implement secure application
protocols on the level of Web applications, including message confidentiality and authentication
services, by exposing trusted cryptographic primitives from the browser. In this talk we will
demonstrate the Cryptosense Analyzer and describe our recent results.
Security Analysis for Cryptographic APIs
Graham Steel
c
Cryptosense 2013
Cryptographic Security Today
Companies use more and more crypto to

protect sensitive data, oer new business
models, comply with regulations,..
Crypto algorithms are now very
powerful
But cryptographic systems are
complex : hard to congure
securely
And they are fragile : one mistake
enough to create a vulnerability
Graham Steel 2/ 32
Cryptography in Practice v1
PM talks to client to understand security goals and threats
Graham Steel 3/ 32
Engineer and PM discuss,

decide formal requirements of crypto schemes
Graham Steel 4/ 32
Engineer consults relevant literature for provably secure scheme
Graham Steel 5/ 32
Engineers get together and plan some fun crypto stu

Graham Steel 6/ 32
Literature consulted
Graham Steel 7/ 32
Literature augmented by attacks on the system
Graham Steel 8/ 32
PM instructed by client : Solution must use PKCS#11 hardware
Graham Steel 9/ 32
PM instructs engineer : Use PKCS#11 API

Graham Steel 10/ 32
Graham Steel 11/ 32

RSA Public Key Cryptography Standard (PKCS) 11
PKCS #1 describes the RSA encryption algorithm, padding etc.
PKCS#11 Describes cryptoki : cryptographic token interface
Ubiquitous in industry for authentication tokens, smartcards

(and HSMs, other devices, . . .)
Version 1.0 of PKCS#11 1995, current version 2.20 2004
Graham Steel 12/ 32

Sessions, PINs, Handles, Attributes
Application programs open a session with a token

Opening a session requires a PIN
Keys (and other objects) accessed by handles
Keys have attributes to control usage
Graham Steel 13/ 32

Graham Steel 14/ 32
Generating keys with PKCS#11
A key template is a partial specication of key attributes

Templates are used for creating, manipulating, and searching for
objects
C GenerateKey :
new n,k
T h(n, k); T
Graham Steel 15/ 32

Setting Key Attributes
C SetAttributeValue :
T , h(n, k) h(n, k); T
T can specify new values for any attributes, but may cause
CKR TEMPLATE INCONSISTENT, CKR ATTRIBUTE READ ONLY
Graham Steel 16/ 32

Wrap and Unwrap
Wrap :
h(x1 , y1 ), h(x2 , y2 ); wrap(x1 ), {y2 }y1
extract(x2 )
Unwrap :
new n1
h(x2 , y2 ), {y1 }y2 , T ; unwrap(x2 ) h(n1 , y1 ); extract(n1 ), T
Graham Steel 17/ 32

Graham Steel 18/ 32
Key Usage
Encrypt :
h(x1 , y1 ), y2 ; encrypt(x1 ) {y2 }y1
Decrypt :
h(x1 , y1 ), {y2 }y1 ; decrypt(x1 ) y2
Graham Steel 19/ 32

PKCS#11 Security
Section 7 of standard :
1. Access to private objects on the token, and possibly to
cryptographic functions and/or certicates on the token as well,
requires a PIN.
2. Additional protection can be given to private keys and secret
keys by marking them as sensitive or unextractable. Sensitive
keys cannot be revealed in plaintext o the token, and
unextractable keys cannot be revealed o the token even when
encrypted
Rogue applications and devices may also change the commands
sent to the cryptographic device to obtain services other than what
the application requested [but cannot] compromise keys marked
sensitive, since a key that is sensitive will always remain
sensitive. Similarly, a key that is unextractable cannot be modied
to be extractable.
Graham Steel 20/ 32
Graham Steel 21/ 32
Graham Steel 22/ 32
Clulow, CHES 2003
Graham Steel 23/ 32

Prevent generation of decrypt and wrap keys..
Intruder knows : h(n1 , k1 ), h(n2 , k2 ), k3

State : sensitive(n1 ), extract(n1 ), extract(n2 )
Set wrap : h(n2 , k2 ) ; wrap(n2 )
Set wrap : h(n1 , k1 ) ; wrap(n1 )
Wrap : h(n1 , k1 ), h(n2 , k2 ) {k2 }k1
Set unwrap : h(n1 , k1 ) ; unwrap(n1 )
new n3
Unwrap : h(n1 , k1 ), {k2 }k1 h(n3 , k2 )
Wrap : h(n2 , k2 ), h(n1 , k1 ) {k1 }k2
Set decrypt : h(n3 , k2 ) ; decrypt(n3 )
Decrypt : h(n3 , k2 ), {k1 }k2 k1
[Delaune, Kremer & S., CSF 2008]
Graham Steel 24/ 32

Graham Steel 25/ 32

Device Supported Functionality Attacks found
Brand Model s as cobj chan w ws wd rs ru su Tk
Aladdin eToken PRO wd
Athena ASEKey
Bull Trustway RCI wd
Eutron Crypto Id. ITSEC
Feitian StorePass2000 rs
Feitian ePass2000 rs
Feitian ePass3003Auto rs
Gemalto SEG
MXI Stealth MXP Bio
RSA SecurID 800 rs
SafeNet iKey 2032
Sata DKey rs
ACS ACOS5
Athena ASE Smartcard
Gemalto Cyberex V2 wd
Gemalto SafeSite V1
Gemalto SafeSite V2 rs
Siemens CardOS V4.3 B ru
Graham Steel 26/ 32

The future of Cryptosense
Analyzer has been deployed in testing PKCS#11-compatible

HSMs, which have a more sophisticated attribute policy, and
support many conguration options
In use at an aircraft manufacturer, two major bank European
banks and two national security agencies
The spin-o company was created in September 2013 with
250ke of funding (French Ministry of Research Prize)
Working on MS-CAPI/CNG, Java JCA/JCE, OpenSSL
engine, some proprietary APIs
Aim to generalise architecture to produce versions of the
analyzer for custom APIs
Graham Steel 27/ 32

New Application Areas
Cryptosense can be applied to any cryptographic API, for example :

Mobile phone Secure Elements
Smartmeters, smartgrid infrastructure
Authentication tokens (and associated backends)
Automotive (ECUs, . . .)
Industrial sensors, control devices
Web Crypto API (W3C)
Graham Steel 28/ 32

W3C Crypto API
Drafted by a W3C including all major browser vendors

First Working Draft 25 June 2013
Idea is to replace Javascript crypto with a browser API that
manages keys and oers access to crypto services
So what about key management ?

Key wrap and Unwrap are dened.... but importKey and exportKey
are not, yet.
Watch this space !
Graham Steel 29/ 32

Cryptosense Generator

Graham Steel 30/ 32

Cryptosense Monitor

!

Graham Steel 31/ 32

Summary
RSA PKCS#11 : Many attacks, many approaches to securing

Cryptosense Analyzer : an automated audit tool
Working on other common APIs, and a Monitor tool.
Future work : Cryptosense Generator (make a version of the
Analyzer for a specifc API)
www.cryptosense.com
Graham Steel 32/ 32

Session 4 - Talk 3
Ana Cavalli
(Institut Mines-Telecom)
Application of passive testing techniques to

secure interoperability testing
Ana Cavalli
App. of passive testing techniques to secure interop. testing
Vita:
Ana R. Cavalli received the Doctorat d'Etat es Mathematics Sciences and
Informatics in 1984 from the University of Paris VII, France. From 1985 to
1990, she was a staff research member of CNET(Centre National d'Etudes des
Telecommunications), where she worked on software engineering and formal
description techniques. Since 1990, she joined as professor the Institut
Telecom/Telecom SudParis (TSP). Currently, A.R. Cavalli is responsible at TSP of
the research group "Protocols and Services Validation" and the Software-
Networks department. She is a member of the CNRS Samovar research
laboratory. She has been chair of several conferences (IFIP FORTE/PSTV, IEEE
ICNP, IFIP TESTCOM, IFIP CFIP, IEEE ICST) and published more than 200 papers.
Her research interests include validation methods for Internet protocols and
services; methods for conformance, interoperability and security testing;
monitoring techniques based on testing and the application of these
methodologies to industrial applications.
Ana Cavalli
App. of passive testing techniques to secure interop. testing
Abstract:
Security and interoperability have been identified as essential by developers,
integrators, users and operators of systems and networks that have to comply with
strong reliability requirements. These systems and networks are required to be more
and more open and new technology is designed to facilitate the interoperation
between these networks composed of heterogeneous, communicating devices. Since
the environment may be potentially hostile and contain malicious opponents, it is
crucial to define frameworks to enforce secure interoperability. By secure
interoperability we mean the property that two or more entities need to possess in
order to communicate or exchange information even though they have different
security policies. In this lecture, we will present a formalism to specify the
interoperability of security policies, a tool for the implementation of these policies and
passive testing techniques used to check that security requirements are satisfied by the
implementation in order to insure a secure interoperability. The application of these
techniques will be illustrated by their application to a real case study, a Multisource
Information System.
Application
o f passive t esting
techniques
t o secure
interoperability testing
Ana Cavalli
Khalifa Toumi
Ins2tut Mines- Telecom/
TELECOM SudParis

Planning
Conformance tes2ng
Ac2ve tes2ng techniques
Fault models
Passive tes2ng (Monitoring)
A case study (Inter-Trust project)
2
Conformance testing
Conformance tes2ng:
to check that an implementa2on conforms to a
specica2on
to check that the implementa2on sa2ses some
expected proper2es
Faults detected :
output faults, if the implementa2on transi2on
produce a wrong output
transfer faults, if the implementa2on transi2on go in a
wrong state
mixtes faults, both output and transfer faults 3

What is active testing ?
IUT Ac4ve Tester Verdict:
PASS,FAIL,
INCONC.

Formal Test
Specica4on Suites
It is assumed that the tester controls the implementa2on

Control means: aRer sending an input and aRer receiving an output,
the tester knows what is the next input to send
The tester can guide the implementa2on towards specic states 4
Automa2c test genera2on methods can be dened
Formal methods for
test generation
Objec4ves
To op2mize tests produc2on by reduc2on of
2me and cost
HandcraRed tests have a high cost
Test suites for real protocols are in average
very huge

5
To improve faults coverage
Limitations of active testing
Non applicable when no direct access to the

implementa2on under test
Non controllable interfaces
Interference on the behaviour of the
implementa2on
Example: components tes2ng
6
Components Testing
Test in context, embedded tes2ng:
Tests focused on some
components of the system, to
avoid redundant tests
Interfaces semi-controllables
In some cases it is not possible
Environment
to apply ac2ve tes2ng
a bc c b a
ib
C ia
Internal
Message
Context Module
A
Embedded Module
7
Why passive testing?
Conformance tes4ng is essen4ally focused on verifying
the conformity of a given implementa4on to its
specica4on
It is based on the ability of a tester that s2mulates the
implementa2on under test and checks the correc2on of the
answers provided by the implementa2on
Closely related to the controllability of the IUT
In some cases this ac2vity becomes dicult, in par2cular:
if the tester has not a direct interface with the implementa2on
or when the implementa2on is built from components that have to
run in their environment and cannot be shutdown or interrupted (for
long 2me) in order to test them
8
Passive Testing based on Invariants
-Test by invariants
Simple (Output) invariant
Deni2on : invariant in which the test is an output
Meaning : immediatly aRer the sequence prambule there is
always the expected output
Example :
(i1 / o1) (i2 / o2)
(preambule in blue, expected output in red)

9
Test by invariants : Obligation (Input) invariant
Deni2on : invariant in which the test is an input

Meaning : immediatly before the sequence preamble there is
always the input test
Example :
(i1 / o1) (i2 / o2)
(preamble in blue, test in red)
10
Test by invariants : succession
invariant
Deni2on : I/O invariant for complex proper2es (loops )

Example :
the 3 invariants below build the property : only the third i2 we
meet is followed by o3
(i1 / o1) (i2 / o2)
(i1 / o1) (i2 / o2) (i2 / o2)
(i1 / o1) (i2 / o2) (i2 / o2) (i2 / o3)
11
PASSIVE vs ACTIVE TESTING
IUT Ac4ve Tester Verdict:
PASS,FAIL,
INCONC.

Formal Test
Specica4on Suites
+&-:
Possibility to focus on a specific part of the
specification
Automatic test generation
" May modify (crash) the IUT behavior
IUT
PO Passive Tester Verdict:

Trace
Collection PASS,FAIL,
INCONC.
System User
System Specica4on
+&-:
No interferences with the IUT 12
System modelisation is not necessary
" No test generation

Application of passive
testing (monitoring)to
secure interoperability
The Inter-Trust project
q The projet has as a main objective the development of different
techniques to insure secure interoperability
q Case study based on the interoperability of two information
systems belonging to two differents nations (Nation 1 and Nation
2) that need to interoperate
q We have used the language ORBAC to describe the security
properties and the tool MMT to verify the properties using
monitoring
q The proposed scenario is the following:
q Nation 2 need to access some resources of Nation 1
q These resources can be confidential, secret or public and
they can be accessed by different intelligence officers
q We need to check that the resources of Nation 1 are
accessed by the authorized intelligence officers of Nation 2
14

OrBAC language
qOrganization Based Access Control OrBAC
q Abstraction of security policies: abstract rules
q Role, View, Activity
q Instantiation of security policies: concrete rules

q Subject, object, action
q Context:
q Spatial context, Temporal context,
User-declared context, Prerequisite
context and Provisional context
q Example: Permission(org,role,activity,view,context)
15
MotOrBAC Tool
qMotOrBAC Functionalities
q Edition, simulation and analysis of the security rules
q Delegation management
q Entity hierarchies
q Conflict management
Plug-ins interface
16
Plug-in1 Plug-in2 ... Plug-ink
Intelligent System Architecture
17
General Architecture
18
Inter-Trust Platform
Administra2on
AC Interoperability
model
templates security policy
Plateform
MotOrBAC INTER-TRUST
AdOrBAC VPO
API OrBAC Network
Sning the
Probe MMT communica2on
Analyzing traces,
Engine MMT detec2ng aaacks System2
and
19
verifying proper2es
Plateform
Result les (HTML) INTER-TRUST
19
Interoperability Scenario (1)
1. Specica2on of the local policy of each par2cipant
2. Deriva2on of the interoperability policy

3. Edi2on of the policy with MOtOrBAC and resolu2on of
conicts
4. Ini2aliza2on of the service web service for managing
the policy and the execu2on of the MMT tool
5. Star2ng the collabora2on
6. Analyzing the traces and crea2on the result les 20

1. Specica2on of the local policy of S1
ORO is the Intelligence Officer (high level)
ORT is the Intelligence Officer (simple

level)
Roles Deni2on DRM is the Directorate of Intelligence
SYSTEM
OR can be an ORO or an ORT or a DRM 21

1. Specica2on of the local policy of S1
manage: read and / or write information
read: read an information
write: write an information
Ac2vi2es deni2on notify: send a notification message
tag : add a label
tag_interop: add an interoperability label

for shared data
22
1. Spcica2on of the local policy of S1
IR_TS: Very Secret Information.
IR_S: Secret Information.
IR_C: Confidential Information.

Views deni2on
IR_R: Restricted Information.
IR_NC: Unclassified Information.
IR: Information
23
The following contexts are defined in

S1:
default_ctx: default context, always on
false_ctx : context never activated

Context deni2on
aprs_ajout_IR: context activated after
the addition of intelligence information
Need_to_know: context activated if

some inputs are checked 24
Example 1:
An ORT has the right to manage intelligence informa2on if the
informa2on has a security level Conden2al or lower.

Rule1:permission(S1, ORT, manage, IR_C, defaut_ctx)

25
Exemple 1:
2. 1 VPO crea2on Creating a Virtual Private Organization
(VPO) between S1and another S2 nation
called VPO_S2_to_S1
Each en2ty in the view V of
the system S1 will belong to
the view IR in the VPO Exemple 2:
use(VPO_S2_to_S1, O, V)
use(S1, O, V) and
2. 2 Mapping between sub_view(S1, V, IR) and
en22es tag_interrop(O, true)
26
A subject S will keep the Example 3: 26

same role in the VPO empower(VPO_S2_to_S1, S,
ORT) empower(n1, S, ORT)
Example 1:

An ORT in S2 has the right to read the intelligence informa2on of S1 if
the security level of this informa2on is Conden2al or lower.

Rule: permission(VPO_S2_to_S1, ORT, read, IR_C,
need_to_know)
27
3. Edi2on of the policy with MOtOrBAC and resolu2on of
conicts
28
4.1 Ini2aliza2on of the service web service for managing the policy
29
Monitoring Results
30
Conclusions
Passive tes2ng (monitoring) techniques are well adapted to
validate security proper2es:
they can be used for vulnerabili2es detec2on, intrusion
detec2on, etc.
They can be used for the systems supervision
They are complementary of ac2ve tes2ng techniques
31
References
1. Pramila Mouaappa, Stephane Maag and Ana Cavalli, "IOSTS based Passive Tes2ng approach for the Valida2on of
data-centric Protocols",12th Interna2onal Conference on Quality SoRware (QSIC 2012), Xian, China, 27th-29th August 2012.

2. Nahid Shahmehri, Amel Mammar, Edgardo Montes de Oca, David Byers, Ana Cavalli, Shanai Ardi and Willy Jimenez,
"An Advanced Approach for Modeling and Detec2ng SoRware Vulnerabili2es",
Journal Informa2on and SoRware Technology, vol 54, issue 9, September 2012.

3. Anderson Morais and Ana Cavalli, "A Distributed Intrusion Detec2on Scheme for Wireless Ad Hoc Networks",
27th Annual ACM Symposium on Applied Compu2ng (SAC'12), March 25-29, 2012, Riva del Garda (Trento), Italy

4. Fayal Bessayah, Ana Cavalli, A Formal Passive Tes2ng Approach For Checking Real Time Constraints, 7th Interna2onal
Conference on the Quality of Informa2on and Communica2ons Technology, September 29th 2010, Porto, Portugal.

5. Csar Andrs, Stephane Maag, Ana Cavalli, Mercedes G. Merayo, Manuel Nunez, "Analysis of the OLSR Protocol
by using formal passive tes2ng", APSEC 2009, December 2009, Penang, Malaysia.

6. Felipe Lalanne, Stephane Maag, Edgardo Montes de Oca, Ana Cavalli, Wissam Mallouli and Arnaud Gonguet ,
An Automated Passive Tes2ng Approach for the IMS PoC Service, 24th ACM/IEEE Interna2onal Conference
on Automated SoRware Engineering, November 2009, Auckland, New Zealand.

7. Ana Rosa Cavalli, Azzedine Benameur, Wissam Mallouli, Keqin Li, A Passive Tes2ng Approach for Security Checking and its
Prac2cal Usage for Web Services Monitoring, invited paper, NOTERE 2009, 29-June 3-July, 2009, Montral, Canada.

8. Ana Cavalli, Stephane Maag and Edgardo Montes de Oca, A Passive Conformance Tes2ng Approach for a Manet
Rou2ng Protocol, The 24th Annual ACM Symposium on Applied Compu2ng SAC'09, March 9-12 2009, Hawaii, USA.
32
9. Ana R. Cavalli, Edgardo Montes De Oca, Wissam Mallouli, Mounir Lallali, Two Complementary Tools for the Formal
Tes2ng of Distributed Systems with Time Constraints, The 12-th IEEE/ACM Interna2onal Symposium on Distributed
Simula2on and Real Time Applica2ons (DS-RT 2008), October 27-29, Vancouver, Canada.

10. Wissam Mallouli, Fayal Bessayah, Ana R. Cavalli, Azzedine Benameur, Security Rules Specica2on and
Analysis Based on Passive Tes2ng, The IEEE Global Communica2ons Conference (GLOBECOM 2008),
November 30 - December 04, New Orleans, USA.

11. J.-M. Orset, B. Alcalde and A. Cavalli, An EFSM-Based Intrusion Detec2on System for Ad Hoc Networks, ATVA 05,
Taipei, Taiwan, October 2005.
12. E. Bayse, A. Cavalli, M. Nez, and F. Zadi. A passive tes2ng approach based on invariants: applica2on to the wap.
In Computer Networks, volume 48, pages 247-266. Elsevier Science, 2005.

13. Csar Andrs, Mara-Emilia Cambronero, Manuel Nez:
Formal Passive Tes2ng of Service-Oriented Systems. IEEE SCC 2010: 610-613

14. Csar Andrs, Mercedes G. Merayo, Manuel Nez: Mul2-objec2ve Gene2c Algorithms: Construc2on and
Recombina2on of Passive Tes2ng Proper2es. SEKE 2010: 405-410

15. Csar Andrs, Mercedes G. Merayo, Manuel Nez: Passive Tes2ng of Timed Systems. ATVA 2008: 418-427
33
Session 4 Talk 4
Wissam Mallouli
(Montimage)

Wissam Mallouli
Vita:
Dr. Wissam Mallouli is currently a research & development engineer at
Montimage France. He received his Masters degree from the Evry Val
dEssonne University in 2005 and his PhD in computer science from Telecom
and Management SudParis (France) in 2008. His topics of interest cover formal
testing and monitoring of functional behaviours and security aspects of
distributed systems and networks. He worked in several European and French
research projects. He also participates to the program/organizing committees
of numerous national and international conferences. He published more than
25 papers in conference proceedings, books and journals. More details can be
found on his webpage http://www.mallouli.com.
Wissam Mallouli
Abstract:
Network monitoring is a laborious challenging task that is vital for a network operator, a service
provider or a corporate network infrastructure in order to keep the network operation stable,
smooth and safe. Monitoring provides valuable real time and historical information to understand
the network usage trends and dynamics and thus detect misbehaviours and attacks. The
vulnerabilities introduced by this open world: Critical infrastructures are more than ever open to
the Internet, the dematerialization of corporate IT and the success of cloud services are pushing
towards proactive mechanisms for detecting and preventing anomalies. In this context, Deep
Packet Inspection (DPI) is considered as a catalyser in the shift towards advanced monitoring. DPI
is the process of capturing network traffic, analysing and inspecting it closely to determine
accurately what is really happening in the network. In this presentation, we will present an
events-based network monitoring solution part of MMT tool that inspects network traffic against
a set of security properties denoting both security rules and attacks. In the context of DIAMONDS,
this solution has been applied to an industrial case study provided by Thales Group that consists
of a set of QoS-aware ad-hoc radio communication protocols. It will be adapted to test secure
interoperation in the context of INTER-TRUST project.
Network Monitoring for
Security Checking
Wissam Mallouli Montimage

SASSI workshop September 20th 2013
wissam.mallouli@montimage.com
What is network monitoring
Process of observing or inspecting the Can be used to
network at different points Understand the behavior of the
network
With the objective of Network planning & resource
optimization
Drawing operation baselines Detect faults and abnormal operation
Produce reports Network security (Intrusion & Attack
Notify on abnormal operation Detection)
Performance, quality & SLA
Provide input to network monitoring
management CRM, Marketing
Sit above traffic measurements

Gather traffic measures and
performance/security indicators
Analyze and correlate the
measures in order to make a
diagnosis
2
Network monitoring: Basics
3
Need for network monitoring
Diagnose & react
Remote User
Typical problem
Remote user arrives at
Regional Offices
regional office and
experiences slow or no
response from corporate web
server
Where to begin?
Where is the problem?
What is the problem?
What is the solution? WWW Servers
Without proper network

monitoring, these questions are
difficult to answer
Corp Network
4
Network monitoring: Components
Interface for real-time monitoring.
Store collected data in a database for post
Monitoring Monitoring
application
analysis (history, reporting ). Application or
DataBase
Gathers, aggregates and correlates

collected information (events)
Security
analysis unit
Notify remote events Analysis Unit
This performs the measurements

Collects the required information for Measurement Measurement Measurement
Measuremen
t unit
analysis. Unit Unit Unit
This is where the measurements will be

performed.
Observation Observation Observation
Observation The more observation points we have, the Point Point Point
points more accurate data we get.
7
Deep Packet Inspection
Based Monitoring
What is DPI
Application classification
Traffic attributes extraction
What is DPI
Technology consisting of digging deep into the packet
header and payload to inspect encapsulated
content
Content may be spread over many packets
Packet Header Layers Packet Payload / Application Layers
L2 L3 L4 L5 L7
Email (SMTP, POP3, IMAP)
Internet Transport
Web (HTTP/S)
Ethernet Protocol Layer
Instant Messaging (IM)
(IP) (TCP/UDP)
Peer-to-Peer (P2P) Applications
Deep Packet
Inspection
11
Why to DPI
Network Visibility
Understand how bandwidth is utilized
What is the application mix
Who is using what, where and when?
Traffic Management (Application Control)
Block undesired traffic (spam, worms, etc.)
Prioritize and shape traffic (limit P2P, QoS, QoE)
Advanced policy enforcement
Zero Facebook, OTT services, per application policy rules
Security
Understand network attacks
Core component in next generation firewalls
Etc.
12
Inside DPI
Group packets belonging to the
same session
Events & Attributes extraction Application classification
Detect application type (Skype,
Bittorrent, etc.) or application
family
Event analysis and parsing Considered as the core of DPI
DPI Engine
Protocol decoding and attribute

extraction
Parse the packet structure (this
Classification depends on the protocol &
application)
Protocol
Get protocol attributes (IP @, port
Sessionizer numbers, )
models
Get session attributes
Events and attributes may involve
Packet different packets
Processing Attached file of an email
13
From classification to attributes and
events extraction
Application classification is a first step towards accurate
traffic information extraction
How can we get the HTTP method (Get, Post, etc.) if we dont
know the type of the traffic
When the application type is known decoding becomes easy
What are traffic attributes?
Meta-attributes: timestamp, data source etc.
Protocol field derived from the packet data: IP@, attachment
size, encoding type, etc.
Flow parameter: packet mean size, inter-arrival delay, packets
lost, reordered, etc.
The application class can be considered as a traffic attribute
14
Attribute extraction with DPI
With the extraction capability, DPI can provide
input for security analysis
Imagine the network as a database and DPI as

an engine to extract data from this database!
15
Network as a Database
Select flow_id Where (application = email

AND attachment is executable)
Select user_id, perceived_quality Where

(application = Video AND protocol = RTP
AND perceived_quality < 1)
16
Security Monitoring with DPI
Using MMT
Abstract description
Challenges
Security properties
The HBGary Hack
HBGary - experts in computer security
computer forensics and malware analysis tools to enable the detection,
isolation, and analysis of worms, viruses, and trojans
implementing intrusion detection systems and secure networking
performs vulnerability assessment and penetration testing of systems and
software
rootkit.com is a respected resource for discussion and analysis of rootkits
CEO Aaron Barr wanted to unmask Anonymous
Those responsible for co-ordinating the group's actions, including the denial-
of-service attacks that hit MasterCard, Visa, and others late last year
Anonymous response
HBGary's servers were broken into, its e-mails pillaged and published to the
world, its data destroyed, and its website defaced. As an added bonus, a
second site owned and operated by Greg Hoglund, owner of HBGary, was
taken offline and the user registration database published
(source) http://arstechnica.com/tech-policy/news/2011/02/anonymous-speaks-the-inside-story-of-the-hbgary-hack.ars/1
18
The HBGary Hack
Attack Method Attacked System Vulnerability Lost Assets
1 SQL Injection CMS on HBGary CMS with missing usernames, e-mail

Federal's website, validity check of SQL @ & password
hbgaryfederal.com Parameters hashes
2 Password cracking Password hashes from 1 Hashes without salt, clear text passwords
using rainbow tables weak passwords
3 Unauthorized use of E-mail, Twitter accounts, Password double use Email accounts of
passwords from 2 and LinkedIn accounts of HBGary officials
HBGarry officials
4 Unauthorized use of Machine running Password double use Non-superuser account
passwords from 2 support.hbgary.com of HBGary official
5 Privilege escalation Machine running Privilege escalation Full access to HBGary's

support.hbgary.com vulnerability, system not system, gigabytes of
up to date backups and research
data
6 Social engineering Machine running Integrity of rootkit.com
rootkit.com
19
The HBGary Hack:
Where can security monitoring help?
Attack Method Attacked System Vulnerability Lost Assets
1 SQL Injection CMS on HBGary CMS with missing usernames, e-mail

Federal's website, validity check of SQL @ & password
hbgaryfederal.com Parameters hashes
2 Password cracking Password hashes from 1 Hashes without salt, clear text passwords
using rainbow tables weak passwords
SQL injection: In top ten most known vulnerabilities
3 Unauthorized use of E-mail, Twitter accounts, Password double use Email accounts of
(Top ten vulnerabilities
passwords from 2 are responsible
and LinkedIn accounts of for 60% of software
HBGarybugs!)
officials
HBGarry officials
4 Unauthorized use of Machine running Password double use Non-superuser account
passwords from 2 support.hbgary.com of HBGary official
Multi-lines security system: Is it possible to detect security attacks (SQL
5 injection in this case)
Privilege escalation by running
Machine inspecting thePrivilege
data encoded
escalation in inbound
Full accesstraffic?
to HBGary's
support.hbgary.com vulnerability, system not system, gigabytes of
up to date backups and research
data
6 Social engineering Machine running Integrity of rootkit.com
rootkit.com
20
Security monitoring with DPI:
The concept:
Detect the occurrence of events on the network
Input provided by DPI
Event can be: packet arrival, HTTP POST request, etc.
Inspect and analyze the succession of events to detect
properties
Property: Succession of events that are linked with time and
logical constraints
If we detect event A, then we MUST detect event B after 10 seconds
The idea:
Monitor the network looking for the occurrence of
properties.
21
Security monitoring using DPI:
Example: SQL injection
www.abcd.com/page?name=Select * Where 1
The events to be detected
HTTP GET request
URL parameter contains SQL statement
The property
It is not allowed to have a URL parameter containing SQL
statement in an HTTP GET request
If the property is detected on the network then most
probably there is an attack attempt!
Nice Theory! But very challenging
22
Security monitoring using DPI
Challenges
The number of events that can occur on a network is
huge!
Solution: Use DPI for the events extraction
Group events/attributes by application and add new ones
when needed
The expressivity of the properties (need to combine
time and logical constraints)
Complex analysis and processing especially in high
bandwidth links
Optimization techniques, multi-core implementations, smart
traffic filtering
23
Properties Expressivity
Considering security monitoring, properties can be used to express:
A Security rule describes the expected behavior of the application
or protocol under-test.
The non-respect of the Security property indicates an abnormal behavior.
Set of properties specifying constraints on the message exchange
i.e. the access to a specific service must always be preceded by an authentication phase
An Attack describes a malicious behavior whether it is an attack
model, a vulnerability or a misbehavior.
The respect of the Security property indicates the detection of an
abnormal behavior that might indicate the occurrence of an attack.
Set of properties referring to a vulnerability or to an attack
A big number of requests from the same user in a limited period can be considered as a
behavioral attack
24
A security property is composed of 2 parts:
A Context
A condition to verify
The Context and Condition of a property are

composed of:
Simple events
Conditions on attributes (IP @ equal to 1.2.3.4)
Complex events linked by
Logical operators (AND/OR/NOT)
Chronological operator (AFTER/BEFORE)
25
A security property is composed of 2 parts:
A Context
A condition to verify
The Context and condition of a property are

Ifcomposed of:
an HTTP Response is received (this is the context)
Simple events
Then an HTTP
Conditions on attributes
request should(IPhave
@ equal
beento 1.2.3.4)
received before
Complex events linked by
(condition to verify)
Logical operators (AND/OR/NOT)
Chronological operator (AFTER/BEFORE)
26
https://github.com/montimage/MMT_Security
Montimage Monitoring Tool (MMT)

MMT Operator
Monitoring & reporting
Web technology User defined
Management of multiple probes Reports and views
Traffic View
Config
DB Plugins
MMT Probe
Add analysis
Analysis modules
modules
Quality Security &
Traffic
Functional HW/SW Probe
Monitoring Monitoring Analysis Can be installed
on dedicated HW
MMT Extract
Deep Packet Inspection functionalities Software library
Traffic classification (600+ protocols) (SDK)
Protocol decoding & attributes extraction Can be integrated
Extraction of metrics in 3rd party SW
Protocol Plugins Add plugins 27

Thales Case Study: Ad-hoc
waveform networking protocols
Technical challenges
Automatic network : no initial planning
Network continuity whatever are the stations in the network
on the move automatic network re-organization and operation
End-to-end heterogeneous user services transmission : voice,
messages
Decentralized mesh network. no Base Station
Nodes collaborate to ensure connectivity and participate in the
routing task
Main scope of radio protocol case study: vulnerability analysis

based on over-the-air info exchanged at physical layer
Frames analysis, data alteration, replay, denial of service,
tampering and misuse, and combination of the threats
29
Network threats and vulnerabilities
Detection of potential attacks
Link spoofing, Data alteration, Flooding attack, Blackhole
attack, Denial of service, Replay
Data alteration
MAC PDU MAC PDU
header header
Node A Node B
intrusive
30
Testing architecture
SMARTESTING MONTIMAGE
Security test generation model and test purposes Specification of 19 security properties
specification Client /Server architecture
Generation of test scenarios denoting attacks using Certify- Notification of exchanged PDUs
it Parsing and extraction of relevant information
FSCOM Online analysis of captured PDUs and detection of attacks
TTCN3 test cases specification occurrences
Test execution using TT-Workbench
31
Security Rules Specification : Example
Security requirement: Every node must periodically send a notification
message that includes the list of its neighbors on its allocated service slot.
Attack scenario 1: A malicious node sends a message on a non-allocated
service slot. This provokes slot reallocation. If done repeatedly, it provokes
denial of service.
Specified security properties
If one node receives two successive MSG_SPHY_DATA_IND messages
from the same source, then these two messages must to be separated
by 50 slots (Prop 1).
If one node receives two MSG_SPHY_DATA_IND messages from different
sources, then these two messages must have two different slot ids (Prop
2).
32
Analysis results: Attack Scenario 1
33
Inter-trust overall objectives
Develop a dynamic and scalable framework

to support trustworthy services in heterogeneous networks and
devices
based on the enforcement of interoperable and changing security
policies
Addressing the needs of developers, integrators and operators
to develop and operate systems in a secure trusted manner
dictated by negotiated security policies through dynamic security SLAs
Separate the security concerns from the functional
requirements => AOP 35
Project Overall Objectives
Validation
Validate the architecture, techniques and tools
developed using two completely different case
studies with complex, high-demanding critical
services
V2X communications
E-Voting
Perfectly illustrate the importance and cross-
domain applicability of the INTER-TRUST's results
36
The Inter-trust global architecture
15-16/11/2012 Inter-Trust KoM 37

Modelling security
policies

Negotiating
security policies

Dynamically
generates aspects
to be woven
Based on the
negotiated policy

Interprets the
negotiated policy
aspects woven
into the application

Injects code
Captures application
events
Detects non compliance
of security requirements

Performs protection and

mitigation strategies

Stand-alone monitoring
and testing tools

V2x case study
Dynamic Route Planning
Contextual Speed Advisory
ITS security objectives (ETSI)

Messages Integrity,
confidentiality, authenticity
Access control
Privacy
Etc.
45
THANK YOU
Wissam Mallouli
wissam.mallouli@montimage.com
46
Some of the material used in these slides come
from the Internet
Thanks to them
47

SASSI13 Proceedings

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SASSI13 Proceedings

Uploaded by

Copyright:

Available Formats

Security Assessment for

Systems, Services and

Germany's National Cyber Security Strategy

Alliance for Cyber Security

De-facto standard for information

Information hub among agencies in charge

Ensure rapid response to serious incidents

Phishing-Mails im Namen von Paypal an Eset-Kunden

Telekom: Router warnt bei Bot-Befall

Quelle: BSI Schwachstellenampel

41% angreifbar mit Microsoft XML Core Services ( 3 CVEs)

CVE - Common Vulnerabilities and Exposures (CVE)

Quelle: Symantec, BSI/FKIE

Progression der mobilen Schdlinge

Quellen: Gdata, BFK, Symantec, BSI

Cooperation platform for

Central information pool Exchange of

Institutions of special public interest attack vectors,

Voluntarily registered participants

Main goals of Cyber Alliance:

At the moment there is no central and dedicated situation

To fulfill this requirement the BSI is creating a situation

Why we create a overview?

up-to-date dynamic national focus online

proven continuous objective central

Who is the Target group?

But also institutions that have direct influence on

Development and implementation will be step by step

Presentation on multiple abstractionon levels

Target group Information

Areas of future development

First draft of BSI's Cyber Security Situation Overview will

Federal Office for

Phone: +49 228 99

Stig Torsbakken Nets SIRT

Risk management in Nets is organized around the 3 lines of defence model

Operational Control, policy and Audit

Group Management (GM)

Nets SIRT security incidents 2013

NemID Suspicious user

Non-target specific malware

We know that malware is part of our internal network

73 (35%) exposures (i.e. malware downloads, unverified infections)

Nets SIRT security incidents 2013

DDoS attack April 10

Gruppe tager ansvar

En gruppe, der via Twitter har taget

Norway target of APT1

From the Mandiant APT1 report

Security testing in Nets

In numbers Nets Norway

Scanningless scanning: 365

Risk analysis of new services

How is this possible?

25-30 security officers in Nets in total

Heavy use of consultants and 3rd party vendors

Security testing in Nets

Vulnerability scanning VLAN

Quarterly external scan

Quarterly internal scan VLAN 157 VLAN 2

Security testing in Nets

Software vulnerabilities, non-updated software

Missing authentication/source verification

Information leakage (verbose error messages etc.)

(Note: we do not do reverse Engineering, but we use some RE methods)