Professional Documents
Culture Documents
01 SWSV Overview PDF
01 SWSV Overview PDF
Introduction
Overview of V&V techniques
3
Stereotypes
“Testing is destructive.”
“Testing is just pushing buttons
and supplying values randomly.”
“Testing is boring.”
4
V&V (and testing) in reality
V&V (and testing) is creative!
How is this working? How can I prove it works?
How should it work? How can it fail?
V&V
6
Useful resources (download now!)
IEEE standards
o 24765-2010 Systems and SW engineering – Vocabulary
o 29148-2011 Requirements engineering
o 29119 Software testing
• Part 1 Concepts and definitions
• Part 2 Test processes
• Part 3 Test documentation
International Software Testing Qualifications Board (ISTQB)
o Foundation Level Syllabus (2011)
o Glossary of Testing Terms
Hungarian Testing Board (HTB)
o Glossary / Kifejezésgyűjtemény (magyar fordítás)
7
MOTIVATION
8
Different kinds of faults
9
Software is the cause of problems
„Defibtech issues a worldwide recall of two of its defibrillator
products due to faulty self-test software that may clear a
previously detected low battery condition.” (February 2007)
10
How many bugs do we have to expect?
Source: K-R. Hase: „Open Proof in Railway Safety Software”, FORMS/FORMAT Conference, December 2-3, 2010, Braunschweig, Germany
11
Distribution and cost of bugs
12
V&V: Verification and Validation
Verification Validation
„Am I building the system right?” „Am I building the right system?”
Check consistency of Check the result of the
development phases development
Conformance of designs/models Conformance of the finished
and their specification system and the user requirements
13
OVERVIEW OF V&V TECHNIQUES
14
Learning outcomes
15
Typical steps in development lifecycle
Requirement
analysis System Schedule, sequencing
System engineer depends on lifecycle model!
specification
Architecture
design
Architect
Module
design
Module
Developer,
implementation coder
System
integration
Test
System
delivery
engineer
Operation,
maintenance
16
Requirement analysis
Requirement Task V&V criteria V&V technique
analysis
- Checklists
System Defining functions, - Risks
specification actors, use cases - Failure mode and
- Criticality
effects analysis
Architecture
design
Module
design
Module
implementation
System
integration
System
delivery
Operation,
maintenance
17
System specification
Requirement
Task V&V criteria V&V technique
analysis - Completeness
Defining functional - Reviews
System - Unambiguity
specification and non-functional - Static analysis
requirements - Verifiability
- Simulation
Architecture - Feasibility
design
Module
design
Analysis
Module Reality
implementation
Design Implementation
System space
integration Modeling 18 Implementation
- structuring
System Designing space
delivery - abstraction
- decomposition
Operation,
maintenance
Architecture design
abstraction
Requirement Analysis
analysis Design space
Mapping
System Structuring
(automated)
specification design space
and mapping
Architecture
design
Implementation space formality
Module
design
Module
implementation Task V&V criteria V&V technique
System - Decomposing - Function coverage - Static analysis
integration modules - Conformance of - Simulation
System - HW-SW co-design interfaces - Performance,
delivery - Designing - Non-functional dependability,
Operation,
communication properties security analysis
maintenance
Module design (detailed design)
Requirement
analysis
System modellje
Rendszer model Requirement spec.
System Formal y n
specification Automatikus
Automated model
verification modellellen
checking
Architecture
design
OK Counter-
Module example
design
Module
implementation
Task V&V criteria V&V technique
System
integration - Designing detailed - Correctness of - Static analysis
behavior critical internal - Simulation
System
delivery (data structures, algorithms and - Formal verification
algorithms) protocols - Rapid prototyping
Operation,
maintenance
20
Module implementation
Requirement
analysis
System
specification
Operation,
maintenance
System integration
Requirement
Task V&V criteria V&V technique
analysis - Conformance of
System
- Integrating modules integrated
- Integration testing
specification - Integrating SW with behavior (incremental)
Architecture
HW - Verifying
design communication
Module
design
Module
implementation
System
integration
System
delivery
Operation,
maintenance
22
System delivery and deployment
Requirement
analysis
System
specification
Architecture
design Task V&V criteria V&V technique
Operation,
maintenance
23
Operation and maintenance
Requirement
analysis
Module
design
Module
implementation
System
integration
Mini-lifecycle
System for each
delivery
modification
Operation,
maintenance
24
V&V TECHNIQUES IN
CRITICAL SYSTEMS
25
Learning outcomes
26
Safety-critical systems
27
Certification
28
Safety concepts
Safety function
o Intended to achieve or maintain a safe state
Safety integrity
o Probability of a safety-related system satisfactorily
performing the required safety functions under all
stated conditions and within a stated period of time
Safety Integrity Level (SIL)
o Based on risk analysis
o Tolerable Hazard Rate (THR)
29
Basics of determining SIL
Risk analysis -> THR -> SIL
Frequency of
hazardous event System Software
safety safety
integrity integrity
level level
4 4
Risk THR SIL 3 3
2 2
Consequence of
1 1
hazardous event
0 0
31
Example: Process (V model)
Operation,
maintenance
Well-defined
Architecture Integration test System
design design integration phases
Module
implementation
32
Example: Techniques (EN 50128)
o M: Mandatory
o HR: Highly recommended (rationale behind not using
it should be detailed and agreed with the assessor)
o R: Recommended
o ---: No recommendation for or against being used
o NR: Not recommended
33
Example: Document structure (EN50128)
System Development Phase Software Maintenance Phase
System Requirements Specification Software Maintenance Records
System Safety Requirements Specification Software Change Records
Software Planning Phase
System Architecture Description
Software Development Plan
System Safety Plan Software Assessment Phase
Software Quality Assurance Plan
Software Configuration Management Plan Software Assessment Report
Software Verification Plan
Software Integration Test Plan Software Requirements Spec. Phase
Software/hardware Integration Test Plan Software Requirements Specification Software Validation Phase
Software Validation Plan Software Requirements Test Specification Software Validation Report
Software Maintenance Plan Software Requirements Verification Report
34
Example: Responsibilities (EN 50128)
Organization Person
SIL 0:
ASS
DES, VER, VAL
SIL 1 or 2:
ASS
DES VER, VAL
SIL 3 or 4: MGR
ASS DES: Designer (analyst,
DES VER, VAL architect, coder, unit
tester)
or: VER: Verifier
MGR
VAL: Validator
ASS ASS: Assessor
DES VER VAL MAN: Project manager
35
BACKGROUND MATERIAL
(For reference only, recommended to come back at the end of the course
to see how many techniques are familiar)
36
IEC 61508 V&V methods
37
IEC 61508 V&V methods – Testing
38
IEC 61508 V&V methods – Static analysis
39