Professional Documents
Culture Documents
Pantel Slides PDF
Pantel Slides PDF
Marc Pantel
1/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Plan
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
2/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Plan
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
3/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Main purpose: Safety critical systems Main approach: formal specication and verication Problems: expressiveness, decidability, completeness, consistency
Marc Pantel
4/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Needs:
methods and tools to ease their development algebraic and logic theoretical fondations proof of transformation and verication correctness links with certication/qualication
Marc Pantel Certication and qualication concerns 5/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Needs:
methods and tools to ease their development algebraic and logic theoretical fondations proof of transformation and verication correctness links with certication/qualication
Marc Pantel Certication and qualication concerns 5/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
6/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Plan
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
8/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
A bit of wording
Requirement: What the end user expects from a system
High level: focus on end users needs (user provided)
Translate proled UML to RTSJ; C to PowerPC Generate test inputs and expected outputs from a system specication Prove the absence of runtime errors Compute a precise estimation of WCET Schedule activities
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
A bit of wording
Requirement: What the end user expects from a system
High level: focus on end users needs (user provided)
Translate proled UML to RTSJ; C to PowerPC Generate test inputs and expected outputs from a system specication Prove the absence of runtime errors Compute a precise estimation of WCET Schedule activities
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
A bit of wording
Requirement: What the end user expects from a system
High level: focus on end users needs (user provided)
Translate proled UML to RTSJ; C to PowerPC Generate test inputs and expected outputs from a system specication Prove the absence of runtime errors Compute a precise estimation of WCET Schedule activities
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
A bit of wording II
Verication: System fullls its requirements explicit specication Validation: System fullls its requirements implicit human needs Certication: System (and its development) follows standards (DO-178, IEC-61508, ISO-26262, . . . ) Qualication: Tools for system development follows standards Certication and qualication: System context related
Marc Pantel
10/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
A bit of wording II
Verication: System fullls its requirements explicit specication Validation: System fullls its requirements implicit human needs Certication: System (and its development) follows standards (DO-178, IEC-61508, ISO-26262, . . . ) Qualication: Tools for system development follows standards Certication and qualication: System context related
Marc Pantel
10/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
A bit of wording II
Verication: System fullls its requirements explicit specication Validation: System fullls its requirements implicit human needs Certication: System (and its development) follows standards (DO-178, IEC-61508, ISO-26262, . . . ) Qualication: Tools for system development follows standards Certication and qualication: System context related
Marc Pantel
10/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Development tools: Tools whose output is part of airborne software and thus can introduce errors (same constraints as the developed system). Verication tools: Tools that cannot introduce errors, but may fail to detect them (much softer constraints: black box V & V). No proof of error absence category
Marc Pantel
12/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Criteria 3: A tool that, within the scope of its intended use, could fail to detect an error (TQL-5 for DAL A). Still no proof of error absence category (might be TQL-2 for DAL A).
Marc Pantel Certication and qualication concerns 13/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Criteria 3: A tool that, within the scope of its intended use, could fail to detect an error (TQL-5 for DAL A). Still no proof of error absence category (might be TQL-2 for DAL A).
Marc Pantel Certication and qualication concerns 13/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Criteria 3: A tool that, within the scope of its intended use, could fail to detect an error (TQL-5 for DAL A). Still no proof of error absence category (might be TQL-2 for DAL A).
Marc Pantel Certication and qualication concerns 13/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Criteria 3: A tool that, within the scope of its intended use, could fail to detect an error (TQL-5 for DAL A). Still no proof of error absence category (might be TQL-2 for DAL A).
Marc Pantel Certication and qualication concerns 13/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Common documents
Marc Pantel
14/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
15/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Some comments
Standards were designed for systems not tools: Adaptation required MCDC not mandatory for tools, but similar arguments might be required Traceability of all artefacts in the development, relate requirements, design and implementation choices Purpose is to provide condence Both cooperative and coercive approach Any verication technology can be used, from proofreading to automatic proof if condence is given Choose the strategy and technologies that will best reduce risks
Marc Pantel Certication and qualication concerns 16/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Some comments
Standards were designed for systems not tools: Adaptation required MCDC not mandatory for tools, but similar arguments might be required Traceability of all artefacts in the development, relate requirements, design and implementation choices Purpose is to provide condence Both cooperative and coercive approach Any verication technology can be used, from proofreading to automatic proof if condence is given Choose the strategy and technologies that will best reduce risks
Marc Pantel Certication and qualication concerns 16/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Some comments II
Must be applied as soon as possible (cost reduction) Small is beautiful (simplicity is the key) Certication authorities need to understand the technologies Cross-experiments are mandatory (classical w.r.t. formal methods)
Marc Pantel
17/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Some comments II
Must be applied as soon as possible (cost reduction) Small is beautiful (simplicity is the key) Certication authorities need to understand the technologies Cross-experiments are mandatory (classical w.r.t. formal methods)
Marc Pantel
17/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Plan
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
18/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Classical technologies:
Document independant proofreading (requirements, specication, implementation) Test
Unit, Integration, Functional, Deployment level Requirement based test coverage Source code test coverage Structural coverage, Decision coverage, Multiple Condition Decision Coverage (MCDC)
Marc Pantel Certication and qualication concerns 19/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Transformation case
Transformation specication: Structural/Behavioral Proof of transformation correctness Links with certication/qualication
Marc Pantel
20/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Tool development, verication and qualication plans User requirements Tool requirements (human proofreading) Test plan (requirements based coverage, code coverage verication) Implementation and test application
Marc Pantel
21/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
23/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Integration
Elementary Tool
XML Inputs
Elementary Tool
Java Front-end
Ocaml Wrapper
Ocaml
Automatic Extraction
Code
Logs
XML Outputs
Elementary Tool
Marc Pantel
24/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
26/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Open questions ?
What are: User requirement for a transformation/verication ? Tool requirement for a transformation/verication ? Formal specication for a transformation/verication ? Test coverage for a transformation/verication ? Test oracle for a transformation/verication ? Qualication constraint for transformation/verication languages ? Best strategy for tool verication (once vs at each use) ?
Marc Pantel
27/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
GeneAuto feedbacks
From the certication perspective: Very good but...
Still some work on qualication of the proof assistant tools
Proof verier Program extractor
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
29/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
29/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Specication verication: Prove the semantics equivalence between source and target in a trace link Soon to be started PhD thesis at Airbus
Marc Pantel
30/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Early feedbacks
Separation of concerns:
Industrial partners: Specication, Implementation, Implementation verication (mainly syntactic) Academic partners: Specication verication (semantics)
Very good subcontracting capabilities Almost no technology constraints on the industrial partner (classical technologies) Good scalability Easy to analyse syntactic error reports Enables to modify generated code and links Parallel work between syntactic and semantics concerns
Marc Pantel Certication and qualication concerns 31/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Work in progress
Positive rst experiments on simple use cases from GeneAuto But requires some grayboxing (expose parts of the internals)
Flattening of statecharts Either very complex specication (doing the attening) Or express the xpoint nature of implementation (in the specication)
Require full scale experiments Require exchange with certication authorities Require qualied syntactic verication tool (OCL-like, but simpler) Require explicit relations between syntactic and semantics work Require explicit description of semantics in metamodels
Marc Pantel Certication and qualication concerns 32/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Plan
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
33/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Required activities
Specify user requirements Specify tool architecture (elementary tools and their assembly) Specify tool level requirements (elementary tools and their assembly) Specify functional test cases and results Choose verication strategy:
Tool verication or Result verication Integration and unit tests (eventually with test generators and oracles) Proof reading of tool source or test results Formal verication of the verication tool itself (i.e. Coq in Coq, Compcert in Coq, . . . )
Marc Pantel Certication and qualication concerns 35/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Abstraction kind
Translate to non standard semantics Compute recursive equations Compute xpoint of equations
Fixpoint algorithm Abstract domains and operators Widening, narrowing
Check that properties are satised on the abstract values Produce user friendly feedback (related to product and its standard semantics)
Marc Pantel
36/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Deductive kind
Produce proof obligations (weakest precondition, verication condition, . . . ) Check the satisfaction of proof obligations
Proof term rewriting to simpler language Split to different sub-languages (pure logic, arithmetic, . . . ) Apply heuristics to produce a proof term Check the correctness of the proof term Produce failure feedback or proof certicate (related to product and its standard semantics)
Marc Pantel
37/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
User-friendly feedback: Code generation based on trace Marc Pantel Certication and qualication concerns links
38/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
User-friendly feedback: Code generation based on trace Marc Pantel Certication and qualication concerns links
38/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
User-friendly feedback: Code generation based on trace Marc Pantel Certication and qualication concerns links
38/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
User-friendly feedback: Code generation based on trace Marc Pantel Certication and qualication concerns links
38/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Verication of the properties of the abstract domains (join, meet, operators, , widening, narrowing, monotony, . . . )
Proof reading Automated test generation with oracles Formal specication and proof
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Verication of the properties of the abstract domains (join, meet, operators, , widening, narrowing, monotony, . . . )
Proof reading Automated test generation with oracles Formal specication and proof
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Verication of the properties of the abstract domains (join, meet, operators, , widening, narrowing, monotony, . . . )
Proof reading Automated test generation with oracles Formal specication and proof
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Verication of the properties of the abstract domains (join, meet, operators, , widening, narrowing, monotony, . . . )
Proof reading Automated test generation with oracles Formal specication and proof
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Marc Pantel
40/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Mainly scientic work and a lot of publications Brings condence but paperwork is not enough Mechanized is better but still not enough Functional user level tests still mandatory currently Mixed system verication experiments (both tests and static analysis) Reverse analysis of existing systems
Marc Pantel
41/42
Safe MDE concerns Certication and Qualication Application to Code generation tools Application to Static analysis tools
Synthesis
Technical exchange with certication authorities mandatory Cross experiments and reverse engineering experiments mandatory Verication strategy must be designed early to choose the right architecture and trace information Semi-formal (even formal) requirements must be written as soon as possible
Marc Pantel
42/42