Systems

Engineering
Management
Selected Aspects

Lecture Relevant Reading

qTextbook: Chapters 18 and 19
qTextbook Sections: 3.10, 4.8, 5.8, and 5.9.

Agenda
Recap of last session
Systems Engineering Management
Verification and Validation
Reviews and Audits

Recap of last lecture

We considered:

Systems Analysis
Optimisation and modelling
Trade studies

Systems Engineering Management

Why?
So we can deliver:

The right system with required quality

At the right time and at the right location

Within budget ideally with minimum expenditure

Management
Planning
Organising
Implementing
Controlling
Leading

SYSTEM ENGINEERING MANAGEMENT PLAN (SEMP)

SEMP is a key management document for Systems

Engineering

Brings together all of the Systems Engineering activities

Generally deals with 3 aspects (per MILSTD-499A)

Technical Program Planning and Control

SE Process

Engineering Specialty Integration

SYSTEMS ENGINEERING MANAGEMENT PLAN

PROGRAM MANAGEMENT PLAN

SYSTEMS
ENGINEERING
MANAGEMENT
PLAN

TECHNICAL PROGRAM
PLANNING AND CONTROL
ORGANISATION
PROG MGMT STRUCTURE
ENGINEERING STRUCTURE
ACCOUNTABILITY
INTERNAL/EXTERNAL
INTERFACES
PLANNING ACTIVITIES
SEMS/SEDS
SOW/CDRLs
REVIEWS
ES SCHED/MILESTONE
SPEC/ICD TREE
SYSTEM TEST
TECHNICAL CONTROL
ACTIVITIES
TPM
ACCOUNTABILITY
DESIGN COMPLIANCE
CONFIG BASELINE
SCHEDULE
DESIGN-TO-COST
DESIGN FOR
ENVIRONMENT

SYSTEM ENGINEERING PROCESS

SYSTEM REQUIREMENTS
DEVELOPMENT
ALLOCATION
VERIFICATION
ENGINEERING ANALYSIS
TRADE STUDIES
DESIGN SYNTHESIS
PROBLEM RESOLUTION

ENGINEERING SPECIALTY
INTEGRATION
POLICY/GUIDELINES
SPECIALITY GROUP ORG
INTERNAL INTERFACES
FUNCTIONAL ACCOUNTABILITY
WORKING INTERFACES
TASK/PROCEDURES
MILESTONES
INTEGRATED TEST PLAN
HUMAN FACTORS PROGRAM PLAN

INTERFACE MANAGEMENT
INTERFACE CONTROL
DOCUMENTATION
INTERFACE CONTROL
WORKING GROUP

SAFETY PROG PLAN

ILS PLANS
RELIABILITY/MAINTAINABILITY PLAN
OTHER PLANS

Work Breakdown Structure

A product or task based hierarchy of work
required to complete the project
Forms the basis of subsequent planning,
budgeting and control of project
Many possible WBSs for a project communication essential to get agreement on
preferred one

DEVELOPMENT OF WBS
The whole does more than the sum of the parts

Product Breakdown
Structure (PBS)

Subsystem A

Sub
system
B
System
Components
(Subsystems)
held together
by glue
(Integration)

Shows the
components which
form the
system

Subsystem C

System

Subsystem D

The individual
system components

Work Breakdown
Structure (WBS)
All work
components
necessary to
produce a
complete
system

System

D
Management

Source: NASA Systems Engineering Handbook

Work to produce
the individual system
components

Systems
Engineering

I&V

ILS

Work to integrate the

components into a system

The whole takes more work than the sum of the parts

TECHNICAL CONTROL
INTERFACE ENGINEERING
TPM (TECHNICAL PERFORMANCE PLAN)
CM (CONFIGURATION MANAGEMENT)
REVIEW AND AUDITS

INTERFACE ENGINEERING

INTERFACE ENGINEERING MANAGEMENT

Requirements Analysis
SYSTEM

External Interfaces

and

System Design / Analysis

Internal Interfaces

INTERFACE - The boundary between two

elements of a system, or between two systems
Ensure systems are functionally and physically
compatible through interface control

INTERFACE MANAGEMENT CONCEPT

Design interfaces to optimize overall design of

system
Keep interfaces simple simpler integration
Define : format, protocols, connections
(physical, electrical etc.)
Control interfaces definition and changes
Verification of interfaces - testing, modeling
and simulation, fit checks

TECHNICAL PERFORMANCE
MEASUREMENT

WHAT IS IT?
Set of activities and measures that provide
insight into the system technical solution.
Carried out throughout the life cycle.
Measures are established early and then tracked
throughout.

MOEs, MOPs and TPMs

Measures of Effectiveness (MOE)

Measures related to operational/mission success

Established from the client/user perspective
Focus on critical Mission performance needs,
independent of any particular solution.

Measures of Performance (MOP)

Measures that relate to a system's operation

Derived from the MOEs
From suppliers viewpoint

MOEs, MOPs and TPMs

Technical Performance Measures

Measures indicating how well a system is satisfying

its performance requirements
Derived from MOPs
Focus on critical technical parameters

Examples:
Weight, reliability, power required, human factors,
throughput, complexity, availability, accuracy, range etc.

CONFIGURATION
MANAGEMENT

10

CONFIGURATION MANAGEMENT
DEFINITION
CM is the process that identifies the functional and physical characteristics
of an item during its life cycle, controls changes to those characteristics,
and provides information on the status of change controls.

IT INVOLVES:
Configuration identification
Configuration control
Configuration status accounting
Configuration audit

DEFINITIONS

Baseline :
A baseline is an established configuration definition from
which changes are tracked and documented.
Functional baseline:
This baseline defines the technical and user requirements and
allocates these to functional areas. It consists of type A
system/segment specification or similar.
Allocated baseline:
Consists of development specification and defines the
performance requirements of each configuration item (CI).
This baseline is the basis for detailed design during the
development process.

11

DEFINITIONS

Product baseline:
This baseline is established by the detailed design
documentation for each CI. It is the basis for hardware
fabrication, software coding and CI delivery. This baseline,
in effect defines the form (physical characteristics), fit
(interfaces), function (correct operating characteristics) and
the performance and test requirements for acceptance.

=> FORMAL CHANGE

CONTROL START

BASELINE ESTABLISHED

TECHNICAL BASELINE EVOLUTION

Mission
Need
Statement

MRR

MDR

Functional
Baseline

System
Spec
Type A

SRR
Segment
Spec

Segment
Spec

Segment
Spec

Type A

Type A

Type A

Allocated
Baseline

Prime Item
Design To
Spec
Type B

Prime Item
Design To
Spec
Type B

Prime Item
Design To
Spec
Type B

Implementation aspects
of design complete

End Item
Design To
Spec
Type B

End Item
Design To
Spec
Type B

End Item
Design To
Spec
Type B

End Item
Build To
Spec
Type C

Design
Disclosure
Drawing

Major architecture aspects

of design complete

SFR
SDR

PDR

Product
Baseline
Realisation aspects of design complete
As-Built
Baseline
Fabrication and Test complete
As-Deployed
Baseline

Operational capability demonstrated

CDR

Diagram
Etc

Manuals

SAR

ORR

12

FORMAL CHANGE CONTROL

Managed and authorized e.g. by a change control

board (CCB)
Control affected through the change process e.g.
using Engineering Change Proposals (ECPs)
Performance Impacts
Cost Impacts

ECP

Schedule Impacts
Interface Impacts

CM DOCUMENTATION
CM plan
Change Board meeting minutes
Change control forms
ECP or equivalent documentation
Requests for deviation / waiver

13

Reviews and Audits,

V&V

REVIEWS AND AUDITS

14

Technical Reviews
u Used as a gate to ensure that one stage has completed

satisfactorily before going on to the next stage help

establish current project baseline.
u They enable stakeholder observation of the technical
progress and, if needed, approval to proceed
w
w
w

Help communicate approaches to design etc.

Help demonstrate that we can/cannot meet requirements
Help figure out risks, progress and status

u May be at the system or subsystem and lower level

Technical Reviews
u Must have clear objectives for each review
u Must enable assessment of progress against established

requirements
u Typically planned around key project milestones
u Indicate path to resolve any actions/issues resulting
from the review.

15

Typical Review

Entry Criteria

Review

Go/No go,
Baselines,
Actions

Success Criteria

Example Entry Criteria

u Evidence of successful predecessor review
u Agreed to review agenda and success criteria
u Relevant technical artifacts e.g.:
w
w
w
w

Requirements documentation including traceability matrix

Design documentation
Operations concept
Risk and hazard analyses, and so on

u Relevant plans such as safety, risk management,

specialty area plans, and so on.

16

Audits
u Often used to independently determine risk and

potential problems
Many times after a project is already in trouble

u Early audits provide opportunities for more effective

remedies to problems
u Audits compare actual performance to required
performance using performance very generally here

Two Major SE Audits

u

Functional configuration audit (FCA)

Pre-requisite to Physical Configuration Audit (PCA)

Ensures that item functions as required
Conducted on CIs
Ensure that CI documentation accurately reflects actual
functional characteristics

u Physical configuration audit (PCA)

w

Ensures that :
a) as built configuration identical to product configuration
identification
b) acceptance testing requirements adequate
c) any differences are reconciled

17

Verification and Validation

Lecture Relevant Reading

u Textbook Chapter 6
u Bahill and Henderson (2004), Requirements

Development, Verification and Validation Exhibited in

Famous Failures.
u Textbook: Sections 3.10, 4.8, 5.8, and Appendix B

18

Verification and Validation

u What is the difference?
w

Verification seeks to establish if we have built the system right focus on intermediate items e.g. design, code etc.
checks the system against specification
Validation seeks to establish if we have built the right system focus is on requirements and final system
checks that the system is what was needed

19

Verification Methods
u Testing
u Walkthroughs, Inspections
u Analogy
u Analysis
u Demonstration
u Other?

20

Verification Planning
u Verification Planning considers, inter alia:

Verification method for each requirement

Verification strategy - categories of verification etc.
Organisation responsible for verification planning and
execution
Disposal of verification results
Configuration management
Assets requirement for verification
Test and their dependencies
Schedule, etc.

Verification Planning (Cont)

u Various documents are used to capture the results of

planning, and subsequent testing. These include:

w
w
w
w
w
w

Verification master Plans

Test Plans
Test Procedures
Intergration Plans and Procedures
Test Reports
Others as appropriate.

21

Validation

Source: Larsen and Buede

22

Summary
u We have considered selected aspects of Systems

Engineering Management
u Remember that technical management supports overall
programme/project management
u We discussed some of the reviews and audits used in
SE

Total Systems Interven0on (TSI)

23

Lecture Relevant Reading

Flood RL and Jackson MC (1991), Total Systems Interven0on

Some Systems Approaches:

Systems Engineering
SoI Systems Methodology Checkland
Viable System Diagnosis Staord Beer
Systems Dynamics Jay Forrester
Strategic Assump0on Surfacing and Tes0ng
Mason and Mitro
Cri0cal Systems Heuris0cs Ulrich
Team Syntegrity Staord Beer

24

Some Issues:
Which methodology to use when?
Will one methodology be adequate to capture the
issues presented by complex situa0ons?
If not, then how do we decide which ones to use?
One answer: Total Systems Interven0on Jackson
and Flood.

Total Systems Interven0on

TSI is broader than hard or soI system
methodologies it allows us to gure out which
methodology suits our context.
It uses metaphors to iden0fy the class of problem
and the major issues.
It encompasses various system thinking
methodologies.

25

Principles of TSI
Complex systems are dicult or impossible to understand using
one model.
They should be inves0gated using a range of metaphors.
Organiza0onal issues and problems highlighted by the metaphors
can be linked to appropriate systems methodologies to guide
applica0on.
Dierent system metaphors and methodologies can be used in a
complementary way to address dierent aspects of the system.

TSI
Creativity

Choice

Implementation

Task

Highlight concerns,
issues, and
problems

Choose an appropriate
systems-based intervention
methodology/ies

Arrive at and
implement specific
change proposals

Tools

Systems metaphors The "system of systems

methodologies" and
knowledge of the strengths
and weaknesses of different
methodologies

Outcome

Dominant and
dependent
concerns, issues
and problems metaphors

Systems
methodologies
employed according to
the logic of TSI

Dominant and dependent

Highly relevant and
methodologies chosen for use coordinated change to
improve the situation

26

TSI Metaphors

Machine - closed systems.

Organic - open systems.
Neurocyberne0c (brain)- viable systems
Culture
Poli0cal
Prison

Poli0cal Metaphor
Looks at individuals and groups as compe00ve and
involved in the pursuit of power.
Three views:
unitary
pluralist
coercive.

The poli0cal metaphor focuses on issues of

interests, conict and power using the these
views.

27

Problem Par0cipant Classica0on

Interests

Unitary
Common objectives a
well integrated team.

Pluralist
Diverging group
interests with the
organisation as a
mutual focal point
loose coalition.

Coercive
Oppositional and
contradictory interests
rival forces

Conflict

Rare and transient.

Inherent, but may have

positive aspects.

Inevitable and likely to

lead to radical change
of the whole structure.

Power

Replaced by
relationships such as
leadership and control.

Medium through which

conflict of interest may
be resolved.

Unequally distributed
thus allowing
domination,
subjugation and so on.

Two Dimensions of a Problem Context

It is useful to group problem contexts according to
the following two dimensions:
Systems - as simple or complex
Par0cipants - as unitary, pluralist or coercive

28

Simple and Complex Systems

Attribute
Number of system
elements

Small

Simple Systems

Complex Systems
Large

Interactions between
elements

Few

Many

Attributes of elements

Predetermined

Not predetermined

Interaction between
elements

Highly organised

Loosely organised

Behaviour

Governed by well-defined laws

Probabilistic

Evolution

Does not evolve

Evolves over time

Nature of sub-systems

Do not pursue their own goals

Are purposeful and

generate their own goals

Interaction with
environment

None

Interacts strongly

System of Systems Methodologies

(SOSM)
Unitary
Simple

Pluralist

Operations research

Social systems design

Systems analysis

Strategic assumption surfacing and

testing

Systems engineering

Coercive
Critical systems
heuristics

Systems dynamics

Complex

Viable system diagnosis

Interactive planning

General system theory

Soft systems methodology

Socio-technical systems thinking

Contingency theory

29

Methodology Choice Examples

Systems methodology
(examples)

Assumptions about
Problem Contexts

Underlying metaphors

System dynamics

Simple-Unitary

Machine/Team

Viable systems diagnosis

Complex-Unitary

Organism/Brain/Team

Strategic assumption
surfacing & testing

Simple-Pluralistic

Machine/Coalition/Culture

Interactive planning

Complex-Pluralistic

Brain/Coalition/Culture

Soft systems methodology Complex-Pluralistic

Organism/Coalition/Culture

Critical systems heuristics

Machine/Organism/Prison

Simple-Coercive

To Recap
choice

creativity

issues
Unitary
Simple

Pluralist

Operations research

Social systems design

Systems analysis

Strategic assumption surfacing and

testing

Systems engineering

Coercive
Critical
heuristics

systems

Systems dynamics

Complex

Viable system diagnosis

Interactive planning

General system theory

Soft systems methodology

Socio-technical systems thinking

Contingency theory

implementation

30

Summary
We have:
Looked briey at CMMI a method of improving the
maturity of an organisa0on
Introduced a meta-methodology, TSI, that aims to
help us apply appropriate system methodologies to
our context.

That is the end of the material covered in this

subject

31