Download as ppt, pdf, or txt
Download as ppt, pdf, or txt
You are on page 1of 78

Aerospace Systems Engineering

The Fuzzy Front End


Dr. Daniel P. Schrage
Professor and Director, CASA, CERT, & PLMC
Dr. Dan DeLaurentis
Asst. Professor, ASDL
School of Aerospace Engineering
Georgia Institute of Technology
Atlanta, GA 30332-0150

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Presentation Outline
• Introduction to Systems Engineering, the Systems
Engineering Process and Systems Analysis
• Modern Systems Engineering and the Quality
Revolution
• The Five Lean Principles as Guiding Principles for
Modern Systems Engineering
• Integrated Product/Process Development (IPPD)
through Robust Design Simulation (RDS) for the Fuzzy
Front End to Identify Customer Value and the Value
Stream
• The Goal for Perfection through creation of a Virtual
Stochastic Life Cycle Design Environment (VSLCDE)
• A Robust Design Simulation (RDS) Example
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Systems Engineering
• Systems Engineering has been defined as an
interdisciplinary engineering management process to
evolve and verify an integrated, life cycle balanced set
of system solutions that satisfy customer needs (SE
Fundamentals, DSMC, 1999)
• Systems Engineering methods and tools were
developed in the early 1960s to decompose and
breakdown complex Aerospace Systems, e.g. Ballistic
Missile, Launch Vehicles, Aircraft
• These methods and tools contributed greatly to winning
the “Space Race” and the “Cold War”
• However, a Modern Approach to Systems Engineering
must reflect the Quality Revolution which has driven
industry for the past 20 years
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
The Systems Engineering Process
• As developed for the Department of Defense (DoD) the
Systems Engineering Process includes three major
elements:
– Requirements Analysis
– Functional Analysis and Allocation
– Synthesis
• Systems Analysis and Control include the techniques
and tools to analyze and control the Systems
Engineering Process
• The Systems Engineering Process is applied to each
stage of life cycle development, one level at a time
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
The DoD Systems Engineering Process
(SE Fundamentals, DSMC, 1999)
P
R
O
C
System Analysis
E
Requirements and Control
S
Analysis (Balance)
S
Requirements
I Loop
N
Functional Analysis
P
Allocation
U
T Design
Loop
Verification
Synthesis

PROCESS OUTPUT
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
The Systems Engineering Process
Process Input
• Customer Needs/Objectives/
Requirements
- Missions
- Measures of Effectiveness
- Environments System Analysis
- Constraints Requirements Analysis
• Technology Base • Analyze Missions & Environments & Control
• Output Requirements from Prior • Identify Functional Requirements (Balance)
Development Effort • Define/Refine Performance & Design
• Program Decision Requirements Constraint Requirement
• Requirements Applied Through • Trade-Off Studies
Specifications and Standards Requirement Loop • Effectiveness Analysis
• Risk Management
Functional Analysis/Allocation • Configuration Management
• Decompose to Lower-Level Functions • Interface Management
• Allocate Performance & Other Limiting Requirements to • Performance Measurement
All Functional Levels - SEMS
• Define/Refine Functional Interfaces (Internal/External) - TPM
• Define/Refine/Integrate Functional Architecture - Technical Reviews
Design Loop
Synthesis
• Transform Architectures (Functional to Physical)
• Define Alternative System Concepts, Configuration
Verification Items & System Elements
• Select Preferred Product & Process Solutions
• Define/Refine Physical Interfaces (Internal/External)
Related Terms:
Customer = Organization responsible for Primary Functions Process Output
Primary Functions = Development, Production/Construction, Verification, • Development Level Dependant
Deployment, Operations, Support Training, Disposal - Decision Data Base
Systems Elements = Hardware, Software, Personnel, Facilities, Data, Material, - System/Configuration Item
Services, Techniques Architecture
- Specification & Baseline
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Why Systems Analysis?
• Systems Analysis is a scientific process, or methodology, which
can best be described in terms of its salient problem-related
elements. The process involves:
– Systematic examination and comparison of those alternative
actions which are related to the accomplishment of desired
objectives
– Comparison of alternatives on the basis of the costs and the
benefits associated with each alternative
– Explicit consideration of risk
• NASA, DoD, and Industry are realizing that more emphasis must
be placing on enhancing systems analysis at the front end of
the life cycle using modern systems engineering approaches

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Lean Principles for Modern Systems Engineering
• Systems Engineering developed in the early 1960s for a
top down hardware decomposition approach
• Systems Analysis and Control used to track and evaluate
implementation
• Work Breakdown Structure (WBS) identified work
packages for pulling in the complete supply chain
• Software Engineering was developed in 1980s along a
parallel path
• Quality Revolution of the 1980’s revealed the need for a
quality emphasis, e.g. Concurrent Engineering, IPPD, JIT
Six Sigma and Lean Manufacturing
• Quality Emphasis has cumulated into a set of Lean
Principles , as first identified in the Womack and Jones
Book on Lean Thinking
• Modern Systems Engineering should start with the Lean
Principles as Guiding Principles
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Evolution of Systems Engineering and Software
Engineering Standards
2000+

ISO
15288
Systems 1994 1998 2000+
Engineering
EIA/IS ANSI/EIA ANSI/EIA
1994 632 632 632
1974 Mil-Std- 1998
1969 499B
Mil-Std-
499A 1994 EIA/IS 731 SE
Mil-Std- Capab. Model
499 IEEE
1220 1998
Others ...
(Trial Use) IEEE
1220
IEEE
1220
Updates

1995 1998?
Software 1988 ISO/IESC US
Engineering 12207 12207
Dod-Std-
2167A
1997
1987 1994 1996
Dod-Std- Mil-Std- IEEE 1498 J-Std-
1703 498 /EIA 640 016
1988 (Draft)
Dod-Std-
7935A

Application of standards is the realm of Systems Engineers


Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
Source: INCOSE Systems
www.asdl.gatech.edu Engineering Handbook
Quality Revolution - Where Competition is Today
Cost Advantage
Cheap Labor
Hi Volume, Lo Mix Production
Quality
Statistical Process Control
Variability reduction
Customer Satisfaction Time-to-Market
Cycle time Comparison (JIT)
Integrated Product/Process Development
Product/Process Simulation
Hi Skill adaptable Workforce
Manufacturing Product Variety
Enterprise
Cost Independent of Volume
Flexibility
Agile and Lean
Commercial/Military Integration
Virtual Companies
Company Goodness
Environment
1960 1970 1980 1990 2000
NCAT Report, 1994
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Japanese Auto Industry Used Concurrent Engineering To
Make Design Changes Earlier Than U.S. Auto Industry with Reduced Cycle Time

Japanese/U.S. Engineering Change Comparison

U.S. Company
Number of Engineering Product
Changes Processed

Japanese
Company

90%
Total Japanese
Changes Complete
Months

Months
Months

Months
Job #1
20-24

14-17

1-3

+3
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
The Quality Engineering Process
provides Recomposition

Knowledge Feedback

Seven
Seven Quality Robust
Robust Statistical
Quality Statistical
Customer Management
Management Function Design
DesignMethods
Methods Process
Function Process
and
andPlaning
Planing Deployment (Taguchi,
(Taguchi,Six
Six- - Control
Deployment Control
Tools
Tools Sigma,
Sigma,DOE)
DOE)
Off-Line Off-Line Off-Line On-Line
Off-Line Off-Line Off-Line On-Line
• Identify •Variation •Hold Gains
•Needs Important Experiments
•Continuous
Items
•Make Improvement
Improvements

Having heard the “voice of the customer”, QFD prioritizes where improvements are
needed; Taguchi provides the mechanism for identifying these improvements

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Lean Principles as Guiding Principles for
Modern Systems Engineering
• Establish and Specify value: Value is defined by customer in
terms of specific products & services, preferably as a Benefits to
Cost Ratio (BCR)
• Identify the value stream: Map out all end-to-end linked actions,
processes and functions necessary for transforming inputs to
outputs to identify and eliminate waste (Value Stream Map or
VSM)
• Make value flow continuously: Having eliminated waste, make
remaining value-creating steps “flow”
• Let customers pull value: Customer’s “pull” cascades all the way
back to the lowest level supplier, enabling just-in-time production
• Pursue perfection: Pursue continuous process of improvement
striving for perfection

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Source: James Womack and Daniel T. Jones, Lean Thinking (New York: Simon & Schuster, 1996).
Relationship of Lean Principles to Systems and
Quality Engineering Activities
• Value is established based on Systems Engineering activities, such as
requirements definition and functional analysis and allocation, and
Quality Engineering activities, such as use of the seven management
& planning tools and Quality Function Deployment (QFD) to Define
the Problem and Establish Value, through the identification of an
Overall Evaluation Criterion (OEC)
• The Value Stream is next determined through system synthesis &
analysis for Generating and Evaluating Alternatives for establishing
customer focused life cycle activities along a timeline, often domain or
agency specific
• Make Value Flow through decision-making to track, system analyze
and control the OEC periodically along the life cycle process, e.g.
earned Value,
• Let Customers Pull Value to apply the Lean Principles throughout the
System Work Breakdown Structure (WBS) to sub-contractors, vendors
and suppliers, e.g. the Supply Chain Integration
• Pursue Perfection is to apply robust design and 0ptimization
approaches for Process Improvement toward attaining Six Sigma,
which is accomplished by shifting the mean to the target and
Dr. Daniel P. Schrage
variability reduction , e.g. Statistical Process Control techniques
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
The Relationships between Requirements, DoD
Acquisition, RDTE, and Industry Design Processes
Operational MNS Phase 0 Phase I ORD Phase II Phase III
Req’ts. Documents
Mission Area Concept Program Definition Engineering and Production, Demilitarization
Analyses (MAAs) } Exploration
&
Risk Reduction
Manufacturing
Development
Deployment and
Operational Support
&
Disposal
DoD Acquisition
Process Phases M.S. 0 M.S. I M.S. II M.S. III

Advanced Concept Technology


Demonstrations (ACTDs)

Advanced Technology Manufacturing


Demonstrations (ATDs) (7.8)
Technology

S&T Categories
and RDT&E
6.1 6.2 6.3 6.4

Basic Exploratory Advanced Engineering


Research Development Development Development

Industry Design Phases

Product Conceptual Preliminary Detailed


Design Phases Design Design Design

Increasing Fidelity of Analysis and Test

Process System Parameter Tolerance Statistical Process


Design Stages Design Design Design Control

Off Line On Line


Quality Quality
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Interaction Between the Defense Acquisition System, the
Requirements Generation System, and the PPBS
(Latest DoD 5000.2)

FUNDING & REQUIREMENTS


 Full funding commitment occurs when systems-
Technology
Opportunities & level work commences
User Challenges  Maintain requirements flexibility early to facilitate
cost-performance trades

X D
Advanced
D
System
D
System
C Production
IOC
Concept Development Integration Demo Readiness &
Production & Support
Exploration
IPR IPR LRIP Deployment
IPR
Risk Reduction & Demonstration
Funding BA 2 or 3 BA 3 BA 4 BA 5 BA 5/Prod Production/O&M
Research Category 6.2/6.3a 6.3a 6.3b 6.4 6.4

Exploratory Advanced Engineering Production Production


Development Development Development Readiness/
IOT&E, LRIP

Requirements MNS Initial ORD ORD

BA: Budget Activity


MNS: Mission Need Statement
ORD: Operational Requirements
Document Full Program Funding
LRIP: Low Rate Initial Production
IOT&E: Initial Operational Test and through outyears
Evaluation

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
NASA’s Life Cycle Process Model
(2nd Generation RLV Risk Reduction Solicitation)

Phase 1 Phase 2 Production Operations

Effectiveness Issues
Requirement Issues Design Issues Verification Issues

Design Accepted
Selected
Reqmts Product
NASA System Detailed Design
Reqmt Verification Operations
Commercial Analyses Design Design
DOD
Customer Baseline
Reqmts Approved
System Technology Design
System
Model Performance
Analyses
Measures Production

Safety
Reliability Define Scale, Enable Full
Define Priority
Refine for Fidelity, Test Scale
Affordability Commercial
Risk
Environment Development
Reduction
Mission Convergence for Tech With 20%
Needs
Demos Margin
Performance

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
A Value Example: Military Transport Aircraft
Overall Evaluation Criterion (OEC)

 ( Ai )   ( MCI )   ( Si )
OEC 
 ( LCC )

Ai 
MTBF
MCI 

Payload  Dis tan ce
Time

MTBF  MTTR WEmpty  WFuel

Si = 1 - PDPHPK LCC = RDTE + PC + O&S +DC

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Coninuous RDS along the System Life Cycle to link the
“fuzzy front end” to the “process capability approaches”
Continuous Product Improvement / Innovation
Uncertainty Risk Management/Reduction

Overall Fuzzy Front End

Evaluation
Criterion Upper Specification
(OEC)
Response

OEC Target

Lower Specification

Bring the Development Process Approach Six-Sigma,


Define Distributions Under Control, C p =1 1<C p <2

System Definition System Integration Manufacturing


& System Design
(Preliminary/Parameter) (Detail/Tolerance) (On-Line Quality)
Tech. Development
(Conceptual/System)

Traditional C p and C pk Approach for Continuous, On-line Process Improvement

Overall Upper Specification

Evaluation
Criterion
(OEC)
Response OEC Target
Lower Specification

Six-Sigma Achieved,
Cp = 2

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
Initial Distribution Reduced Variability and Improved Mean Response
CASA/CERT/PLMC
www.asdl.gatech.edu Time
The VSLCDE- Key Characteristics
The purpose of VSLCDE is to facilitate design decision-
making over time (at any level of the organization) in the
presence of uncertainty, allowing affordable solutions to be
reached with adequate confidence. It is a research testbed.
 Virtual . . . Simulation-based system life-cycle prediction
 Stochastic . . . Time-varying uncertainty is modeled; temporal decision-making
 Life-Cycle . . . the design, engineering development, test, manufacture, flight test,
operational simulation, sustainment, and retirement of a system. The operational
simulation includes virtual testing, evaluation, certification, and fielding of a vehicle in the
existing infrastructure, and tracking of its impact on the economy, market demands,
environment.
 Design . . . Implies that the environment’s main role is to provide knowledge for use by
decision-makers, especially for finding robust solutions
 Environment . . . Implies the support of geographically distributed analyses and people
through collaboration tools and data management techniques
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Concept Exploration
Alternative Concepts
Analysis of Alternatives
M System Level Requirements O
N R
S D
System Analysis
Requirements and Control
Analysis (Balance)
Requirements
Loop
Functional Analysis
Allocation
Design
Loop
Verification
Synthesis

Tech
MS 0 Review MS 1

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
What is IPPD?
• Integrated Product/Process Development (IPPD) is a
management methodology that incorporates a systematic
approach to the early integration and concurrent application of
all the disciplines that play a part throughout a system’s life
cycle (Technology for Affordability: A Report on the Activities of the Working
Groups to the Industry Affordability Executive Committee, The National Center for
Advanced Technologies (NCAT), January 1994)
• IPPD evolved out of the commercial sector’s assessment of what
it took to be world class competitive in the 1980s
• The DoD has required IPPD and the use of IPTs where practical
throughout the DoD Acquisition Process for Major Systems
(DoD 5000.2R)
• Conduct of IPPD requires Product/Process Simulation using
Probabilistic Approaches
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Traditional Design & Development Using only a Top Down
Decomposition Systems Engineering Process

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Life Cycle Cost Gets Locked In Early for Complex
Systems using only Systems Engineering Decomposition

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Concurrent vs Serial Approach
Design & Test &
high Support
Development Production

Cost of
Change
N um ber of D esign C hanges

IPPD
Focus

Serial
Approach
Concurrent
Engineering
Approach

low

Time

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
IPPD Requires the Computer Integration of Product and Process Models and
Tools for System Level Design Trades and Cycle Time Reduction

CONCEPTUAL
DESIGN
(SYSTEM)
SYSTEM SYSTEM
PROCESS FUNCTIONAL
RECOMPOSITION DECOMPOSITION
Process Product
Trades Trades
PRELIMINARY PRELIMINARY
DESIGN DESIGN
(PARAMETER) (PARAMETER)

INTEGRATED
COMPONENT Process PRODUCT COMPONENT
Product
PROCESS Trades PROCESS FUNCTIONAL
Trades
RECOMPOSITION DEVELOPMENT DECOMPOSITION

DETAIL DETAIL
DESIGN DESIGN
(TOLERANCE) (TOLERANCE)
Process Product
Trades Trades
PART PART
PROCESS FUNCTIONAL
RECOMPOSITION DECOMPOSITION
MANUFACTURING
PROCESSES
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Georgia Tech Generic IPPD Methodology
COMPUTER-INTEGRATED ENVIRONMENT
QUALITY TOP-DOWN DESIGN SYSTEMS
ENGINEERING METHODS DECISION SUPPORT PROCESS ENGINEERING METHODS

7 M&P TOOLS AND REQUIREMENTS


ESTABLISH
QUALITY FUNCTION THE NEED & FUNCTIONAL
DEPLOYMENT (QFD) ANALYSIS

PRODUCT DESIGN DRIVEN


PROCESS DESIGN DRIVEN

DEFINE THE PROBLEM SYSTEM DECOMPOSITION


&
FUNCTIONAL ALLOCATION
ESTABLISH
VALUE

ROBUST DESIGN GENERATE FEASIBLE


ASSESSMENT & SYSTEM SYNTHESIS
ALTERNATIVES THROUGH MDO
OPTIMIZATION

EVALUATE
ALTERNATIVE

ON-LINE QUALITY
ENGINEERING & SYSTEM ANALYSIS
MAKE DECISION &
STATISTICAL
PROCESS CONTROL

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Georgia Tech Generic IPPD Methodology
• Methodology provides a procedural design (trade-off iteration)
approach based on four key elements:
– Systems Engineering Methods and Tools (Product design
driven, deterministic, decomposition approaches; MDO is usually based
on analytic design approach)
– Quality Engineering Methods and Tools (Process design
driven, nondeterministic, recomposition approaches; MDO is usually
based on experimental design approach)
– Top Down Design Decision Process Flow (Provides the design
trade-off process required for Complex Systems)
– Computer Integrated Design Environment(Information
Technology driven to provide a collaborative interactive environment)
• Methodology has been implemented through Robust
Design Simulation (RDS) for a number of applications
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Georgia Tech Graduate Program in Aerospace
Systems Design & Analysis
• Initiative kicked of in the early 1990s based on IPPD Approach
• Education executed in the School of Aerospace Engineering and
research program executed through the Center for Aerospace
Systems Analysis (CASA) and its two major Laboratories, The
Aerospace Systems Design Laboratory (ASDL) and The Space
Systems Design Laboratory (SSDL)
• Currently has approximately 140 graduate students with over 80
% U.S. citizens – top students from top universities
• Research Program currently at approximately $6M per year
including four faculty and 15 research engineers, plus 100 GRAs
• Program is built on probabilistic approaches for implementing
IPPD through Robust Design Simulation
• Goal is to develop, verify and validate, in collaboration with
industry and government, a Virtual Stochastic Life Cycle Design
Environment (VSLCDE)
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
A System Integration and Practice-Oriented M.S.
Program in Aerospace Systems Design & Analysis

Semester I Semester II Summer

Design Methods/Techniques ISE/PLMC


Aerospace Disciplinary Propulsion Development
Systems Electives Systems
Engineering Design
Special
Project
Applied
Systems Applied
Systems
Design
Design II Design
Design IIII
Safety
By Design
Advanced Advanced Product
Design Design Life Cycle
Methods I Methods II Management
Internship
Design Tools/Infrastructure
Mathematics (2 Required) Other Electives

Dr. Daniel P. Schrage Legend: Core Classes Elective Classes


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Aerospace Systems Design Integrated
Education & Research Philosophy
Government Industry

Partners: • Methods Formulation Relevant Partners:


ONR • Supports Basic Research Problems GEAE
NASA • Implementation of Methods RRA
AFRL Data & Tools LMTAS
NRTC Funding Boeing
Funding Sikorsky

Aerospace Systems
Design Laboratory

Methods
Methods Students
Students
Classroom Implementation

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Complex System Formulation initially taught In Aerospace
Systems Engineering using an Integrated Set of Simple Tools

T e c h . A lte rn a tiv e
QFD Id e n tific a tio n
HOW s a lt. c o n c e p ts Baseline 1 st Option 2 nd Option
Engine Type MFTF Mid-Tandem Turbine Bypass
Fan
Fan 3 Stage 2 Stage No Fan

criteria
Combustor Conventional RQL LPP
Nozzle Conventional Conventional + Mixer Ejector
Acoustic Liner Nozzle
Aircraft None Circulation Hybrid Laminar
Technologies Control Flow Control
W e ig h ts M o rp h o lo g ic a l M a trix

P u g h E v a lu a tio n M a trix

M ADM B est
A lte rn a tiv e
S u b je c tiv e E v a lu a tio n
(th ro u g h e x p e rt o p in io n ,
s u rv e y s , e tc .)
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Current Complex System Formulation Projects in
Aerospace Systems Engineering Course (Fall 2003)
1. AIAA Graduate Student Missile Design Competition “Multi
Mission Cruise Missile Design”
2. AHS Student Design Competition for “Design Certification
Mountain Rescue Helicopter”
3. NASA Identified Complex System of Systems Problem: “Future
Air Transportation Architecture - A System of Systems
Problem”
4. NASA Specific Complex System Problem: “Space Shuttle
Derivative: What it takes to make it Safe and Flyable”
5. NASA Identified Complex System Problem: “Two Stage
Turbine Based Combined Cycle (TBCC) Space Access
Launch Vehicle”
6. NASA Aerospace Vehicle Systems Technology Office Student
Design: “Quiet Supersonic Business Jet and Transport”

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Current Complex System Formulation Projects in
Aerospace Systems Engineering Course (Fall 2003)
7. Homeland Security and Coast Guard Initiative: “Feasibility of
Accelerating the Integrated Deepwater System (IDS): A
Network Centric Complex System”
8. Missile System Technical Committee: “Long Range Liquid
Target Vehicle (LRLTV)”
9. University Student Design Competition: “International
Micro Aerial Vehicle (MAV) Competition”
10. Complex System Formulation for: “Boeing 7E7
“Dreamliner” Commercial Transport”
11. Complex System Formulation for : “Morphing UCAV
Aircraft”
12. NASA Aerospace Vehicle Systems Technology Office Student
Design Competition for: “Unmanned Air Vehicle Systems
and Technologies: Replacement for Helios”

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Roadmap to Affordability Through
Robust Design Simulation

Robust
RobustDesign
DesignSimulation
Simulation
Subject to

Design
Robust Solutions
Design&&Environmental
Environmental
Technology Constraints
Constraints
Technology
Infusion
Infusion
Objectives:
Physics-
Physics- Schedule
Based Economic Budget
Based Synthesis
Simulation Operational Life-Cycle
Modeling
Modeling & Sizing Environment Reduce LCC
Analysis
Increase Affordability
Activity
Activityand
and Increase Reliability
Process-
Process- .....
Based
Based
Modeling
Modeling Economic
Economic&& Impact
Impactof
ofNew
New
Discipline
Discipline Technologies-
Technologies-
Uncertainties
Uncertainties Performance
Performance&&
Schedule
ScheduleRisk
Risk Customer
Satisfaction
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Synthesis & Sizing
is the key for translating Mission into Geometry
Safety

Safety
Aerodynamics Economics
Aerodynamics Economics
Geometry

Synthesis & Sizing


S&C Mission Manufacturing
S&C Manufacturing

Integrated Routines
Table Lookup Increasing
Sophistication and
Structures Performance Complexity
Conceptual Design Tools
(First-Order Methods)
Approximating Functions
Direct Coupling of Analyses Propulsion
Structures Performance

Preliminary Design Tools


(Higher-Order Methods)

Propulsion
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Aircraft Life Cycle Cost Analysis (ALCCA) -
including Economic Cash Flow Analysis
PA
YM
PR E NT AIR
SC ODU S C LIN
HE E
HE CT DU
AIRCRAFT D U I ON
LE LE
WEIGHTS
ENGINE RDT & E
THRUST & WGHT. COSTS
LABOR MANUFACTURER
RATES
AIRCRAFT UNIT
CALCULATE YES MANUFACTURER ROI VS PRICE
MANUFACTURING MANUFACTURER CASH-FLOW
COSTS
PRODUCTION
COSTS CASH-FLOW
QUANTITY ROI
LEARNING AVERAGE
CURVES COST

NO

AIRCRAFT MISSION
PERFORMANCE Airline Production
FUEL, INSURANCE Yield Quantity
DEPRECIATION RATES AIRLINE RO I
OPERATING
LABOR & BURDEN
RATES COST

PR ICE
INDIRECT

DIRECT TA
COSTS

COSTS

RE
VE XR
NU AT
E E

AIRLINE AIRLINE
CALCULATE YES ROI VS PRICE
AIRLINE ROI RETURN ON
INVESTMENT
NO
N NT
IO
ISIT E Y ME E DS
TOTAL QU UL A H
AC HED EP SC
Dr. Daniel P. Schrage OPERATING PR PR.
SC
Georgia Institute of Technology
Atlanta, GA 30332-0150
COST
&
DE CASA/CERT/PLMC
www.asdl.gatech.edu
Risk & Uncertainty are Greatest at the Front
Known Unknowns correspond to Risk and Known Probability Distribution

KNOWNS

KNOWN-UNKNOWNS

UNKNOWN-UNKNOWNS

CONCEPT VALIDATION FULL PRODUCTION DEVELOPMENT


SCALE
DEVELOPMENT

UnKnown Unknowns correspond to Uncertainty and Unknown Probability Distribution


Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
The Five Step RDS Process
2
Determine System Feasibility 1
Problem Definition

Design Space Model


Constraint Fault Tree Identify objectives, constraints,
x
P(feas)
1 design variables (and associated
Xi = Design Variable side constraints), analyses,
FPI(AIS) or Monte Carlo x uncertainty models, and metrics
Ci = Constraint 2
AND
x
3
C1 C2 C3 C4

3
Examine Feasible Space Constraint
Cumulative Distribution
Functions (CDFs)
N P(feas) Y
P

Design Space Model


< small
x
1
FPI(AMV) C1
5 or P
Decision Making x
2 Monte Carlo
• MADM Techniques C2
• Robust Design Simulation P
• Incorporate Uncertainty Models x Relax
3 Y
Constraints?
C3

4
Technology Identification/Evaluation/Selection (TIES)
• Technology Selection • Identify Technology Alternatives Relax Active Y
• Resource Allocation Obtain New CDFs Constraints
• Collect Technology Attributes ?
• Robust Design Solution
• Form Metamodels for Attribute Metrics P N
through Modeling & Simulation
• Incorporate Tech. Confidence Shape Fcns. Ci
• Probabilistic Analysis to obtain CDFs for the Old Tech.
Alternatives New Tech.

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Interactive RDS Environment
FPI
FP/ IMC CDF

or Requirement 2
Criterion 2
100%
DISCIPLINARY RSEs

Probability
Aero
0%
Structures Objective JPDM

Weights Criterion 1
or Requirement 1
Etc. Concept Space

Metrics/Objectives
Technology

 Responses
Space Requirements

Metrics/Objectives
Space

 Responses
SYNTHESIS & SIZING

Metrics/Objectives
 Responses
Constraints

Constraints

Constraints
1

Dynamic TWR

Contour ² % $/ RPM

Dr. Daniel P. Schrage RSEs Plots


Georgia Institute of Technology
Atlanta, GA 30332-0150
-1

-1 SW
TOFLm od
SLNm od

1
CASA/CERT/PLMC
www.asdl.gatech.edu
Two Examples on Application
of IPPD Through RDS
System of Systems: CDSE Process for FCS
FST Team in Phase I
Derivative Program: F-18C Conversion to
F-18E/G

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
FST Process Methodology
for FCS Phase I
Concept Development
& Systems Engineering
Methodology Incorporates IPPD Principles, QFD, Analysis,
Engineering Simulation, Systems Engineering Tools, and
Force-on-Force Simulation
(This presentation does not reflect
the current thinking of the FCS LSI)
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
The Full Spectrum Team(FST): 2408W-C07

One of Four Teams in FCS Phase I

SBA/SMART
Soldier Systems C4ISR

Weapons/ FCS Robotics


Platforms

Deployment /
O&O Sustainment

Dr. Daniel P. Schrage Working Closely With Government Labs


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC A0042
www.asdl.gatech.edu
Network Centric System Design of a Lethal Brigade

The FST Brigade is designed to fight with precision fires and high lethality
• Diverse, overlapping fires and
sensor coverage at all echelons Command
Vehicle Sensor systems
• Near and far fires with area and
precision effects Tethered UAV Command
HQ External
Vehicle
• Multiple layered sensor RSTA
augmentation
coverage Vehicle Command Brigade
Vehicle
echelon
MM Radar
NLOS

Typical Vehicle Sensors Tethered UAV


Command MM Radar BDE Air Defense
EO/IR sensor/sight Vehicle Scout
Vehicle ARV Marsupial
laser detectors NLOS BG
glint detectors UGV
NetFires
Medium UAV

RSTA Battle Group


Marsupial RSTA Small UAV NLOS
UGV Vehicle 9 ton variants
Mortar
Command
Marsupial Vehicle Scout NetFires
UGV Vehicle ARV Battle Group
Infantry echelon
Carrier Command
NLOS RSTA Battle Unit
Mortar
Soldier Systems Vehicle ARV

Stinger BlkIIE
NLOS Vehicles
HUMMWV SUV Soldier
Systems
Mortar LOS/BLOS Assault Battle Unit Objective Crew Served Weapon
command vehicles
Dr. omitted echelon ARV
Daniel P. Schrage
Georgia Institute of TechnologyWeapon systems Common Missile
CASA/CERT/PLMC
Objective Crew Served Weapon
Atlanta, GA 30332-0150 External
www.asdl.gatech.edu augmentation
Concept Development & Systems
Engineering (CDSE) Process
• Incorporates key aspects of modern systems engineering approaches,
and lends itself to iteration
– Requirements Flowdown
– Engineering Trades / Analysis
– Force Effectiveness Modeling and Simulation
– Risk Mitigation
• Allows full exploration of need identification and problem definition,
concept development, and concept selection—prior to system
definition and design
• Facilitates group work and utilizes modern software based tools
• Allows full incorporation of increasingly detailed simulation-based
analyses and designs
• Smoothly extends throughout engineering and manufacturing phases

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Integration of QFD, Morph and Pugh Products
Into CDSE Process
HOWs
QFD 1-4 Tech. Alternative
Identification
HOWs alt. concepts Engine Type
Baseline
MFTF
1st Option
Mid-Tandem
2nd Option
Turbine Bypass
Fan

Pugh
Fan 3 Stage 2 Stage No Fan

criteria
Combustor Conventional RQL LPP

Evaluation Nozzle

Aircraft
Conventional

None
Conventional +
Acoustic Liner
Circulation
Mixer Ejector
Nozzle
Hybrid Laminar

Matrix Technologies Control Flow Control

Morphological Matrices
Weights

Multi Attribute
Decision
QFD Methodology Best
“Context” Alternative
Rationale
Subjective Evaluation
(through expert opinion,
Preliminary
surveys, etc.)
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Full Spectrum Team FCS Concept
Development & Systems Engineering (CDSE) Process

Selected Force
1 2
End Start Guidance - QFD 1 Guidance - QFD (2-4)
Concepts Iteration
Pugh Force 
AUTL 
Missions
Concepts 
TRADOC Docs 
Tasks
Selection Matrix MOEs 
MNS 
Functions

Capabilities

7
Force 3
Design Guidance Systems
Effectiveness
Alternative 1 
Organizational Capabilities
Simulations
O&O 
Operational
Engineering
Recomposition

Decomposition

Morph Matrices -
6 Selection of
Force Concepts Pugh Force Consistent Sets
Alternatives 1 Concepts of Systems and
Selection Matrix Technologies
5
4
Systems Concept 2 Technology /
Set 1 Characteristics
Concept 1 Subsystems
Characteristics Options

Legend Process
Process isis Parallel
Parallel Missions / Technology
Products Decision Systems Concept IPT
and
and Iterative
Iterative Scenarios Trees
Technology
Dr. Daniel IPT
P. Schrage Requirements IPT All IPTs
CASA/CERT/PLMC
A0031t
Georgia Institute of Technology
Atlanta, GA 30332-0150
www.asdl.gatech.edu
Focus on Requirements
Flowdown through QFD
AUTL QFD4
Commander
National Imperatives /

Missions AUTL Combat


Centric Co-Owned
Tasks
Warfare with Concepts IPT
Army Vision

HOWs (Title) Customer


Customer Importance

Assessment

How
HOQ

Direction of Improvement
HOWs (Title) Customer
Functions System

Customer Importance
Assessment

How
What

Technologies
HOQ

Capabilities
Missions
WHATs (Title)

AUTL

Direction of Improvement
HOWs (Title) Customer

What

Customer Importance
Assessment

AUTL Combat
How
HOQ
WHATs (Title)

Centric Functions
HOWs (Title) Customer
How Much

Customer Importance
Assessment


Direction of Improvement

How
Organizational Difficulty HO Q

Tasks

System Capabilities
What
Assessment

Commander
Technical

WHATs (Title)
How Much Direction of Improvement


Weighted Importance

What
Relative Importance Organizational Difficulty
Assessment
Technical

WHATs (Title)
Morpho-
How Much


Weighted Importance

Relative Importance
Organizational Difficulty

Assessment
Technical
logical
How M uch


Organizational Difficulty
Weighted Importance

Assessment
Relative Importance

Technical
Weighted Importance

Relative Importance Matrix


Requirements CDSE ITERATION 2
IPT Incorporate Four Detailed Determine Force
Characteristics to Select
Mission Scenarios Perform Functions Technologies
and Combine
into Candidate
Candidate Concepts
Criteria / MOEs

Concepts
ALTERNATIVE CONCEPTS
Importance
Customer

Pugh Concept Selection

Example

Physical Dimensions 1.0 A U A U U

Portability 5.0
A
Concept Criteria

Cost 8.0 A U A T U

Performance 0.0 A U U T T

Size 9.0
B

A T U U T

Weight 0.0 H T T U U

Durability 0.0 A U A U U

Life 4.0
C

A T A U U

Failures 3.0 A U U T U

st Best Concepts Determined in Process That Trades


+ Pugh Sums

- Pugh sums

Be
+ Pugh tally

Technologies, Concepts and Requirements


0

Dr. Daniel P. Schrage


CASA/CERT/PLMC
- Pugh tally
0

Georgia Institute of Technology


Atlanta, GA 30332-0150
www.asdl.gatech.edu
Representative FCS Trade Studies
• Communications Trades • Weapons Trades
– Peer-to-Peer and Client-Server – Guns and Missiles
– Islands and Backbones – 105 and 120 BLOS/LOS
– Frequency Bands and Bandwidth – Precision and Area Fires
– QoS (Latency, Availability, Bandwidth)
• Sensor Trades
• C2 Trades – Radar, EO/IR, and Ladar
– Legacy, New, Organic, Joint
– Systems (UAV, UGV, Mast, Tethered)
– Initiative and Control
– On Board and Off Board Fusion
• Platform Trades – ATR and Clutter (False Alarms)
– Tracks and Wheels
– Weight Studies (16t, 9t, 6t)
• Survivability Trades
– Modularity and Commonality Studies – Active and Passive
– Hybrid and Conventional Propulsion – Collective and Individual
– Turbine and Diesel – Links and Nodes
– Wheel and Body Motors • HMI Trades
– Active and Passive Suspension – Autonomy, Responsibility, Workload
• Force Trades – Commonality, Simplicity
– UAVs and UGVs – Motion and Maneuver
– Manned and Unmanned • Logistics
– Mounted snd Dismounted
– OPTEMPO and Sustainment
– BLOS / NLOS / LOS Mix
– Deployment (Weights, Times, Pulses)
– BU Size and Composition
– Prognostics, Distribution, Log Support
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
The Trade Space is Defined
by a Full Spectrum of Scenarios
Sensitivity Environment
analysis will Scenario 1
help drive to Solution
“The FCS
?
Solution” ce
an
rm
rfo
pe

“The Potential
in

FCS FCS
e
th

Solution” Solution
s
ti
ha
W

Potential
Scenario 2
FCS
Solution
Solution
Scenario 3
Solution
Threat

Mission
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Scenario Driven OMS/MP Development
Bosnia FST concept robustness is tested against a
FST Scenarios: Scenario broad range of missions & scenarios which gain
Forcing Function Korean Desert Storm validity from extensive quantitative & qualitative
for Full Spectrum Scenario Scenario
Chad wargaming & analysis
Force Development
Scenario
Tabletop
MAPEX
JANUS Operational
Initial Force Four Wargaming Examination
Four
Scenario
Concepts CASTFOREM
Initial
Specific
1&2 Wargaming
OMSMPs
OMS/MP
Inputs Scenario Weighting

Full Spectrum
Force Concepts Force
Concept Composite OMS/MP
(Version 4.1) (Scenario Specific Consolidation)
Wargaming Feedback

Force Concept Was Developed Iteratively & in Parallel With OMS/MP Refinements
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Evolution of FCS Force Concepts
Concept 1 Concept 2
 Heavy reliance on robotic ground vehicles  Heavy reliance on NLOS range engmts / UAVs
 More RSTA and assault  Significantly more payload volume
 Deployable by current rotary wing aircraft 2012 FUE  Mostly 18-ton vehicles

 Max vehicle weight 10 tons  Peer-to-peer information architecture


 Client-server information architecture  Based on Small Unit Operations comms tech
 Evolutionary comms and software systems  New software services

Baseline
 Blending of Concept 1 and 2--Robotics and NLOS engmts
 6-ton ARVs, 9-ton CVs, 16-ton max vehicle weight
 Helo vertical envelopment with smaller vehicles
2012 FUE  16 ton limit for C-130 deployment on unimproved runways
 Hybrid information architecture
 Peer-to Peer with communication islands

 Based on Small Unit Operations comms technology


 New software services

BLK I / BLK II
 BLK I - Employ available weapons systems
- Most vehicles manned
2008 FUE BLK I - 9-ton vehicles become 16 ton
2013 FUE BLK II  BLK II – Employ advanced weapons systems
- Many advanced robotic systems
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
 Same information and comms arch. as Baseline CASA/CERT/PLMC
www.asdl.gatech.edu
Concept Baseline Alternatives Summary

Concept 1 Concept 2
Sensors • Smaller Number of UAVs • Larger Number of UAVs
• Large Number of Modestly Capable • Small Number of Highly Capable Ground
Ground Sensors (ARVs) Sensors (RSTA Vehicle)

Information • Client Server • Peer-to-Peer


• Evolutionary Software • Distributed, Dynamic
Deciders • Highly Automated • Highly Automated Commander-
Commander-Centric Centric (Group/Brigade)
(Unit/Group)

Actors • Full Spectrum, but Weighted for the • Full Spectrum, but Weighted for
Red Zone Beyond the Red Zone
• Robotic ARV, DF, Netfires, NLOS • Robotic Netfires, Small UGV
• Manned C2, CV, and Infantry • Manned C2, Infantry Carrier, Direct
Carrier Fire, RSTA and Short Range NLOS
• CH-47 Transportable (<10 tons) • C-130 Transportable (<18 tons)
• Many Dispersed 4, 6, and 9 Ton • Common Chassis with
Vehicles (~650) Modules (~340)
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Baseline 2012 Concept
Network Centric Functional Overview
Global Hawk SAR / MTI
Satellite Imagery Soldier
A160
FoPen MUAV ELINT MUAV SAR / MTI
Sensors Organic UAVs, UGVs (EO/IR, FoPen, SAR, MTI)
Radar
SUAV FoPen
Ladar Organic Manned R&S with Tethered Sensors
R&S Vehicle SUAV
Tethered EO / IR EO / IR
Aided Target Recognition
LOS Red Zone

Beyond Red Zone


ARV
EO / IR Augmenting UAVs and Satellites
A0054

S
erviceLa
ye ie
rV
ie
w D
istrib
ute
dCom
m
man
dan
dCo
ntro
lSe
rvic
e
ics R
ealT
im
imea
ndH
ardR
ealT
im
imeP
roce
sso
rs

A
cto
rs D
e

S
e

In
cid

fo
e

rm
a
rs

rvic
esL
a

tio
y

nL
e
r

a
yer
S
enso
rs

S
e A
c
ir
A
P
Pla

irS
Sp
n
la
n

a
p
a
n
n

c
in

e
c
g
in
g

eCC
2
2 In
E
Ex
e
x

te
In
c
eu
tio
c
u

llig
te
n
tio

e
llig
en
n
n
I
n O

c
e
c
fo
In
p
O

e
p
rm
fo
er
a
e
ra

CC
a
rm

o
mo
tio
tio
a
n
tio

bb
m
S
a
u
S
up
n

a
p
n
tio
n
s
s

ttS
p
e
o
p
o
Srv
e
rtrt
ic
rv
S
o

ic
ld

a
ie

rf
ig
F
E
E

ee T
rS

h
te
ire
F
ff
fe
y
s

c
fe

T
te

rC
s&
ire
s&
t
s
c

ra
m

2S
e

ts

in
ra
&
In
te

in
in
&
g
in
S
im
S
rfa

rvic
es

,g
c

S
A
A

R
im
e

itu
S
w
a

u
w

,Re
la
u
a
itu
re
a
tio
a
ne
re

h
ehe
tio
la
n

a
e
n R
tio
n R
s
s
e

nn
tio
s

ra
s
s

a
r
s
e
c
e

l,
a
o
c
S
u
S
n
o
n
n
ru
v
a
n
e
rv

l, N
is
a
e
is
lila

o
N
o
S
s
n
a
s
a
c
illa

n
n
u
S
ta
p
u
ta
pp
nc
e
n

c
o
p
n
c
c

c
o
e& C
e

ti
c a
tic
rtrt
4
IS
e& C
M
M

a
4
aa

ll
R
IS
nn
S
RS
ag
a
g

O
y
e
m

S
s
te
y
s
e
m

S
m
te
e
m
n
e
n

ta
fffic
O
s
s
tt

n
ta
e
fic
n
e
d
d
T
a
o
T
rd
a
o
o
rd
olsls
V
a
n
e
hiclle
dCC o
eM gm
ntrol
t Organic Multi-Layer Peer-to-Peer Coms
u C o m m onC 2S erv ic e s
Syn c h
roniza tion& N aviga tio n& D is
trib ute d

Opportunistic Ground, UAV, and Satellite Links


r Terra inReaso
nin
g MC G & I Pers on a
liza tio
n

Info System
S c
hedu ling P os ition ing C o
llabora tiveO ps
i
C
omm
unica
tio
nsL
aye
r t Inform ationM a na
gem e n
t(I M )S e rvices
CommonO bje
c t
y KnowledgeBaseDataS tore S tore Geos pa tia
lData Pla n Forc
eS tru ctu re P ro file IM(

Rad iosa n
dn etw
ork
ing M
ultic
ast Pro
xy
Q
P
o
S&
o
lic
y
D
is
semin
atio
nIntere
s tM
gmtC ompo
nen
tFe
dera
tio
nInte
rop
era
b
b
)
ilityIM(a
-kerne ls C o
m p
utingP
latformSe
rvic
es
)

-con trolle
rs Ne
two
rk/C
omm
unic
atio
nsS
erv
ice
s S
ens
orArra
ysa
ndF
usion
-pro cess ors

Integrated Software/Hardware Open Architecture


In
dustryS
tand
arda
n ta
dS te-o
f-
f-the
-ArtT
echn
iq sfo
ue rHte
e ro
gen
eou
sCo
m tiin
pu ng
A
pp1 A
AA
ppp
p
pp
3
13 App2 A
pp2 App3
f1
f1 f1 f1 App4 A
p p4
f1 f2 f2
f1 f2
T
OOLS
App5 ANDS
TAND
ARD
S
“M
idd
leW
are
”/O
pera
tingS
yste
m S
ervice
s: f1

Distributed Warfighter Services


Im
ple
m e
nta
tio
noftheRE
ALTIM
ECOMP
UTIN
G M
eta-Henviro nment
OPERATINGE
NVIR
O N
MEN
T S A G E
U M L
A u
toco ders
SE
IandEmergin
gProce
sse
s M &Ssim ulatio
ns
IndustryS
tan
dardsand
a
ndProce
dures Syste
m Inte
gL a
b
esta
blishe
dprotoco
l

Brigade, Battle Group, Battle Unit


BDE
HQ
RSTA
Group
Battle
Group
NLOS
Group
LS
Group
Deciders Common Relevant Operating Picture
Decision and Planning Aids
Precision and Area NLOS Fires
Actors Precision BLOS / LOS Weapons
Net Fires
Soldier
System
Armed Reconnaissance Vehicles OCSW on all Combat Vehicles
105 NLOS
Mix of Manned and Unmanned Platforms
Infantry / Soldier Systems
Infantry Carrier / C2
120mm Mortar NLOS
Dr. Daniel P. Schrage / CV Large Number of Stowed Kills
CASA/CERT/PLMC
Systems
Georgia Institute of Technology105 LOS / BLOS
Atlanta, GA 30332-0150
www.asdl.gatech.edu
FCS Phase I Results
• Talented team pulled together through a generic IPPD
methodology extended to address the ‘system of systems’
problem for FCS CDSE
• An integrated set of simple tools proved useful for the first
iteration through the process
• Success of the Concept Development and Systems
Engineering (CDSE) Process developed and utilized is
summarized by the rankings by the government’s independent
assessment team
• Elements of the process are being continued by the Boeing-
SAIC Lead System Integrator (LSI) through the current FCS
Phase

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Future Combat System (FCS)
Government Independent Assessment Team’s
Phase 1 competitive scores (awarded Nov 2001)

Full Gladiator FocusVision


CONCEPT CATEGORY Spectrum Boeing (Lockheed/ (GDLS/
Team TRW) Raytheon)
Deployment 1 st
2 (tie)
nd
2 (tie)
nd
2nd (tie)
Situational 1st 4th 3rd (tie) 3rd (tie)
Understanding
Lethal Effects 1st 2nd (tie) 2nd (tie) 2nd (tie)
Protection 1st (tie) 4th 1st (tie) 2nd
Sustainability 1st 2nd (tie) 2nd (tie) 4th
Transition to New 1st 4th 2nd 3rd
Mission
OVERALL 1st 4th 2nd 3rd

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
F/A-18 Example
Example Application
• Link the appropriate aircraft sizing/synthesis and economic tools plus probabilistic methods
to create testbed environment; model the F/A-18C (using substantiation data for validation)
• With F/A-18E/F requirements (Ref. AIAA Paper 98-4701) as drivers, look at relation of
technology metrics on requirements mathematically

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Expanding Missions: The F/A-18E/F
Air
Maritime Air Close Day/ All
Fighter Defense
Air Combat Recce Air Night Weather
Escort Suppres-
Superiority Fighter Support Attack Attack
sion

F-14D F/A-18 A-6F


NATF A/B/C/D

F/A-18 E/F

Ref. Young, et.al. AIAA-98-4701, 1998.

How can such multi-role vehicles be


examined as potential solutions for the
war-fighter with respect to technologies,
requirements, and design constraints ?
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Process
The traditional process of identification of an overall objective to be
optimized is replaced by the following process:

1) Using Response Surface Method to mathematically represent combined


requirements-technology-configuration space

2) search for alternatives (configuration changes plus technology infusion)


that satisfy requirements and constraints (TIES method)

3) simultaneously, optimize on desirements within this feasible space


(continuous) or set (discrete) then, perform sensitivity studies to show the
perturbation of the solution due to possible changes in requirements and
design variables.

Thus, the customer/decision maker has information with regards to the choice between
tolerating a relaxation in requirements or accepting achievable performance levels

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
F/A-18C Basic Geometry

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Primary Mission- Fighter Escort
Actual
Modeled ( )
Intermediate Thrust Climb
Cruise at Optimum Mach and Altitude
42,550 ft
(39,910 ft)
(40,000 ft) 41,300 ft
(37,796 ft) 39,300 ft
(38,904 ft)
38,100 ft

Reserves:
20 minutes Loiter at S.L.
plus 5% of T/O Fuel Combat at 10,000 ft
Start & Taxi, Accelerate to Climb Speed 2 minutes at Maximum Thrust
4.6 minutes at Intermediate Thrust, SLS Mach 1.0 (missiles retained)

Combat Radius =311 nmi


Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Alternate Missions- Addressing Multi-role
Capability
• Requirements can include performance against a wide variety of missions

• Vehicle sizing proceeds based on a primary mission and then fallout performance of the sized vehicle on
alternate missions is computed and tracked
Metrics/Objectives
 Responses

Primary
Constraints

Mission
Responses

Alternative
Mission
Responses
Example: Given a vehicle sized
for Air Superiority (A-S) mission, Constraints
compute the performance for
Interdiction mission as A-S
requirements change Requirements, Vehicle Chars., or Technologies

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Alternate Mission: Hi Hi Hi
Actual
Modeled ( )
Intermediate Thrust Climb Cruise at Optimum Mach and Altitude
43,200 ft
(40,000 ft)
38,550 ft
40,600 ft
(37.911 ft) (40,000 ft)

Reserves:
20 minutes Loiter at S.L.
plus 5% of T/O Fuel Combat at Best Altitude
Start & Taxi, Accelerate to Climb Speed 5 minutes at Maximum Speed
4.6 minutes at Intermediate Thrust, SLS Mach 1.0 (missiles retained)

Combat Radius =505 nmi


Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Correlation of Drag Polars for Varying Mach Numbers
Altitude = 36,089 ft
1.6

1.4

1.2

1
Cl 1.8
1.5
0.8 1.2
Cl

0.9
0.6
0.2
0.6

0.4

0.2

0
0.00000 0.10000 0.20000 0.30000 0.40000 0.50000 0.60000
Cd
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Propulsion Modeling
F404-GE-402 Augmented Turbofan Engine
• The F404-GE-402 is an increased General Specifications:
performance derivative of the F404 • Thrust: 17,700 lb
and is used in the F/A-18C • SFC (max A/B): 1.74 lbm/lbf-hr
• Features a dual-spool mixed flow • SFC (IRP): 0.81 lbm/lbf-hr
turbofan architecture, 3X7X1X1
• Airflow (SLS): 146 pps
turbomachinery configuration
• F404 Engine performance deck • Weight: 2,282 lb
based on installed engine data for • Length: 159 in
the F/A-18C • Diameter: 35 in
• Engine performance data source:
“F/A-18C Substantiating
Performance Data with F404-GE-
402 Engines” Report
MDC91B0290
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Weight Breakdown- Validation
F/A18C Weight Breakdown Comparison
Group F/A18C Baseline Model
• Sizing/Synthesis Code Used: Wing 3,919 3,918
Tail Group 1,005 1,006
FLight OPtimization System Body 5,009 5,009
(FLOPS) Alighting Gear 2,229 2,228
Propulsion Group
Engines 4,420 4,417
• F/A-18C Baseline Modeled in Engine Section
Gear Box 921 922
FLOPS calibrated against actual Controls
Starting System
substantiation data from Fuel System 1,078 1,078
manufacturer Flight Controls 1,061 1,062
Auxiliary Power Plant 206 206
Instruments 84 84
• Highly accurate model (errors Hydraulics 351 352
Electrical 592 592
in weights less than 1%) Avionics 1,864 1,864
Armament, Gun, Launchers, Ejectors 948 948
Furnishings, Load/Handling, Contingency 631 631
Air Conditioning 641 642
Crew 180 180
Unusable Fuel 207 207
Engine Fluids 114 115
Chaff, Ammunition 252 252
Miscellaneous 58 58
Operating Weight Empty 25,770 25,771
Missiles 1,410
(2) AIM-7F 1,020
(2) AIM-9L 390
Mission Fuel 10,860 10,857
Takeoff Gross Weight 38,040 38,038
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Economic Assumptions
• MALCCA (Military Aircraft Life Cycle Cost Analysis) in-house
code used to determine notional aircraft economics

• Baseline File created starting with defaults based on the military


aircraft assumptions (primarily sourced from F-15 and F-16 data)

Inflation Factor 3.3%


Dollar Year 1994
Year of Program Initiation 2000
Final Year of Production 2023
# Operational Vehicles 2530 units
System Economic Life 20 years
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Wind Over Deck
Recovery Wind Over Deck Launch Wind Over Deck

Wind Over Deck Wind Over Deck

Aircraft Weight
Aircraft Weight

Aircraft Arresting Gear Cat Plus


Airspeed
Touchdown Speed Performance A/C Thrust
Required

Speed
Speed

•Aircraft Touchdown Speed = 1.05 * Vapp


•Airspeed Required = Calculated Liftoff Speed
•Arresting Gear Performance Calculated at Limit Capacity

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Responses and Top Level Requirements
Example Responses:
Responses/Desirements

R1
Metrics/Objectives

Gross Weight
R2 Probability of Survival
Lethality
R3 O+S
Acquisition Cost
R4
Approach Speed (constraint)
R5 TOFL (constraint)
Constraints

R6
T/W and W/S may belong in
R7
either the requirements or the
Req.1 Req.2 Req.3 Req.4 Req.5 Req. 6 Req.7 Req.8 responses section - depending
Range Payload PS tloiter turn rate fW wtW Mach on how the programs are set up
Top Level Requirements
This approach de-emphasizes the geometry of an aircraft, and instead focuses on the mission requirements.
However, it does require a baseline aircraft configuration. Geometry and Technologies are fixed, while
Requirements vary. Each vector of top level requirements maps to a specific mission.

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Requirement RSEs for Notional F/A-18C
Performance Capabilities Weights Economics
Takeoff Landing
Alternate Combat Specific Combat Combat Approach Landing Field Takeoff Field Wind Over Wind Over Takeoff Gross Operation & First Unit RDT&E
Wing Loading Thrust/Weight Wing Span Mission Range Life Cycle Cost
Excess Power Turn Rate Turn Radius Speed Length Length Deck Deck Empty Weight Weight Support Cost Cost
(lb/ft 2) (ft) Cost cents
(nm) (ft/s) (deg/s) (ft) (ft/s) (ft) (ft) (ft/s) (ft/s) (lbs) (lbs) (mil. $/yr) (mil. $)
(mil. $) #AC * hr * lb

95.89077

37.03584

740.7152
0.841592

1364.336

11844.52
3.746093

4088.045
152.8745

4184.198

17.36432

30061.56

42958.04
-0.99871

42.98329

5330.668

62.87922

0.792245

4380.113
-7.06871
-39.8082

4841.269

5885.437

3777.587
0.866574
0.749297

4897.278
45.0536

36214.7
23727.3

31626.8

55799.5
1014.9
1062.1

1787.6

53.187
16241
137.2

72.927
0.666

33.67

2.732
1.088

510.8

128.9
4.891

182.8
9072

2708
3137

5776

9345
68.1

40.3
-5%
 Radius
+40% -10%
Ult. Load Fact. Combat Mach #
+10% -10% +10% 0 500 1000 -5%
 Payload
(lbs)
Thrust
+40% -5%
Area
+30% 0
stealth
(lbs)

500 1000 0
# Aux. Tanks
(lbs)

2244 4488
Fuel Consumption

-15%
Specific
0%

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Requirements Exploration: F/A-18C Design Contours
Horiz Vert Factor Current X Grid Density Update Mode
Radius -0.8888 20 x 20 Immediate
ULF 0
CmbMach 0
DPayld
Thrust
-1
-0.8888
Slide bars control variable values
Area -0.857
DStealth -1
Response Auxtnk -1
SFC 1
contours may Response Contour Current Y Lo Limit Hi Limit
O&S 5500 5031.4066 ? 5500
be set here TOGW 40000 37137.19 ? 40000
LDWOD 15 4.8129793 ? 15 Constraints are set here
TOWOD 0 -20.93512 ? 0
Vapp 153 152.42056 ? 153
Ps 695 700.21755 695 ?
AltRng 920 1238.6386 920 ?

21420 Vapp

Takeoff
Wind Over Deck

Thrust Ps
(lbs.)
Landing
Wind Over Deck

TOGW
14535
360 Area (ft^2) 520
White area indicates available design space, while filled areas
Dr. Daniel P. Schrage
Georgia Institute of Technology
indicate areas which violate set constraints CASA/CERT/PLMC
Atlanta, GA 30332-0150
www.asdl.gatech.edu
Effects of Increase in Combat Radius Req.
21,420 Vapp

TOWOD

Thrust LDWOD

Ps

TOGW

Decreasing 14,535
380 Area 520 Increasing
Feasible 21,420 Vapp

Space
Combat
Thrust
TOWOD Radius
LDWOD

Ps
Reqmt.
TOGW
14,535
380 Area 520
21,420
Vapp

Thrust TOWOD
LDWOD

Ps

TOGW
14,535
380 Area 520
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Horiz Vert Factor Current X Grid Density Update Mode
Radius 0.964 20 x 20 Immediate
ULF 0.71
CmbMach 0
DPayld -1
Thrust 0.88888
Area 0.857
DStealth -1

Exploring the Space:


Auxtnk -1
SFC 0.3333
Response Contour Current Y Lo Limit Hi Limit

Capturing the F/A-18E/F !


O&S 5130 5475.7833 ? ?
TOGW 45000 47224.344 ? ?
LDWOD 30 26.563468 ? 30
TOWOD 15 13.613377 ? 15
Vapp 151 150.2058 ? 151
t_Radius 12656.5 10828.741 ? ?
t_Rate 3.8115 4.0987387 ? ?
Ps 780 807.60615 780 ?
AltRng 1540 1545.1224 1540 ?

21420 TOWOD

Vapp Alternate
Ps Range
Turn Radius
Turn Rate
TOGW

Thrust
(lbs.)

O&S
14535
380 520
Dr. Daniel P. Schrage Area (ft^2)
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
RDS Example: TOGW Req. for Notional F/A-18 (1)
Scenario 1: Conservative tech. improvements gives low confidence of meeting requirement
k_CDo
k CDo
k_CDi
k CDi
k_WingWt
k WingWt.

-0.06 -0.05 -0.03 -0.02 0.00 -0.06 -0.05 -0.03 -0.02 0.00 -0.13 -0.10 -0.07 -0.04 -0.01

k_FusWt
k FusWt. k_HTWt
k HTWt. k_VTWt
k VtWt.

-0.12 -0.09 -0.06 -0.03 0.00 -0.11 -0.08 -0.05 -0.02 0.01 0.00 0.25 0.50 0.75 1.00

Ov erlay Chart Forecast: Ov erlap

Frequency Comparison 10,000 Trials Cumulativ e Chart 1 Outlier


.025 1.000 10000

Achieved
TOGW
.019
TOGW
.750 Probability of Satisfying
TOGW Req. =~1%
.013 .500

.006 Anticipated .250


Anticipated Req.
TOGW Req.
.000 .000 0

31,500.00 32,375.00 33,250.00 34,125.00 35,000.00 -1,000.00 0.00 1,000.00 2,000.00 3,000.00
Certainty is 99.99% from -1,000.00 to +Infinity

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
RDS Example: TOGW Req. for Notional F/A-18 (2)
Scenario 2: Aggressive tech. improvements gives higher confidence of meeting requirement
k_CDo
k CDo
k_CDi
k CDi
k_WingWt
k WingWt.

-0.10 -0.09 -0.07 -0.06 -0.04 -0.10 -0.09 -0.07 -0.06 -0.04 -0.17 -0.14 -0.11 -0.08 -0.05

k_FusWt
k FusWt.
k_HTWt
k HTWt.
k_VTWt
k VtWt.

-0.17 -0.14 -0.11 -0.08 -0.05 -0.16 -0.13 -0.10 -0.07 -0.04 0.00 0.25 0.50 0.75 1.00

Ov erlay Chart Forecast: Ov erlap

Frequency Comparison 10,000 Trials Cumulativ e Chart 19 Outliers


.023 1.000 10000

Achieved Probability of Satisfying


.017 TOGW .750
TOGW TOGW Req. =~45%

.011 .500

.006 Anticipated .250


Anticipated Req.
TOGW Req.
.000 .000 0

31,500.00 32,250.00 33,000.00 33,750.00 34,500.00 -1,500.00 -625.00


0250.00 1,125.00 2,000.00

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
RDS Example:O&S $ as an Anticipated Requirement (1)
Scenario 1: Conservative tech. improvements gives low confidence of meeting requirement
k_CDo
k CDo
k_CDi
k CDi
k_WingWt
k WingWt.

-0.06 -0.05 -0.03 -0.02 0.00 -0.06 -0.05 -0.03 -0.02 0.00 -0.13 -0.10 -0.07 -0.04 -0.01

k_FusWt k_HTWt k_$O&S


k FusWt. k HTWt. k $O&S

-0.07 -0.06 -0.04 -0.03 -0.01


-0.12 -0.09 -0.06 -0.03 0.00 -0.11 -0.08 -0.05 -0.02 0.01

Forecast: Overlap
Overlay Chart P(Achieved-Anticipated)
5,000 Trials Cumulative Chart 189 Outliers
Frequency Comparison
1.000 5000
.022
Probability of Satisfying
Achieved .750 $O&S Req. =~2%
.017 $O&S/flthr
$O&S
.500
.011

.250
.006 Anticipated
Anticipated Req.
$O&S Req.
.000 0
.000
-100.00 0 100.00 300.00 500.00 700.00
6,500.00 6,775.00 7,050.00 7,325.00 7,600.00

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Example: $O&S as an Anticipated Requirement (2)
Scenario 2: Aggressive tech. improvements gives higher confidence of meeting requirement
k_CDo
k CDo
k_CDi
k CDi
k_WingWt
k WingWt.

-0.10 -0.09 -0.07 -0.06 -0.04 -0.10 -0.09 -0.07 -0.06 -0.04 -0.17 -0.14 -0.11 -0.08 -0.05

k_FusWt
k FusWt.
k_HTWt
k HTWt.
k_$O&S
k $O&S

-0.17 -0.14 -0.11 -0.08 -0.05 -0.16 -0.13 -0.10 -0.07 -0.04 -0.11 -0.09 -0.08 -0.06 -0.05

Overlay Chart Forecast: Ov erlap


P(Achieved-Anticipated)
Frequency Comparison 5,000 Trials Cumulativ e Chart 9 Outliers

.020 1.000 5000


Probability of Satisfying
Achieved .750
$O&S Req. =~50%
.015 $O&S/flthr
$O&S
.010 .500

.005 Anticipated .250


Anticipated Req.
$O&S Req.
.000 .000 0

6,500.00 6,750.00 7,000.00 7,250.00 7,500.00 -600.00 -300.00 0.00 300.00 600.00

Dr. Daniel P. Schrage


Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu
Summary and Conclusions
• Systems Engineering has its roots in Aerospace
• Modern Systems Engineering must reflect the Quality Revolution
• The Five Lean Principles should serve as guiding principles for
a Modern Systems Engineering Approach
• Integrated Product and Process Development (IPPD) required in
the Fuzzy Front End to Establish Value for subsequent principle
implementation: Value Stream, Process Flow, Customer Pull,
and Perfection
• Georgia Tech has developed an IPPD through Robust Design
Simulation (RDS) environment that allows a Probabilistic
Approach for the Fuzzy Front End
• The transition of the IPPD through RDS to a Virtual Stochastic
Life Cycle Design Environment is necessary for full
implementation of the Five Lean Guiding Principles
Dr. Daniel P. Schrage
Georgia Institute of Technology
Atlanta, GA 30332-0150
CASA/CERT/PLMC
www.asdl.gatech.edu

You might also like