systems

Article
Economic Analysis of Model-Based
Systems Engineering
Azad M. Madni * and Shatad Purohit
University of Southern California, Los Angeles, CA 90089-1192, USA; shatadkp@usc.edu
* Correspondence: azad.madni@usc.edu; Tel.: +1-213-821-1001




Received: 26 December 2018; Accepted: 15 February 2019; Published: 20 February 2019


Abstract: In the face of ever-increasing complexity of systems and system development programs,
several aerospace, automotive, and defense organizations have already begun or are contemplating
the transition to model-based systems engineering (MBSE). The key challenges that organizations face
in making this decision are determining whether it is technically feasible and financially beneficial
in the long-run to transition to MBSE, and whether such transition is achievable given budgetary
constraints. Among other cost drivers of this transition, are a new digital infrastructure, personnel
training in MBSE, and cost-effective migration of legacy models and data into the new infrastructure.
The ability to quantify gains from MBSE investment is critical to making the decision to commit to
MBSE implementation. This paper proposes a methodological framework for analyzing investments
and potential gains associated with MBSE implementation on large-scale system programs. To this
end, the MBSE implementation problem is characterized in terms of: system complexity, environment
complexity and regulatory constraints, and system lifespan. These criteria are applied to systems
in twelve major industry sectors to determine MBSE investment and expected gains. Results from
this cost-benefit analysis are used to justify investment in MBSE implementation where warranted.
This approach is generic and can be applied to different sectors for economic evaluation of costs and
benefits and justification of transition to MBSE if warranted.

Keywords: model-based systems engineering; return-on-investment; economic analysis; MBSE

methodology; MBSE tools; document-centric systems engineering

1. Introduction
In the face of ever-increasing complexity of systems and system development programs, several
major aerospace and defense organizations are turning to model-based systems engineering (MBSE) to
reduce system and program complexity. These organizations are specifically interested in determining
whether MBSE is worth the investment and whether the transition to MBSE is affordable given budgetary
constraints. Among other cost considerations, this transition requires a new digital infrastructure and
cost-effective migration of legacy models and data into the new environment [1]. An MBSE investment
also needs to cover costs associated with evaluating candidate MBSE methodologies, infrastructure
implementation/extension, personnel training, model development and verification, model curation,
and configuration management. While investment in MBSE implementation is made in the early
stages of the system life cycle, the gains are invariably realized in the latter stages of the system life
cycle. The factors that contribute to these gains include early defect detection, model and data reuse,
technical and programmatic risk reduction, improved communication, supply chain efficiency, product
line definition, and standards conformance. Therefore, the different industry sectors need the ability
to quantify the benefits of MBSE investment to justify investing in MBSE. Despite the lack of such
quantitative data, some industry sectors (e.g., aerospace, energy, and automotive), that already have
systems engineering (SE) practices in place, have begun to transition to MBSE. Other industry sectors

Systems 2019, 7, 12; doi:10.3390/systems7010012 www.mdpi.com/journal/systems

Systems 2019, 7, 12 2 of 18

(e.g., consumer electronics, healthcare, and construction) that do not have SE practices in place are
turning to SE as the preferred means to manage complexity in their respective markets. Other industries
are beginning to explore possible on-ramps to accelerate MBSE implementation without disrupting their
ongoing operations. However, several industries that are currently following traditional SE practices are
looking for evidence to justify MBSE investment. This recognition provides the motivation for this paper.
This paper is organized as follows. Section 2 presents industry sectors that stand to benefit from
SE practices. Section 3 reviews the limitations of the traditional SE approach. Section 4 discusses the
potential benefits of the MBSE approach a. Section 5 presents an overview of the study approach
and evaluation framework for economic analysis of MBSE implementation in the different sectors.
Section 6 presents an economic analysis of the traditional SE approach. Section 7 presents an economic
analysis of the MBSE approach. Section 8 compares and contrasts the economics of the two approaches
in terms of their investment costs. Section 9 examines MBSE implementation in terms of key problem
characteristics and their impact on MBSE investment and expected gains. Section 10 discusses how
problem characterization of systems in different sectors can be used to gauge the benefits of MBSE
implementation in those sectors, and then applies the approach to those sectors. Section 11 presents
conclusion and discusses future MBSE advances along with their investment cost implications and
expected payback.

2. Industry Sectors that Stand to Benefit from SE

Industries that stand to benefit from systems engineering can be conveniently grouped into twelve
sectors: transportation and mobility, aerospace and defense, industrial equipment, energy and utilities,
architecture and construction, life sciences, high-tech, marine and offshore, financial and business
services, consumer goods and retail, natural resources, and consumer packaged goods and retail.
Table 1. presents the industry sectors and associated systems. The transportation and mobility
sector includes original equipment manufacturers (OEMs) of cars, light and heavy trucks, buses,
trains, motorcycles, racecars, and suppliers. The aerospace and defense industry sector includes
commercial and military aircraft, satellites and exploration systems. The industrial equipment sector
includes specialized manufacturing machinery, industrial robots, machine tools, 3D printers, power
and fluidic equipment, fabricated metal and plastic products, and heavy mobile machinery. The energy,
processes, and utilities sector consists of power generation and distribution systems. The architecture
and construction industry deals with commercial and government infrastructures, buildings and
cities. Life science includes applications of cyber–physical–human systems in pharmaceutical,
biotechnology, medical devices, and patient care. The high-tech industry consists of consumer
electronics, telecommunication, computing and network, technology suppliers, contract manufacturing
services, security, control and instrumentation, and semiconductors. The marine and offshore sector
comprises the shipbuilding industry and water transportation systems such as commercial ships, naval
carriers, offshore vessels, yachts and workboats. The financial and business services sector includes
systems related to banking, insurance and financial markets. The consumer and retail industry sector
encompasses systems related to home, lifestyle, and fashion. The natural resources sector spans
mining, oil, gas, agriculture and forestry industries. The consumer goods and retail sector span the
food and beverage industry, beauty and personal care, household products, packaging suppliers, and
general retailers.
In these industry sectors, there are primarily two types of organizations: (a) organizations in
which SE practices are being followed and who are trying to decide whether to transition to MBSE;
(b) organizations wanting to adopt SE practices. The latter are interested in determining whether to
adopt traditional SE or pursue MBSE. Examples of the former are aerospace, defense, and energy.
Examples of the latter are high-tech industries.
Systems 2019, 7, 12 3 of 18

Table 1. Industry sectors relevant to systems engineering (SE) [2].

Industry Sectors Associated Systems

Transportation and Mobility
Aerospace and Defense
Industrial Equipment
Energy, Processes and Utilities
Architecture and Construction
Life Science
High-Tech
Marine and Offshore
Financial and Business Services
Consumer and Retail
Natural Resources
Consumer Packaged Goods and Retail
Original equipment manufacturers (OEMs) of cars, light and heavy
trucks, buses, trains, motorcycles, racing cars, and suppliers
Commercial and military aircrafts, satellites, and exploration systems
Specialized manufacturing machinery, industrial robots, machine
tools, 3D printers, power & fluidic equipment, fabricated metal &
plastic products, and heavy mobile machinery
Power generation and distribution systems
Commercial and government infrastructures, buildings, and cities
Pharmaceutical, Biotechnology, medical devices, and patient care
Consumer Electronics, telecommunication, computing & network
technology suppliers, contract manufacturing services, security,
control & instrumentation, and semiconductor industry
Ship, yacht & workboat industry, and navy
Banking, insurance and financial markets
Home, lifestyle and fashion industry
Mining, oil, gas, agriculture and forestry industry
Food, beverages, personal care, household products, packaging
suppliers, and general retailers

Systems in the different industry sectors differ in complexity, the characteristics of the operational
environment in which they operate, and their lifespan. For example, aircraft systems have high
complexity, long lifespans, and a strict regulatory environment. On the other hand, consumer
electronics have relatively low complexity, short lifespans, and comparatively a loose regulatory
environment. Since industry sectors differ from each other along these dimensions and the amount
of investment and potential gains can be expected to vary across industry sectors, the MBSE strategy
for each industry sector needs to be addressed on a case-by-case basis (i.e., it needs to be customized).
In the following sections, we examine the different economic factors that impact MBSE implementation
decision in the different industry sectors.

3. Limitations of Traditional (i.e., Document-Centric) Systems Engineering

In the traditional systems engineering approach, thousands of documents are generated during
a system’s life cycle. For large, complicated systems such as aircraft, satellites and power plants,
documents generated during program execution are numerous and include requirements specification,
requirement traceability, master equipment list, concept of operations (ConOps), operational view
diagrams, interface description documents, design structure matrix, test scenarios, test specifications,
and deployment plans [3]. Furthermore, interface control documents (ICDs) by themselves could
comprise multiple documents categorized by the type of interface. For example, mechanical interface,
thermal interface and information interface have separate ICDs to avoid conflicts among individuals
from different disciplines working on the same project. Furthermore, there could be hundreds of
components with each requiring the development of an ICD. Consequently, for complex systems
there could be thousands of documents that need to be created and maintained throughout the
system life cycle. These documents are used by systems engineers during the various stages of the
system development process. Specifically, they help with determining what information needs to be
communicated by one team to another during system development [4]. Since the data represented in
these documents do not have explicit dependencies, a change in one document needs to be reflected
manually in all the other affected documents. This manual process is not only lengthy but also prone
to error. Furthermore, when documents are used to facilitate communication, it is difficult to verify
their completeness and consistency, and to surface conflicting or contradictory information. Finally,
since documents are written in natural language (i.e., sentences and paragraphs), they are not in a
 Systems 2018, 6, x FOR PEER REVIEW 4 of 18
Systems 2019, 7, 12 4 of 18

conflicting or contradictory information. Finally, since documents are written in natural language
(i.e., sentences
suitable and
form for paragraphs),
formal theyand
verification are testing
not in a[4].
suitable
Figureform for formaldepicts
1 graphically verification and testing
traditional [4].
systems
Figure 1 graphically
engineering depicts traditional
and model-based systems engineering and model-based systems engineering.
systems engineering.

(a)
(b)
Figure 1. (a)
Figure Traditional
1. (a) Traditionalsystems
systems engineering; (b)model-based
engineering; (b) model-based systems
systems engineering.
engineering.

4.
4. Promise
Promise of
of Model-Based
Model-Based Systems
Systems Engineering
Engineering
MBSE
MBSE covers
covers several
several important
important but but notnotall
allsystems
systemsengineering
engineeringfunctions functions[1,5].
[1,5]. MBSE
MBSE provides
provides
opportunities
opportunities to capture and harmonize information from multiple disciplines within an
to capture and harmonize information from multiple disciplines within anintegrated
integrated
digital
digital model
model of of the
the system
system [6–8].
[6–8]. Since
Since dependencies
dependencies across across multiple
multiple disciplines
disciplines are are explicitly
explicitly
addressed in the model, a change made in one part of the model
addressed in the model, a change made in one part of the model is automatically reflected in all is automatically reflected in the
all
the impacted parts of the model. This feature makes it possible to maintain
impacted parts of the model. This feature makes it possible to maintain consistent and up-to-date consistent and up-to-date
information
information in in the
the integrated
integrated digital
digital model.
model. In In addition,
addition, completeness,
completeness, consistency,
consistency, traceability
traceability and and
contradiction
contradiction checks
checks can can bebe performed.
performed. As As important,
important, MBSE MBSE enables
enables the the replication
replication of of models
models in in aa
hierarchy. For example, model parameters, interfaces, structure and
hierarchy. For example, model parameters, interfaces, structure and behaviors at the system level behaviors at the system level can
be
canreplicated and integrated
be replicated and integrated at theatsubsystem
the subsystem or theorcomponent
the component level. level.
For theForsame problem
the same with
problem
traditional document-centric systems engineering, a separate document
with traditional document-centric systems engineering, a separate document would have to be would have to be manually
created
manuallyat each
createdlevel at in the level
each hierarchy.
in theInhierarchy.
sharp contrast,
In sharp MBSE allowsMBSE
contrast, the use of models
allows the use from higher
of models
level
fromin hierarchy
higher leveltoinrapidly
hierarchydevelop models
to rapidly at lower
develop levels.atAslower
models important,
levels.documents
As important, are not viewed
documents
as data; rather, they are viewed as data repositories. With MBSE, data
are not viewed as data; rather, they are viewed as data repositories. With MBSE, data can be encoded can be encoded into models,
thereby providing
into models, an opportunity
thereby providing toanintegrate
opportunity the system model across
to integrate life cyclemodel
the system processes,
across andlife
thereby
cycle
promote
processes,reuse.
and thereby promote reuse.
MBSE
MBSE has achieved aa positive
has achieved positiveReturn
ReturnononInvestment
Investment (ROI)
(ROI) for forthe the
US US Submarine
Submarine Fleet.Fleet.
The
The Submarine Warfare Federated Tactical System1 (SWFTS)
Submarine Warfare Federated Tactical System1 (SWFTS) is a rapidly evolving, COTS-based combatis a rapidly evolving, COTS-based
combat
system system
of systemsof systems
(SoS) (SoS)
acquiredacquired
by the by US
the US
Navy Navy forfor deploymentacross
deployment acrossthetheFleet.
Fleet. TheThe SWFTS
SWFTS
program
program integrates the combat system from component systems produced by multiple programs of
integrates the combat system from component systems produced by multiple programs of
record. The Program Executive Office for Submarines practices
record. The Program Executive Office for Submarines practices continuous process improvement in continuous process improvement
in
an an unending
unending quest quest to improve
to improve efficiency
efficiency and deliver
and deliver greater greater
capability capability
to the fleetto despite
the fleet despite
increasing
increasing budget pressures. As part of this quest, in 2011 the SWFTS
budget pressures. As part of this quest, in 2011 the SWFTS SoS engineering team started a three-year SoS engineering team started a
three-year
program to program
transition to transition
from the from the traditional
traditional document-based
document-based systems systems engineering
engineering process
process to
to
model-based systems engineering (MBSE) process utilizing UML and SysML That transition was
model-based systems engineering (MBSE) process utilizing UML and SysML That transition was
successfully
successfullycompleted
completedand and is now
is now producing
producing a positive return return
a positive on investment (ROI) for(ROI)
on investment the customer.
for the
SWFTS
customer. SWFTS is a level-of-effort contract, so the ROI manifests in a combination ofnumber
is a level-of-effort contract, so the ROI manifests in a combination of increased increasedof
baselines developed, improved quality of systems engineering products,
number of baselines developed, improved quality of systems engineering products, and expanded and expanded scope of work
within
scope constant
of work funding. Since thefunding.
within constant transition to SysML-based
Since the transition MBSE, the average number
to SysML-based MBSE, of thebaselines
average
produced on a monthly basis has increased by approximately 30%.
number of baselines produced on a monthly basis has increased by approximately 30%. The number The number of subsystems and
combat system variants
of subsystems and combat integrated
systeminvariants
these baselines
integrated hasinincreased by 60%.has
these baselines Over the sameby
increased time
60%.period,
Over
the complexity of an average baseline, as measured by numbers
the same time period, the complexity of an average baseline, as measured by numbers of of requirements and defined interfaces,
requirements and defined interfaces, has grown at 7.5% per annum. The efficiencies achieved
Systems 2019, 7, 12 5 of 18

Systems 2018, 6, x FOR PEER REVIEW 5 of 18

has grown at 7.5% per annum. The efficiencies achieved through the transition to MBSE have also
through
enabled the the transition
SWFTS to MBSE have
SoS engineering teamalso enabled theadditional
to accomplish SWFTS SoS engineering
tasks team to accomplish
that had previously fallen
additional tasks that had previously fallen below the funding line. One measure of this
below the funding line. One measure of this growing engineering scope is that the number of discrete growing
engineering scope is that the number of discrete systems engineering products produced
systems engineering products produced by the team has increased by over 60%, with most generated by the
team has increased by over
directly out of the SoS model [9].60%, with most generated directly out of the SoS model [9].

5. Study
5. Study Approach
Approach
OurOur overall
overall studystudy approach
approach is on
is based based on creating
creating an evaluation
an evaluation frameworkframework
Figure 2 forFigure 2 for
economic
analysis of MBSE implementation. Inputs to this evaluation framework include: data on MBSE tools in on
economic analysis of MBSE implementation. Inputs to this evaluation framework include: data
theMBSE
form oftools
tool in the form of and
specifications toolreports;
specifications
lessonsand reports;
learned lessons experts
by industry learnedwho by industry
had goneexperts
throughwho
thehad goneof
process through the process
implementing of implementing
MBSE; and existing MBSE;
literatureandonexisting literature
high-level on high-level
economic economic
justification of
MBSE. The literature on MBSE tools helped with understanding the amount of training needed in of
justification of MBSE. The literature on MBSE tools helped with understanding the amount
training needed systems
general-engineering, in general-engineering,
engineering, modeling systems engineering,
languages, modeling The
and methodology. languages,
MBSE tooland
methodology.
specifications andThe MBSE
reports tool specifications
provided information on andthe reports
amount provided
of effort,information on the
cost, and talent amount
required to of
implement MBSE. In addition, keynote talks from government and industry experts were used as aand
effort, cost, and talent required to implement MBSE. In addition, keynote talks from government
industry
source experts
of lessons werealong
learned used with
as aanecdotal
source ofinformation
lessons learned alongtowith
pertaining anecdotal
investments andinformation
returns
pertaining to investments and
associated with traditional SE and MBSE.returns associated with traditional SE and MBSE.

Figure
Figure 2.
2. Model-based systems
Model-based systems engineering
engineering (MBSE)
(MBSE) Economic
Economic Analysis
Analysis Approach.
Approach.

TheThe
overall methodology
overall methodologyaccounts for interest
accounts rates, inflation
for interest rates, rates and risk
inflation premium
rates and risk in evaluating
premium in
viability of MBSE implementation. The evaluation framework comprises:
evaluating viability of MBSE implementation. The evaluation framework comprises: SE life cycle SE life cycle processes;
guidelines
processes; for guidelines
tailoring SEfor lifetailoring
cycle processes
SE life for different
cycle industries;
processes key MBSE
for different implementation
industries; key MBSE
problem characteristics such as system complexity, environment complexity,
implementation problem characteristics such as system complexity, environment complexity, and system lifespan thatand
help assesslifespan
system MBSE value proposition
that help for different
assess MBSE industries; parametric
value proposition for different cost curves for
industries; traditionalcost
parametric
systems engineering and MBSE; popular MBSE tools and their coverage of
curves for traditional systems engineering and MBSE; popular MBSE tools and their coverage the SE life cycle; process
of the
Design Structure Matrix (DSM)-based approaches for process analysis and
SE life cycle; process Design Structure Matrix (DSM)-based approaches for process analysis process sequencing; andand
database
process of sequencing;
lessons learned andon major government
database programs.
of lessons learned on Analysis of system life
major government cycle processes
programs. Analysisis of
necessary to ensure that the framework applies to a wide range of industry sectors.
system life cycle processes is necessary to ensure that the framework applies to a wide range of Since each industry
sector has somewhat
industry different
sectors. Since processes,
each industry analyzing
sector processesdifferent
has somewhat from theprocesses,
perspectives of sequence,
analyzing processes
dependencies, and parallelism and evaluating their temporal characteristics
from the perspectives of sequence, dependencies, and parallelism and evaluating their is essential for calculating
temporal
thecharacteristics
net present worth of investments
is essential and returns.
for calculating the net Forpresent
example, there
worth ofcould be several
investments andreal-world
returns. For
scenarios
example, in which the transition
there could be several to real-world
MBSE has already
scenarios begun; in such
in which cases, the framework
the transition to MBSE has can be
already
applied to ensure maximum returns on investments already made.
begun; in such cases, the framework can be applied to ensure maximum returns on investments
In our made.
already study, we parameterized the MBSE implementation problem using the system’s internal,
externalInandourtemporal
study, we(i.e., lifespan) characteristics.
parameterized These characteristics
the MBSE implementation problem were selected
using becauseinternal,
the system’s they
external and temporal (i.e., lifespan) characteristics. These characteristics were selected because they
enable comparison of different industries in terms of the value that MBSE implementation can
 Systems2019,
Systems 2018,7,6,12x FOR PEER REVIEW of18
6 of 18

provide. With this parameterization, we discovered that there are some industry sectors which may
enable comparison
not benefit of different
significantly from industries in terms of theinvalue
MBSE implementation theirthat MBSE
current implementation
circumstance, but can
withprovide.
rapidly
With this parameterization, we discovered that there are some industry
changing internal and external characteristics of systems in those sectors, MBSE implementation sectors which may not benefit
significantly
may well turn fromoutMBSE to be implementation
beneficial in in thetheir current
future. These circumstance,
three MBSEbut with rapidly changing
implementation problem
internal
characteristics also provided insights into how much to spend on MBSE implementation inturn
and external characteristics of systems in those sectors, MBSE implementation may well the
out to be beneficial
different in the future.
industry sectors These
in relative three With
terms. MBSEthis implementation
parameterization, problem we werecharacteristics also
able to create
provided insights
clusters of industryintosectors,
how much whichto spend on MBSE implementation
could potentially benefit from MBSE in theimplementation.
different industry sectors
Applying
inthis
relative
MBSE implementation problem characterization to each sector, it became apparent that which
terms. With this parameterization, we were able to create clusters of industry sectors, not all
could potentially
sectors derived the benefit
samefrom MBSE
benefit fromimplementation. Applying This
MBSE implementation. this MBSE implementation
was expected because problem
based on
characterization
the systems in to each
each sector,the
sector, it became apparent that notproblems
MBSE implementation all sectorsare derived the same
different. The benefit
clusteringfromof
MBSE implementation. This was expected because based on the systems
industry sectors allowed us to address MBSE implementation for each cluster as a whole. Thus, a in each sector, the MBSE
implementation
MBSE approachproblems employed arein different.
one industryThe clustering of industry
sector within a clustersectors
can allowed
be applied us to
to address MBSE
other industry
implementation for each
sectors in the same cluster. cluster as a whole. Thus, a MBSE approach employed in one industry sector
withinEvaluation
a cluster can be applied to other industry sectors in the same cluster.
metrics in Figure 2 are associated with the variables that we want to estimate using
Evaluation metrics
the methodology. in Figure
The metrics 2 are associated
encompass with the
both generic variables
measures such thatas we want
effort, to estimate
cost, and cycleusingtimes
the
as well as domain-specific measures. The results of the evaluation are primarily in the formasof
methodology. The metrics encompass both generic measures such as effort, cost, and cycle times
well as domain-specific
graphs and charts. These measures. The results
results contribute of the evaluation
actionable insights into are when,
primarily where in the
andform
to whatof graphs
degree
and charts. MBSE
to adopt These results contribute actionable
to economically justify MBSE insights into when,
investments andwhere and to
returns forwhat degree to system
a particular adopt
MBSE to economically justify MBSE
development project, or a family of projects. investments and returns for a particular system development
project, or a family
Figure of projects.
3 presents the key steps in the overall methodology for MBSE economic analysis. It
Figure 3 presents
begins with characterizing the key the
stepsMBSE
in the implementation
overall methodology problemfor MBSE
in termseconomic analysis.
of system It begins
complexity,
with characterizing
environment the MBSE
complexity, andimplementation
system lifespan. problem
These in terms of are
attributes system complexity,
applied environment
to different industry
complexity, and systemtolifespan.
sectors are considered produce key These attributes areSystem
characteristics. applied to different
evaluation industry sectors
is performed are
using these
considered to produce key characteristics. System evaluation is performed
characteristics to produce the characteristics of each sector. These evaluation results along with using these characteristics
tobudget
produce the characteristics
constraints are used to of each
perform sector. These evaluation
cost-benefit analysis.results along with
Investments, budget
payback and constraints
their time
are used
frame toused
are perform cost-benefit
in evaluating analysis.
benefits and Investments, payback
costs. If the benefits and their
justify the costtime of frame
MBSE are used in
investment,
evaluating
the next steps in the methodology are to plan, schedule and cost the transition to MBSE.inAfter
benefits and costs. If the benefits justify the cost of MBSE investment, the next steps the
methodology
making sure are thattothe
plan,
costsschedule
are within andbudget
cost theconstraints,
transition to theMBSE.
plan is After making
executed. Onsure
the that
other the costsif
hand,
are
the benefits do not justify the cost, the organization stays with the status quo (i.e., traditionalthe
within budget constraints, the plan is executed. On the other hand, if the benefits do not justify SE
cost, the organization stays with the status quo (i.e., traditional SE practices).
practices).

Figure 3.3.MBSE
Figure MBSEeconomic analysismethodology:
economic analysis methodology: process
process flow.flow.
Systems 2019, 7, 12 7 of 18
Systems 2018,
Systems 6, x6,
2018, FOR PEER
x FOR REVIEW
PEER REVIEW 7 of7 18
of 18

6.
6. Economic
Economic Analysis
Analysis
6. Economic of
of Traditional
Analysis Traditional System
System
of Traditional Engineering
Engineering
System Approach
Approach
Engineering Approach
Traditional
Traditional
Traditional systems
systems
systems engineering
engineering
engineering approaches
approaches
approaches are
arearedocument-centric,
document-centric,
document-centric, and document-centric
andanddocument-centric
document-centric
approaches
approaches
approaches tend
tend to
tend have
to to hidden
havehave costs.
hidden
hidden To
costs. illuminate
costs. To To the
illuminateeconomics
illuminate thethe of document-centric
economics
economics of of SE approaches,
document-centric
document-centric SE SE
we present
approaches, anweimportant
present curve
an in
importantFigure 4.
curve This
in curve
Figure explains
4. This the
curve percentage
explains
approaches, we present an important curve in Figure 4. This curve explains the percentage cost for cost
the for each
percentage life cycle
cost for
stage
each forlife
life
each systems
cycle stage
cycle engineering
forfor
stage systems projects
systems based on
engineering
engineering aprojects
statistical
projects based
basedanalysis
on on conducted
a statistical on the
analysis
a statistical US Department
conducted
analysis conducted on on
thethe
of Defense
USUS Departmentprograms,
Department as
of Defense reported
of Defense by
programs,
programs, the Defense
as asreported Acquisition
reported by by
thethe University
Defense
Defense [3].
Acquisition From
Acquisition these
University
Universityfigures,
[3].[3]. we
FromFrom
can
thesededuce
these that
figures,
figures, investments
wewe cancan in
deduce the
deduce early
thatthat phases
investments of
investmentssystem
in in life
thethe cycle,
early
early are comparatively
phases
phasesof ofsystem
systemlower
lifelifethan
cycle, they
areare
cycle,
are in the later phases.
comparatively
comparatively lower
lower Furthermore,
thanthanthey theyareconcept,
arein inthedesign
thelater and
later development
phases.
phases. phase cumulatively
Furthermore,
Furthermore, concept,
concept, designaccount
design andand
for 20% of thephase
development
development total
phaselife cycle cost. account
cumulatively
cumulatively The remaining
account forfor
20% 80%
20% ofofthe
of the thetotal
totalcost occurs
lifelife
cycle incost.
cost.
cycle the production,
The remaining
The remaining testing,
80%80% of of
operations,
thethe
costcost support,
occurs
occurs in the maintenance,
production,
in the production, and
testing,disposal
testing, phases.
operations,
operations, support,
support, maintenance,
maintenance, andand
disposal
disposal phases.
phases.

60 60
Percentage Life Cycle Cost Against Time

Percentage Life Cycle Cost Against Time

50 50

40 40

30 30

20 20

10 10

0 0
Concept
Concept Design
Design Develop
Develop Prod/Test Operations
Prod/Test Operations
through
through
disposal
disposal
TimeTime

Figure 4. Committed
Figure
Figure 4.Committed
4. life
Committedlifecycle
life cost
cycle
cycle against
costcost time
against
against time forfor
time each
for
each stage
each
stage stage
[3]. [3].[3].

From
From an
Froman analysis
analysis
an analysis of
of cost
cost in
in the
of cost the system
system
in the system life cycle
lifelife
cycle
cyclefor
forfortraditional
traditional
traditional SE, we
SE,SE,
wewe can create
cancan
create an approximate
an an
create approximate
approximate
cost
cost profile
profile
cost for
profile SE
forfor initiatives
SE SEinitiatives
initiatives(Figure
(Figure 5). In
5). 5).
(Figure this
In In figure,
thisthis
figure,
figure,the ordinate
thethe
ordinate
ordinate (i.e.,
(i.e., the vertical
thethe
(i.e., vertical axis)
axis)
vertical represents
represents
axis) represents
normalized
normalized
normalizedsystem
system
systemlife
lifecycle
cycle
life cost,
cycle and
cost,cost,the
and and abscissa
the (i.e., (i.e.,
abscissa
the abscissa the(i.e.,
horizontal
thethe axis) represents
horizontal
horizontal axis) time. As
represents
axis) represents shown
time. AsAs
time.
in this
shown figure,
shownin in SE
thisthis investment
figure,
figure,SE SE is relatively
investment
investment low in thelow
is relatively
is relatively early
low instages
thethe
in ofearly
early the system
stages of life
stages cycle.
thethe
of system Specifically,
systemlifelife
cycle.
cycle.
for conceptual
Specifically, fordesign
Specifically, for and preliminary
conceptual
conceptual design
design design,
andand investments
preliminary
preliminary design, areinvestments
design, relatively
investments low, but
areare for detailed
relatively low,
relatively low,design,
butbut
forfor
manufacturing,
detailed design,
detailed production,
design,manufacturing, and operation
manufacturing, production,
production,theandinvestment
operation
and operation increases
thethe sharply.increases
investment
investment increasessharply.
sharply.

Figure
Figure5.5.Traditional
Figure 5. Traditional
Traditional SEcost
SE cost
SE curve.
cost curve.
curve.

7.
7. Economic
Economic Analysis
Analysis
7. Economic of
of MBSE
Analysis MBSE Approach
Approach
of MBSE Approach
For MBSE
ForFor
MBSE
MBSE projects,
projects, it
it is
projects, isit difficult
difficult to
to find
is difficult find
to cost
cost
find curves
curves
cost similar
similar
curves to
to those
similar those
to for traditional
forfor
those traditional SE in
in the
SE SE
traditional thethe
in
literature.
literature. This
This
literature. is
This understandable
is understandable
is understandable because
because
becauseMBSE
MBSE
MBSEis relatively
is relatively new
new
is relatively newand,
and, therefore,
therefore,
and, therefore,economic
economic analysis
analysis
economic analysis
data
data for
data MBSE
forfor
MBSE
MBSEinitiatives
initiatives is
is not
initiatives not available.
available.
is not Therefore,
Therefore,
available. Therefore,in our
in study
ourour
in study we
wewe
study took
took aa different
took different approach
approach
a different approachto
to to
approximate
approximatecost curves
cost forfor
curves MBSE
MBSEprojects. Specifically,
projects. wewe
Specifically, combined
combined historical data
historical (evidence)
data (evidence)
from a space
from mission
a space with
mission current
with MBSE
current coverage
MBSE coverageof the system
of the lifelife
system cycle processes
cycle processesusing thethe
using ISOISO
Systems 2019, 7, 12 8 of 18

approximate cost curves for MBSE projects. Specifically, we combined historical data (evidence) from
a space mission with current MBSE coverage of the system life cycle processes using the ISO 15288
process standard. The source of the former was a keynote address at the 2016 No Magic World
Symposium. The keynote speaker spoke specifically about the scale of the investment to implement
MBSE to “save” NASA’s Mars mission with an overall project cost of $327 M [10]. According to him,
an initial MBSE investment of $5–10 M from the project budget could have immeasurably helped
that project. We gathered additional evidence from a project manager from the industrial equipment
sector. He indicated that on a $40 M equipment development project, basic MBSE implementation took
under 1% additional investment to cover requirement management, functional allocation, traceability,
document generation, verification, and validation. However, his team did not employ MBSE to cover
system analysis, simulation and detailed integration with the design and development processes of the
system life cycle. From these sources, we were able to make an educated inference about the relative
cost of MBSE investment for a system development project. Even so, it is not possible to identify how
such costs should be distributed (i.e., how investments should be made) over the system life cycle.
Therefore, to analyze MBSE investment over the system life cycle. Specifically, we examined MBSE tool
coverage of system life cycle processes. According to ‘System Life Cycle ISO/IEC/IEEE 15288 2015(E):
Processes in System Life Cycle,’ there are four major processes that characterize the system life cycle:
technical processes, technical management processes, organizational project-enabling processes, and
agreement processes [11]. These four major processes consist of 30 sub-processes. Each sub-process
consists of activities, and each activity consists of tasks. We evaluated the coverage of MBSE tools for
system life cycle processes for each task using four popular MBSE tools (e.g., No Magic Cameo Systems
Modeler, IBM Rational Rhapsody, Eclipse Papyrus and Eclipse Capella) that are representative of the
life cycle coverage provided by MBSE tools. The data associated with SE life cycle process coverage by
MBSE tools provides important insights into the temporal spend profile for MBSE investment needed
to cover the system life cycle.

7.1. System Life Cycle Processes

Our study evaluated the selected MBSE tools against ISO 15288 processes (which define the
system life cycle and coverage requirements). Once again, we decomposed the processes into activities
and activities into tasks (Table 2). There are 14 technical processes, 54 activities and 240 tasks that
resulted from the decomposition [11]. It was rather straightforward to determine whether or not the
exemplar MBSE tools addressed each task. Based on the total number of tasks in a given process and
the number of tasks covered by the MBSE tool, we were able to calculate the percentage coverage by
each tool of the corresponding technical process. Since technical processes are the primary focus of
MBSE tools, our study focused mostly on technical processes. The remaining processes (i.e., technical
management, organizational project enabling processes and agreement processes) were analyzed using
system life cycle process model Figure 6, and processes dependency structure matrix (DSM) shown in
Figure 7a,b.
 Systems 2018, 6, x FOR PEER REVIEW 9 of 18
Systems 2019, 7, 12 9 of 18

Table 2. System life cycle ISO/IEC/IEEE 15288 2015(E): Processes in system life cycle [11].
Table 2. System life cycle ISO/IEC/IEEE 15288 2015(E): Processes
Technical Management in system life cycle [11].
Organizational Agreement
Technical Processes
Processes Project-Enabling Processes Processes
Technical Management Organizational
Technical
1. Business orProcesses
Mission Analysis 1. Project Planning Process Project-Enabling
1. Life Cycle Model Agreement Processes
1. Acquisition
Processes Processes
Process 2. Project Assessment and Management Process Process
2. Stakeholder Needs & Control Process 2. Infrastructure 2. Supply Process
1. Business or Mission 1. Project Planning Process 1. Life Cycle Model 1. Acquisition Process
Requirement Definition
Analysis Process 3. Decision Management Management
Management Processes
Process
2. Project Assessment and 2. Supply Process
2. Process
Stakeholder Needs & Process
Control Process 2. 3.Infrastructure
Portfolio Management
3. System Requirements
Requirement 3. 4. Risk Management
Decision Process Processes
Management
Definition Process
Definition Process Management
Process Process 3. 4.Portfolio
Human Resource
3. System Requirements
4. Architecture Definition 4. RiskConfiguration
5. Management Process Management
Management Process
Process
Definition Process 5. Configuration 4. Human Resource
Process Management Process 5. Quality Management
4. Architecture Management Process Management Process
5. Design Definition
Definition Process Process 6. 6. Information
Information 5. Process
Quality
6. System
5. Design Analysis
DefinitionProcess
Process Management
Management Process
Process 6.Management
KnowledgeProcess
Management
7. Implementation
6. Process
System Analysis Process 7. 7. Measurement
Measurement Process 6.
Process Process
Knowledge
7. Implementation
8. Integration Process
Process 8. 8. Quality
Quality Assurance
Assurance Process Management Process
8. Integration Process
9. Verification Process Process
9. Verification Process
10. Transition Process
10. Transition Process
11. Validation Process
11. Validation Process
12. Operation
12. Operation Process
Process
13. Maintenance
13. Process
Maintenance Process
14. Disposal
14. Disposal Process
Process

Figure
Figure6.6.Simulink modelofofsystem
Simulink model systemlifelife cycle
cycle processes.
processes.
Systems 2019, 7, 12 10 of 18
Systems 2018, 6, x FOR PEER REVIEW 10 of 18

(a)

(b)
Figure7.7.(a)
Figure (a)Processes
Processes dependency
dependency structure
structure matrix;
matrix; (b) sequenced
(b) sequenced processes
processes dependency
dependency structure
matrix processes.
structure matrix processes.

Our
Our task
task evaluation here is
evaluation here is based
based on on research
research papers
papers from
from the
the SE
SE literature,
literature, our
our hands-on
hands-on
experience
experience with MBSE tools, available MBSE tool documentation, and models that we developed
with MBSE tools, available MBSE tool documentation, and models that we developed asas
needed to complete the evaluation. Our study accounted for plugins which allow these
needed to complete the evaluation. Our study accounted for plugins which allow these MBSE toolsMBSE tools to
integrate with third-party tools related to requirement engineering, PLM, CAD integration,
to integrate with third-party tools related to requirement engineering, PLM, CAD integration, analysis,
simulation, design optimization,
analysis, simulation, real-time and
design optimization, embedded
real-time development,
and embedded publishing,
development, reviewing and
publishing,
collaboration [12,13]. Figure 8a presents the percentage coverage of technical processes
reviewing and collaboration [12,13]. Figure 8a presents the percentage coverage of technical by MBSE tools.
Figure 8b presents
processes by MBSEan MBSE
tools. cost 8b
Figure curve.
presents an MBSE cost curve.
Systems 2019,
Systems 7, 12
2018, 6, x FOR PEER REVIEW 11 of 18
11 of 18

Business or mission…
Stakeholder needs or…
System requirement…
Architecture definition
Design definition
System analysis
verification
validation
0% 20% 40% 60% 80%

Papyrus Rhapsody Capella Cameo

(a) (b)
Figure
Figure 8. Percentage
8. (a) (a) Percentage coverage
coverage of system
of system life cycle
life cycle processes
processes by MBSE
by MBSE tools;
tools; (b) MBSE(b) cost
MBSE curve.
cost curve.
7.2. Sequencing System Life Cycle Processes
7.2. Sequencing System Life Cycle Processes
After quantifying the coverage provided by MBSE tools of system life cycle processes,
After quantifying
we identified where these thetools
coverage provided by
are employed MBSE
in the tools life
system of system
cycle. life
To cycle processes,
this end, we
we created a
Simulink ®
identified whereofthese
Model all 30tools are employed
processes Figure 6inand
thetheir
system life cycle.
interfaces To the
using this processes
end, we created a in
identified
the Simulink
® Model of all 30 processes Figure 6 and their interfaces using the processes identified in the
INCOSE SE Handbook [3]. We then converted this model to a Processes Dependency Structure
INCOSE SE Handbook
Matrix (DSM) shown in Figure [3]. We then converted
7a. From this DSM,
the Processes modelittobecame
a Processes Dependency
possible Structure
to evaluate the parallel,
Matrix (DSM) shown in Figure 7a. From the Processes DSM, it became possible to evaluate the
sequential, and coupled processes Figure 7b.
parallel, sequential, and coupled processes Figure 7b.
In our study, we also created an Excel sheet with Visual Basic program to manipulate Process DSM,
In our study, we also created an Excel sheet with Visual Basic program to manipulate Process
andDSM,
sequence processes in accord with their coupling, dependency, and parallelism. Figures 6 and 7a,b
and sequence processes in accord with their coupling, dependency, and parallelism. Figures 6,
present system life cycle
and 7a,b present process
system model,
life cycle Process
process DSM,Process
model, and sequenced
DSM, andProcess
sequencedDSM,Process
respectively.
DSM,
respectively.
7.3. Cost Curves for Systems Engineering Programs with MBSE Approach
7.3. Costsequencing
After Curves for Systems Engineering
the DSM, it becamePrograms with
evident MBSE
that Approach
MBSE tools today are focused primarily on the
front end of the
After system life
sequencing thecycle.
DSM,Furthermore, for SE
it became evident projects
that MBSEthat
toolsuse an MBSE
today approach,
are focused projects
primarily on can
be expected
the front toendmake higher
of the systeminvestment
life cycle. in conceptual for
Furthermore, design and preliminary
SE projects that use an design
MBSEstages. In these
approach,
projects
upfront can be expected
engineering phases,toconsiderable
make higherinvestment
investmentgoes
in conceptual design and design
into the preliminary preliminary
stage.design
Figure 8a
stages.percentage
indicates In these upfront
coverageengineering
of MBSEphases,
tools forconsiderable
system life investment goes into
cycle processes. the preliminary
In addition, with MBSE,
substantial cost savings can be expected during the latter stages of design. Since thecycle
design stage. Figure 8a indicates percentage coverage of MBSE tools for system life MBSEprocesses.
approach is
In addition,
likely to reducewiththe MBSE,
number substantial
of defectscost savings
in latter can beofexpected
stages design, during the that
it follows latterthe
stages
costofindesign.
these later
Since the MBSE approach is likely to reduce the number of defects in latter stages of design, it
stages will be significantly lower in comparison to the traditional SE approach. Figure 8b presents an
follows that the cost in these later stages will be significantly lower in comparison to the traditional
approximate cost curve associated with the MBSE spend profile. As seen in this figure, MBSE cost is
SE approach. Figure 8b presents an approximate cost curve associated with the MBSE spend profile.
greater in conceptual and preliminary design phases. In fact, the majority of the cost associated with
As seen in this figure, MBSE cost is greater in conceptual and preliminary design phases. In fact, the
MBSE implementation
majority can be expected
of the cost associated with MBSEto be in the preliminary
implementation can bedesign phase.
expected to be in the preliminary
design phase.
8. Economics of MBSE versus Traditional SE
8.The
Economics of MBSE
comparison versus Traditional
of investments SE MBSE and traditional (i.e., document-centric) SE is
between
depictedThein Figure 9a. The
comparison of blue curve indicates
investments betweenthe MBSEcostand
of MBSE projects,
traditional (i.e.,while the red curveSE
document-centric) indicates
is
the depicted
traditional
in SE project
Figure 9a. cost across
The blue the system
curve indicateslifethe
cycle.
cost At
of the
MBSEconceptual and preliminary
projects, while the red curve design
indicates
stages, MBSEthe traditional
projects requireSEgreater
projectinvestment
cost across thanthe traditional
system life SEcycle. At the
projects. conceptual
However, andlatter
in the
preliminary
stages of designdesign
MBSEstages, MBSE
projects projects
require require greater
substantially lower investment
investment than traditional
than SE projects.
traditional SE projects.
For However,
economicin the latter stages
justification, of design
the net presentMBSE
worthprojects
(NPW)require substantially
of investment lower
should investment
be strictly lessthan
than the
traditional SE projects. For economic justification, the net present worth (NPW)
NPW of gains resulting from the MBSE approach in the system life cycle. Or, the net present worth of of investment
should be strictly less than the NPW of gains resulting from the MBSE approach in the system life
savings should be greater than the net present worth of a predefined MBSE investment earlier in the
life cycle. Figure 9b presents the difference in investments between MBSE and traditional SE and the
expected savings from MBSE implementation.
 Systems 2018, 6, x FOR PEER REVIEW 12 of 18

cycle. Or, the net present worth of savings should be greater than the net present worth of a
predefined MBSE investment earlier in the life cycle. Figure 9b presents the difference in
investments
Systems 2019, 7, 12 between MBSE and traditional SE and the expected savings from MBSE 12 of 18
implementation.

(a) (b)
Figure9. 9.
Figure (a)(a) Comparison
Comparison of between
of costs costs between traditional
traditional SE and
SE and MBSE; MBSE; (b)
(b) difference difference and
in investments in
investments
savings betweenand savings SE
traditional between traditional SE and MBSE.
and MBSE.

As
Asnoted
notedearlier,
earlier,withwith MBSE,
MBSE, organizations invariably spend
organizations invariably spend more
moremoney
moneyearly earlyininthe
thelifecycle
lifecycle
and when making upfront investment. Due to the time value of
and when making upfront investment. Due to the time value of money, additional expenditures money, additional expenditures early
in the system life cycle should be considerably less than the gains from
early in the system life cycle should be considerably less than the gains from the latter stages in the the latter stages in the life
cycle phase.
life cycle Discounting
phase. Discounting andand converting
converting gainsgainsinto present
into present worth
worth provides
provides ananaccurate
accurate measure
measureof
return-on-investment
of return-on-investment (ROI).
(ROI). Clearly,
Clearly,gains
gainsshould
shouldbe besignificantly
significantly more
more than investments
investmentsto tomake
make
MBSE
MBSEimplementation
implementation successful.
successful. From From approximate
approximate calculations
calculationson onsample
sampleprojects,
projects,the
theNPW
NPWofof
traditional
traditionalSE SEprojects
projectsand and MBSE
MBSE projects
projects are
are found to be comparable.
However,
However,there there exists
exists aa tipping
tipping point
point in MBSE implementation,
implementation,where wherethe theNPW
NPWof ofinvestment
investment
exceeds the
exceeds the gains from from MBSE.
MBSE.This Thispoint
pointcould
could happen
happen because
because of excess investment
of excess investmentearlyearly
in theinlife
the
cycle,
life or lower
cycle, or lower than expected
than expected gains laterlater
gains in thein thelifecycle. TheThe
lifecycle. ROIROIalsoalso
dependsdependson the timing
on the of
timing
ofinvestments
investments and and gains. TheThe
gains. investment
investment NPW NPW for traditional
for traditionalSE andSE MBSE
and MBSE are calculated from two
are calculated from
equations.
two equations.The equation
The equation to calculate Investment
to calculate Investment NPWNPW approximation
approximation for Traditional SE is SE
for Traditional P =is
−k
P = ∑𝐹nk=01F+ k ( i
1 + i ≈
) ≈
36.6597 C.
36.6597 The C. equation
The to
equation calculate
to Investment
calculate NPW
Investment approximation
NPW for
approximation MBSE for
n −k
is P =is P = ∑
MBSE 𝐹 k=10 + Fki(1 +≈i)34.7564
≈ 34.7564 C; where,
C; where, C is assumed
C is assumed to betoa be a scaling
scaling constant,
constant, n is nnumber
is numberof
of time interval for investments considered to be 10 from the cost curves,
time interval for investments considered to be 10 from the cost curves, i is overall interest rate for i is overall interest rate for
time
timeinterval
interval considered
considered to be 3.5%, 3.5%, andand 𝐹Fk isisinvestment
investment made after kk time time intervals
intervalsdepicted
depictedininthethe
cost
costcurves
curvesas asnormalized
normalized systemsystem life life cycle cost.
Because
Becauseapproximate
approximate NPW NPW investment for traditional traditional SE SE and
andMBSE
MBSEare arecomparable,
comparable,the theresults
results
ininthis
thiscase
caseare
areinconclusive
inconclusive relative
relative to to
which
which is the preferred
is the preferredoption. ThisThis
option. recognition drives
recognition the need
drives the
need
for for a deeper
a deeper analysis analysis of factors
of factors related related
to MBSE to MBSE investment
investment and gains.
and gains.

9.9.Economic
Economic Analysis
Analysis of
of MBSE
MBSE Implementation
AsAsdiscussed
discussedininthe previous
the previous section, SE initiatives
section, that that
SE initiatives employ an MBSE
employ approach
an MBSE requirerequire
approach greater
upfront
greater investment (i.e., in the(i.e.,
upfront investment earlier stages
in the of the
earlier system
stages of life
the cycle)
systemthan
lifeneeded with needed
cycle) than traditional
withSE.
However, such initiatives can be expected to produce greater gains in the latter stages
traditional SE. However, such initiatives can be expected to produce greater gains in the latter stages of the system life
cycle.
of theTherefore,
system life ancycle.
analysis of factors
Therefore, anrelated to early
analysis investment
of factors relatedintoMBSE
early and factors related
investment in MBSE to gains
and
in the latter
factors stages
related from in
to gains MBSE can be
the latter expected
stages fromtoMBSE
provide
canuseful insights
be expected to for economic
provide justification
useful insights forof
economic
MBSE justification of
implementation. MBSE
Figure 10 implementation. Figure
presents key factors 10 presents
related to MBSE key factors related
investments to MBSE
and gains.
investments and gains.
 Systems
Systems 2018,
2019, 6, x FOR PEER REVIEW
7, 12 1313
of of
18 18

Figure
Figure 10. MBSE
MBSEfactors
factorsrelated
related to investments
to investments and gains.
and gains.

9.1. Factors
9.1. Related
Factors to to
Related MBSE Investment
MBSE Investment
MBSE
MBSEinvestment
investmentcovers coversa anumbernumberofofcosts costsincluding
includingthe thecostcostofofMBSE
MBSEprocess
processdefinition,
definition,
infrastructure cost, training cost, and model-related costs. Each
infrastructure cost, training cost, and model-related costs. Each cost is described next. cost is described next.
Process
Processdefinition
definitioncost costis isa akeykeyconsideration
considerationinincalculating
calculatingthe thecost
costof ofadopting
adoptingananMBSE MBSE
methodology for organization-wide implementation. MBSE
methodology for organization-wide implementation. MBSE process definition depends process definition depends on the MBSE on the
methodology
MBSE methodology selected. There are several
selected. There MBSE methodologies,
are several such as INCOSEsuch
MBSE methodologies, Object-Oriented
as INCOSE
Systems Engineering Method (OOSEM), Object-Process Methodology
Object-Oriented Systems Engineering Method (OOSEM), Object-Process Methodology (OPM), (OPM), IBM Rational UnifiedIBM
Process
Rational Unified Process for Systems Engineering (RUP SE) for Model-/Driven Vitech
for Systems Engineering (RUP SE) for Model-/Driven Systems Development (MDSD), Systems
Model-Based
Development System (MDSD),Engineering
Vitech (MBSE)
Model-Based Methodology,
System JPL State Analysis
Engineering (MBSE) (SA)Methodology,
[14]. Implementing a
JPL State
particular
Analysismethodology and generatingamodels
(SA) [14]. Implementing usingmethodology
particular that methodology and have differentmodels
generating costs associated
using that
with them (i.e., not all MBSE methodologies cost the same).
methodology have different costs associated with them (i.e., not all MBSE methodologies cost the
Infrastructure cost is another cost category. Infrastructure cost consists of licenses, equipment,
same).
environments, processing,
Infrastructure cost isand collaboration.
another Infrastructure
cost category. cost forcost
Infrastructure this consists
study is assumed
of licenses,to be specific
equipment,
to environments,
projects. In general, infrastructure cost is incurred at the enterprise
processing, and collaboration. Infrastructure cost for this study is assumed level, but that can be shared
to be
among projects to quantify costs specific to each project.
specific to projects. In general, infrastructure cost is incurred at the enterprise level, but that can be
Training
shared amongcost projects
is a distinct category.
to quantify It involves,
costs specific training on tools, training on modeling languages,
to each project.
and training
Training cost is a distinct category. It involves, trainingcost
employees in systems engineering and MBSE. Training on involves the cost for
tools, training on learning
modeling
curves and organization level resistance to change.
languages, and training employees in systems engineering and MBSE. Training cost involves the
costModel-related
for learningcosts curvesare and
a major source of level
organization costs.resistance
Model development
to change. efforts include identifying the
goal, purpose, and scope
Model-related costs of
arethe model,source
a major improving model
of costs. Modelcapabilities,
development defining the intent
efforts include ofidentifying
use, and
configuring the model for the intended number of users. Building
the goal, purpose, and scope of the model, improving model capabilities, defining the intent federated, centralized or hybrid
of use,
models have different costs [4]. The scale of the model is also
and configuring the model for the intended number of users. Building federated, centralizeda cost modifier. Unifying the modelor
format
hybrid hasmodels
its ownhave cost different
[10]. Maintaining
costs [4]. model
The scaleconsistency
of the modelis a source
is alsoofacosts. Building models
cost modifier. Unifying arethe
not enough; model verification needs to be performed at each stage
model format has its own cost [10]. Maintaining model consistency is a source of costs. Building of model development to ensure
model
models correctness
are not and credibility
enough; model [4].verification
Identifying criteria
needs to of model verification
be performed at and
eachdetermining
stage of model the
right level of model
development to abstraction
ensure model havecorrectness
their own costs and [1]. Identifying
credibility [4].opportunities
Identifying of model of
criteria trading
model
and executing it in the right context also requires effort. In the case
verification and determining the right level of model abstraction have their own costs [1]. of MBSE, an increase in the number
of Identifying
stakeholdersopportunities
working on central of model digital models
trading andmakes
executingthe models
it in themorerightprone
context to errors and require
also requires effort.
model curation efforts. Model curation consists of gauging model
In the case of MBSE, an increase in the number of stakeholders working on central digital characteristics, selecting models
models
formakes
implementation,
the models more and devising
prone to modelerrors andpolicies [4]. model
require Configuration management
curation efforts. Model efforts
curation include
consists
maintaining ownership, managing data rights and distribution of rights,
of gauging model characteristics, selecting models for implementation, and devising model policies re-use or copyrights.
[4]. Configuration management efforts include maintaining ownership, managing data rights and
9.2. Factors Related to MBSE Gains
distribution of rights, re-use or copyrights.
The factors that pertain to gains from MBSE include early defect detection, reuse, product
9.2.definition,
line Factors Related
risk to MBSE Gains
reduction, improved communication, usage in supply chain and standards
conformance [1,4,5,10]. It is well known that the later in the system life cycle that defects are detected,
The factors that pertain to gains from MBSE include early defect detection, reuse, product line
the greater the cost of correcting the defects. The ability of MBSE to identify defects early in the system
definition, risk reduction, improved communication, usage in supply chain and standards
conformance [1,4,5,10]. It is well known that the later in the system life cycle that defects are
Systems 2019, 7, 12 14 of 18

life cycle can contribute to significant cost savings and thereby provide significant gains [4]. Another
key factor is the ability to use legacy models and data during the latter stages of system life cycle to
eliminate extraneous activities and reduce rework. The ability of MBSE to exploit legacy models and
data also contributes to economic justification. Since individual projects may not have the luxury of
time to work on data reuse, additional measures are needed at the enterprise management level to
incentivize projects to achieve this reuse. In particular, commercial aircraft and automobiles which
have product lines with variants can benefit from model and data reuse. MBSE can be expected to
provide significant gain to define product line characteristics because of single, centralized configurable
system models [4]. The adoption of MBSE can be expected to increase confidence while reducing
risks related to both processes and products. Also, with increasing system complexity, it is becoming
increasingly difficult to account for emergence. Therefore, risk reduction in complex systems is a
significant gain [1]. With centralized digital repositories and data analysis capability, MBSE promises
to increase communication efficiency and reduce feedback loops. These characteristics can potentially
provide important gains (e.g., speed up the process) during the system life cycle. MBSE also provides
opportunities to share data and knowledge in timely fashion, which translates into increased supply
chain efficiency [5]. Additionally, MBSE provides cost savings through quick traceability mechanisms
that are built into today’s modeling environments. However, the ever-increasing complexity of systems
in multiple domains is making it increasingly difficult to conform to standards.
From the analysis of factors related to both investments and returns/gains, it became apparent
that MBSE needs to derive value from a variety of systems engineering activities across the system
life cycle to produce convincing economic justification. In other words, using model-based approach
solely for requirements engineering is not enough. The use of integrated digital models for data
analysis, trade studies, decision support, and communication is necessary throughout the detailed
design, implementation, production, maintenance, retrofit, operation, and retirement stages of the
system life cycle.

10. Assessing MBSE Implementation Benefit

As noted previously, the factors associated with MBSE investment and expected gains depend
largely on the system’s intrinsic characteristics such as complexity, operational environment
characteristics, and system lifespan. The system’s complexity can be considered to be a function
of the number of unique components in the system, the interactions between them, the amount of
knowledge required for developing the system, and the amount of information needed to describe the
system [15,16]. A system’s environment can be characterized in terms of the number of stakeholders
and the number of external entities the system needs to interact with which are outside the system’s
boundary, and the regulatory environment. In other words, the system’s environment consists of
stakeholders, external systems which impose regulatory and interface constraint, and applicable
standards that the system needs to conform to. This factor includes system interaction with humans or
entities outside the system boundary. A system’s lifespan can be characterized by the system’s useful
life. The system lifecycle spans the time required for concept formulation, development, production,
operation/utilization and retirement stages. Before transitioning to operation, a system must conform
to the various applicable standards.
We parameterized MBSE implementation problem using systems complexity, environment
complexity, and system lifespan as distinguishing characteristics. We then identified those industry
sectors in which maximum gains are likely to be achieved with the adoption of MBSE. Figure 11a–c
presents MBSE implementation problem in terms of system complexity, environment complexity, and
system lifespan for 12 industry sectors. The constructed scale of system complexity, environmental
complexity and lifespan are divided into three segments-high, medium and low. For example,
aerospace systems such as airliners have more than ~100,000 parts (high), which is more complex
than high-tech industry sector systems like copying machines which have ~2000 parts (medium).
The systems used by household consumers on daily basis which have up to ~300 parts (low) [17].
Systems 2018, 6, 12
2019, 7, x FOR PEER REVIEW 15
15of
of 18

(low) [17]. Here, complexity of systems is determined on the basis of number of unique components,
Here, complexity
interactions of systems
between is determined
components, amounton the basis of number
of knowledge of unique
required components,
to develop interactions
the system and
between components, amount of knowledge required to develop the system
amount of information needed to describe the system. High, medium, and low scales of and amount of information
needed to describe
environmental the system.
complexity High, medium,
are evaluated based on and thelow scales of
number of stakeholders,
environmentalexternal
complexity are
systems
evaluated based on the number of stakeholders, external systems which impose
which impose regulatory and interface constraint, and applicable standards that the system needs to regulatory and
interface to
conform constraint,
related toand applicable
particular standards
industry that
sector. the system
Systems withneeds to conform
a relatively shortto related
useful to particular
lifespan of 0 to
1industry
year aresector. Systems
attributed withscore
to low a relatively short useful lifespan
in parameterization comparedof 0 to
to systems
1 year arewith
attributed to low
a lifespan score
of more
in parameterization
than 30 years. compared to systems with a lifespan of more than 30 years.

(a) (b)

(c)
Figure 11. MBSE
Figure 11. MBSEimplementation
implementation problem
problem difficulty
difficulty in industry
in industry sectors:
sectors: (a) (a) sectors
industry industryon
sectors onconstructed
subjective subjectivescale
constructed
of systemscale of system
complexity, complexity,
environment environment
complexity, complexity,
and lifespan; and
(b) industry
sectors on quantitative
lifespan; (b) industryapproximate
sectors on scale of system complexity,
quantitative approximate environment
scale ofcomplexity, and lifespan;
system complexity,
(c) visual representation
environment of MBSE
complexity, and implementation problem
lifespan; (c) visual in industry sectors
representation along
of MBSE three axes.
implementation
problem in industry sectors along three axes.
11. Conclusions
This paper has presented a methodological framework for economic analysis of MBSE
11. Conclusions
implementation that can be applied to different sectors. MBSE and traditional SE approaches are
This paper has presented a methodological framework for economic analysis of MBSE
compared in terms of upfront investment and expected gains. Compared to traditional SE, MBSE
implementation that can be applied to different sectors. MBSE and traditional SE approaches are
 Systems2019,
Systems 2018,7,6,12x FOR PEER REVIEW 16ofof1818
16

compared in terms of upfront investment and expected gains. Compared to traditional SE, MBSE is
isshown
showntotorequire
require greater
greater upfront
upfront investment
investment withwith gains
gains showing
showing up inup
theinlatter
the latter
stagesstages
of the of the
system
system life cycle.
life cycle. ThreeThree
key key
MBSE MBSE implementation
implementation problemcharacteristics
problem characteristics(i.e.,
(i.e., system complexity,
complexity,
environment
environmentcomplexity
complexityandand
system lifespan)
system are identified
lifespan) as the major
are identified as thediscriminators of the different
major discriminators of the
sectors when
different it comes
sectors whentoitMBSE
comesimplementation. These factors
to MBSE implementation. arefactors
These used toare evaluate
used tothe different
evaluate the
industry sectors in terms of MBSE costs and MBSE returns Figure 12.
different industry sectors in terms of MBSE costs and MBSE returns Figure 12.

Figure 12.12.
Figure 3D3D
visual
visualrepresentation ofMBSE
representation of MBSEimplementation
implementation problem
problem in industry
in industry sectors.
sectors.

By
Byparameterizing
parameterizing the the
MBSE implementation
MBSE implementationproblem in termsinof terms
problem system of complexity, regulatory
system complexity,
and
regulatory and operational environment complexity, and system lifespan, we were able industries
operational environment complexity, and system lifespan, we were able to identify those to identify
that stand
those to derive
industries thestand
that most tovalue fromthe
derive themost
adoption
valueoffromthe MBSE approach.
the adoption As shown
of the in Figure 12,
MBSE approach. As
MBSE
shown caninbeFigure
expected 12,toMBSE
greatlycan
benefit the transportation
be expected to greatly and mobility,
benefit theaerospace and defense,
transportation energy,
and mobility,
processes
aerospaceand andutilities
defense, andenergy,
naturalprocesses
resourcesand industries.
utilities With the increasing
and natural resources complexity
industries. of systems
With the
inincreasing
the marinecomplexity
and offshore, architecture, engineering and construction,
of systems in the marine and offshore, architecture, engineeringlife sciences, and industrial
and
equipment sectors,
construction, life MBSE
sciences,canand
potentially playequipment
industrial an important role throughout
sectors, MBSE canthe system lifeplay
potentially cycle.
an
Industry sectors such as financial and business services and high-tech are
important role throughout the system life cycle. Industry sectors such as financial and business not likely to benefit from
MBSE as much
services as the other
and high-tech are industries.
not likely toFrom ourfrom
benefit study, systems
MBSE related
as much as to
thehome,
otherlifestyle
industries.andFrom
fashionour
and consumer packaged goods and retail are likely to benefit less than the
study, systems related to home, lifestyle and fashion and consumer packaged goods and retail are other sectors.
More
likely recently,
to benefit MBSE
less than isthebeing
otheraugmented
sectors. with interactive storytelling approaches to maximize
stakeholder participation
More recently, MBSEinis upfront engineering
being augmented with[18–20]. Thisstorytelling
interactive added capability
approacheswill tohave costs
maximize
associated
stakeholder participation in upfront engineering [18–20]. This added capability will have into
with it as well as benefits. The costs will be associated with mapping system models costs
system stories
associated withthat arewell
it as interactively
as benefits.executed
The costs in virtual
will be worlds
associated or simulations
with mapping [19,20].
systemThemodels
gains are
into
elimination of rework
system stories that are and extraneous executed
interactively design iterations
in virtual that typically
worlds occur when[19,20].
or simulations not all stakeholders
The gains are
have been ableofto contribute
elimination rework and to upfront engineering
extraneous design[18]. These costs
iterations that and benefitsoccur
typically can bewhen
incorporated
not all
instakeholders
our evaluationhave framework.
been able Ultimately,
to contribute thetoability
upfrontto quantify
engineering factors
[18].related
These to MBSE
costs andinvestments
benefits can
and returns will enable
be incorporated in ourprediction
evaluation of framework.
ROI from investments
Ultimately,inthe MBSE on large-scale
ability to quantifysystem
factorsprograms
related to
and product development initiatives.
MBSE investments and returns will enable prediction of ROI from investments in MBSE on
large-scale system programs and product development initiatives.
Systems 2019, 7, 12 17 of 18

In sum, the MBSE value the proposition for a particular industry and the affordable transition to
model-based approach are the two key challenges that need to be addressed in quantifiable terms before
committing to MBSE implementation. By identifying when and where the system life cycle processes
with MBSE impact occur and then quantifying the factors that contribute to the investments and
gains, organizations can make informed decisions. By parameterizing individual SE projects, we can
identify where maximum gains are likely to be achieved with the adoption of MBSE. Future research
can expand the framework to take additional factors into account and provide more comprehensive
economic justification for MBSE in different industries.

Author Contributions: Conceptualization, A.M.M. and S.P.; Methodology, A.M.M. and S.P.; Validation, A.M.M.
and S.P.; Formal analysis, A.M.M. and S.P.; Investigation, A.M.M. and S.P.; Resources, A.M.M.; Data curation, S.P.;
Writing—Original Draft preparation, A.M.M. and S.P.; Writing—Review and Editing, A.M.M.; Visualization, S.P.;
Supervision, A.M.M.; Project administration, A.M.M.
Funding: This research received no external funding.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Carroll, E.; Malins, R. Systematic Literature Review: How Is Model-Based Systems Engineering Justified?
Sandia National Laboratories. 2016. Available online: https://www.incose.org/docs/default-source/
enchantment/161109-carrolled-howismodel-basedsystemsengineeringjustified-researchreport.pdf?
sfvrsn=2&sfvrsn=2 (accessed on 4 December 2018).
2. Transforming Industries, Markets and Customer Experiences, Dassault Systèmes® . Available online: https:
//ifwe.3ds.com/ (accessed on 4 December 2018).
3. Walden, D.D.; Roedler, G.J.; Forsberg, K.; Hamelin, R.D.; Shortell, T.M. Systems Engineering Handbook: A
Guide for System Life Cycle Processes and Activities; John Wiley & Sons Inc.: Hoboken, NJ, USA, 2015.
4. Architecture and Systems Engineering: Models and Methods to Manage Complex Systems (2017) Course 3
Model-Based Systems Engineering: Documentation and Analysis, Massachusetts Institute of Technology
Online Professional Education Program. Available online: https://sysengonline.mit.edu/course-3/
(accessed on 7 August 2017).
5. Madni, A.M.; Sievers, M. Model-Based Systems Engineering: Motivation, Current Status, and Needed
Advances. In Proceedings of the Conference on Systems Engineering Research, Redondo Beach, CA, USA,
23–25 March 2017.
6. Ramos, A.L.; Ferreira, J.V.; Barcelo, J. Model-based systems engineering: An emerging approach for modern
systems. IEEE Trans. Syst. Man Cybern. C Appl. Rev. 2012, 42, 101–111.
7. Stoewer, H. Model-based systems engineering (MBSE): Missing link in the digital enterprise strategy?
In Proceedings of the INCOSE MBSE Workshop, Los Angeles, CA, USA, 26 January 2014.
8. Friedenthal, S.; Oster, C. Architecting Spacecraft with SysML: A Model-Based Systems Engineering Approach;
CreateSpace Independent Publishing Platform: Scotts Valley, CA, USA, 2017.
9. Zingarelli, M.A.; Wright, S.R.; Pallack, R.J.; Mattos, K.C. SWFTS—System Engineering Applied to Submarine
Combat Systems. Available online: https://www.navalengineers.org/SiteCollectionDocuments/2010%
20Proceedings%20Documents/ETS%202010%20Proceedings/Zingarelli.pdf (accessed on 4 December 2018).
10. Technology & Enterprise Architecture: Economical Modeling: Minimizing Effort, Maximizing Modeling
Return on Investment (ROI), Speaker: Michael Vinarcik—Booz Allen Hamilton, May 24th 2016 Keynote
from the No Magic World Symposium. 2016. Available online: https://vimeopro.com/nomagic/no-magic-
world-symposium-2016-presentations/video/170588353 (accessed on 4 December 2018).
11. IEEE Guide—Adoption of ISO/IEC TR 24748-2:2011 Systems and Software Engineering—Life Cycle
Management—Part 2: Guide to the Application of ISO/IEC 15288 (System Life Cycle Processes)
(24748-2-2012). Available online: https://standards.ieee.org/standard/24748-2-2012.html (accessed on
4 December 2018).
12. No Magic Documentation. Available online: https://www.nomagic.com/support/documentation (accessed
on 12 December 2018).
Systems 2019, 7, 12 18 of 18

13. Madni, A.M.; Sievers, M. Model Based Systems Engineering: Motivation, Current Status, and Research
Opportunities. Syst. Eng. 2018, 21, 172–190.
14. Estefan, J.A. Survey of model-based systems engineering (MBSE) methodologies. INCOSE MBSE Focus
Group 2007, 25, 1–12.
15. De Weck, O.; Roos, D.; Magee, C.; Vest, C. Engineering Systems Meeting Human Needs in a Complex Technological
World; MIT Press: Cambridge, MA, USA, 2011.
16. Kolmogorov, A.N. Combinatorial Foundation of Information Theory and the Calculus of Probability.
Russ. Math. Surv. 1983, 38, 29–40.
17. Miller, G.A. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing
Information. Psychol. Rev. 1956, 63, 81–97. [PubMed]
18. Eclipse Capella Documentation. Available online: https://www.polarsys.org/capella/ (accessed on
12 December 2018).
19. Madni, A.M. Expanding Stakeholder Participation in Upfront System Engineering through Storytelling in
Virtual Worlds. Syst. Eng. 2015, 18, 16–27.
20. Madni, A.M.; Spraragen, M.; Madni, C.C. Exploring and Assessing Complex System Behavior through
Model-Driven Storytelling. In Proceedings of the 2014 IEEE International Conference on Systems, Man, and
Cybernetics (SMC), San Diego, CA, USA, 5–8 October 2014.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).