Software Process Simulation over the Past Decade:
Trends Discovery from A Systematic Review
He Zhang Barbara Kitchenham
School of Computer Science School of Computer Science
and Engineering, UNSW and Mathematics,
National ICT Australia Keele University, UK
he.zhang@nicta.com.au barbara@cs.keele.ac.uk
ABSTRACT
Software Process Simulation (SPS) research has increased
since 1998 when the first ProSim Workshop was held. This
paper aims to reveal how SPS has evolved during the past
10 years based on the preliminary results from the system-
atic literature review of SPS publications from 1998 to 2007.
Trends over the period showed that interest in continuous
modelling was decreasing and interest in micro-processes
was increasing. Hybrid models were based primarily on sys-
tem dynamics and discrete event simulation and were all
implemented by vertical integration.
Categories and Subject Descriptors
D.2.9 [Software Engineering]: Management—Software pro-
cess models
General Terms
Management
Keywords
Software process simulation, systematic literature review
1. INTRODUCTION
Software Process Simulation (SPS) was introduced into
the software engineering domain by Abdel-Hamid’s pioneer-
ing work [1]. In the last two decades, it emerged as an
effective tool to help evaluate and manage changes made to
software projects and organisations. As a major research
event, the ProSim series conference1 has taken place since
1998, and focuses on up-to-date theory and practice of SPS
research in addressing real-world problems.
Recently, we undertook a systematic review to revisit and
update the state-of-the-art of SPSM research, to summarise
the 10-years (1998-2007) progress, and to discover trends for
1
International Workshop on Software Process Simulation Modelling
(ProSim) and International Conference on Software Process (ICSP)
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
ESEM’08, October 9–10, 2008, Kaiserslautern, Germany..
Copyright 2008 ACM 978-1-59593-971-5/08/10 ...$5.00.
345
Dietmar Pfahl
Simula Research Laboratory
Department of Informatics,
University of Oslo, Norway
dietmarp@simula.no
our future research activities in this domain. The process
and preliminary results of this review have been reported
in [5]. This paper instead presents the underlying trends
over the decade derived from the review results.
2. THE SYSTEMATIC REVIEW
This section briefly describes the method applied in this
review and the studies identified (see [5] for more details).
2.1 Method
This research follows Kitchenham’s methodological guide-
lines for systematic reviews [3]. Three researchers were in-
volved, including one principal and one secondary reviewer,
plus the third acting as the expert panel. The research ques-
tions of this review are the following:
Q1. What are the purposes and motivations for SPS in the
last decade’s practice?
Q2. Which simulation paradigms have been applied in the
last decade, and how popular are they in SPS?
Q3. Which simulation tools are available for SPS and have
been in use in the decade? And how popular are they?
Q4. On model level, what are problem domains and model
scopes focused on by simulation models?
Q5. On parameter level, what are the output variables con-
sidered when developing simulation models?
Q6. Which simulation paradigm is the most appropriate for
a specific SPS purpose and scope?
The time frame of sources for this study is constrained
to the period from 1998 to 2007. As the ProSim series
are regarded as the most important forum of SPS, the re-
lated sources, i.e. ProSim, ICSP, Journal of Systems & Soft-
ware (JSS) and Software Process Improvement & Practice
(SPIP), are the primary data sources for this research.
2.2 Studies
The pilot review identified four categories of primary study:
A. Software process simulation models or simulators;
B. Process simulation modelling paradigms, methodologies,
and environments;
C. Applications, guidelines, and solutions for adopting pro-
cess simulation in software engineering practice;
D. Experience reports of SPS research and practice.
Table 3: Paradigms applied in primary studies

1998
1999
2000
2002
2003
2004
2005
2006
2007
These four categories are not a mutually exclusive. They
focus on different aspects of SPS research, and may give an-
swers to the research questions from different perspectives. SD l l l l l l l l l
DES l l l l l l l l l
SBS l l l l l
Table 1: Sources identified as primary studies KBS l l l
QSIM l l l l

total
RPG l l l
1998

1999

2000

2002

2003

2004

2005

2006
2007
ABS l l
Proc. 15 13 21 0 32 27 24 8 8 148 DTS l
JSS 11 0 12 0 0 0 0 0 0 23 Stig. l
SPIP 0 10 0 7 5 7 7 2 2 38 Emrg. l
selected 13 9 14 7 16 10 13 6 8 96 Number 4 2 5 5 4 5 5 4 4

In total, 209 papers have been published in the ProSim Trend 1 System dynamics and discrete-event simulation formed
sources. They form a comprehensive body of knowledge of the main stream of SPS paradigms.
software process simulation. By carefully reviewing their
titles, abstracts, keywords, conclusions, 96 articles were se- Table 2 shows the number of Category A studies (simu-
lected and identified as the primary studies. The number of lation models) per paradigm and per year. Note that some
papers per year and source are summarised in Table 1. studies employed more than one paradigm in their research.
The bottom line indicates the number of hybrid models
3. TRENDS DISCOVERY each year, which integrate two or more modelling techniques
above it. The majority of process models for simulation each
Trend means “the general movement over time of a sta- year were built using SD or DES (the top two rows), includ-
tistically detectable change” (Merriam-Webster Dictionary). ing their combinations appearing at the bottom.
This section attempts to detect the interesting ‘movement’
or ‘change’ derived from the review results over the decade. Trend 2 New simulation paradigms continued to be intro-
duced into SPS research.
3.1 Paradigms
Table 3 presents a more comprehensive view of all primary
studies (Category A/B/C/D). Unlike Table 2, the study
Table 2: Paradigms applied in simulation models numbers are represented with the solid dots, which make
it clear which paradigms were applied each year. The four
total
1998
1999
2000

2002

2003
2004
2005
2006
2007

rows at the top are the paradigms discussed in KMR’s paper

in 1998, which are separated from the others by a solid line.
SD 4 5 7 1 4 3 5 2 1 32 The emerging paradigms were introduced to SPS research
DES 1 2 3 1 2 2 4 1 2 18 since 2000, and their application became dominant at 2004.
SBS 1 1 1 3
KBS 1 1 1 3
3.2 Granularities
QSIM 1 1 2 As two complementary types of software process, macro-
RPG 1 1 2 process and micro-process research (identified by Osterweil [4]),
ABS 1 2 3 are applicable to SPS research as well. However, the system-
DTS 1 1 atic review reflects that there are three different granularity
Stig. 1 1 levels that process simulation models can focus on: namely
Hybrid 1 1 2 1 1 3 1 10 system, process, and entity levels. They are supported by
different simulation paradigms.
SD: system dynamics DES: discrete-event simulation
At system level , a software process is modelled as an
SBS: state-based simulation KBS: knowledge-based simulation
overall system. The process behaviour is described by a
RPG: role-playing game ABS: agent-based simulation
set of external parameters that continuously vary over time.
Stig.: stigmergy QSIM: qualitative/semi-quantitative simulation
This granularity corresponds to the macro-process research
Emrg.: emergent DTS: discrete-time simulation
and is supported by the conventional continuous simulation.
Software processes are modelled and observed with more
details at process level , where the executed tasks with the
In 1998, there were only three simulation paradigms em- resources required and their sequential relationships can be
ployed by the Category A studies (models) published in the tracked during simulation. The modelling on this level is
first ProSim workshop (Table 2). They were SD, DES and supported by discrete simulation.
KBS. Kellner, Madachy and Raffo (KMR) discussed four
types of simulation in their seminal paper in ProSim com- Trend 3 Continuous modelling gradually lost its dominant
munity [2]. However, our review identified ten simulation position in SPS research in comparison with discrete
paradigms from the primary studies (listed in Table 3). approaches during the decade.

346

Table 4: Paradigms in support of granularities
QSIM
RPG
KBS
ABS
DES
SBS
SD
System l l m Hybrid modelling employs more than one paradigm in de-
Process l l l m
Entity m l
l - inherently supported m - condition applying
Table 5: Granularity levels of simulation studies
1998
1999
2000
2002
2003
2004
2005
2006
System 4 5 7 2 4 3 5 3
Process 2 2 4 3 3 3 4 1
Entity 2 2
When revisiting the top two rows of Table 2, both SD
and DES supported simulation studies of software process
each year. Although the total number of SD models is larger
than any others’, it is not difficult to observe a decreasing
trend of SD application in the decade, by absolute number
or percentage. The studies using SD dominated in the early
years (prior to 2000). Recently, the number of published
studies using SD has decreased and approached the number
of studies using DES. It implies that simulation research has
become more interested in micro-process modelling.
Process level simulation models are based on the conven-
tional life-cycle or sequential process modelling. However,
it is difficult to simulate team effects by process level mod-
elling through aggregating individuals. This becomes an is-
sue when modelling a process with many participants, such
as the open source developer community. Accordingly, mod-
elling and simulation technology at entity level requires
quite different characteristics from the technology used on
process level. Simulation at entity level consumes more re-
sources to track all entities and their relationships individu-
ally than the approaches using aggregation.
Table 4 indicates for each simulation paradigm the in-
herently supported research granularity level(s) as identified
from the review. Here, RPG is a special case, its supported
granularity level often depending on its combined approach.
Trend 4 Most of newly introduced paradigms enhanced the
research capability at the micro-process level.
All newly introduced paradigms are listed at the right side
of the vertical line (between KBS and QSIM) in Table 4
(compared with the four discussed by KMR at leftmost). If
we refine the scope of micro-process to cover the entity gran-
ularity level, then most of them are capable of the process
or entity level research corresponding to the micro-process.
Trend 5 In recent years, micro-processes have been attract-
ing more simulation research.
Table 5 records the number of published studies on each
granularity level over years. When applying the extended
concept of micro-process, since 2004 the number of micro-
process simulation models has been no less than macro-
process models. It is also an enhancement of Trend 3.
3.3 Integration
Like the interaction between macro-process and micro-
Emrg.
process [4], process modelling on the three granularity levels
DTS
Stig.
is not mutually exclusive, and sometimes combined.
l veloping a process simulation model. The systematic re-
l l view concludes that hybrid process simulation models have
attracted interest as a possibility to avoid the limitations
of applying single modelling method, and more realistically
capture complex real world software processes.
Trend 6 System dynamics and discrete-event simulation were
the most common combination for hybrid modelling.
total
2007
The bottom line of Table 2 shows the number of hybrid
models published per year. All these 10 models were at least
1 34 based on the combination of SD and DES, or even more.
3 25
4 Trend 7 Hybrid process simulation modelling concentrated
on vertical integration.
There exist two broad approaches to develop a hybrid
process model: vertical integration, which primarily im-
plements discrete modelling at the lower process level, then
continuously calculates the process factors and incorporates
the feedback loops at system level; and horizontal integra-
tion, in which the sub-processes or phases within a large
scale or complex software process may be modelled using
different approaches respectively and sequentially, and the
data flow has to be converted at the interface in transition.
We found all the hybrid models reported in primary studies
were implemented by vertical integration.
4. CONCLUSION
The method, process and preliminary results from the sys-
tematic review on software process simulation has been re-
ported in [5]. This paper further presents the in-depth find-
ings of SPS research by discovering the trends underneath
the review results over the past decade. In addition, we
also gain the new insights about SPS research on modelling
granularity and integration.
The results from this review can help both insiders’ and
outsiders’ perception and understanding of SPS research.
These trends derived from the preliminary facts help suggest
possible directions for future SPS-related research.
5. REFERENCES
[1] T. K. Abdel-Hamid and S. E. Madnick. Software
Project Dynamics: An Integrated Approach. Prentice
Hall, Englewood Cliffs, NJ, 1991.
[2] M. I. Kellner, R. J. Madachy, and D. M. Raffo. Software
process simulation modelling: Why? what? how?
Journal of Systems and Software, 46(2/3):91–105, 1999.
[3] B. Kitchenham. Guidelines for performing systematic
literature reviews in software engineering. Technical
report, School of Computer Science and Mathematics,
Keele University, 2007.
[4] L. J. Osterweil. Unifying microprocess and
macroprocess research. In Software Process Workshop,
Beijing, 2005. Springer.
[5] H. Zhang, B. Kitchenham, and D. Pfahl. Reflections on
10 years of software process simulation modelling: A
systematic review. In International Conference on
Software Process, Leipzig, Germany, 2008. Springer.
347