See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/320742803

Proposal for a Scientific Software Lifecycle Model

Conference Paper · November 2017

DOI: 10.1145/3144763.3144767

CITATIONS READS
0 1,474

2 authors, including:

Anshu Dubey
Argonne National Laboratory
79 PUBLICATIONS   1,156 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

FLASH Code View project

Code Comparison View project

All content following this page was uploaded by Anshu Dubey on 04 January 2018.

The user has requested enhancement of the downloaded file.

 Proposal for a Scientific Software Lifecycle Model
Anshu Dubey
Mathematics and Computer Science Division
Argonne National Laboratory
Lemont, IL 60439
Flash Center for Computational Science
University of Chicago
Chicago, IL 60637
Email:adubey@anl.gov
Abstract—Improvements in computational capabilities have
lead to rising complexity in scientific modeling, simulation, and
analytics and thus the software implementing them. In addition,
a paradigm shift in platform architectures has added another
dimension to complexity, to the point where software produc-
tivity (or the time, effort, and cost for software development,
maintenance, and support) has emerged as a growing concern for
computational science and engineering. Clearly communicating
about the lifecycle of scientific software provides a foundation
for community dialogue about processes and practices for various
lifecycle phases that can improve developer productivity and soft-
ware sustainability—key aspects of overall scientific productivity.
While the mainstream software engineering community have
produced lifecycle models that meet the needs of software projects
in business and industry, none of the available models adequately
describes the lifecycle of scientific computing software. In par-
ticular, software for end-to-end computations for obtaining sci-
entific results has no formalized development model. Examining
development approaches employed by teams implementing large
multicomponent codes reveals a great deal of similarity in their
strategies. In earlier work, we organized related approaches into
workflow schematics, with loose coupling between submodels for
development of scientific capabilities and reusable infrastructure.
Here we consider an orthogonal approach, formulating models
that capture the workflow of software development in slightly
different scenarios, and we propose a scientific software lifecycle
model based on agile principles.
Index Terms—scientific computing, software engineering, soft-
ware lifecycle
I. I NTRODUCTION
The topic of software lifecycles for business and commercial
software is well researched with many models that meet the
needs of different types of projects. A lifecycle model decom-
poses software development into distinct phases, where each
phase has its own requirements, specifications, and methods.
Many reasons make such decomposition into phases desirable.
For example, each phase can control its own quality and result
in higher quality software overall. Similarly, in larger projects,
phases can help define roles for developers and bring clarity
to the development process. Some standard lifecycle models
are waterfall [12], where each stage is completed before the
next stage can begin; V-shaped [17], which is an extension
of the waterfall model that also incorporates testing phases
for each development stage; iterative [14], where development
Lois Curfman McInnes
Mathematics and Computer Science Division
Argonne National Laboratory
Lemont, IL 60439
Email:curfman@mcs.anl.gov
stages are cycled over for subsets of requirements until the
final product is obtained; spiral [4], where iterations occur for
ongoing and new requirements; big bang [16], where develop-
ment occurs without defined process; and agile, which allows
cycling through any group of phases emphasizing incremental
changes [1]. (See [18], [13] for a general description of various
software lifecycle models.)
Because of the unique requirements of scientific software,
a mismatch exists between the needs of scientific software
developers and the theory of mainstream software engineering.
Aspects of some lifecycle models apply; for example, the gen-
eral philosophy of the agile approach fits well. But the methods
used in nonscientific software under the agile approach do not
fit nearly as well for scientific code. The biggest challenge
in having well-defined methods and timelines for scientific
software development is that often the numerical methods and
abstractions being used in implementations are themselves
subject of research, and therefore not fully specified ahead
of time. There have been efforts to adopt the agile model
for research-driven software. For example, the TriBITS [2]
effort has produced a package that also incorporates an agile
lifecycle model for research-driven software development.
The model addresses concerns of software that downstream
becomes a component in a larger software collection. The
Blue Brain Project [8], [10] is adapting this model for their
own computational needs. In general this model is suitable
for software that implements research ideas and becomes a
reusable component in other larger collections of interoperable
software.
However, many projects exist in scientific domains where
the primary software objective is to be the means for con-
ducting research instead of being the product or the goal
of the research. End-to-end simulation codes fall into this
category. They may use libraries and other third-party software
as components, but the codes have different usage models
and user expectations. In most successful scientific software
development projects, there is an implicit understanding of
the software lifecycle, even if is not articulated. In earlier
work [7] presented as an idea paper at the WSSSPE work-
shop in 2016, we devised schematics of scientific software
development workflows with a view toward engaging the
community in examining this important aspect of software
productivity. Here we refine ideas from the earlier work and observa)on
  Phenomena
 of
 
Interest
 
take the next step toward devising a lifecycle model applicable model
 
to simulation software and data analytics associated with model
 fidelity
 
Valida)on
  Equa)ons
 
scientific simulations. Our methodology follows a three-step
model
 fidelity
 
process. Requirement gathering, design, implementation, and output
 
discre)ze
 
verification & validation are four distinct phases in a typical
Numerical
 
development cycle for all kinds of software. In the first step Verifica)on
 
Solvers
 
we map the activities during the development of scientific op)miza)on
 
 
accuracy
  accuracy
 
Performance
 
 stability
 
 stability
 
software to these well known and well understood phases. In
the second step we examine the existing lifecycle models and Implementa)on
 
evaluate their applicability to the conceptualized workflows.
In the third step we use the insights from the first two steps
to propose a lifecycle model that covers important aspects of
Fig. 1. Workflow for developing multiphysics software: overall perspective.
scientific software development and maintenance.
II. W ORKFLOW FOR S CIENTIFIC S OFTWARE
D EVELOPMENT
model,
 framework
 
Requirements
 
 
 
We begin by looking at the workflow for typical multi- gathering
 
data,
 expecta3ons
 
workflow
 
physics scientific software development projects. Examples of
such development projects include FLASH [6], Uintah [3], approxima3ons,
 numerics
 
Design
  storage,
 cura3on,
 retrieval,
 analysis
 
Pluto [11], Ramses [15], Cactus [5], and many more. All of steps
 in
 scien3fic
 process
 
these codes use high performance computing (HPC) platforms
solvers,
 infrastructure
 
for running simulations. Figure 1 captures essential features Implementa3on
  algorithms,
 data
 structures
 
of the workflow that has been implicit during development tools,
 interfaces
 

in many such projects. All boxes in the figure can be, and
 order,
 correc3on
  valida3on,
 observa3ons
usually are, under research. The research topic may be of
general interest going beyond the project, such as numerical
methods being applied, or it may be driven by the needs of the
project itself. Many feedback loops in the workflow indicate
ongoing research and refinement in corresponding sections of
the workflow based on insights gained during the project.
The process starts with devising a mathematical model for
the phenomena of interest. The equations are discretized, and
numerical methods are devised and implemented for solving
the equations. Here the workflow for scientific software de-
velopment begins to diverge from that of mainstream software
development. The verification of scientific software addresses
not only expected behavior, but also convergence and stability
of the numerical solvers. A failure in either takes the workflow
back to numerical solvers, which may need to be revised or
redesigned. Similarly, validation of output against observations
may reveal that the discretization or approximations used in
the mathematical model are inadequate, which in turn can
completely reset the workflow to the first step.
Figure 1 illustrates a rough mapping of the workflow for
software development of scientific capabilities onto four basic
phases: requirements gathering, design, implementation, and
verification. Setting aside issues such as release, maintenance,
and user support, these phases apply to any standard software
process. What differs in the realm of scientific software is the
feedback among various phases. From the perspective of these
distinct phases of development, the workflow can be simplified
into interactions among the phases as shown in Figure 2. Each
phase in the figure shows entities that are resolved in the
corresponding phase.
convergence,
Verifica3on
valida3on
provenance
Fig. 2. Interaction among development phases for multiphysics software:
overall perspective.
Another important aspect of scientific software functionality
is reusable infrastructure, or using a loose definition of the
word framework (e.g., see [9]), the entity that provides basic
services (such as data structures related to discretizations and
data layout, parallelization, and I/O), enables composability,
and allows orchestration of calculations. A flexible and exten-
sible framework is a critical component of scientific software,
with unique requirements in its development cycle, therefore
deserving its own separate workflow and design space. Frame-
work development comes closest to other general business
and commercial software, in that the control flow from one
phase to another is linear, as shown in Figure 3. This reusable
infrastructure is the most stable part of the resulting scientific
software, and once it has been implemented, a change to
the framework is a major undertaking. Modifications to a
framework would normally require starting at the requirement
gathering phase. Not surprisingly, the diagram of interaction
among phases is also linear, as shown in Figure 4.
We also examine the scientific process workflow from the
perspective of data used in computational science, which
may be data generated by or input to simulations, or ob-
servational data used for validation, or all used together for
advancing scientific understanding. Some of the processes
 Processing
 
Services
 Provided
 
Needed
 
model
 

Separa)on  of   Data  Layout/  

Interoperability   Storage   Accessibility   Analy)cs  
Concerns   Interface  

Implementa)on  
Implementa)on  
robustness  
reliability  
Verifica)on   Verifica)on  

Fig. 3. Workflow for developing multiphysics software: infrastructure. Fig. 5. Workflow for developing multiphysics software: data management
and analytics.

interoperability  concerns   data  volume  

Requirements       Requirements       needed  analy3cs  
interfaces  
gathering   gathering  
process  constraints   accessibility  

separa3on  of  concerns   storage  modes,  accessor  func3ons  

Design   data  scoping  and  ownership   Design   data  scoping  and  retrieval  algorithms  
encapsula3on   modifica3on,  control  

backbone   archive,  analysis  

Implementa3on   interfaces   Implementa3on   interfaces  

helper  func3ons   helper  func3ons  

interoperability   tes3ng  

Verifica3on     expected  behavior   Verifica3on     expected  behavior  

robustness   robustness  

Fig. 4. Interaction among development phases for multiphysics software: Fig. 6. Interaction among development phases for multiphysics software: data
infrastructure. management and analytics.

are similar to computations, e.g., much of the analysis starts
with a hypothesis, models are mathematical, and analysis
may involve numerical methods also used in simulations.
Some other processes are different, for example archiving
and retrieval, which have no equivalent in simulations. The
schematic in Figure 5 captures the workflow characteristics of
scientific data management and analysis, and Figure 6 shows
the corresponding phase interactions. More feedback loops
exist for data analytics than in framework design because
insights and inferences can lead to modifying or replacing
algorithms used in analysis (similar to the feedback loops for
scientific capabilities, shown in Figure 2).
III. E XISTING S OFTWARE L IFECYCLE M ODELS
We now examine existing software lifecycle models to see
what, if any, mapping is possible between the models and the
development workflow of scientific software.
a) Waterfall model: is the simplest of the software
lifecycle models, and it is also the least applicable to scientific
software. The main reason is that it relies upon the a relation-
ship between phases where the next phase cannot begin until
the first phase is complete. Because scientific software is by
design meant for exploration of new ideas and insights, linear
progression is incompatible with its goals. Framework design
and development comes the closest to being able to follow this
model.
b) V-shaped model: differs from the waterfall model
in having testing phases corresponding to each development
phase. Because it also needs to have no unknown requirements,
it has similar drawbacks for adoption by scientific software as
the waterfall model.
c) Iterative model: operates by allowing the waterfall
model to proceed for a subset of requirements, and then going
back to the beginning for the next set of requirements. It
overcomes one problem of the other two models discussed
so far in that it allows going back to the first phase. However,
it still lacks the flexibility of permitting evolving requirements
that can happen in scientific software.
d) Spiral model: is a refinement of the iterative model
where phases are repeated for previously implemented require-
ments as well as new requirements over and over until the
project objectives are met. However, this model is still not
adequate for scientific software, because, as seen in Figure 2,
feedback loops exist among multiple phases in the workflow,

Initial Development the implementation box. Normally implementation phase will
Requirements Capability
Integration of Design
New Research gathering Addition
Verification
Implementation
validation
Release
Maintenance
Ongoing Testing
Distribution
User Support
Issues and Bug Resolution
Fig. 7. Overall lifecycle model for scientific software derived by analyzing
workflow in Figure 1 and mapping it to lifecycle phases in Figure 2. Figures 4
and 6 show details of interactions among development phases of infrastructure
and analysis.
so the spiral may end up folding back on itself.
e) Big bang model: is the model in which the vast
majority of scientific software development projects have
operated until recently. This model does not have a well
defined process or requirements and thus is inherently risky.
This model can be acceptable for small projects with just a
few developers; however, as demonstrated by the many failed
large projects in the scientific world, it clearly does not apply
to any moderate to large project.
f) Agile model: comes closest to being applicable to
scientific software development because it allows cycling
through any group of phases and emphasizes incremental
changes. Its philosophy applies, though many of the methods
that implement the philosophy do not. For example, sprints
have very little use in software that is used for research and
is being researched.
IV. P ROPOSED L IFECYCLE M ODEL
The proposed lifecycle model for scientific software, shown
in Figure 7, is derived from agile methodology and includes
steps beyond the initial development cycle discussed in Section
II. The initial development phase is taken from Figure 2, since
the scientific capabilities of multiphysics software have the
most demanding interaction among phases. The other aspects
of scientific software (infrastructure, shown in Figure 4 and
analysis, shown in Figure 6) have a subset of complexity
of interaction among phases. Two-way arrows represent tight
coupling and feedback loops that exist among various phases
in the development cycle. Note that any of the arrows or phases
can be nullified in a traversal of the cycle depending upon the
need of the project.
Boxes outside the development phase represent later stages
in the software lifecycle, with their arrows pointing to the
phase where they are more likely to plug into the development
cycle. For example, the two way arrow between maintenance
and release boxes indicates user interactions with issues and
bugs reported back. Sometimes the issues may be resolved
with discussion; at other times a new implementation may
be needed, hence the arrow from the maintenance box to
resolve most issues and bugs reported, however, occasionally
the severity of the issue may cause going back earlier in
the development cycle, to design or even to requirement
gathering phases. The diagonal arrows among the boxes permit
escalation of development complexity as needed. Similarly,
capability addition is normally expected to plug into the
development cycle at the design phase, while integration of
new research is likely to cause going back all the way to
requirements gathering. Because this model permits nullifying
arrows and phases as needed, it provides the flexibility of
bypassing one or more phases for either capability addition or
integrating new research if needed. Therefore, for any stage in
software development, the cycle can be made simpler or more
complex depending upon the needs of the moment. Similar to
the agile methodology, our model supports frequent releases
whenever there is a stable code version.
V. C ONCLUSIONS AND F UTURE W ORK
Through the process of mapping typical workflows for
development of scientific software, especially as it applies
to the most complex multiphysics codes, we have unraveled
the dependencies and feedback loops within the lifecycle of
such software. We have synthesized a lifecycle model that,
by permitting null instances of phases and connecting arrows,
unifies many complex workflows into a simple schematic. This
lifecycle model captures the essential features and phases of
the most complex scientific software development.
One aspect of scientific software that we have not addressed
in our current model is that of refactoring existing software.
The software in question could be a legacy code or a well
constructed software that nevertheless has to be refactored be-
cause of the exigencies of platform requirements. An important
common feature of such development is that the new structure
needs to be built while retaining large chunks of original
code. This approach provides a path to incremental adoption,
necessary in most scientific refactoring projects. Considering
a lifecycle model for refactoring will be a next step.
ACKNOWLEDGMENTS
This work was supported by the U.S. Department of
Energy Office of Science Office of Advanced Scientific
Computing Research. The submitted manuscript has been
created by UChicago Argonne, LLC, Operator of Argonne
National Laboratory (Argonne). Argonne, a U.S. Department
of Energy Office of Science laboratory, is operated under
Contract No. DE-AC02-06CH11357. The U.S. Government
retains for itself, and others acting on its behalf, a paid-up
nonexclusive, irrevocable worldwide license in said article to
reproduce, prepare derivative works, distribute copies to the
public, and perform publicly and display publicly, by or on
behalf of the Government. The Department of Energy will
provide public access to these results of federally sponsored
research in accordance with the DOE Public Access Plan.
http://energy.gov/downloads/doe-public-access-plan.
 R EFERENCES
[1] Agile methodology. http://agilemethodology.org/.
[2] R. A. Bartlett, M. A. Heroux, and J. M. Willenbring. Overview of
the tribits lifecycle model: A lean/agile software lifecycle model for
research-based computational science and engineering software. In E-
Science (e-Science), 2012 IEEE 8th International Conference on, pages
1–8. IEEE, 2012.
[3] M. Berzins, J. Luitjens, Q. Meng, T. Harman, C. Wight, and J. Peterson.
Uintah - a scalable framework for hazard analysis. In TG ’10: Proc. of
2010 TeraGrid Conference, New York, NY, USA, 2010. ACM.
[4] B. W. Boehm. A spiral model of software development and enhance-
ment. Computer, 21(5):61–72, 1988.
[5] Cactus Computational Toolkit, 2013.
[6] A. Dubey, K. Antypas, M. Ganapathy, L. Reid, K. Riley, D. Sheeler,
A. Siegel, and K. Weide. Extensible component-based architecture for
FLASH, a massively parallel, multiphysics simulation code. Parallel
Computing, 35(10-11):512–522, 2009.
[7] A. Dubey and L. McInnes. Idea paper: Software lifecycle for scientific
simulation software. Working Towards Sustainable Software for Science:
Practice and Experience (WSSSPE4), http://wssspe.researchcomputing.
org.uk/wp-content/uploads/2016/06/WSSSPE4 paper 16.pdf.
[8] M.-O. Gewaltig and R. Cannon. Current practice in software devel-
opment for computational neuroscience and how to improve it. PLoS
Comput Biol, 10(1):e1003376, 2014.
[9] D. E. Keyes, L. C. McInnes, C. Woodward, W. Gropp, E. Myra, M. Per-
nice, et al. Multiphysics simulations: Challenges and opportunities.
The International Journal of High Performance Computing Applications,
27(1):4–83, 2013.
[10] H. Markram. The blue brain project. Nature Reviews Neuroscience,
7(2):153–160, 2006.
[11] A. Mignone, C. Zanni, P. Tzeferacos, B. van Straalen, P. Colella, and
G. Bodo. The PLUTO Code for Adaptive Mesh Computations in
Astrophysical Fluid Dynamics. The Astrophysical Journal Supplement
Series, 198:7, Jan. 2012.
[12] K. Petersen, C. Wohlin, and D. Baca. The waterfall model in large-scale
development. In International Conference on Product-Focused Software
Process Improvement, pages 386–400. Springer, 2009.
[13] Robert Half International. 6 basic SDLC methodologies:
Which one is best? https://www.roberthalf.com/technology/blog/
6-basic-sdlc-methodologies-the-pros-and-cons.
[14] I. Spence and K. Bittner. What is iterative development? https://www.
ibm.com/developerworks/rational/library/may05/bittner-spence/.
[15] R. Teyssier. Cosmological hydrodynamics with adaptive mesh refine-
ment. A new high resolution code called RAMSES. Astronomy and
Astrophysics, 385:337–364, Apr. 2002.
[16] Tutorials Point. SDLC: Big bang model. https://www.tutorialspoint.
com/sdlc/sdlc bigbang model.htm.
[17] Tutorials Point. SDLC: V-model. https://www.tutorialspoint.com/sdlc/
sdlc v model.htm.
[18] Tutorials Point. Software development life cycle tutorial. http://www.
tutorialspoint.com/sdlc/index.htm.

View publication stats