Built-in software quality

in Agile development

Mehdi Saleh

Subject: Information Systems

Corresponds to: 30 hp
Presented: VT 2018
Supervisor: Görkem Pacaci
Department of Informatics and Media
Abstract

During the past decade, the amount of software code in automotive

industry has been increased annually by millions lines of codes.
(Businessinsider.com, 2018) This has forced many automotive OEMs
to perform an organizational transition, being more software focused
rather than experts within hardware development. Thus, software
development methodologies and practices become essential to
maintain a successful organizational switch. Waterfall and Agile are
the two most popular software development methodologies. Waterfall
as the traditional one, is a progressive method. The progress flows in
one direction, and upon the completion of previous process. For
example, user requirement must be frozen before development, and
software test only happens after compete development. Waterfall can
be referred as long-plan-driven methodology. Agile is another software
development methodology with more iterative approach and possible
change of requirements; with incremental delivery. Volvo Cars has
experienced software development transformation, from traditional
waterfall to agile development.
Agile introduces freedom to requirement change and iterative delivery.
However, such a liberty should not disrate software quality. Especially
in automotive industry that deals with human safety and security. In
such an iterative environment, there is a higher risk to compromise on
quality checks before each delivery, due to short intense lead times.
There will not be enough time for intensive quality assurance activities
after each iteration. However, the OEM still requires high quality
software and cannot sacrifice quality for more frequent deliveries, as

1
this could result in unexpected penalty costs and failure to satisfy legal
requirements. The solution to downsize the intensive quality check
after each iteration, is to improve development quality during the
development and building quality into development processes. This is
what we refer as “Built-in quality”; the quality that is built during
software artefacts development on a continuous basis.
Built-in quality can be a huge challenge for organizations with
developers from different background and quality mind-set, who are
benefiting from different quality methods and tools.
This study conducted at Volvo Cars during Agile transformation time
and the main objective was to connect and emphasize the importance
of built-in quality in agile software development. In this study we look
at existing challenges that decrease quality of software artifacts
during the development. Thus, by prevailing those challenges we can
improve the software quality during iteration delivery. Such an
improvement decreases the amount of intensive quality check after
each iteration. Additionally we look at guidelines and tools that used
by different development teams to improve software artifacts’ quality.
We also investigate how quality assurance engineer can support built-
in quality during agile transformation.

Keywords
Built-in quality, Agile development, SAFe, Just in time reporting,
Building quality in code, building quality in requirement specification

2
Table of content

1. Introduction .................................................................................. 4
1.1. Background ..............................................................................4
1.1.1. Traditional quality assurance activities .......................... 9
1.1.2. Agile development and SAFe framework ........................9
1.2. Problem statement ................................................................ 11
1.3. Research questions ................................................................ 11
1.4. Delimitation ........................................................................... 12
1.5. Outline ................................................................................... 12
2. Theoretical framework ................................................................ 14
2.1. Quality, Internal Quality and Built-in software quality ...... 14
2.2. Agile way of working ............................................................. 15
2.3. Built-in quality method and tools ......................................... 17
2.3.1. Built-in quality into requirement .................................. 17
2.3.2. Built-in quality into code ............................................... 20
3. Study design ............................................................................... 23
3.1. Research method ................................................................... 23
4. Data collection ............................................................................ 24
4.1. Strategy ................................................................................. 24
4.2. Questionnaire ........................................................................ 24
4.2.1. Demographics ................................................................. 24
4.2.2. Main questions ............................................................... 24
4.3. Collected Data ....................................................................... 26
5. Findings and conclusions ............................................................ 32
5.1. Summary of the findings and conclusion .............................. 32
5.2. Future work ........................................................................... 35
References......................................................................................... 36
Appendix ........................................................................................... 39
A Appendix: Software Development Standards .............................. 39
B Appendix : Quality assurance documents .................................... 40

3
1. Introduction

This chapter discusses the background and the purpose of the study.
It describes the importance and the relation between agile
development and built-in quality. It defines the problem area along
with the related research questions. The delimitations of the thesis are
also be presented and an outline is listed at the end.

1.1. Background

At the beginning let’s look at the definition of “Software” in automotive

industry and how it works within different hardware components.
An Electronic Control Unit (ECU) is an embedded system in
automotive electronics that controls one or more electrical systems or
subsystems in the vehicle; Like Engine Control Module (ECM), Brake
Control Module (BCM) or Central System Display (CSD). An ECU
includes both hardware and software.
“Software” in automotive, is the embedded application that runs on
every ECU in the vehicle.
But why does the OEM demand on agile software development
methodology? The answer to this question is directly connected to
highly competitive market and customers’ expectation.
Customers frequently request for better features and functions.
Customers’ needs may also change during the development period.
Apart from these, technology also changes frequently with much faster
pace than old days. To meet customers’ expectations and being the
brand of choice in a highly competitive automotive market, a software

4
development methodology is needed with shorter release and delivery
time for new functions and features.
Previously, Waterfall methodology was broadly used for software
development and has been used at Volvo Cars. Figure 1 shows
waterfall software process model (Ajmer Rajasthan, 2013) “Its name is
derived from structural specification. Every phase comes after a phase
is completed and tasks can be divided according to phases. The output
of one phase becomes input of next phase but we have the option to
revisit phases in the next cycle.” (Ajmer Rajasthan, 2013)

Figure 1 - Iterative waterfall software process model (Ajmer R, 2013)

In waterfall, quality assurance activities are performed after

completion of each phase. Such as joint reviews between software
supplier and Volvo Cars quality assurance team.

5
In past decade, Agile as incremental software development
methodology has been introduced and commonly used in different
industries. Agile is based on iterative deliveries with focus on
customer’s need and freedom to change. In such a development,
customer may change or modify the requirements. (Figure 2)

Figure 2 - Incremental software process model ((Ajmer R, 2013)

This methodology enables the OEM to deliver new features and

functions more frequently. However, in Agile, it will not be possible to
fully assess the product quality in every fixed interval with lengthy
and time-consuming legacy quality assessment activities. In that case

6
the quality activities will lag behind the deliveries and become
ineffective. (Figure 3) Activities like joint reviews, comprehensive
documentation and complete risk assessment prior to development.

Figure 3 - QA activities; Waterfall vs Agile

To overcome with this challenge in agile development, the majority of

quality assurance activities should be integrated (built-in) with the
development activities.
Development team is responsible for the quality and building quality
into the product from early stage of development and quality must be
a non-segregated element from development activities and assured
within software artefact continuously. These artefacts may refer as
requirement specification, architect, design, code and test.
When discussing built-in quality, one need to emphasize the
importance of automated feedback system. These systems are used to
measure the quality of the software artefact and provide with instant

7
reporting during the development. The developer can instantly update
his/her work product to achieve the desire quality of work.
For example, deep nesting is one of the characteristics of code
complexity. The automated reporting system measures code nesting
and provides with instant reporting. If nesting was above that
maximum threshold (based on company quality policy), the developer
should refactor and improve his/her software code.
Volvo Cars has already started the agile transformation journey and
adaptation to agile software development methodology.

8
1.1.1. Traditional quality assurance activities

At Volvo Cars, majority of software development are outsourced to

external suppliers. To ensure supplier quality of work, Volvo Cars
requested certain standards certification from the supplier, such as
ISO/IEC 12207 (Appendix A.1) and Automotive SPICE (Appendix A.2).
Volvo Cars also requested the supplier to prepare and present several
documents during four software joint reviews. In these software joint
reviews, the supplier represents documentation related to design,
requirement traceability, test plan and quality assurance plan. The
primary intention was to qualify supplier’s development process.
Appendix B is the list of these documents and the purpose of them.

1.1.2. Agile development and SAFe framework

Volvo Cars has introduced Scalable Agile Framework (SAFe) for
software development.
Full Safe in figure 4 (SAFe for Lean Enterprise, 2018) demonstrates
different levels in SAFe framework. Developers (suppliers or in-house
development team) are primarily working on team level. They can use
different agile methodologies such as Scrum or XP. In a large
enterprise environment like Volvo Cars, often the work of several
teams can be integrated to an Agile Release Train (ART). ART is part
of a large solution that is ideally delivers a specific function to large
solution. “The Agile Release Train (ART) is a long-lived team of agile
teams which, along with other stakeholders, develops and delivers
solutions incrementally, using a series of fixed-length Iterations
within a Program Increment (PI) time box. The ART aligns teams to a
common business and technology mission. Each ART is a virtual

9
organization (50 – 125 people) that plans, commits, and executes
together. ARTs are organized around the enterprise’s significant Value
Streams and exist solely to realize the promise of that value by
building Solutions that deliver benefit to the end user.” (Agile Release
Train - SAFe f for Lean Enterprise, 2018)

Figure 4 – Full SAfe (www.scaledagileframework.com)

For large solution with several product streams, SAFe framework

supports organization to prevail challenges like short lead time and
continues integration with sustainable deliveries on team, ARTs (Agile
Release Train) or program level.

10
1.2. Problem statement

Volvo Cars is moving toward Agile and significant increase of in-house

software development, while maintaining supplier deliveries. Volvo
Cars decided to use SAFe development framework. Traditionally,
software quality assurance had risk of degrading processes and late
design modifications, considering that majority of software in
automotive industry are safety-critical. In order to mitigate this risk,
quality assurance is desired to be an integrated activity, where every
engineer is responsible for their product quality with the help of
quality assurance engineer. Such a mindset allows building quality
into the software product all the time thus mitigating the
aforementioned risk. However, relevant methods are needed to
support this mind set, e. g. automated feedback systems, systematic
educational practices, etc.
There are different methods and automated reporting tools available
to support built-in quality. The aim of this study is to investigate the
existing challenges to build software quality into the product on
continues basis and what methods and tools there are for such a
support and to determine how quality assurance engineer can support
to improve it.

1.3. Research questions

Below are the research questions in order to serve the purpose of this
study.

11
 What are the existing challenges and activities related to building
software quality into the product on a continuous basis within
Volvo Cars electronic and software department?
 What are the existing guidelines and automated software quality
measurement systems used in Volvo Cars electronic and software
department?
 How quality assurance engineer can support and improve built-in
quality in large enterprise agile environment?

1.4. Delimitation

This is a case study with focus on company built-in quality methods

and tools for following software artefacts.
1- Requirement specification
2- Code development
This report will not cover architect, design, test and documentation.
Apart from that, the study will not suggest the best practise for Volvo
Cars.

1.5. Outline

This dissertation contains a total of five chapters, and it has been

organised in the following way. The first chapter provides a general
view of research study and problem statement among with research
questions. Chapter two is a brief explanation of software built-in
quality and agile transformation in a theoretical framework. Chapter
three describes the research methodology. Data collection and

12
questionnaire displayed in chapter four and at the end; chapter five is
the place for discussions and conclusions.

13
2. Theoretical framework
This chapter explains the concept of built-in quality. In addition, we
will look at some methods and tools for automated feedback system for
improving software artefact quality.

2.1. Quality, Internal Quality and Built-in software quality

There are several definitions for Quality. Here we look at few of them.
 “The degree to which a set of inherent characteristics fulfils
requirements.” (SS-EN ISO 9000:2000)
 “Fitness for use” (Joseph Juran, 1953)
 “Conformance to requirements (requirements meaning both the
product and the customer's requirements)” (Philips Crosby, 1979)
 “Quality of a product (part of service) is the ability to satisfy and
preferably exceed the customer needs and expectation” (Bergman
Klefsjö, 2004)
Quality means improving services, products, processes and systems, to
assure that the organization products or services are fault free and
satisfy customers’ need.
Internal Quality in software development is the degree of the design
excellence of software artefacts that comprise the software product.
Built-In Quality refers to the practice when the internal quality of
software is built-in the software development artefacts on a continuous
basis with help of automated measurement systems and guidelines.
Crucially, internal quality of software (figure 5) has a decisive impact
on product quality. The benefits of built-in quality practice are: (V.
Antinyan, etal. ,2017)

14
  Significantly reduced number of defects
 Minimized risk of late design modification
 250-500 % reduced maintenance time

Figure 5 – Internal quality ISO/IEC 9126-1:2000

2.2. Agile way of working

Agile Software development methodology focuses on iterative and

incremental deliveries. Unlike long-plan-driven development,
requirements are not fixed and frozen before the development.
Customer may come with new requirement(s) during the development.
Customer is one of the main stakeholders and customer collaboration
is one of the essential elements of agile manifesto. Agile manifesto
emphasizes on following statements. (The Agile Manifesto, 2001)
 Individuals and interactions over processes and tools
 Working software over comprehensive documentation
 Customer collaboration over contract negotiation
 Responding to change over following a plan

15
Agile methodology emphasizes on Empowered team. A self-organized
knowledgeable team that is responsible and capable of high quality
product delivery.
As stated earlier, Agile proved to be a successful methodology for
software development. Faster delivery in shorter iteration, delivering
customer value on every
iteration, with the
possibility to change of
requirement; increase
customer satisfaction.
Figure 6 is a comparison
between agile and
waterfall methodologies.
In Agile, developers are
within an empowered, Figure 6 – Agile vs Waterfall
self-organized team
without traditional management and fixed deadlines. Agile team is
responsible for software quality and there will not be a dedicated
separate team looking after quality and quality assurance activities.
Many software companies found agile methodology as a suitable way
of working with better performance and moving toward this change.
To build a quality product, every software product artefact, including
requirement specification, architect, model, code, test and
documentation, must be well qualified during the development. For
example writing bad requirement may lead to wrong architect and
wrong code. Necessary methods and tools are needed to support this
success.

16
2.3. Built-in quality method and tools

This section refers to previous studies and findings around built-in

quality into requirements writing and code development. And then
looks at existing tools for measuring these two development artefacts.

2.3.1. Built-in quality in requirement

Unclear requirement specification can be misinterpreted and thus will

be the cause of incorrect design and late design modification (Kamata
and Tamai, 2007). But what is a good or bad requirement? And is there
any measurement to measure the quality of software requirement
specification?
Detail study to answer those questions is not the objective of this
thesis. However, the answer is yes; there are differences between good
and bad requirement specification. There are several studies and
papers for building quality into software requirement specification and
there are several frameworks to measure it.
Below is the list of important factors for building quality into
requirement specification.
 Purpose
Is the purpose for the requirement clearly stated?
Does the requirement add value?
 Complete
Does the requirement contain all relevant information?
Including use cases, modes etc.
Are all requirements simple and one at a time (atomic)?

17
 Are all prerequisites clearly defined?
 Clear
Does the requirement contain enough information? (Boolean
expressions like AND, OR, should be kept as simple as possible)
It is recommended that all statements written in positive terms.
Does each requirement state WHAT the system SHALL do and
under which conditions?
Interpretation of the requirement must be free of contradictions.
 Verifiable
It should be possible to test all requirements through system
demonstration, inspection and analysis.
Does the requirement define what quantity shall be measured
and in what unit?
 Traceable
Can the requirement be traced back to the requirement it
originates from?
Requirement should have a unique ID.
 Requirement vs Implementation
Is the requirement as free of solutions as possible?
(Focus on WHAT the system shall do. Avoid describing HOW the
system will do something)
Is the requirement feasible / realistic to achieve in terms of Cost,
implementation & added value?
The question is how to measure textual requirements quality. Briand
et al. (1996) proposed a set of internal quality properties for software
artefacts. An overview presented in Figure 7

18
In the left side, there are measure aspects including complexity,
coupling and size, and the right side is the list of measures to quantify
those aspects. For example large number of conjunctions and/or vague
phrases indicates the high complexity of the requirement.

Figure 7 – Requirements internal quality measurement

model

Below table (Table 1) presents internal properties of textual

requirement and definitions and measures with method for the
measuring of given measure (V Antinyan et al., 2016)

19
 Table 1 – Measures and measurement method

Some companies like Volvo Cars developed their own internal system
to provide with just-in-time feedback to requirement writer, based on
above or similar methods. These kinds of methods empowers the
systems responsible to get instant feedback while specifying the
requirement.

2.3.2. Built-in quality in code

There are intensive studies around code complexity in past decade as

it was the main characteristic of code internal quality. Code complexity
is the major aspect of internal quality with direct effect on code
maintainability (Banker et al. 1993)

20
Nesting depth, control statement, misleading comments, multi
developers, multi-tasking, frequent changes and complex

Figure 8 – influence of code characteristics on complexity. (Antinyan V et al., 2017)

requirements, are among of important code characteristics that

influence code complexity. There are several tools and methods to
measure the complexity and provide with just-in-time reporting.
Figure 8 is the result of a code complexity study with 100 participants
from seven companies and two universities. (Antinyan V et al., 2017)
The figure represents the influence of code characteristics on
complexity.
The order from left to right shows the important of influence factor in
code complexity in this study.

21
There are several metrics and measurement systems that can give
automated feedback on software complexity; like SonarQube, Pylint
and Pycodestyle.

22
3. Study design

This chapter is rather brief and is to explain the approach to conduct

the research at Volvo Cars, to be able answering the research
questions, presented in chapter 1.3

3.1. Research method

The research is a case study and quantitative approach. The objective

was to find the challenges and existing methods within the
organization. This case study is around existing quality process and
prediction cannot be made. The interviews conducted were centered on
built-in quality to get a detailed understanding of the participants’
sentiments. The questions asked were open ended allowing for the
possibility to explore and dig further into the interviewee’s responses.
“Open-ended questions allow the respondent to express an opinion
without being influenced by the researcher.” (Foddy, W, 1992)

Interviews conducted with eight key engineers. Thematic analysis was

used to generate patterns from individual ideas and interviewees
practices. Those patterns were prioritized and marked based on
number of interviewees with similar idea and then used for analyses.
In the end of this report the findings from Volvo Cars will be discussed
and whether the research questions were answered or not.

23
4. Data collection
This chapter represents the strategy and questioners that I used
during interviews for data collection.

4.1. Strategy

They strategy is to conduct total of eight interviews for requirements

writing and source code development. It is a semi-structured interview
with open-ended questions.

4.2. Questionnaire

The interview consists of three sections and planned for 30-45 minutes.

4.2.1. Demographics

The first part of the interview investigated the participant

demographics. Three fields were given for demographics information.
1- What is your role?
2- Can you briefly state the main activities of your role?
3- How many years of experience do you have in your job?

4.2.2. Main questions

The second and main part the interview structured in a way to achieve
following objectives that are directly link to research questions.
 Main challenges and activities to increase built-in quality of
source code or requirement.
 Existing standard and guidelines to improve quality.

24
  Metrics and measures for built-in quality.
To investigate above mentioned objective, following interview
questionnaire was designed and used.
1- Can you please describe the main challenges that decrease the
quality of Software artefact?
2- What activities do you have that intend to increase the quality of
Software artefact?
3- How often do you perform them?
4- How much do you think it helps to increase code quality?
5- What guidelines and standards do you use for improving Software
artefact quality?
6- What metrics or measurement systems do you have that can give
automated feedback on Software artefact quality?
7- How much do you thinks it helps?
8- What decision criteria are defined based on measurements to
trigger corrective actions?
9- What do you think is missing from the current state of measures,
guidelines, and tools?
Above nine questions are directly link to research problem and allow
us to draw patterns to answer the research questions.

4.2.3. Invitees and data collection

The interviewees have been selected by departments’ managers and

the interviews conducted in person and one to one meeting. Data
collected by interviews during interview and compiled after completion
of all interviews.

25
4.3. Collected Data

Total eight interviews conducted from eight participants including five

developers that were responsible for software source code development
and three system responsible who were in charge of software
requirements specification. As mentioned in delamination chapters,
the study was only around two software product artefacts.
 Requirement specification
 Source code development
The demographic result in table 2 shows that the majority of the
interviewees had more than 6 years’ experiences in their field of
working. Due to nature of the business, 3 out of 5 developers were
model developers rather than source code programmer. They generate
models based on requirements specification and benefit from auto
generation code tool.

Role Main activity Experience

Software developer (5) Code developer (2) <3 years (1)

System responsible (3) Model developer (3) 3-5 years (1)

Requirement specification
6-10 years (4)
(3)
>10 years (2)

Table 2 – Demographic information

26
Below Table 3 is the list of main challenges that decrease the quality
of software artefact. (Second question of questionnaire)

Main Challenges Description

Lack of communication to know how the function and

interfaces work with other requirements and providing crisp
Uncertain, complex
information to next level. Next level can be requirement
and unclear
realization or software architect.
requirement (3)
If the requirement is very high level and not broken,
supplier may understand it differently.
Matlab is not adequate for test development and has lack of
Lack of good test effectiveness to find defects. Hard to mock in Matlab to find
platform (2) problems.
Lack of Continues Integration (CI) machinery.
It is more about delivery than quality with fix deadline,
Delivery vs qualified budget and content. Quality is the only thing that you can
delivery (2) sacrifice. In general, focus is on faster deliveries rather than
qualified deliveries. It is a development cultural issue.
Unclear system solution and interfaces between signals (in
signal database). High complexity and coupling between
Unclear systems
ECUs because of the old functions. This is more an architect
solution and
issue that may change in future platforms. However,
interfaces(1)
currently unclear requirement and complex ECUs coupling
is a challenge for good quality.
Occasional team member recruitments that requires large
Personal rotation and effort for training and communication. Every time a new
staff turn over (1) person joins the team, time and effort is needed to adopt to
the new environment, company culture and processes..
Dependencies of tools create a complex development tool
chain. This complexity can result in certain tools not
Tools dependencies (1) working when a new operating system or other tools are
introduced. Bugs and problems can be introduced due to
updates of these tools or operating systems.
Many testers are expected to write code for automating
Lack of software their tests. But these testers traditionally were not coders.
development This is a car manufacturer company and many developers
knowledge (1) came from mechanical background with limited experience
of software development.
Windows 7 workstation performance are not fit for Software
development. This problem make developers to take
Hardware
shortcuts. To avoid writing complete test and even to think
performance (1)
how to write a test that take shorter run time and skip
Table 3 – Main challenges to decrease the quality of software artefact
pushing suite.

27
Activities that intend to increase the quality of software artefact is
shown in Table 4. The second column is the total number of positive
responses and the third column is the total number of interviewees
that the activity was applicable to their working software artefact.
They perform those activities whenever there is new requirement or
software release. Whenever new code is going to be merged. At least
once in every sprint. A sprint is the development iteration and usually
two weeks long.

All the interviewees believed those activities helps in different ways. They
mentioned following statements.
 It will be much easier for developers to find issues early before
integration to complete software.
 It is good but there is a lot of room to make it better.
 It helps a lot in a long run, keeps the high maintainability.
 Learning from each other and correcting each other mistakes

Total
Activities Number number of
interviewee
Peer review - design review 8 8

Statistic code analyser 2 4

Unit test 3 4

Visualizing testing chain 1 4

Integration test 2 4

Test coverage 3 4
1
Knowledge transfer within the team 4

Table 4 – Activities that intend to increase the quality of software

artefact

28
Next question was about guideline and standards that they use for
improving software artefact quality. Below table 5 is the list of
interviewees’ feedbacks. In this table the first column is the guidelines
and standards that one or more interviewees mentioned during the
interviews and second column is the number of interviewees who
benefit from this guideline and standard.

Number of
Guideline and Standards
interviewees

Volvo Cars guidelines for creating simulating

3
model

Internal coding guidelines 2

Awesome framework internal guidelines. This is

1
an internal build guideline

Internal Wiki modelling guidelines 1

Review template for filling review records.

1
Spelling error, logical error and et

Table 5 –Guideline and Standards

29
Table 6 is the list of metrics and measurement systems that can give
automated feedback on software quality artefact. The column two is
the number of interviewees that use related tool.

Number of
Tool name
interviewees

Cyclomatic complexity for code 3

SonarQube for code static analysis 1

Pylint and Pycodestyle for static analysis that gives

automatic fast feedback for code violations, writing 1
styles, and nesting depth.

Code coverage tool for unit test. Pass/fail for unit test 1

In-house developed tool to indicate which models have

1
been reviewed or when are they going to be reviewed.

Table 6 – Metrics and measurement systems

30
Table 7 shows the list of items that are missing from the current state
of measures, guidelines, and tools. Second column indicates number of
interviewees who mentioned the missing item during interviews.

Number of
Items
interviewees

Measures and guidelines 4

Continues integration complete platform 2

User guide documentation for tools and benefits 1

Developers training 1

Table 7 – What is missing from current state of measures, guidelines and

tools

31
5. Findings and conclusions
This chapter is the summery of finding and conclusions. At the end it
presents the future work.

5.1. Summary of the findings and conclusion

All interviews were open discussions for open equations and findings,
to allow the interviewees to discuss any uncovered area. All of
interviewees were well experienced in their field of expertise. Built-in
quality was a very interesting subject for all, especially during agile
transformation at Volvo Cars.
The first and important observation was the finding in main
challenges. The expectation was to find more about code complexity
characteristics and technical subjects. Although some of the main
challenges were technical challenges, but majority of them were
around communication and business non-technical concerns. Some of
the points like “delivery vs qualified delivery” was more personal and
team dependent.
However, this is an improving opportunity for Volvo Cars during agile
transformation. Because communication and self-organized team is
one of the agile essentials. Such conclusion proves that agile
transformation was the right decision for Volvo Cars to overcome with
these challenges. It was interesting to see how hardware performance
can challenge software product quality. It shows that quality
engineers must look at bigger picture to be able to help more
efficiently. It should not be only about code and requirement, it is
about overall software quality life cycle. More communication with

32
other stakeholders including higher management can resolve some
business-related challenges like personal rotation.
Based on answers to second question, almost non-of the suggested
activities that intend to increase the quality of the software artifact
are directly related to exiting challenges in fist question. Table 8 shows
the relation between challenges and activities suggested by each
interviews.

Main Challenges Activities to increase the quality of the software

Uncertain, complex and Peer review -

Unit test
unclear requirement design review

Knowledge
Lack of good test Peer review -
transfer within
platform design review
the team

Delivery vs qualitied Peer review - Statistic code

Unit test
delivery design review analyser

Unclear systems solution Peer review -

Unit test
and interfaces design review

Personal rotation and

staff turn over

Peer review - Visualizing

Tools dependencies
design review testing chain

Lack of software
development knowledge
Unit test isolated
from main Statistic code
Hardware performance Integration test
development analyser
machine
Table 8 – Relation between challenges and activities

Once again, agile is an opportunity to elaborate on sharing best

practices within the different units and teams.

33
Another finding from this study is that the quality assurance engineer
can support and possibly improve built-in quality at Volvo Cars. This
can be achieved by meeting development team and understanding
their challenges and activities. It may not be the same in different
organizations. Hence, quality assurance engineer should not assume
that all companies operate in same way with similar challenges.

For guidelines and standards, the study indicates that large enterprise
solution with large number of development teams, requires centralized
guidelines and standards. This action can save lot of time and money,
and to some extent overcomes with staff turnover challenge. New or
transferred developer can be trained easier and best practices can be
discussed between teams.

When we use different metrics and measurements systems for

automated feedback on software artefact quality, time and energy are
needed for mastering these tools. It has been observed that lack of
awareness on guidelines and systems, leads to poor quality of software
artefact. In case of centralized quality guidelines, measurement
systems and standards, quality assurance engineer can master the
tools with focus of selected guideline and standard. In this case
training and empowering with best practices can be more effective and
easier.

34
5.2. Future work

The first step was to study limited number of developers. However,

this study can be much more efficient if we increase number of
interviewees and decode more findings. We can then eliminate top
challenges by providing related activities. More interviews are needed
to understand existing guidelines and measurement systems. Then
providing with best practices for everyone in development team.
Secondly, we need to expand the study to other software artefacts,
including architecture, design, test and documentation.

35
References

Ajmer, Rajasthan (2013) A Comparative Study between Iterative

Waterfall and Incremental Software Development Life Cycle Model
for Optimizing the Resources Using Computer Simulation, 978-81-
920249-7-4/13/ 2013 IEEE, pp. 189

Antinyan V, et al. (2017) Empirical Software Engineering,

https://link.springer.com/article/10.1007/s10664-017-9508-2
[Accessed 15 May 2018].

Antinyan V et al. (2016) A method for automated reviews of textual

requirements, Elsevier

Antinyan V et al. (2017) Evaluating code complexity triggers, use of

complexity measures and the influence of code complexity on
maintenance time. DOI 10.1007/s10664-017-9508-2

Banker RD et al. (1993) Software complexity and maintenance costs.

Commun ACM 36(11):81– 95

Bergman Klefsjö (2004) Quality from Customer Needs to Customer

Satisfaction, Studentlitteratur AB, ISBN-10: 9144041667

Briand, L.C. , Morasca, S. , Basili, V.R. , 1996. Property-based software

engineering measurement. IEEE Trans. Softw. Eng. 22, 68–86 .

Businessinsider.com (2018),

36
<https://www.businessinsider.com/how-many-lines-of-code-it-takes-
to-run-different-software-2017-2?r=US&IR=T&IR=T> [Accessed 25
August 2018].

Foddy, W, 1992, Constructing Questions for Interviews and

Questionnaires: Theory and Practice in Social Research. Cambridge:
Cambridge University Press.
IEC-15504 (2014),
<https://en.wikipedia.org/wiki/ISO/IEC_15504#cite_note-1> [Accessed
23 March 2018].

ISO/IEC/IEEE 12207:2017. Standards catalogue. International

Organization for Standardization. November 2017. Retrieved 21 June
2018.

Kamata, M.I. , Tamai, T. , 2007. How does requirements quality relate

to project success or failure? In: Requirements Engineering
Conference, 2007. RE’07. 15th IEEE International, pp. 69–78 .

Philips Crosby (1979) Quality is Free, New York: McGraw-

Hill. ISBN 0-07-014512-1.

R. Lawrence (2006) Three-way cultural change: introducing agile

within two non-agile companies and a non-agile methodology, IEEE 0-
7695-2562-8

37
SAFe f for Lean Enterprise (2018)
<https://www.scaledagileframework.com/#> [Accessed 25 April 2018].
Agile Release Train - SAFe f for Lean Enterprise (2018)
<https://www.scaledagileframework.com/agile-release-train/>
[Accessed 15 August 2018].

The Agile Manifesto (2001) < http://agilemanifesto.org/>[Accessed 15

August 2018].

38
Appendix

A Appendix: Software Development Standards

A.1 ISO/IEC 12207

ISO/IEC 12207 “establishes a set of processes for managing the
lifecycle of software, including processes and activities applied during
the acquisition, supply, configuration, operation, maintenance, and
disposal of the software system, with each process having an
associated set of outcomes. The standard, however, does not prescribe
a specific software life cycle model, development methodology, method,
modelling approach, or technique." (ISO/IEC/IEEE 12207:2017)

A.2 SO/IEC 15504 - SPICE

“SO/IEC 15504 Information technology – Process assessment, also
termed Software Process Improvement and Capability Determination
(SPICE), is a set of technical standards documents for the computer
software development process and related business management
functions. It is one of the joint International Organization for
Standardization (ISO) and International Electrotechnical Commission
(IEC) standards, which was developed by the ISO and IEC joint
subcommittee, ISO/IEC JTC 1/SC 7.” (IEC-15504, 2014)

39
B Appendix : Quality assurance documents

1- Software Development Plan (SWDP) to demonstrate scope,

resources aligned with management and how team looks like. If
supplier used new/reused modules, software lifecycle and it
activities, company processes in how to work with Software and
how to maintain Software that is in production. First doc to go to
when introducing new team members
2- Software Risk Management. How the supplier collect risk
information? How often review them to mitigate them? And which
ones are shared with Volvo Cars
3- Software Development Schedule (SWDS) Overall planning for the
activities mentioned in SWDP
4- Software Progress Report (SWPR) What was done and where are
we heading. Milestones.
5- Software Configuration Management Plan (SWCP) Where to find
work products belonging to this project. Configuration item list.
Who reaches them? Baselines, change management and problem
resolution
6- Software Quality Plan (SWQP) Quality goals for this project.
Reviews, percentage of unit testing? QA organization?
7- Software Requirements Traceability Matrix (SWRT). To
understand which requirement are implemented and tested with
one/many test cases. The relation traceability between test and
code and requirement.
8- Software Design Description (SWDD) High level architecture
and modules and low level for how the modules works. Very

40
 important to understand construction when adding/removing
functionality or bug fixing.
9- Hardware/Software Interface Specification. Hardware and
Software interface for memory mapping.
10- Software Product Specification (SWPS) How to recreate a certain
and configure files. To make sure that knowledge will be
transferred in team.
11- Software Version Description (SWVD). What was done in this
version compare to last one? Problems, what works what doesn't.
12- Software Test Plan (SWTP). Test organization. Test environment,
test levels, automatic tests / manual tests.
13- Software Test Description (SWTD). How a requirement is
tested, which test steps. If test case verifies requirement
14- Software Test Report (SWTR) Evidence that SW will have all
desired requirements. Correctly respond to all inputs. use it to as
part of decision of SW is good enough to release at Volvo
15- Software Worst-Case Analysis Report (SWWCA) In real-time
computing (in vehicle), the worst-case execution time is often of
particular concern since it is important to know how much time
might be needed in the worst case to guarantee that the algorithm
will always finish on time.
16- SPICE assessment, Process maturity is only one part that
contributes to product quality. The more structured one works, the
lower is the likelihood to introduce systematic faults.

17- SPICE assessment action plan. Who drives this to make sure
that improvements will be done? (effective if independent quality
organization)

41