An in depth study of quality assurance in software

development

James Lofting1 Dr. Peter Bednar1,2

[up811674@myport.ac.uk] [peter.bednar@port.ac.uk]
[1]
School of Computing, University of Portsmouth, UK
[2]
Department of Informatics, Lund University, Sweden

Abstract. The abstract should summarize the contents of the paper in short
terms, i.e. 150-250 words.

Keywords: First Keyword, Second Keyword, Third Keyword.

1 Introduction

In the empirical study addresses three main research questions around the topic of
software quality assurance (QA). The topic of choice arose from a practical experi-
ence in the form of a placement year working in the QA team in a large IT depart-
ment. The experience resulted in an increasing interest of QA in IT.

The research questions are as follows;

 What is the current understanding of QA activities in software development?
 What are the different aspects/applications of QA activities in software de-
velopment?
 What are the advantages and disadvantages of a QA activities in software
development?

The overall aim of the study is to increase the quality of software by increasing
awareness and understanding of software quality assurance. A key assumption made
by the study is that there is a lack of understanding and engagement in QA activities/
process in the IT industry. The scope of the research is limited to quality assurance in
software development and software development lifecycle. QA processes in the de-
velopment of systems for different industries are included, for example healthcare,
education but QA outside the domain of software development, for example web
development, is outside scope.
8 key stakeholders were interviewed over a period of 2 weeks, each interview took
between 30 and 90 minutes. The stakeholders where made up of academics, business
analysts, developers, a project manager and a solution architect with varying amounts
of experience in the IT industry, ranging from 1 year to 30 years’ experience. Partici-
pants were aged between 21 and 55 years of age working in different industries, in-
2

cluding insurance, banking, academia and manufacturing. 3 stakeholders has less than
2 years’ experience working in IT.

2 Background

Background to each finding – software testing, perception of QA and

definition of QA and QA.

3 Empirical study

3.1 Quality assurance as software testing

The empirical study found several examples in which software quality assurance was
thought of as software testing.
Research question 1 investigated the understanding of software QA activities. The
majority of comments made during interviews referred to software testing.
Out of 72 comments recorded during interviews, 49 were sorted into 4 emergence
themes. The software testing theme comprised of 26 comments, the largest by far; the
next largest theme being audit/regulatory with 13 comments. The results of interview
question 1 (“When I say software QA activities, what kind of things/activities come to
mind?”) harmonises with this finding as almost half (45%) of all comments recorded
referred specifically to testing, showing testing makes up a large proportion of QA
activities known by participants in the sample. Specifically, the participant stated
different types of testing; 43% of all comments in the testing theme stated a type of
testing (for example unit testing, usability testing, pen. testing). However, interview
question 4 (“How would you define Software QA?”) offers a contradictory result with
T2 receiving 4 comments followed to T1, 3 comments. This is puzzling as overall
thematic analysis would suggest T1 would receive the largest number of comments,
whereas in reality this is not the case.
Comments were not limited to software testing, 13 comments referred to audit/reg-
ulatory processes and 10 referring to requirements gathering/management processes.
 3

Frequency of comments describing each theme (Interview

Questions 1-5)
T1
(Software Testing)

T2
(Requirements Gathering/
Management)

T3
(Audit/Regulatory)

T4
(Organisational Viewpoint/
Business Value)

0 5 10 15 20 25 30

Fig. 1. Figures shows the frequency of comments in each theme for research question 1 (inter-
view questions 1-5).

This is an interesting yet unsurprising finding because testing is arguably the

largest QA process (or set of processes) and requires a large amount of resource. Dur-
ing the placement most IT colleagues believed that QA were involved predominantly
with process audits, as well as testing. However, QA is not only concerned with pro-
cesses involved in the actual testing of software, but also with documenting the test-
ing process providing the project stakeholders with proof and progress as well as an
audit trail. Documenting of the testing process was not mentioned specially by partici-
pants in the study, however, some made general comments stating QA can be seen as
adding bureaucracy, but not specifically in the context of testing.
This shows that participants only referred to direct testing processes when referring
to QA, overlooking other QA processes designed to further assure quality, for exam-
ple documenting the software testing process.

3.2 Disadvantage of QA - Perception of QA process/people

Exploration of research question 3 (“What are the advantages/disadvantages of QA

activities in software development?”) found that the perception of QA process/people
is a key disadvantage of QA. An assumption made when devising this research was
that there is a lack of engagement in QA activities on some IT projects, due to prior
experience working in a QA role.
4

Interviews uncovered several disadvantages eluding to IT colleagues perception of

QA process and people, potentially leading to lack of engagement in QA activities.
For example, some participants stated QA can sometimes be seen as out of touch with
the rest of IT, introducing bureaucracy. Others stated colleagues did not see the value
in adhering to QA process potentially further causing disengagement and comple-
menting other perceptions.
Of the 7 disadvantages identified by the empirical study, 4 referred to perception of
QA process/people (T4) as seen in table 1, which when compared to disadvantages
referring to other themes, is a large amount. One disadvantages stated that IT col-
leagues do not see value in completing QA activities (“People involved in IT do not
see the value in completing/complying with QA activities/processes”), another stating
that QA people add bureaucracy (“The perception of QA people adding bureaucracy
instead of the actual activities themselves”). This shows a clear differentiation be-
tween QA people and QA activities in the context of lack of engagement in QA activ-
ities, stating it may not be the activities themselves, but stigma attached to the QA
people causing lack of engagement.

Table 1. Advantages and disadvantages identified in research question 2 (interview questions

6-7).
The T T T T
me 1 2 3 4
Ad- 4 1 1 0
van
t-
age
s
Dis- 1 0 2 4
ad-
van
t-
age
s
Due to 3 of 4 disadvantages stated in T4 commenting directly on perceptions of QA
people mean the perception of QA people is a key disadvantage of QA and could
contribute to the supposed lack of engagement in QA activities. Advantage of QA -
Software testing

The study found that most advantages reported referred to software

testing. This could indicate that software testing is the QA process that
offers the most advantages.
When participants in the sample were asked to outline advantages of software QA,
a total of 6 advantages were identified, 4 of which were categorized under the soft-
ware testing theme (as seen in table 1). This finding could mean participants gen-
uinely believe software testing offers the most advantages, however, the high number
 5

of advantages regarding software testing coincides with findings from research ques-
tion 1 (current understanding of QA) where most comments also related to software
testing. It could be the case that when participants were asked the question “What are
the advantages of software QA?” they initially thought in terms of software testing
and simply gave advantages of testing. Either way, this shows a high level of under-
standing of software compared to other QA processes.
In any case this shows that although some IT colleagues hold misconceptions
around software QA processes and people, participants also recognized several advan-
tages derived from QA activities.

4 Critical Discussion/Conclusions

Perception of QA processes and people is a plausible reason for the lack of engage-
ment in QA. This phenomenon was evident during the placement year spent in QA
with feedback from IT colleagues suggesting some thought QA actually slowed pro-
cesses via documentation that was deemed unnecessary causing some to think QA
people are out of touch with project level procedures. This has obvious negative im-
plications for software quality and may also negatively impact the interaction between
QA people and other IT colleagues. This mean this phenomenon may be less or more
prominent in different organizations due to structural and personal differences within
the industry.

Testing undoubtedly offers several advantages, however, the notion that most of
the advantages of QA are derived solely from testing processes (as reported by partic-
ipants) is not supported by knowledge gained from the placement experience. Audits
can help the “learn and improve” process of IT project teams, auditors and the wider
IT function as well as ensuring regulation/legislation is adhered to. Furthermore,
process assurance can save time via the implementation of efficient processes, reduc -
ing costs.

A large number of advantages stated by participant related to the software testing

theme, coinciding with previous findings that participants think of QA mostly in
terms of software testing. It is therefore plausible that participants reported advan-
tages of QA in terms of software testing as this was the first thing, and sometimes
only aspect of QA that came to mind. Participants’ practical experience of QA activi-
ties also suggests this, with most reported having experience of testing activities. This
finding coincides with that of research question 1 as the “Software Testing” theme
received the largest number of comments.
6

Confusion between quality assurance and quality assessment was evident throughout
the empirical study. On several instance participants spoke about quality assurance
but actually described quality assessment, especially when referring to software test-
ing. This is further demonstrates the lack of understanding of QA. Quality assurance
refers to processes which assure the quality of software during the development of
software, whereas quality assessment refers to processes that assess the quality of
software after the development process. It is important to make the distinction be-
tween quality assessment and assurance as they serve fundamentally different pur-
poses. Quality assurance aims to achieve the highest quality software in the form of
processes during the development of software. Quality assessment measures the qual-
ity of software against a set a predefined criteria after the software is developed. Con-
fusion between the two concepts could negatively affect the quality of software and
give an inaccurate indication of software quality. However, although 4 disadvantages
relating to the perception of QA people/process is significant when compared to other
findings in the empirical study, the finding is not statistically significant in the context
of the IT industry.

Software testing was a common theme throughout the empirical study. A potential
reason for this is level of experience of the participant in the sample. Some partici -
pants had little experience working in the IT industry as well as limited exposure and
involvement in QA practices. Others were not involved in QA practices at all. None
of the participants investigated had significant experience of QA practices in a large
IT project. This is important when reflecting on findings of the empirical study; the
research does not exclaim that colleagues working in software testing or QA do not
understand their roles. Instead, in conducting the research, we can see several exam-
ples of practitioners that appeared to lack knowledge of quality assurance.
.

Please note that the first paragraph of a section or subsection is not indented. The first
paragraphs that follows a table, figure, equation etc. does not have an indent, either.
Subsequent paragraphs, however, are indented.

Sample Heading (Third Level). Only two levels of headings should be numbered.
Lower level headings remain unnumbered; they are formatted as run-in headings.

Sample Heading (Forth Level). The contribution should contain no more than four
levels of headings. The following Error: Reference source not found gives a summary
of all heading levels.
 7

Table 1. Table captions should be placed above the tables.

Head Example Font

ing size
level and
style
Title
14
(cen-
tered
Lecture Notes point,
bold
)
1st-
12
level
head-
1 Introduction point,
bold
ing
2nd-
10
level
2.1 Printing Area point,
head-
bold
ing
3rd-
10
level Run-in Heading in Bold.
point,
head- Text follows
bold
ing
4th-
10
level Lowest Level Heading.
point,
head- Text follows
italic
ing

Displayed equations are centered and set on a separate line.

x+y=z (1)

Please try to avoid rasterized images for line-art diagrams and schemas. Whenever
possible, use vector graphics instead (see Error: Reference source not found).
8

50
45
40 Data
35 A
30
25
20
15
10
5
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Fig. 1. A figure caption is always placed below the illustration. Short captions are centered,
while long ones are justified. The macro button chooses the correct format automatically.

For citations of references, we prefer the use of square brackets and consecutive num-
bers. Citations using labels or the author/year convention are also acceptable. The
following bibliography provides a sample reference list with entries for journal arti-
cles [1], an LNCS chapter [2], a book [3], proceedings without editors [4], as well as a
URL [5].

References
1. Author, F.: Article title. Journal 2(5), 99–110 (2016).
2. Author, F., Author, S.: Title of a proceedings paper. In: Editor, F., Editor, S. (eds.)
CONFERENCE 2016, LNCS, vol. 9999, pp. 1–13. Springer, Heidelberg (2016).
3. Author, F., Author, S., Author, T.: Book title. 2nd edn. Publisher, Location (1999).
4. Author, F.: Contribution title. In: 9th International Proceedings on Proceedings, pp. 1–2.
Publisher, Location (2010).
5. LNCS Homepage, http://www.springer.com/lncs, last accessed 2016/11/21.