Testing and QA

on Development stack with Agile

methodologies

B INH N GUYEN T HAI

Etvs Lornd University,
Department of Cartography and Geoinformatics
July 3, 2014

Agile methodologies from testing and QA perspective

Lets discuss briefly what is the problem with traditional waterfall development methodology? The biggest disadvantage is it assumes that every
requirement are identified before any coding and design occurs, involving
Test Design phases:
1. Identify test conditions and test cases.
2. Design test cases.
3. Test strategies.
4. Risk evaluation.
5. Incident management.
By disintegrating waterfall model into smaller and shorter interations, we
are capable of react flexibly and gradually as user requirements are clarified
as we develop.
Agile software development methodology is all about releasing new
functionalities in smaller cycles based on small and highly efficient team(s)
which are self-organized. A sprint can be considered as a cycle or iteration,
usually takes 1 to 6 weeks. Each sprint has its task, after a sprint have
been design, its invulnerable, meaning that task can not be thrown or
added during a sprint. A sprint involves all the software development steps
1

and components including testing and QA. Agile methodologies shall be

integrated into a companys circulation gradually, meaning features and
diciplines are developed and acquired in small steps.
However, can we answer these questions before starting to implement
Agile methodology with testing and QA:
Does testing and QA will effect cost values in prototype development?
During prototype development, a software may contains defects and
its acceptable during a demo, because our primary task is to convince
stakeholders of our capabilities and prove that the concept is good.
Nevertheless, do we need testing and QA at this stage? Can we make
a prototype software with reusable components? How will testing
and QA increase development costs?
What is the learning curve and costs for testing and QA?
For an un-experienced team, with lack of testing and QA knowledge,
the learning curve could take time and efforts, does Agile methodologies gives any best practices for that? Agile manifesto only gives
corner stones for development and management issues, to answer this
question precisely is a challenge for every organization. It depends
on the team, project and development stack providing handy tools
supporting testing and QA.
Do we need technical and/or lead developers familiar with QA?
We may assume that a self-organized and disciplined team have already posseses testing and QA knowledge. What happens with a
team having a lead developer or technical leader, the only one person
familiar with testing and QA, how much will it effect on quality values
of the software, leads us to the seond question.
Do we need testing and QA across development stack?
Lets take an MVC architecture providing all the tools for testing all 3
layers, but what happens with the rest? Integration test, compatibility
test, communication test across services residing on different platforms
and so on. Can we guarantee our test coverage and say that its bullet
proof? Can we consider our development stack as an isolated entity
within a large software eco-system?
Tools supporting QA?
Your code is stored in a code versioning system (GIT, SVN, Perforce, etc.),
with classical branch, tags layout. Unfortunately most of us think of it
as a file storage with versioning capability. A good repository layout
could save time and costs. A good repository layout is a quality factor.
Along with versioning system, we also need a continous integration

system where all the automated tests runs on every commit, notify
resposible people if there is any failure, break, error.
Opensource your software components will increase risk
Opensource is a wonderfull fenomena, but one of its drawback is
quality. Many people are working on the same code-base. These
developers are taking measures to implement their tests first then write
their code using TDD to increase code coverage, however they test
are usually stuck at unit level, defects commonly occur at component
level. How to prevent this?
We may conclude that to achieve a well-tested and high quality software we must know the capabilities of our development stack, other tools
supporting QA described in the following sections.

What is a Development Stack

The term is quite familiar with developers, however problems and quality
aspects become vital when your are working with more than two people. Therefore, you must get familiar with the building blocks of your
development stack. We will particularly take a closer look at web-based
development stacks based on MVC architecture.
Unlike programming and development methodologies, to get familiar
with a development stack is quite a challange. It could be complex, difficult
to setup, with long learning curve. Lets take a look at each building block
or component individually:

2.1

Server-side language

There are many web-frameworks based on both object oriented and scripting languages. Among object oriented languages JAVA is the most wellknown and used. The reason is fairly simple, because the language itself
follow clearly stated standards and technologies, not to mention many big
companies like Google, Oracle have committed themselves with it like Playframework or Oracle JEE and lets not forget Spring-framework. However,
in order to develop a web application in Java may take some time and lot
of efforts, which is not affordable in some cases, hence web-frameworks
based script languages are more suitable for startup solutions and prototype
applications like Ruby on Rails, eg.: Twitter was originally written in Ruby
on Rails, later ported to JVM. There are some interesting developments
going on mixing the flexibility of scripting languages with JVM architecture
like Groovy or design a new functional language like Scala. The above

mentioned languages and related frameworks are multiplatform, but lets

not forget Microsoft out of the survey, who has improved a lot recently on
their Web-framework (MVC 5), unfortunately its not multiplatform. Of
course, there is a work-around called Mono, but its still behind with a few
versions of .NET framework.
All of the frameworks support testing and QA support for a developer
in most of the cases. We must find a close-to-perfect combination of tools to
achieve the best quality of our software, which will be discussed in the next
chapters.

2.2

Database engine

A web application is simply a set of HTML pages containing information in

it. These information are published, modified or deleted which is stored in
a database. The number of databases to choose from is vast, depending on
the nature of data, usage of information, structure of data storage we use
different kind of databases:
1. Graph database: Neo4J
2. Document database: MongoDB, CouchDB
3. Relational database: PostgreSQL, Oracle, MSSQL, DB2
4. Text based database: Hadoop HDFS, Google Big Table
The most important quality factor when working with databases is: consistency on both structural (schema) and tupple (content) levels. How to
achieve and maintain it? The solution could be setting up rules and quality
aspects for both input, output and intermediate data. As for structural integrity schema migration feature must be supported by development stack
on database engines.

2.3

Web-Server and load balancer

The web server is responsible for serving your web-application, behind

it there is usually a load-balancer, who is responsible for dispatching request among application instances located on clusters. At first glance, its
a simple architecture design, but things may get complicated when a webapplication is communicating with other applications or services, this is
where integration and component testing is vital.
Getting familiar with development stack, we know where to test, but
what tools and development methodologies exists for improving our test
coverage and quality of our software?

Figure 1: Development Stack

TDD and BDD

3.1

Test Driven Development

At first we might think of it as a software development methodology where

tests that drive your development flow. However TDD is much more than
that, having 5 steps:
1. Developer must write tests before coding.
2. Developer runs those tests, obviously with failure.
3. Developer actually implements the code related to test cases.
4. Developer run those tests again, until all the tests have passed.
5. Developer can send the code for review, refactor, optimization.

3.2

Behavior Driven Development

BDD is a software development process based on TDD focusing on the

behavioral specification of software components. Test in BDD are designed
to specify a desired behavior of the current software module. Usually a
behavior specification contains:
Title
Narrative, a shortly introduce who is the actor, what are the features or
business values.
Acceptance criteria for each scenario.
5

For example:
Feature: Addition
In order to avoid silly mistakes
I want to be told the sum of two numbers
Scenario: Add two numbers
Given I have entered 50
And I have entered 20
When execute Addition
Then the result should be 70
A step definition code is needed, for example:
Given I have entered (.*) do |n|
adder = Adder.new
adder.push(n.to_i)
end
execute Addition do
adder.execute
end
Its your choice which you method do you prefer. Tests itself does not
improve our code as well as tests does not find bugs for you. However a
developer uses TDD, must think, before coding. A developer must think,
how a function, a class, a module and a component should work, what side
affects (state transformations) will it make and so on. All these thoughts are
implemented in test cases. TDD is a safety belt for your software.

Mock-Objects

Mock objects are simply simulated objects, simulating the behavior of real
objects in a controlled manner. Mock objects are created by developers to
test certain behaviors of an object. When do we need them:
1. Slow runs.
2. Difficult to produce.
3. Producing non-deterministic results.
4. Does not yet exists.
5. Intended only for testing purposes.

By using Mock objects, developers are focusing on testing the behavior of

the system, without worrying about its dependencies. Most of the web
application heavily rely on database communications, which is expensive,
because tests must run fast using mock objects could reduce communication
costs drastically.

Continous Integration

A continous integration system (CI) are systems that build, test software
automatically and generate report. They can run slow, data-intensive tests
or verification of proper performance test on different platforms. Users are
getting immediate feedback of current or previous status and stages of the
software.
The primary advantage of using a CI system from QA aspects is immediate feedback on any defect. Both developer team and testing team can
react to this, localizing the source of that defect is fairly simple, because on
the report, we know, which modules test case(s) have been broken. Also
metrics are generated from automated testing like:
1. code coverage
2. code complexity
3. feature completeness
Continous integration itself does not give you quality at all. The core of
a good continous integration system is the work-flow that developer team
has set-up for themselves. A good layout of code repository is also very
important, for example product owners request the current version of the
software is quite simple: we take the latest stable version of the software
and deliver it, however when product owners wants a version with certain
set of features included under-development and bug-fixed functionalities,
we may have a hard time to produce it, CI is useless if the repository layout
is not configured properly.

Conclusion

Despite the knowledge of software testing and QA there are many other
methodologies, architectures, techniques and tools we must get familiar
with especially in a rapidly changing world of Agile development. Testing
and QA questions and answers on a development stack itself is crucial
to fullfill a good quality software. QA engineers, disciplined team and
management is required.
7