Continental Test Lab

August 2014

Interior Infotaiment and Connectivity R&D Romania

 Agenda

1Smoke Testing
2Regression Testing

Interior Infotaiment and Connectivity R&D Romania

2 11-03-26 Autor, © Continental Automotive Romania SRL
Smoke Testing

Interior Infotaiment and Connectivity R&D Romania

3 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Smoke Testing – Definition
Smoke Testing, also known as “Build Verification Testing”, is a type of software testing that
comprises of a non-exhaustive set of tests that aim at ensuring that the most important
functions work.

The results of this testing is used to decide if a build is stable enough to proceed with further
testing.

The term ‘smoke testing’, it is said, came to software testing from a similar type of hardware
testing, in which the device passed the test if it did not catch fire (or smoked) the first time it
was turned on.

Smoke testing can be used in all test levels

Interior Infotaiment and Connectivity R&D Romania

4 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Smoke Testing
Smoke testing covers most of the major functions of the software but none of them in depth.

The result of this test is used to decide whether to proceed with further testing.

If the smoke test passes, go ahead with further testing.

If it fails, halt further tests and ask for a new build with the required fixes.

If an application is badly broken, detailed testing might be a waste of time and effort.

Interior Infotaiment and Connectivity R&D Romania

5 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Smoke Testing
Smoke test helps in exposing integration and major problems early in the cycle.

It can be conducted on both newly created software and enhanced software.

Smoke test is performed manually or with the help of automation tools/scripts.

If builds are prepared frequently, it is best to automate smoke testing.

As and when an application becomes mature, with addition of more functionalities etc, the
smoke test needs to be made more expansive.

Sometimes, it takes just one incorrect character in the code to render an entire application
useless.

Interior Infotaiment and Connectivity R&D Romania

6 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Smoke Testing - Advantages
It exposes integration issues.

It uncovers problems early.

It provides some level of confidence that changes to the software have not adversely
affected major areas (the areas covered by smoke testing, of course)

NOTE
Do not consider smoke testing to be a substitute of functional/regression testing.

Interior Infotaiment and Connectivity R&D Romania

7 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
 Agenda

1Smoke Testing
2Regression Testing

Interior Infotaiment and Connectivity R&D Romania

8 11-03-26 Autor, © Continental Automotive Romania SRL
Regression Testing

Interior Infotaiment and Connectivity R&D Romania

9 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Regression Testing - Definition
Regression testing is a type of software testing that intends to ensure that changes
(enhancements or defect fixes) to the software have not adversely affected it.

The likelihood of any code change impacting functionalities that are not directly associated
with the code is always there and it is essential that regression testing is conducted to make
sure that fixing one thing has not broken another thing.

During regression testing, new test cases are not created but previously created test cases
are re-executed.

Interior Infotaiment and Connectivity R&D Romania

10 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Regression Testing - Definition
Regression testing can be performed during any level of testing (Unit, Integration, System,
or Acceptance) but it is mostly relevant during System Testing.

In an ideal case, a full regression test is desirable but oftentimes there are time/resource
constraints.

In such cases, it is essential to do an impact analysis of the changes to identify areas of the
software that have the highest probability of being affected by the change and that have the
highest impact to users in case of malfunction and focus testing around those areas.

Due to the scale and importance of regression testing, more and more companies and
projects are adopting regression test automation tools.

Interior Infotaiment and Connectivity R&D Romania

11 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Regression Testing
 There are three Modules in the Project named Admin Module, Personal Information, and
Employment Module and suppose bug occurs in the Admin Module like on Admin Module
existing User is not able to login with valid login credentials so this is the bug. 

Interior Infotaiment and Connectivity R&D Romania

12 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Regression Testing - How to choose test suites for Regression
Testing?
 On the one hand:

a. It’s often extremely difficult for a programmer to know how a change in one part of
the software will echo in other parts of the software.
AND what’s even worse:
b. Programmers sometimes change the software without even trying to figure out
if their changes might break something.

 On the other hand, testers usually know (or can make an educated guess) which features
will be directly affected by a change.

Interior Infotaiment and Connectivity R&D Romania

13 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Regression Testing - How to choose test suites for Regression
Testing?
 A good approach is to single out 3 groups of test suites for Regression Testing

 The 1st group must contain test suites that check legacy (i.e., existing) features that are
directly affected by a change in software.

 The 2nd group must contain test suites that check legacy features that may somehow
depend on the changed features

 3rd group. As a rule, the majority of test suites don’t fall into the 1st and 2nd groups. But
that majority surely needs to be tested too, because, as we know, a change in one part of
the software can often break other seemingly unrelated part(s) of the software. It’s usually
out of question to execute ALL of those test suites during Regression Testing for a single
release, so what we can do is to set up a plan to execute all the test suites within a defined
period of time – e.g., 6 months.

Interior Infotaiment and Connectivity R&D Romania

14 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Regression Testing

How To Resolve The Contradiction Between Our Limited Resources And The Ever
Growing Number Of Test Suites?

 We can resolve this contradiction by the following means:

1. Prioritization of test suites and test cases

2. Test suite optimization
3. New hires or outsourcing
4. Test automation

Interior Infotaiment and Connectivity R&D Romania

15 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Regression Testing - How To Resolve The Contradiction Between Our
Limited Resources And The Ever Growing Number Of Test Suites?

1. Prioritization of test suites and test cases

 The prioritization of test suites and test cases is the cheapest and, in many cases, most
effective solution to resolve the issue of contradiction.

 If you use test suite/case priorities as the criteria for test suite/case selection for Regression
Testing, you should keep those priorities up-to-date.

 The test suite/case priority must be changed, for example, when a tested feature loses its
importance or, on the contrary, becomes more important.

Interior Infotaiment and Connectivity R&D Romania

16 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Regression Testing - How To Resolve The Contradiction Between Our
Limited Resources And The Ever Growing Number Of Test Suites?

2. Test suite optimization

 Many old test cases can be optimized in two ways:

a. Reduction in the number of test cases

and/or
b. Simplification of the test case execution
 For point a – Very often it makes sense to review test cases that were written some time
ago to see if they make any sense today; for example, some features can be deprecated, or
maybe the test case is a duplicate of another test case.

 In some cases, it makes sense to retire whole test suites.

 For point b – Some test cases can be simplified because new helper tools are written.

Interior Infotaiment and Connectivity R&D Romania

17 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Regression Testing - How To Resolve The Contradiction Between Our
Limited Resources And The Ever Growing Number Of Test Suites?

3. New hires or outsourcing

 In some cases, this makes sense; in some cases, it doesn’t.

 To hire new testers or to outsource testing is the most linear way to resolve the
contradiction.

 There is no universal advice here; it all depends on the company and its type of business.

Interior Infotaiment and Connectivity R&D Romania

18 June 2014 Silvia Tanase, © Continental Automotive Romania SRL