See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/358480404

Software Regression Testing in Industrial Settings: Preliminary Findings from

a Literature Review

Chapter · February 2022

DOI: 10.1007/978-3-030-96147-3_18

CITATION READS

1 319

5 authors, including:

Raul H. Rosero Miranda Omar S. Gómez

Escuela Superior Politécnica de Chimborazo Escuela Superior Politécnica de Chimborazo
29 PUBLICATIONS 133 CITATIONS 102 PUBLICATIONS 724 CITATIONS

SEE PROFILE SEE PROFILE

Raúl Antonio Aguilar Vera César Pardo

Universidad Autónoma de Yucatán Universidad del Cauca
86 PUBLICATIONS 231 CITATIONS 177 PUBLICATIONS 816 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Omar S. Gómez on 10 February 2022.

The user has requested enhancement of the downloaded file.

 Software Regression Testing in Industrial Settings:
Preliminary Findings from a Literature Review

Raúl H. Rosero1[0000-0002-2315-9773], Omar S. Gómez1[0000-0002-2951-3833], Eduardo R. Villa1,

Raúl A. Aguilar3[0000-0002-1711-7016], and César J. Pardo2[0000-0002-6907-2905]
1GrIISoft Research Group, Escuela Superior Politécnica de Chimborazo, Riobamba 060155,
Ecuador
2 Universidad del Cauca, Facultad de Ingeniería Electrónica y Telecomunicaciones, Programa

de Ingeniería de Sistemas, Grupo de Investigación y Desarrollo de Tecnologías de la Informa-

ción (GTI), Popayán, Colombia
3 Facultad de Matemáticas, Universidad Autónoma de Yucatán, Mérida, Yucatán, México

Corresponding author email: rrosero@espoch.edu.ec

Abstract. In the professional field of software development, unit tests are often
used as a verification mechanism of the code produced. With the design and ex-
ecution of test cases, it is possible to verify new or modified code, as well as to
verify existing versions of it. In order to prevent new defects from spreading
over existing versions of the code, it is necessary to strategically re-run different
test cases. This activity is known as software regression testing and can consti-
tute a high percentage of the total cost of the verification process. Researchers
continuously are dealing with making software regression testing efficient. In
order to shed light on the application of software regression testing in industry,
this paper presents a literature review on different aspects of regression testing
applied in industrial settings. As a result, 40 primary studies that report the use
of regression testing in the industry were identified. We observe that the main
regression testing technique used is the selection of test cases followed by the
prioritization of them. The use of combinations of metrics based on coverage,
requirements, risk, defects, cost-efficiency searches is also observed.

Keywords: Software Engineering, Software Engineering in Industry, Software

Regression Testing, Software Verification, Literature Review.

1 Introduction

Software testing constitute an important element in the software development pro-

cess. It contributes to the quality of the software developed or maintained. A particu-
lar type of software testing approach is known as software regression testing, which
aim is to determine a subset of test cases that should be re-executed in order to check
either injection or propagation of defects [1].
During software development or production stages, the software product can be
modified for different reasons, such as the fix of existing defects or the addition of
new functionalities; being required the application of regression tests; activity that
2

according to reports can cost up to 80% of the budget of the software development
project or up to 50% of the maintenance budget [2,3].
The frequency and the moment at which a regression test is applied depends on the
development context. In traditional software development processes, regression test-
ing is run after changes are made, usually at night or before new versions of the prod-
uct are released [4]; on the other hand, in agile development processes, regression
testing is run every time the code is saved and compiled. In any case, the goal of a
regression test is to help ensure that new changes have not affected the behavior or
functionality of the software product.
Different regression testing approaches, as well as tools are documented in the lit-
erature. Examples of these approaches are prioritization, selection, or reduction of a
set of test cases to be executed by this testing approach. However, research on this
topic is still ongoing in industrial settings [5,6]. It is observed a major research in
academic contexts than industrial settings [5].
The main objective of this work is to present a global vision of the main software
regression testing approaches, metrics and indicators reported in the industry. For this,
the application of a literature review [7] is conducted. The rest of the document is
structured as follows: Section 2 presents the relevant aspects regarding software re-
gression testing. Section 3 presents the method used for conducting the literature re-
view; later, in Section 4, the report of the results is discussed. Finally, in Section 5
some conclusions are presented.

2 Overview of Software Regression Testing Approaches

At first glance, conducting regression testing consists in re-test (or re-execute) all
of the available test cases. However, following this approach is expensive and time-
consuming. In order to reduce the “retest all” approach, approaches such as minimiza-
tion, selection, prioritization and optimization have been proposed. Following, we
briefly describe each one of these approaches.

2.1 Regression by minimization

According to [8] and [9], the minimization or reduction consists in the elimination
of redundant test cases. It is defined as following: Given a test suite, P, a set of re-
quirements {r1, r2,….rn}, which must be satisfied to provide the “adequate” test of a
program, and subsets of P1, P2,…, Pn, associated with each of the rj's, such that any of
the pj's belonging to Pi, can be used to meet ri requirements. The problem focuses on
representatively determine a P' of test cases which is subset of P, and that satisfies all
the requirements ri of P.

2.2 Regression by selection

In this approach, the reduction also occurs permanently, but this strategy focuses
on the detection of the modified parts of a software program (code), which normally
works based on static white-box analysis. According to [10], given a program P, the
 3

modified version of P, P' and a suite of tests T, the problem consists in determining a
subset of T, T', with which to test P'.

2.3 Regression by Prioritization

The objective of this approach [11] is to determine and to order a set of tests based
on a characteristic or property such as the ability to detect failures. Its main objective
is to find the ideal permutation of the sequence of test cases. Formally: Given a suite
of tests P, the set of permutations of P, PP and a function from PP to real numbers f:
PP → ℝ, then, find P' that belongs to PP, such that (∀P'') (P'' є PP) (P'' ≠ P') [f (P') ≥ f
(P'')].

2.4 Regression by Optimization

This strategy uses combinations of the aforementioned approaches. The use of

multi-objective optimization and artificial intelligence techniques are also used [12].
A thoroughly review of the mentioned approaches also the discussion of aspects
such as their implementation and domain can be found in [5].

3 Method

In order to examine evidence published in recent years on the subject of software

regression testing in industry and thereby characterize the information found, the
methodology proposed in [7] was used as a guideline.

3.1 Research questions

The purpose of a literature review is to bring together a set of existing knowledge,
in our case, concerning the application of software regression testing in the software
industry. Through posing a set of research questions, we aim to identify aspects as
well as factors of the regression testing approaches applied and evaluated in the soft-
ware industry. The questions posed for this literature review are as follows:

• RQ1 What software regression testing approaches are the most used in industry?
• RQ2 What software regression testing metrics are usually taken into account?
• RQ3 Under what process stage is regression testing usually applied?
• RQ4 In which software domains is regression testing applied?
• RQ5 At what level of verification is software regression testing applied?

3.2 Information sources

To obtain an expanded vision on this subject, different sources of information were

reviewed, including journals and conference proceedings on this subject [13]. Data-
4

bases such as: Scopus, Science Direct, Springer, IEEE Xplore, ACM Digital Library
as well as Google Scholar were considered in the present review.
These databases cover publications within the software engineering field [14]. Un-
published work in progress was not considered due to quality issues (only peer re-
viewed publications were considered).

3.3 Searching criteria

The searching criteria used for this review includes different terminology, the
search string created for this review is as follows:

(“Software regression testing” OR “software regression test”) AND (industry OR

industrial) AND publication year between 1990 and 2020.

We considered works from 1990 to 2020 related to software regression testing in

industry. Another aspect was the language, in this case English was the only language
selected. The initial search returned a result of 480 documents.

3.4 Study selection process

Once executed the first filter by excluding documents with regards their tittle, we
selected 420 documents, in the second filter we discarded documents by their sum-
mary, selecting in this stage 320 documents. We proceeded to read the contents of
these documents, thus selecting 44 documents. We were not able to access the con-
tents of four papers, so we ended up with 40 documents as primary studies. The pri-
mary studies selected cover information of the factors of interest to our review such as
metrics, regression testing approaches, methods used, size of the software applications
and finally the industrial context where the software regression testing is applied.

3.5 Information extraction and synthesis

Using the described procedure, 40 primary studies were selected. These studies
show empirical approaches that consider software regression approaches by prioritiza-
tion, selection and optimization with different application methods. We examined
each of the primary studies with regards the research questions previously presented.
Concerning the software products used by the regression testing approaches reported,
we classified the size of the product as follows: small, medium, and large (S, M, L),
according to the number of lines of code: up to 10k lines of code (LOC) small, from
10k to 100k medium, and large software products greater than 100k. Those works that
did not consider this information were left from the sample.

4 Results

In this section, we briefly summarize the findings with regards each of the research
question previously defined. With regards RQ1 (What software regression testing
 5

approaches are the most used in the industry?), it was observed that the test case se-
lection technique is the most commonly used (24 papers) [18-21,25-27,29,31,32,35-
37,40,41,43,45,47,48,50-54], followed by the prioritization technique (14 papers) [15-
17,22-24,28,30,34,39,43,45,47,50]. We observe that the approach that considers arti-
ficial intelligence methods such as data mining is the least used with 2 papers found
[32,39]. It is worth to mention that the minimization approach is not reported in the
primary studies we examined.
Regarding RQ2 (What software regression testing metrics are usually taken into
account?), we observe that efficiency by means of detecting software defects is the
most used metric with 23 papers [20-23,25,26,28, 30-32,34,36,37,39-43,47,49-51,53].
The second most used metric is the test case reduction metric with 8 papers [16,27,29,
35,36,40,44,48]. We also observe the reduction in execution time metric with 6 papers
[23,34,39,41,45,46]. Other metrics such as precision (4 papers) [34,36,38,40], classes
coverage, and based on requirements are also mentioned to be used.
With regards RQ3 (Under what process stage is regression testing usually ap-
plied?). We observe that software regression testing is mostly applied under a correc-
tive approach (software maintenance stage), with 22 papers [17,18,20,22,24,25,26,
28,29-32,36,38,39,43,44,47,53]. On the other hand, we found 17 papers [15,16,19,21,
23,27,34,35,37,40-44,48,49,54] mentioning to apply software regression testing under
a progressive approach (software development stage).
In the case of RQ4 (In which software domains is regression testing applied?), we
observe that software regression testing approaches have been applied mostly in the
context of information systems (17 papers) [15,21,23-25,28,32,35,36,41,43,45,46,48,
51,52,54] such as: healthcare services, commerce, resource management, finance.
Other application domains observed are telecommunications (10 papers) [16,20,
22,29,30,34,40,47,49,53], software development (web browsers development, 6 pa-
pers [24,25,26,29,32,40]) and Gaming with 4 papers [26,31,33,39]. We also observe
primary studies discussing regression testing in the context of real time systems such
as robotics and manufacturing (4 papers [17,18,19,42]).
Finally, for RQ5 (At what level of verification is software regression testing ap-
plied?), we observe that software regression testing is mainly used at unit testing level
(21 papers) [16,19-23,26,33,34,37,38,39-44,46,48,49,52], acceptance testing (17 pa-
pers) [15,17,18,24,25,27-31,35,36,45,50,51,53,54], and integration testing (2 papers)
[32, 47]. A summary of the aforementioned results is available in Table 1 of Annex
A.

5 Conclusions

Secondary studies like the one here reported are subject to limitations, in this
sense, the findings of our review have limitations related to the research design, the
databases, and the applied bibliometric approaches. Concerning the research design,
the choice of primary studies is based on a searching string, which does reduce the
scope of the study. However, although increasing the number of keywords in the
6

search string may improve the sample’s scope, it also tends to add noise (irrelevant
documents) and makes the sample increasingly challenging from a practical perspec-
tive. However, the primary studies analyzed here correspond to a representative sam-
ple on the main subject here addressed. We also reviewed the literature beyond the
primary studies included in the analysis and integrated important references into the
literature review. Due to a literature review is a demanding endeavor, perhaps possi-
ble recent relevant papers may have been left out from our review.
Our findings show that the software regression testing most applied in industrial
settings is the selection of test cases (RQ1). The metric commonly used is the effi-
ciency in the detection of defects (RQ2). Regression testing is mainly applied in the
maintenance stage of the software process (RQ3). With regards to the application
domains, it is observed that medium-sized information systems are mostly used
(RQ4). Finally, unit tests are commonly used in software regression testing (RQ5).
With regards our previous research [5], where we observed that most of the soft-
ware regression testing approaches are mainly reported under an academic setting,
software regression testing in industry tends to use only a subset of the existing re-
gression testing approaches reported by the academia.

Acknowledgments

The authors thank the Escuela Superior Politécnica de Chimborazo, in particular to

the Faculty of Informatics and Electronics for their support in carrying out this work.
Also thank anonymous reviewers. César Pardo acknowledges the contribution of the
Universidad del Cauca, where he works as an associate professor.

References

1. Rothermel, G., Harrold, M.J.: A safe, efficient algorithm for regression test selec-
tion. In: , Conference on Software Maintenance ,1993. CSM-93, Proceedings. pp.
358–367 (1993).
2. Leung, H.K.N., White, L.: Insights into regression testing [software testing]. In:
Proceedings. Conference on Software Maintenance - 1989. pp. 60–69 (1989).
3. Beizer, B.: Software Testing Techniques (2Nd Ed.). Van Nostrand Reinhold Co.,
New York, NY, USA (1990).
4. Humble J., Farley D.: Continuous delivery: reliable software releases through
build, test, and deployment automation. Pearson Education. (2010).
5. Rosero, R.H., Gómez, O.S., Rodríguez, G.: 15 Years of Software Regression
Testing Techniques — A Survey. International Journal of Software Engineering
and Knowledge Engineering. 26, 675–689 (2016).
6. Rosero, R.H., Gómez, O.S., Rodríguez, G.: Regression Testing of Database Ap-
plications Under an Incremental Software Development Setting. IEEE Access. 5,
18419–18428 (2017).
 7

7. Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., Linkman,
S.: Systematic literature reviews in software engineering – A systematic litera-
ture review. Information and Software Technology. 51, 7–15 (2009).
8. Rothermel, G., Harrold, M.J.: Empirical studies of a safe regression test selection
technique. IEEE Transactions on Software Engineering. 24, 401–419 (1998).
9. Yoo, S., Harman, M.: Regression testing minimization, selection and prioritiza-
tion: a survey. Softw. Test. Verif. Reliab. 22, 67–120 (2012).
10. Rothermel, G., Harrold, M.J.: Analyzing regression test selection techniques.
IEEE Transactions on Software Engineering. 22, 529–551 (1996).
11. Rothermel, G., Untch, R.H., Chu, C., Harrold, M.J.: Prioritizing test cases for
regression testing. IEEE Transactions on Software Engineering. 27, 929–948
(2001).
12. Mohanty, R., Ravi, V., Patra, M.R.: The application of intelligent and soft-
computing techniques to software engineering problems: a review. International
Journal of Information and Decision Sciences. 2, 233–272 (2010).
13. Santos, A., Gómez, O., Juristo, N.: Analyzing Families of Experiments in SE: A
Systematic Mapping Study. IEEE Transactions on Software Engineering. 46,
566–583 (2020).
14. Dieste, O., Grimán, A., Juristo, N.: Developing search strategies for detecting
relevant experiments. Empir Software Eng. 14, 513–539 (2009).
15. Lübke, D.: Selecting and Prioritizing Regression Test Suites by Production Us-
age Risk in Time-Constrained Environments. In: Winkler, D., Biffl, S., Mendez,
D., and Bergsmann, J. (eds.) Software Quality: Quality Intelligence in Software
and Systems Engineering. pp. 31–50. Springer International Publishing, Cham
(2020).
16. Magalhães, C., Andrade, J., Perrusi, L., Mota, A., Barros, F., Maia, E.: HSP: A
hybrid selection and prioritisation of regression test cases based on information
retrieval and code coverage applied on an industrial case study. Journal of Sys-
tems and Software. 159, 110430 (2020).
17. Wu, Z., Yang, Y., Li, Z., Zhao, R.: A Time Window based Reinforcement Learn-
ing Reward for Test Case Prioritization in Continuous Integration. In: Proceed-
ings of the 11th Asia-Pacific Symposium on Internetware. pp. 1–6. Association
for Computing Machinery, New York, NY, USA (2019).
18. Pal, D., Vain, J.: A Systematic Approach on Modeling Refinement and Regres-
sion Testing of Real-Time Distributed Systems. IFAC-PapersOnLine. 52, 1091–
1096 (2019).
19. Correia, D.: An industrial application of test selection using test suite diagnosa-
bility. In: Proceedings of the 2019 27th ACM Joint Meeting on European Soft-
ware Engineering Conference and Symposium on the Foundations of Software
Engineering. pp. 1214–1216. Association for Computing Machinery, New York,
NY, USA (2019).
20. Marijan, D., Gotlieb, A., Liaaen, M.: A learning algorithm for optimizing contin-
uous integration development and testing practice. Software: Practice and Expe-
rience. 49, 192–213 (2019).
8

21. Ali, S., Hafeez, Y., Hussain, S., Yang, S.: Enhanced regression testing technique
for agile software development and continuous integration strategies. Software
Qual J. 28, 397–423 (2020).
22. Wang, X., Zeng, H., Gao, H., Miao, H., Lin, W.: Location-Based Test Case Pri-
oritization for Software Embedded in Mobile Devices Using the Law of Gravita-
tion. Mobile Information Systems. 2019, e9083956 (2019).
23. Tulasiraman, M., Vivekanandan, N., Kalimuthu, V.: Multi-objective Test Case
Prioritization Using Improved Pareto-Optimal Clonal Selection Algorithm. 3D
Res. 9, 32 (2018).
24. Aman, H., Nakano, T., Ogasawara, H., Kawahara, M.: A Topic Model and Test
History-Based Test Case Recommendation Method for Regression Testing. In:
2018 IEEE International Conference on Software Testing, Verification and Vali-
dation Workshops (ICSTW). pp. 392–397 (2018).
25. Akin, A., Sentürk, S., Garousi, V.: Transitioning from Manual to Automated
Software Regression Testing: Experience from the Banking Domain. In: 2018
25th Asia-Pacific Software Engineering Conference (APSEC). pp. 591–597
(2018).
26. Marijan, D., Liaaen, M.: Practical selective regression testing with effective re-
dundancy in interleaved tests. In: Proceedings of the 40th International Confer-
ence on Software Engineering: Software Engineering in Practice. pp. 153–162.
Association for Computing Machinery, New York, NY, USA (2018).
27. Araújo, J., Araújo, J., Magalhães, C., Andrade, J., Mota, A.: Feasibility of using
Source Code Changes on the Selection of Text-based Regression Test Cases. In:
Proceedings of the 2nd Brazilian Symposium on Systematic and Automated
Software Testing. pp. 1–6. Association for Computing Machinery, New York,
NY, USA (2017).
28. Aman, H., Nakano, T., Ogasawara, H., Kawahara, M.: A Test Case Recommen-
dation Method Based on Morphological Analysis, Clustering and the Mahalano-
bis-Taguchi Method. In: 2017 IEEE International Conference on Software Test-
ing, Verification and Validation Workshops (ICSTW). pp. 29–35 (2017).
29. Ramler, R., Salomon, C., Buchgeher, G., Lusser, M.: Tool Support for Change-
Based Regression Testing: An Industry Experience Report. In: Winkler, D.,
Biffl, S., and Bergsmann, J. (eds.) Software Quality. Complexity and Challenges
of Software Engineering in Emerging Technologies. pp. 133–152. Springer In-
ternational Publishing, Cham (2017).
30. EmaShankari, K.H., ThirumalaiSelvi, R., Balasubramanian, N.V.: Industry based
regression testing using IIGRTCP algorithm and RFT tool. In: Lecture Notes in
Engineering and Computer Science. pp. 473–478. Newswood Limited (2016).
31. de Oliveira Neto, F.G., Torkar, R., Machado, P.D.L.: Full modification coverage
through automatic similarity-based test case selection. Information and Software
Technology. 80, 124–137 (2016).
32. Anderson, J., Salem, S., Do, H.: Striving for Failure: An Industrial Case Study
about Test Failure Prediction. In: 2015 IEEE/ACM 37th IEEE International Con-
ference on Software Engineering. pp. 49–58 (2015).
 9

33. Horváth, F., Vancsics, B., Vidács, L., Beszédes, Á., Tengeri, D., Gergely, T.,
Gyimóthy, T.: Test Suite Evaluation using Code Coverage Based Metrics. Pre-
sented at the Proceedings of the 14th Symposium on Programming Languages
and Software Tools (SPLST’15) October (2015).
34. Marijan, D.: Multi-perspective Regression Test Prioritization for Time-
Constrained Environments. In: 2015 IEEE International Conference on Software
Quality, Reliability and Security. pp. 157–162 (2015).
35. Arora, A., Chauhan, N.: A regression test selection technique by optimizing user
stories in an Agile environment. In: 2014 IEEE International Advance Compu-
ting Conference (IACC). pp. 1454–1458 (2014).
36. Fourneret, E., Cantenot, J., Bouquet, F., Legeard, B., Botella, J.: SeTGaM: Gen-
eralized Technique for Regression Testing Based on UML/OCL Models. In:
2014 Eighth International Conference on Software Security and Reliability
(SERE). pp. 147–156 (2014).
37. Gligoric, M., Negara, S., Legunsen, O., Marinov, D.: An empirical evaluation
and comparison of manual and automated test selection. In: Proceedings of the
29th ACM/IEEE international conference on Automated software engineering.
pp. 361–372. Association for Computing Machinery, New York, NY, USA
(2014).
38. Anderson, J., Salem, S., Do, H.: Improving the Effectiveness of Test Suite
Through Mining Historical Data. In: Proceedings of the 11th Working Confer-
ence on Mining Software Repositories. pp. 142–151. ACM, New York, NY,
USA (2014).
39. Marijan, D., Gotlieb, A., Sen, S.: Test Case Prioritization for Continuous Regres-
sion Testing: An Industrial Case Study. In: 2013 IEEE International Conference
on Software Maintenance. pp. 540–543 (2013).
40. Xu, Z., Liu, Y., Gao, K.: A novel fuzzy classification to enhance software regres-
sion testing. In: 2013 IEEE Symposium on Computational Intelligence and Data
Mining (CIDM). pp. 53–58 (2013).
41. Rogstad, E., Briand, L., Torkar, R.: Test case selection for black-box regression
testing of database applications. Information and Software Technology. 55,
1781–1795 (2013).
42. Buchgeher, G., Ernstbrunner, C., Ramler, R., Lusser, M.: Towards Tool-Support
for Test Case Selection in Manual Regression Testing. In: 2013 IEEE Sixth In-
ternational Conference on Software Testing, Verification and Validation Work-
shops. pp. 74–79 (2013).
43. Prakash, N., Rangaswamy, T.R.: Modular based multiple test case prioritization.
In: 2012 IEEE International Conference on Computational Intelligence and
Computing Research. pp. 1–7 (2012).
44. Chen, W., Iqbal, A., Abdrakhmanov, A., Parlar, J., George, C., Lawford, M.,
Maibaum, T., Wassyng, A.: Change Impact Analysis for Large-scale Enterprise
Systems. In: ICEIS (2012).
45. Srikanth, H., Cohen, M.B.: Regression testing in Software as a Service: An in-
dustrial case study. In: 2011 27th IEEE International Conference on Software
Maintenance (ICSM). pp. 372–381 (2011).
10

46. Rogstad, E., Briand, L., Dalberg, R., Rynning, M., Arisholm, E.: Industrial expe-
riences with automated regression testing of a legacy database application. In:
2011 27th IEEE International Conference on Software Maintenance (ICSM). pp.
362–371 (2011).
47. Engström, E., Runeson, P., Ljung, A.: Improving Regression Testing Transpar-
ency and Efficiency with History-Based Prioritization – An Industrial Case
Study. In: Verification and Validation 2011 Fourth IEEE International Confer-
ence on Software Testing. pp. 367–376 (2011).
48. Juergens, E., Hummel, B., Deissenboeck, F., Feilkas, M., Schlögel, C., Wübbeke,
A.: Regression Test Selection of Manual System Tests in Practice. In: 2011 15th
European Conference on Software Maintenance and Reengineering. pp. 309–312
(2011).
49. Wikstrand, G., Feldt, R., Gorantla, J.K., Zhe, W., White, C.: Dynamic Regres-
sion Test Selection Based on a File Cache An Industrial Evaluation. In: 2009 In-
ternational Conference on Software Testing Verification and Validation. pp.
299–302 (2009).
50. Ramasamy, K., Arul Mary, S.: Incorporating varying requirement priorities and
costs in test case prioritization for new and regression testing. In: 2008 Interna-
tional Conference on Computing, Communication and Networking. pp. 1–9
(2008).
51. Sneed, H.M.: Selective Regression Testing of a Host to DotNet Migration. In:
2006 22nd IEEE International Conference on Software Maintenance. pp. 104–
112 (2006).
52. Skoglund, M., Runeson, P.: A case study of the class firewall regression test
selection technique on a large scale distributed software system. In: 2005 Interna-
tional Symposium on Empirical Software Engineering, 2005. p. 10 pp.- (2005).
53. White, L., Robinson, B.: Industrial real-time regression testing and analysis using
firewalls. In: 20th IEEE International Conference on Software Maintenance,
2004. Proceedings. pp. 18–27 (2004).
54. Briand, L.C., Labiche, Y., Soccar, G.: Automating impact analysis and regres-
sion test selection based on UML designs. In: International Conference on Soft-
ware Maintenance, 2002. Proceedings. pp. 252–261 (2002).
 11

Annex A
Table 1. Characterization of the software regression testing approaches used in industry.
RT Verification
Ref. App. domain App. size Process stage Metric
Approach level
[15] Prioritization Acceptance Information system Medium Development RC
[16] Prioritization Unit testing Telecommunications Small Development TCRS
[17] Prioritization Acceptance Robotics Large Maintenance CR
[18] Selection Acceptance Manufacturing Large Maintenance ID
[19] Selection Unit testing Manufacturing Large Development CR
[20] Selection Unit testing Networking Large Maintenance DDE
[21] Selection Unit testing Information system Large Development DDE
[22] Prioritization Unit testing Networking Small Maintenance DDE
[23] Prioritization Unit testing Information system Medium Development DDE, ETR
[24] Prioritization Acceptance Information system Medium Maintenance TCS
[25] Selection Acceptance Information system Large Maintenance DDE
[26] Selection Unit testing Videoconference Medium Maintenance DDE
[27] Selection Acceptance Software testing tool Small Development TCRS
[28] Prioritization Acceptance Information system Medium Maintenance DDE
[29] Selection Acceptance Electronics Large Maintenance TCRS
[30] Prioritization Acceptance Networking Medium Maintenance DDE
[31] Selection Acceptance Gaming Medium Maintenance DDE
[32] Optimization Integration Information system Large Maintenance DDE
[33] Selection Unit testing Web browser Large Maintenance PC
[34] Prioritization Unit testing Telecommunications Small Development DDE, PR, ETR
[35] Selection Acceptance Information system Medium Development TCRS
[36] Selection Acceptance Information system Small Maintenance P, DDE, TCRS
[37] Selection Unit testing Software tool Medium Development DDE
[38] Selection Unit testing Software tool Large Maintenance P, R
[39] Prioritization Unit testing Entertainment Medium Maintenance ETR, DDE
[40] Optimization Unit testing Telecommunications Medium Development DDR, TCRS, P
[41] Selection Unit testing Information system Large Development DDR, ETR
[42] Selection Unit testing Electronics Large Development DDR
[43] Prioritization Unit testing Information system Medium Maintenance DDE
[44] Selection Unit testing Software tool Large Maintenance TCRS
[45] Prioritization Acceptance Information system Large Development ETR
[46] Selection Unit testing Information system Large Development DDR, ETR
[47] Prioritization Integration Telecommunications Large Maintenance DDE
[48] Selection Unit testing Information system Large Development TCRS
[49] Selection Unit testing Telecommunications Large Development DDE
[50] Prioritization Acceptance Software tools Small Maintenance DDE
[51] Selection Acceptance Information system Large Maintenance DDE
[52] Selection Unit testing Information system Large Maintenance CC
[53] Selection Acceptance Telecommunications Medium Maintenance DDE
[54] Selection Acceptance Information system Small Development ReqC
Terminology: TCRS (Test Case Suite Reduction), ETR (Execution Time Reduction), RC = (Risk Cover-
age), CR (Cost Reduction), CC (Class Coverage), PC (Partition Coverage), ReqC (Requirements coverage),
TCS (Test Case Similarity), DDE (Defect Detection Efficiency), (ID = Index of Diagnosability), DDR
(Defect Detection Reduction), P (Precision), R (Recall).

View publication stats