Professional Documents
Culture Documents
Requirements - Traceability - A - Systematic - Review - Assignment 2
Requirements - Traceability - A - Systematic - Review - Assignment 2
net/publication/265807397
Article in International Journal of Software Engineering and Knowledge Engineering · May 2012
DOI: 10.1142/S021819401250009X
CITATIONS READS
92 1,201
6 authors, including:
All content following this page was uploaded by Richard Torkar on 22 February 2023.
1
May 19, 2012 6:13:42pm WSPC/117-IJSEKE 00584
1stReading
2 R. Torkar et al.
1 (i.e. from its origins, through its development and speci¯cation to its, subsequent
2 deployment and use, and through periods of on-going re¯nement and iteration in any
3 of these phases)."
4 The de¯nition of RT by [38] is considered to be a comprehensive de¯nition and is
5 cited by several researchers and the same de¯nition is also quoted at the Software
6 Engineering Institute's website [71].
7 According to [37] there are two aspects of RT: Pre- and post-RS traceability. Pre-
8 RS traceability is concerned with those aspects of a requirement's life from the point
9 before it is included in the software requirements speci¯cation (SRS) [6]. In pre-RS
10 traceability, requirements are related to their origin and other requirements. The
11 origin of requirements includes stakeholders, business rules or previous documents.
12 Post-RS traceability, on the other hand, is concerned with those aspects of a
13 requirement's life from the point after it is included in the SRS [6]. Post-RS trace-
14 ability ensures that all requirements are ful¯lled. In post-RS traceability, require-
15 ments are often related to test cases that ensure that software items satisfy the
16 requirements.
17 In addition to the above two aspects of RT, others have classi¯ed traceability into
18 the following four types [74]:
19
20 . Backward-from traceability: Links requirements to their sources which are in e.g.
21 other documents.
22 . Forward-from traceability: Links requirements to the design and implementation
23 components.
24 . Backward-to traceability: Links design and implementation components back to
25 the requirements.
26 . Forward-to traceability: Links other documents to relevant requirements, e.g.
1 This paper aims to establish the state of the art of requirements traceability as well
2 as to identify the main challenges reported by research and industry. In order to achieve
3 this a systematic review was performed covering the years 19972007. As a second step
4 two case studies were performed to validate and complement the review ¯ndings.
5
6 1.1. Research questions
7 Two research questions will be answered through a series of questions connected to
8 the review and its review protocol:
9
RQ 1. What is requirements traceability based on the state of the art in research and
10
standards?
11
The research on requirements traceability will be carried out mainly by the use of
12
a systematic review [55]. The focus of this systematic review will be on requirements
13
traceability from 19972007. This will help in identifying, clarifying and under-
14
standing requirements traceability in general and requirements traceability techni-
15
ques in particular.
16
17 RQ 2. What are the main factors reported by academia and industry as hindering
18 (proper) implementation of requirements traceability?
19 Some of the factors, for instance lack of coordination between people, failure to
20 follow standards or requirements traceability costs, may obstruct the implementa-
21 tion of requirements traceability policies; however, contributions from researchers
22 will hopefully clarify to what extent, and why, this happens. Interviews will be
23 conducted in industry to ¯nd out if there are any impeding factors which are not
24 reported in the literature.
25
26 1.2. Research methodology
27 A mixed approach using both quantitative and qualitative research methodologies
28 was adopted for this study. Through the systematic review, very much a quantitative
29 method, requirements traceability is mapped as an area through a number of de¯-
30 nitions and a number of issues regarding requirements traceability are presented.
31 Even though the quantitative results were signi¯cant, there was still a need to
32 complement the results with accompanying semi-structured interviews [70] carried
33 out in industry in order to receive an early initial vaidation.
34
35
2. Systematic Review
36
37 A systematic literature review provides a mechanism for evaluating, identifying and
38 interpreting all available research relevant to a particular research question, topic area
39 or phenomenon of interest [55]. Individual studies contributing to the systematic
40 review are known as primary studies while a systematic review is a secondary study.
41 A systematic review compromises three main phases: Planning the review, exe-
42 cuting the review and reporting the review. In the planning phase the need for the
43 systematic literature review is identi¯ed and a review protocol is developed. This
May 19, 2012 6:13:43pm WSPC/117-IJSEKE 00584
1stReading
4 R. Torkar et al.
6 R. Torkar et al.
1 Table 1. Articles on requirements traceability in selected journals and proceedings (the rightmost
column). The second column lists the total number of articles found during the time span
2
(19972007).
3
4 Articles
Articles (RT)
5
6 Journal
IEEE Transactions on Software Engineering (TSE) 751 4
7 ACM Transactions on Software Engineering Methodology (TOSEM) 139 1
8 Springer Requirements Engineering Journal 144 2
9 Springer Innovations in Systems and Software Engineering 48 2
Springer Software and Systems Modeling 106 1
10 Springer Annals of Software Engineering 150 2
11 Conference Proceeding
12 ACM International Conference on Software Engineering (ICSE) 1,272 3
13 IEEE International Conference on Requirements Engineering (RE) 357 12
IEEE International Symposium on Requirements Engineering 120 2
14
Total 3,087 29
15
16
(10) Does it evaluate or compare two or more RT techniques?
17
18 . Study Exclusion Criteria.
19 — If the answer is \no" to question 1 or 2, the study is excluded immediately.
20 — At least questions 3–10 must be answered \yes".
21
22 Study selection process. The study selection process was based on the title, ab-
23 stract and conclusion of the research paper. If it satis¯ed the inclusion criteria the
24 complete research paper was read.
25
Study quality assessment and procedures. The selected articles were then
26
evaluated based on the following criteria:
27
28 (1) Introduction. Does the introduction provide an overview of RT?
29 (2) Method. Was the research methodology clearly de¯ned in the research paper?
30 (3) Results. Were the study results presented in a clear manner? Do the results help
31 to solve an RT problem? What types of validity threats were de¯ned for the
32 results?
33 (4) Analysis. How was data analyzed? What type of analysis techniques were used?
34 If the article contained a framework, then has it been validated in an industrial
35 setting?
36 (5) Discussion and/or conclusion. Were negative ¯ndings properly reported? Were
37 there any limits or restrictions imposed on the conclusions claims?
38
39 Data extraction strategy. In order to obtain the information from each primary
40 study the following data extraction form was used:
41
(1) General information regarding research paper
42
43 (a) Article title
May 19, 2012 6:13:44pm WSPC/117-IJSEKE 00584
1stReading
8 R. Torkar et al.
1 Table 2. (Continued )
2 Ref. Title
3 [57] Recovering Traceability Links in Software Artifact Management Systems using Information
4 Retrieval Methods
5 [25] Advancing Candidate Link Generation for Requirements Tracing: The Study of Methods
[27] Design Traceability
6
[31] Automating Requirements Traceability: Beyond the Record and Replay Paradigm
7 [38] Extended Requirements Traceability: Results of an Industrial Case Study
8 [39] Crafting the Requirements Record with the Informed use of Media
9 [40] From Non-Functional Requirements to Design Through Patterns
[42] TRAM: A Tool for Requirements and Architecture Management
10 [44] Improving Requirements Tracing via Information Retrieval
11 [45] Improving After-the-Fact Tracing and Mapping: Supporting Software Quality Predictions
12 [46] REquirements TRacing On Target (RETRO): Improving Software Maintenance Through
Traceability Recovery
13 [47] A Case Study on Value-Based Requirements Tracing
14 [51] Requirements Tracing
15 [52] XTraQue: Traceability for Product Line Systems
[53] Modeling Functional Requirements to Support Traceability Analysis
16
[54] A Reusable Traceability Framework Using Patterns
17 [56] Reconstructing Requirements Coverage Views from Design and Test Using Traceability Recovery
18 via LSI
19 [58] Recovery of Traceability Links Between Software Documentation and Source Code
[59] Using Scenarios to Support Traceability
20 [60] Enhancing Traceability Using Ontologies
21 [61] Tool Support for Computer-Aided Requirements Traceability in Architectural Design: The Case
22 of DesignTrack
[62] Towards Method-Driven Trace Capture
23 [28] Experiences Using Scenarios to Enhance Traceability
24 [63] Factors In°uencing Requirements Traceability Practice
25 [65] Requirements Traceability: Theory and Practice
[64] Toward Reference Models for Requirements Traceability
26
[66] Pre-Requirement Speci¯cation Traceability: Bridging the Complexity Gap Through Capabilities
27 [68] An Evaluation of Traceability Approaches to Support Software Evolution
28 [69] Improving Software Quality Through Requirements Traceability Models
29 [72] A Framework for Managing Traceability Relationships Between Requirements and Architectures
[73] Scenario Advisor Tool for Requirements Engineering
30 [76] Rule-Based Generation of Requirements Traceability Relations
31 [77] Traceability for System Families
32 [80] Introducing E±cient Requirements Management
[81] Requirements Traceability to Support Evolution of Access Control
33 [83] Tracing Requirements to Defect Reports: An Application of Information Retrieval Techniques
34
35
36
37 The search was carried out in two steps. During the ¯rst step the selected elec-
38 tronic resources were explored online using the search terms de¯ned in the review
39 protocol. Research papers related to RT were downloaded and their details were
40 recorded. In the second step the selected conference proceedings and journals were
41 searched manually year by year (Table 1). Research papers related to RT were ¯rst
42 searched in the records. If they were not found in the records they were then
43 downloaded.
May 19, 2012 6:13:46pm WSPC/117-IJSEKE 00584
1stReading
10 R. Torkar et al.
1 20
2
18
3
4 16
5
14
6
7 12
8
10
9
10 8
11
6
12
13 4
14
2
15
16 0
19
19
19
20
20
20
20
20
20
20
20
17
97
98
99
00
01
02
03
04
05
06
07
18
19 Fig. 1. Year-wise distribution of primary studies on requirements traceability during 19972007.
20
21 Only three research papers [46, 52, 73] were found by manually exploring the
22 sources listed in Table 1. These papers were downloaded and their details recorded.
23 Concerning the distribution of research papers in relation to our selected journals
24 and proceedings please see Table 1 (obviously, all papers in the rightmost column are
25 included in Table 2).
26 Figure 1 represents the yearly distribution of the primary studies during the period
27 19972007. It is evident from the ¯gure that the number of publications on RT
28 peaked in the year 2005. One of the reasons for this increase in research publications
29 (which can already be seen before 2005) is the International Workshop on Traceability
30 in Emerging Forms of Software Engineering held in 2002, 2003, 2005 and 2007
31 (TEFSE'07 is not part of this search since the publications had not been published
32 electronically when the study was executed). At this workshop only research papers
33 related to traceability were presented and later published electronically.
34
35
36 3. Results from and Analysis of the Systematic Review
37
3.1. De¯nitions of requirements traceability (Question 1)
38
39 In [38] RT is de¯ned as, \the ability to describe and follow the life of a requirement, in
40 both a forward and backward direction (i.e. from its origin, through its development
41 and speci¯cation, to its subsequent deployment and use, and through periods of on-
42 going re¯nement and iteration in any of these phases)." This de¯nition is further
43 used by, e.g. [8, 10, 11, 27, 31, 47, 51, 28, 6466, 68, 81]. The de¯nition seems to be
May 19, 2012 6:13:46pm WSPC/117-IJSEKE 00584
1stReading
12 R. Torkar et al.
1 Gotel and Finkelstein, Hamilton and Beeby, DOD-STD-2167A and IEEE Std.
2 8301998 accomplish this.
3 It is evident from Table 3 that Gotel and Finkelstein's de¯nition of RT is dom-
4 inant (more than 80% of the primary studies, which de¯ned requirements trace-
5 ability, used this de¯nition).
6
7
8 3.2. Challenges in requirements traceability (Question 2)
9 In this subsection challenges in requirements traceability will be covered (as found in
10 the primary studies). Any techniques or tools discussed will be covered in subsequent
11 subsections.
12 In [16], the authors discuss the results of a survey conducted by Gotel and Fin-
13 kelstein in 1994 [37]. Various traceability problems were identi¯ed in this survey,
14 such as informal development methods, insu±cient resources, time and cost for
15 traceability, lack of coordination between people which were responsible for di®erent
16 traceable artifacts, lack of training in RT practices, imbalance between bene¯ts
17 obtained and e®orts spent for implementing traceability practices, and failure to
18 follow standards. Cleland-Huang et al. comment that all of these issues are intensi-
19 ¯ed by the challenges of today's distributed development environment. In order to
20 solve some of these challenges they propose a traceability technique called event-
21 based traceability (EBT). This technique creates links between software artifacts
22 after a change request is executed, and, according to the authors, alleviates the
23 coordination e®orts required for maintaining software artifacts. Cleland-Huang et al.
24 also recommends other techniques, e.g. information retrieval (IR), when working
25 with automated RT.
26 In [39], the authors describe traceability problems such as, requirements changed
27 by users, and availability of less contextual information in decision making. Gotel
28 and Morris suggests a new dimension in the ¯eld of RT and media to solve these
29 problems. They propose a theoretical framework which helps to select the appro-
30 priate media for recording requirements-related information.
31 In [5], the authors claim that various project managers and team members per-
32 ceive that RT does not o®er immediate bene¯t to the development process; therefore
33 RT is kept at low priority. They propose a new technique, traceable development
34 contract (TDC), to overcome traceability problems. (TDC is used to control the
35 interaction of development teams and to document traceability relationships.)
36 In [21], the authors claim that manual construction and maintenance of a
37 traceability matrix proves to be costly for various reasons and, hence, it is a common
38 perception that traceability is not feasible from a ¯nancial point of view. In order to
39 solve this problem, dynamic retrieval methods are used to automate the generation
40 of traceability links.
41 In [22], the authors present the problem of link maintenance, i.e. the lack of
42 coordination between team members results in a failure in maintaining links between
43 artifacts. Most of the time developers believe that traceability costs more than it
May 19, 2012 6:13:47pm WSPC/117-IJSEKE 00584
1stReading
14 R. Torkar et al.
1 delivers. However, Cleland-Huang et al. point at the fact that excessive usage of
2 traceability can also lead to confusion and they then present a combination of
3 traceability techniques in a framework named traceability for complex systems
4 (TraCS). They argue that TraCS helps to implement RT practices in a cost e®ective
5 manner and bring signi¯cant value to an organization.
6 Reference [38] report the results of an empirical study related to the problems of
7 RT. They identify a set of problematic questions such as, `Who identi¯es or discovers
8 a requirement and how? \Who was responsible for the requirement to start with, and
9 who is currently responsible?" Who will take care of change(s) in requirements?
10 What will be the e®ect on the project in terms of knowledge loss, if some individual or
11 the whole group leave the company? Gotel and Finkelstein then propose a model
12 using contribution structures to solve these issues. Contribution structures re°ect the
13 network of people who participate in producing artifacts in requirement engineering.
14 This model extends the artifact-based traceability with traceability of people in-
15 volved in requirements engineering. This model is then successfully implemented in a
16 company as a case study [38]. The employees of the company commented that the
17 proposed model identi¯ed the relevant people to rectify the speci¯c problems
18 regarding changes in requirements. The model has successfully been used in decision-
19 making concerning sta® turnover issues.
20 In [9], the authors investigate how practitioners (project managers) decide on
21 when and how to adopt RT. They use a literature review combined with Howard's
22 theory of classical decision making [48] to identify relevant factors for making a
23 decision with respect to traceability. They then validate the factors in a large case
24 study. The main conclusions they reach is that \There is little incentive to use
25 traceability when most of the bene¯ts are outside the project" and that there is a lack
26 of awareness. In addition, they claim that one major obstacle is, \the way software
27 development projects [are] contracted and organized."
28 In [80], the author presents experiences from a project related to requirements
29 management improvement (speci¯cally RT). Initially the problems related to
30 requirements management were identi¯ed in the information transfer from contract
31 to speci¯cation phase as well as inadequate impact analysis of requirement changes.
32 In this regard three requirements management tools, i.e. DOORS, RTM and
33 RequisitePro were evaluated and, ultimately, DOORS was recommended for
34 solving traceability issues. (Requirements traceability tools are further covered in
35 Sec. 3.3).
36 In [63], the author identi¯es environmental, organizational and technical factors
37 in°uencing the implementation of RT. The environmental factors include inability to
38 use available technologies for RT, such as reluctance to manually construct a
39 requirements traceability matrix (RTM) by the employees. Organizational factors
40 include compliance with standards strictly demanding RT like CMMI level 3.
41 Technical factors include ad hoc practices and sta® employed in organizations.
42 Ramesh proposes tools support, training, and change in system development policies
43 to overcome these challenges.
May 19, 2012 6:13:47pm WSPC/117-IJSEKE 00584
1stReading
1 In [47], the authors identify costs associated with traceability as a factor that
2 obstructs its implementation. The traceability costs signi¯cantly increases if a
3 company uses RT tools. The reason for this increase in cost is that existing trace-
4 ability techniques do not di®erentiate between high and low value requirements (the
5 high value requirements are those that are more important as compared to other
6 requirements and vice versa). The authors propose a method called value-based
7 requirements tracing (VBRT) to solve this cost-related issue of RT. The VBRT
8 approach identi¯es high value requirements based on parameters like: Number of
9 artifacts, number of traces (links between software artifacts), and number of
10 requirements. They successfully implement this approach in an industrial case study.
11 In [14, 20], the authors identify that organizations fail to trace non-functional
12 requirements (NFRs) like performance, security and usability. This is, according to
13 the authors, due to the fact that NFRs have a global impact on the software system
14 and extensive network of interdependencies and trade-o®s exist between them. In
15 order to overcome these NFR-related traceability issues an approach called goal-
16 centric traceability (GCT) is proposed. An industry-based experiment was con-
17 ducted to verify this technique. The experimental results indicate that this approach
18 is successful in managing traceability in NFR.
19 In [66], the authors identify one major problem when tracing requirements back to
20 their sources. The system validation testing is performed against requirements;
21 therefore, there should be a technique to trace requirements back to their sources.
22 The authors propose a technique for pre-RS traceability.
23
24 3.2.1. Analysis of challenges in requirements traceability
25
26 Over time various researchers have reported di®erent challenges regarding imple-
menting requirements traceability practices. Among these challenges some can be
27
28 referred to as commonplace, e.g. cost, time, e®ort, and team coordination. In order to
tackle these challenges, various solutions have been proposed such as new trace-
29
ability techniques, frameworks, models, and various automated traceability tools.
30
The challenges related to RT and their solutions can be classi¯ed into three main
31
types on the basis of the primary studies in the review:
32
33 . Challenges addressed by academia (Table 5).
34 . Challenges addressed by academia/industry and veri¯ed in industry (Table 6).
35 . Unsolved issues. Issues which are still unsolved in industry and academia.
36
37 The solutions proposed might not be generic, i.e. suitable for all types of com-
38 panies, however, some general conclusions can be drawn. First, understanding the
39 challenges that might arise when realizing and working with traceability can be seen
40 as a pre-requisite. Second, the cost and value distribution of RT seems to be o®,
41 namely many of the bene¯ts associated with RT span over and beyond any single
42 project, while large parts of the costs associated with establishing and maintaining
43 RT are put on the project. This could hamper the motivation of a project to work
May 19, 2012 6:13:48pm WSPC/117-IJSEKE 00584
1stReading
16 R. Torkar et al.
1
2 Table 5. Primary studies reporting challenges related to requirements traceability addressed by
academia.
3
4 Ref. Issue(s) Proposed solution (not veri¯ed in industry)
5 [5] Requirement traceability does not o®er Traceable development contract.
6 immediate bene¯t to development
process.
7
[16] Informal development methods, Insu±cient Event-based traceability.
8
resources, Time and cost for traceability,
9 Lack of coordination between people,
10 Failure to follow standards.
11 [22] Lack of coordination between team mem- Traceability for Complex Systems (TraCS)
12 bers. Developers think that traceability Framework.
costs more then it delivers. Excessive use
13 of traceability generate more links which
14 are not easy to manage.
15 [21] Manual construction of an RTM is costly. Solved by the use of IR methods.
16 [39] Requirements change by user. Less appro- Media recording framework.
17 priate information is available for mak-
ing decision with requirements.
18
[66] Problems associated with tracing back to Pre-RS requirements traceability technique.
19 their sources.
20
21
22
23 Table 6. Primary studies reporting challenges related to requirements traceability addressed by industry.
24
Ref. Issue(s) Proposed solution (veri¯ed in industry)
25
[9] Adopting RT Increase awareness and adapt organizations
26 to include RT.
27 [14] Failure to trace non-functional requirements Goal centric traceability evaluated by an
28 e.g. security, performance and usability. experiment.
29 [38] Some problematic questions are identi¯ed as Framework of contribution structure.
challenges: Who identi¯es a requirement
30
and how? Who was responsible for the
31 requirement to start with and who is cur-
32 rently responsible? Who is responsible for
33 change(s) in requirements? What will be
the e®ect on the project in terms of
34 knowledge loss if key employers are quit?
35 [47] Cost related to RT. VBRT tested through a case study.
36 [63] Organizational, environmental and technical Best practices given.
37 factors.
38 [80] Requirements management challenges in Requirements management tools like
39 industry projects e.g. inadequate impact DOORS and RequisitePro.
analysis and lack of information transfer.
40
41
42
43
May 19, 2012 6:13:48pm WSPC/117-IJSEKE 00584
1stReading
1 with RT as the bene¯t is not seen immediately. This could be an underlying factor
2 that enhances many of the other challenges identi¯ed.
3
4
3.3. Requirements traceability tools (Question 3 )
5
6 This subsection will cover the requirements traceability tools found in the primary
7 studies. Any traceability techniques discussed in this subsection will be covered in
8 the following subsection.
9
RETRO. Requirements tracing on-target (RETRO) is an RT tool facilitating the
10
automatic generation of requirements traceability matrices (RTM). RETRO uses
11
information retrieval (IR) methods and has a GUI front-end [45, 46, 83].
12
Hayes et al. [46] conducted a case study with 30 graduate-level students in a
13
requirements engineering course. The subjects in this case study were divided into
14
two groups. One group was tracing requirements manually using a RTM, while the
15
other group was using RETRO. Students who had previous experience of traceability
16
were placed in the former group. Both groups had to trace 22 requirements to 52
17
design elements. The results of the case study revealed that the students using
18
RETRO produced the most accurate result.
19
20 Rational RequisitePro. Rational RequisitePro is a requirements management tool
21 developed by IBM. It provides support to save SRS documents and, link requirements
22 to use-case diagrams and test cases. When change to requirements occur Rational
23 RequisitePro identi¯es the corresponding software artifacts that are a®ected.
24 It is currently in use in industry and by researchers, e.g. [17, 18, 16, 22, 15, 44, 45]
25 have all identi¯ed it as a requirements management tool, which also supports
26 traceability.
27
DOORS. DOORS, is a requirements management tool developed by Telelogic (now
28
IBM). It is used to capture, link, trace, analyze, and manage changes to the
29
requirements. It provides ways to create and traverse links between requirements (for
30
example a link can be created by a drag and drop operation). DOORS is also helpful
31
in change management; it immediately °ags the changes that could impact other
32
requirements. In addition to all this DOORS also support dynamic report generation.
33 From 1997 to 2007 several researchers have reported using DOORS as an auto-
34
mated requirements management tool. DOORS has been covered implicitly or
35
explicitly by several researchers [5, 1518, 22, 44, 45, 64, 80].
36
37 DesignTrack. According to [61], DesignTrack is a prototype tool supporting RT. It
38 provides traceability between requirements and architectural design. DesignTrack
39 helps organizations by providing an integrated environment for requirements
40 modeling and speci¯cation. An architect can use this tool to manage issues of design
41 requirements and other design tasks.
42 The primary goals of DesignTrack is to: (i) Provide an integrated environment for
43 designers to manage requirements information along with exploration. (ii) Facilitate
May 19, 2012 6:13:48pm WSPC/117-IJSEKE 00584
1stReading
18 R. Torkar et al.
20 R. Torkar et al.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Fig. 2. Overview of the value-based requirements traceability process. The dashed arrows indicate an
23 explicit or implicit connection between the activities. The actor Project manager can be substituted in
24 some cases by, e.g. a requirements engineer.
25
26
27 irreducible) requirements. A unique identi¯er is assigned to each requirement by the
28 requirements engineer. The result of the requirements de¯nition step is a set of
29 requirements and their IDs.
30 In the requirements prioritization phase all stakeholders assess the requirements
31 based on three parameters, i.e. the value, the risk and the e®ort of each requirement.
32 The result of this phase is an ordered list of prioritized requirements based on the
33 three priority levels.
34 The packaging of requirements is optional and allows a group of architects to
35 identify clusters of requirements. These clusters of requirements help to develop and
36 re¯ne an architecture from a given set of requirements.
37 During the linking of artifacts the team identi¯es traceability links between
38 requirements and artifacts. Important requirements are traced in more detail than
39 other requirements. These important requirements can be identi¯ed from the list of
40 prioritized requirements based on three levels developed in the requirements prior-
41 itization step. A traceability plan is the by-product of this phase.
42 Finally, during the evaluation phase, a project manager can use traces for various
43 purpose such as to estimate the impact of a change.
May 19, 2012 6:13:51pm WSPC/117-IJSEKE 00584
1stReading
22 R. Torkar et al.
1 Heindl and Bi®l [47] conducted a case study using VBRT at Siemens, Austria.
2 The project Public Transport on Demand consisted of 46 requirements that were
3 selected to compare full tracing and VBRT.
4 The results of the case study were:
5
6 (1) Focusing on the most important requirements reduced e®orts as compared to
7 tracing all requirement. The high e®ort spent in requirements tracing is the
8 factor due to which projects do not implement traceability. The e®ort can be
9 reduced using VBRT, which took 35% less e®ort compared to full tracing.
10 (2) It is easier to capture traceability information in earlier phases of the software
11 development life-cycle as compared to capturing traceability in later stages.
12 (For example if no traceability information is maintained during the project.)
13 (3) The prioritization step of VBRT identi¯es the most valuable requirements that
14 are needed to be traced in more detail.
15
Feature-oriented requirements traceability. Artifacts, like software require-
16
ments speci¯cation, design documents, source code and test cases, are produced
17
during software development. When a change request (CR) occurs at any stage
18
during the software development life-cycle, it might be di±cult to discover the
19
software artifacts a®ected by the CR. Additionally, it may be di±cult for software
20
engineers to construct and manage traceability links in general. The approach of the
21
feature-oriented requirements tracing (FORT) technique aims at reducing the dif-
22 ¯culty in managing traceability links by identifying them through prioritized
23
requirements and by constantly considering cost and e®orts [3].
24
In order to understand the concept of FORT some terms need to be introduced as
25
given below [3]:
26
27 . Features are the key characteristics of the product. These features can be classi¯ed
28 on the basis of capabilities, domain technologies, implementation technologies and
29 operating environments.
30 . The process of identifying features and then organizing them in a model called a
31 feature model.
32 . The user visible characteristics that can be identi¯ed as operations, nonfunctional
33 characteristics and distinct services.
34 . The domain technologies represent the way of implementing a service or an
35 operation.
36 . Generic functions or techniques that are used to implement domain functions,
37 services, and operations are called implementation techniques.
38 . The environments in which the applications are in use are called operating
39 environments.
40 . There are three types of relationships in feature modeling. The composed-of
41 relationship is used when there is a whole-part relationship between a feature and
42 its sub-features. In generalization/specialization relationship features are the
43
May 19, 2012 6:13:51pm WSPC/117-IJSEKE 00584
1stReading
24 R. Torkar et al.
6 Granularity TL ETL
7 Low 9 4
8 Medium 49 28
9 High 152 100
10
11
12 The authors in [3] provide empirical evidence related to FORT. Feature-oriented
13 requirements tracing was applied in a case study to a car rental system containing
14 nine components, 49 classes and 152 methods. The results of this case study are seen
15 in Table 10.
16 The case study's result indicate that FORT reduces e®orts for generating trace-
17 ability links with 2472% [3].
18 In short, FORT provides variability information by the use of feature modeling.
19 This variability is helpful to estimate the impact of requirements change. FORT also
20 reduces the e®orts to create traceability links by prioritizing the features and,
21 additionally, provides a tight relationship between requirements and artifacts by the
22 help of an intermediate catalyst.
23 We have made a table between value-based and feature-oriented requirements
24 tracing so as to clearly see the di®erences, Table 11 [3], since the two techniques are
25 fairly similar to each other at ¯rst glance.
26
Pre-RS requirements traceability. In pre-RS tracing, requirements are traced
27
back to their source. These sources are basically the user needs that, in most cases,
28
corresponds to unstructured information. Requirements engineers use interviews,
29
questionnaires or prototyping to gather user needs during the process of require-
30
ments elicitation. These needs are then documented as requirements in an SRS.
31
Traceability between requirements and their sources is one of the biggest challenges
32
33
34 Table 11. Comparison between value-based and feature-
35 oriented requirements traceability.
36 Criteria VBRT FORT
37
Well-de¯ned process Partial support Supports
38 Value-added tracing Supports Supports
39 Feature modeling No support Supports
40 Variability consideration No support Supports
E®orts reduction High High
41 Tool support Yes No
42 Empirical evidence Case study Case Study
43
May 19, 2012 6:13:52pm WSPC/117-IJSEKE 00584
1stReading
1 faced by the research community according to [66]. In system validation the system is
2 validated against the requirements and in the case of a test failing it might be
3 necessary to trace a requirement back to its sources.
4 The approach of pre-RS traceability, as explained in [66], is based on capabilities
5 engineering. Capabilities engineering is a process for developing change-tolerant
6 systems by using functional abstractions known as capabilities.
7 The capabilities engineering process is based on three phases:
8 First, the problem space that represents the conceptual region which is associated
9 with the problem domain [66]. There are two important entities in the problem space:
10 Needs and directives. The needs represent the user's view of the system. The needs
11 specify what is desired of the system from user's perspective, expressed in the lan-
12 guage of the problem domain. Directives are the detailed characteristics of the sys-
13 tem or requirements with context information. There are two purposes of directives
14 in the problem space. The ¯rst is to capture domain information and the second to
15 facilitate progress from problem space to transition space. In the problem domain
16 needs are decomposed into directives. Decomposition is achieved with the help of
17 functional decomposition (FD) graphs and is a directed acyclic graph represented as
18 G ¼ ðV ; EÞ, where V is the vertex and E is the edge. The FD graph's root represents
19 the overall mission of the system and edges represent the directives. The internal
20 nodes between root and leaves are the functional abstractions. The FD graph pro-
21 vides traceability links between needs and directives.
22 Second, we have the transition space which, according to [66], is de¯ned as a
23 collective aggregation of the system view, capabilities and problem domain. The two
24 main entities in the transition space are initial and optimized capabilities. Formu-
25 lation and optimization are two capabilities engineering activities in transition space.
26 The initial capabilities are the functional abstractions with high cohesion and low
27 coupling whereas the optimized capabilities are the constraints of technology feasi-
28 bility and implementation schedules. The formulation activity identi¯es initial
29 capabilities from all possible abstractions present in the FD graph. These initial
30 capabilities show high cohesion and low coupling. Cohesion represents the togeth-
31 erness of elements within the entity and coupling is the interdependences between
32 elements. The aim of the optimization activity is to identify the set which best
33 accommodates the constraints of schedule and technology. The initial capabilities
34 are the input for optimization activities.
35 Finally, there is the solution space that represents the technical area relevant to
36 the system being developed. The important entity in the solution space is the
37 requirement. In the solution space a requirement represents users' expectations from
38 the system and is subject to quality constraints, e.g. testability, veri¯ability, accu-
39 racy, and un-ambiguity [66]. The input to the solution space is the optimized set of
40 capabilities with their directives that are then transformed into requirements. In
41 capabilities engineering based pre-RS traceability, only directives associated with the
42 capabilities chosen for development are transformed into requirements. The rele-
43 vance value can be used to determine critical requirements while transforming
May 19, 2012 6:13:53pm WSPC/117-IJSEKE 00584
1stReading
26 R. Torkar et al.
1 Table 12. Pre-RS traceability using the capabilities engineering (CE) approach.
2 CE phase Entities CE activity Input View
3
Problem space Needs, directives Decomposition User needs User
4 Transition space Initial and optimized Formulation, FD graph System
5 capabilities optimization
6 Solution space Finalized capabilities Transformation Optimized System
(requirements) capabilities
7
8
9
directives to requirements and represents the importance of a directive in achieving
10
the objective of its parent node. For instance, a directive with relevance 1 is mission
11
critical and therefore the requirements associated with this directive are also mission
12
critical.
13
Table 12 provides a summary of the pre-RS traceability process using the capa-
14
bilities engineering approach. In [66] the authors do not provide any empirical evi-
15
dence regarding the success of this approach, however they claim that this approach
16
is very useful in RT.
17
18 Event-based traceability. The event-based traceability approach (EBT) was
19 proposed by Cleland-Huang et al. in [18, 16]. The main reason for developing EBT
20 was to provide maintenance of traceability relationships. Cleland-Huang et al. de¯ne
21 traceability relationships as publisher-subscriber relationship. In this relationship,
22 dependent objects, i.e. artifacts, have to subscribe to their respective requirements on
23 which they are dependent. Whenever a requirement change occurs, an event message
24 is published, which is then noti¯ed to all dependent objects.
25 According to [17], there are three main components which participate in the whole
26 process, requirements manager, event server and subscriber manager. The require-
27 ments manager manages the requirements and publishes the event messages to the
28 event server, whenever a change request occurs. The event server is responsible for
29 three main activities: (i) It handles subscriptions from dependent objects. (ii) It is
30 responsible for listening to event messages received from the requirements manager.
31 (iii) It publishes or forwards event messages to the relevant subscribers. The sub-
32 scriber manager is responsible for listening to the event server and for receiving and
33 managing the event noti¯cations.
34 Event-based traceability handles functional requirements and non-functional
35 requirements, and it provides a solution to the traceability update problem. Addi-
36 tionally, it can be used in combination with requirements management tools like
37 DOORS and Rational RequisitePro according to [16].
38
Information retrieval. The information retrieval [20, 44, 21] approach (IR) is used
39
to automate the generation of traceability links. Commonly used IR methods in-
40
clude: Vector space model (VSM), di®erent probabilistic models and latent semantic
41
indexing (LSI) [58]. Information retrieval methods are based on similarity compar-
42
ison and probabilistic values of two artifacts.
43
May 19, 2012 6:13:53pm WSPC/117-IJSEKE 00584
1stReading
1 According to [68] IR methods include three general steps: (i) Pre-processing. (ii)
2 Analyzing, indexing, creating its representation and archiving. (iii) Analyzing in-
3 coming artifacts using some ranking algorithms.
4 Whenever a pair of artifacts reach a speci¯c rank, it is considered as a candidate
5 link that must be reported to the analyst for a ¯nal decision, i.e. the analyst di®er-
6 entiates between true and false links. IR methods signi¯cantly reduce the e®ort
7 required for creating traceability links between artifacts but, on the other hand, it
8 still requires signi¯cant e®orts by the analyst.
9 IR methods like VSM and LSI are used in the RT tool RETRO [57, 45]. Research
10 on IR methods has focused on tracing functional requirements, however there is one
11 exception where the authors in [14] used IR methods in goal-centric traceability for
12 tracing non-functional requirements (this is covered later in this section).
13
Rule-based approach. The basic purpose of the rule-based (RB) approach is to
14
automatically generate traceability links using rules [76]. There are two traceability
15
rules, i.e. requirement-to-object-model traceability (RTOM) rule and inter-require-
16
ments traceability (IREQ) rule.
17
These rules are used for three speci¯c documents: Requirements statement doc-
18
ument (RSD), use case documents (UCD), and the analysis object model (AOM).
19
RSD and UCD are traced to an AOM by using RTOM rules. IREQ rules are used for
20
tracing between RSD and UCD. The RB approach presents all of the documents and
21
both of the rules in an XML-based format. The RB approach consists of four steps:
22
Grammatical tagging of the artifacts, converting the tagged artifacts into XML
23
representations, generating traceability relations between artifacts, and generating
24
traceability relations between di®erent parts of the artifacts.
25
26 Feature-model based approach. The feature-model based (FB) approach is de-
27 scribed in [67]. In feature modeling, requirements are described as overviews and
28 models as a variability of the product line. A feature model consists of graphs along
29 with nodes and edges, while nodes are the features and edges are the features relations.
30 Each feature represents a property of the product from a customer's point of view.
31 There are three types of features: Functional features, interface features, and param-
32 eter features. Feature relations can be classi¯ed into three categories: Hierarchical
33 relations; in this case most important feature is placed at a high position in the hier-
34 archy. Re¯nement relations, which are used to de¯ne the relations of generalization,
35 specialization and aggregation. Exclude relations, which de¯ne the constraints be-
36 tween variable features which can in°uence the sequence of the decision of the product.
37
Scenario-based approach. According to the proposal by [14, 68] scenarios are
38
used to model system functionality and to generate functional test cases. Scenario-
39
based test cases create a mapping between requirements and other artifacts like
40
design and code. The traceability is established by mapping scenarios with the design
41
elements. Scenarios are created to trace only the interesting cases therefore they
42
might not provide complete coverage. However, scenarios are frequently used by
43
May 19, 2012 6:13:53pm WSPC/117-IJSEKE 00584
1stReading
28 R. Torkar et al.
30 R. Torkar et al.
1 Table 13. Summary of comparison between di®erent traceability techniques. (F and NF, in the fourth
column from left, stands for functional and non-functional, respectively.)
2
3 Empirical
4 Ref. Technique Aspect Type of req. evidence
In
Sc
G
ul
oa
fo
en
en
e-
37
rm
l-c
t
ar
b
-b
at
as
en
io
as
io
38
-b
e
tric
n
ed
a
re
se
tri
39
d
ev
al
40
41 Fig. 3. Descriptive statistics showing the number of times a technique has been `published' according to
42 the primary studies.
43
May 19, 2012 6:13:55pm WSPC/117-IJSEKE 00584
1stReading
32 R. Torkar et al.
1 Table 14. Conclusions drawn from the empirical evidence on traceability techniques.
2 Refs. Technique Conclusions
3
[3] Feature-oriented (1) FORT provides variability information based of feature
4 requirements modeling which is useful to estimate requirements change.
5 tracing (FORT) (2) FORT reduces e®orts by being based on feature pri-
6 oritization.
7 [14, 20] Goal-centric trace- (1) GCT facilitates to manage impact of functional change
ability (GCT) upon the non-functional requirements. (2) GCT helps to
8 manage critical system qualities such as safety, security,
9 reliability, usability and performance.
10 [20, 14, 68] Information (1) It is very hard to maintain links in constantly evolving
11 [57, 25, 44] retrieval (IR) systems. (IR methods facilitate dynamic link generation.)
[21, 4] (2) The results of a case study indicate that \IR provides a
12 practicable solution to the problem of semi-automatically
13 recovering traceability links between code and doc-
14 umentation".
[47] Value-based require- (1) The traceability e®orts are reduced by focusing on most
15
ments tracing important requirements as compared to full tracing. (2) It
16 (VBRT) is easier to capture traceability related information in
17 earlier phases of software development lifecycle. (3) The
18 prioritization step of VBRT identi¯es important require-
ments to be traced in more detail than others. (4) In
19 VBRT important requirements are identi¯ed based on
20 parameters stakeholder value, requirements risk/volatility
21 and tracing costs. (5) Tracing requirements into code at
method level provides more useful and detailed informa-
22 tion than tracing into class level.
23
24
25
26 updating artifacts and resolving traceability links. (The authors claim that a long-
27 term empirical study of event-based traceability is currently under way.)
28 In [22] the authors describe a framework known as TraCS. It uses event-based
29 traceability to trace performance related requirements, however there is no empirical
30 evidence connected to TraCS.
31 The conclusions drawn from examining the empirical evidence of RT are reported
32 in Table 14. These conclusions could be of interest while selecting a particular
33 technique for providing solutions in academia or industry or when developing a
34 framework for requirements traceability.
35 Based on the systematic review results, the traceability techniques can be divided
36 into two types:
37
38 (1) Techniques facilitating pre-RS traceability. This type includes those trace-
39 ability techniques which help to describe the life of requirements when they are
40 not included in the requirements speci¯cation. There is only one technique in
41 this category, i.e. pre-RS requirements traceability (see Sec. 3.4).
42 (2) Techniques facilitating post-RS traceability. This type includes those techni-
43 ques which help to trace the life of requirements when they are included in the
May 19, 2012 6:13:55pm WSPC/117-IJSEKE 00584
1stReading
34 R. Torkar et al.
1
2
3
4
5
6
7
8
9
10
11
12
13
14 Fig. 4. Requirements traceability tools as found by this systematic review.
15
16 categories are requirements management tools providing traceability support, RT
17 tools and other requirements management/traceability tools. The lack of empirical
18 evaluation (validation) of the tools might be a serious indicator.
19 Question 4. What requirements traceability techniques are reported in research
20 literature?
21 The RT techniques reported in the primary studies are discussed in Sec. 3.4. We
22 have grouped traceability according to two aspects of RT as shown in Fig. 5. One
23 interesting ¯nding, which is noticeable through this categorization, is the fact that
24 there is very little research done in the area of pre-RS traceability. In addition it should
25 be noticed that 11 out of 15 RT techniques lack empirical validation either through lab-
26 validation in academia (e.g. experiments), or through industry case studies or pilots.
27
28
5. Interviews Conducted in Industry
29
30 In order to determine RT challenges in industry, interviews were conducted with two
31 software development companies. These interviews are summarized below. The
32 questionnaire used and the response to the questions can be found at [78].
33
34 5.1. Company A
35
36 5.1.1. Introduction
37 Company A is based in Sweden and a world-leading telecommunication company
38 (the interviewee were selected from one of the company's divisions). Company A
39 provides telecommunications equipment and other telecom related services in 140
40 countries. More than 1,000 networks use Company A's products and 40% of all
41 mobile calls are made through systems provided by Company A. Company A is one
42 of few telecom companies that o®er end-to-end solutions for all major mobile com-
43 munication standards. The company is ISO 9000 certi¯ed.
May 19, 2012 6:13:56pm WSPC/117-IJSEKE 00584
1stReading
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Fig. 5. Requirements traceability techniques found by the systematic review.
27
28
29 5.1.2. Interviewee
30 The authors' ¯rst interviewee was person x. She is working as a discipline driver for a
31 whole product unit. She is also responsible for the implementation of RT practices.
32 Person x has been working at Company A for the last ten years and she has spe-
33 ci¯cally worked with RT for the last ¯ve years. The authors' second interviewee was
34 person y. Person y is working as a requirements manager and responsible for overall
35 requirement engineering activities. Person y has been working at Company A for the
36 last nine years and focused on RT for the last 1.5 years.
37
38
39 5.1.3. Traceability practices
40 Persons x any y believe that RT is an important and useful activity within
41 requirements management. There are thirty people working in requirements engi-
42 neering and 35 people are dedicated to RT practices. According to the interviewees
43 traceability is bene¯cial for the company due to several reasons.
May 19, 2012 6:13:57pm WSPC/117-IJSEKE 00584
1stReading
36 R. Torkar et al.
1 One of the major bene¯ts is customer satisfaction. Customer can check and follow
2 the product development progress according to their requirement. Requirements
3 traceability provides a guarantee for requirements engineers and product managers
4 that every requirement is implemented. Requirements traceability helps to ensure
5 impact analysis and derivation analysis. Requirements traceability facilitates impact
6 analysis. Whenever a change request is initiated at any stage we can trace the
7 requirements or design artifacts or test cases that can be a®ected. Similarly in cov-
8 erage analysis RT ensures implementation of all requirements. Interviewees also
9 mentioned that normally there are 200 change requests per project.
10
11
5.1.4. Tool support for traceability
12
13 According to x and y, Company A use an automated requirements management
14 tool named MARS which is developed by IBM speci¯cally for Company A. There
15 is a speci¯c module in MARS which is responsible for traceability. Company A
16 also uses another tool named `Focal Point' for storing the elicited requirements.
17 After entering the requirements into Focal Point the Main Requirement Speci¯-
18 cation (MRS) is generated from it. After generating the MRS, it is entered into
19 MARS.
20 By using MARS, requirements are stored in a central repository. Each require-
21 ment has a number of attributes and a unique identi¯er, i.e. ID, slogan and
22 description. Traceability of requirements is obtained by using MARS and Focal
23 Point as both tools are synchronized. Requirements traceability matrices are gen-
24 erated to cater for RT. These matrices are used to constantly be up-to-date regarding
25 the status of a requirement and whether the requirement is implemented or not.
26 MARS also facilitates change management. Change requests are initiated based on
27 the requirements stored in MARS. A change control board analyzes the change
28 request and decides to accept or reject it. In case of acceptance, changes are made
29 permanent in MARS (versioning is also handled).
30
31 5.1.5. Factors in°uencing requirements traceability
32
33 According to the interviewees traceability is very well implemented in MARS. De-
34 spite this, there are a few issues e.g. the manual decomposition of master require-
35 ments speci¯cation into detailed requirements for RT. Furthermore the maintenance
36 cost of MARS is also very high, therefore Company A is thinking about using another
37 tool in the future.
38
39 5.2. Company B
40
5.2.1. Introduction
41
42 Company B develops both bespoke and market-driven products for its customers.
43 This company is involved in developing application suites and frameworks for mobile
May 19, 2012 6:13:57pm WSPC/117-IJSEKE 00584
1stReading
1 phones based on the Symbian operating system. Company B is neither ISO9000 nor
2 CMMI certi¯ed.
3 In Company B the requirements engineering activities are monitored by a de-
4 partment named Software Development Organization (SDO). The responsibility of
5 SDO is to capture, clarify, process and trace requirements to testing and imple-
6 mentation. There are thirty people working in the SDO; out of which 13 people work
7 with capturing requirements.
8
9 5.2.2. Interviewee
10
11 Interviewee z has been working as a software quality engineer in Company B for the
12 last two years. The current job responsibilities of person z include process develop-
13 ment, requirements matrix measurement and controlling the development projects.
14 Although person z is not directly working with requirements engineering but as a
15 software quality engineer, he has knowledge of requirements engineering activities in
16 Company B.
17
18 5.2.3. Traceability practices
19
In Company B the high level requirements are known as Market Requirement
20
Speci¯cation (MRS). They are decomposed into Product REQuirements (PREQs).
21
The PREQs are further decomposed into a Feature Speci¯cation (FS), which is
22
speci¯ed at a low level. Traceability is maintained by establishing connection
23
between every MRS, PREQ and FS; and vice versa.
24
There is no traceability in design and source code. In testing, each FS is connected
25
to at least one test case which must have passed. The traceability is only maintained
26
within the requirements and testing phases.
27
28
29 5.2.4. Tool support for traceability
30 Company B is using two di®erent tools for managing requirements and test cases.
31 These tools are Requirements Management System (RMS) and Test Management
32 System (TMS). The RMS is used to manage requirements like MRS, PREQ and FS.
33 It also provides traceability support but only against the requirements part. TMS is
34 used to manage and store test cases.
35 There is no link between the RMS and the TMS. The traceability against FS, to
36 test cases, is maintained manually with the help of a Requirements Traceability
37 Matrix (RTM). The RTM is maintained in MS Excel.
38
39
5.2.5. Bene¯ts of implementing traceability
40
41 In Company B, requirements are decomposed into Market Requirements Speci¯ca-
42 tion (MRS) which is further decomposed into PREQ and then to FS. Traceability
43 provides a higher level of details for the software engineers and helps in identifying
May 19, 2012 6:13:58pm WSPC/117-IJSEKE 00584
1stReading
38 R. Torkar et al.
1 the source of the FS. Similarly traceability from MRS to test cases ensures that the
2 feature is `complete'.
3 Traceability helps in impact analysis. In Company B impact analysis is done in
4 several ways. If it is done early in the development life-cycle during PREQ phase;
5 usually by the product manager. The product manager should have su±cient com-
6 petency to declare that this change will a®ect the product in one way or another. But
7 during development, impact analysis is done by the team responsible for managing
8 the component a®ected by the change.
9
10 5.2.6. Factors in°uencing requirements traceability
11
Company B has good traceability support between feature speci¯cations, product
12
requirement speci¯cations and functional speci¯cation according to person z. But at
13
the moment it is very di±cult to determine requirements completion in a project.
14
The requirement completion means that the FS has been developed and veri¯ed for
15
speci¯c project. There is no direct link between RMS and TMS. The FS that is
16
veri¯ed for a particular project is then maintained manually using MS Excel.
17
Therefore, tool support is one factor a®ecting RT.
18
Company B is not maintaining traceability between requirements to design and
19
code phase. The traceability is maintained only between requirements and test cases.
20
Therefore it is very di±cult to identify the components that are a®ected by change
21
requests in design and development phase. There are normally 58 change requests
22
for every twenty features.
23
24
5.3. Analysis of interviews
25
26 The results of the case studies are interesting in the sense that Company A and
27 Company B are fairly di®erent with regards to traceability policies. The analysis of
28 the interview reveals that they can be placed into two di®erent categories [63], i.e.
29 high-end and low-end traceability users.
30 For requirements management, Company A is using a formal tool, MARS, which
31 also provides traceability support. MARS was tailored by IBM for Company A. On
32 the other hand Company B lacks formal methods for traceability. Company B uses
33 automated tools for requirements management (RMS) and test case management
34 (TMS). Although the automated traceability is only maintained in RMS. There is no
35 traceability in design and code. Furthermore, to establish traceability links between
36 RMS and TMS, traceability matrices are manually maintained using MS excel.
37 Company B uses static methods like traceability matrices that are not updated
38 automatically as the system evolves. Therefore, manual traceability matrices lose
39 most of their usefulness after creation (low-end traceability users often exhibit this
40 aspect). Whereas, high-end traceability users recognize the need for maintaining
41 traceability, which helps in re°ecting the current status of the system and generate
42 traceability documentation at any point in the development life-cycle. This practice
43 is exercised by Company A and not by Company B.
May 19, 2012 6:13:58pm WSPC/117-IJSEKE 00584
1stReading
1 As stated by [63], low-end traceability users view traceability as a mandate from the
2 customer, while high-end traceability users consider traceability as an important com-
3 ponent of the overall quality engineering process. In Company A traceability is an
4 integral part of the quality assurance process. Similarly there are approximately 200
5 change request per project in Company A and, hence, they rely on traceability to identify
6 the a®ected software artifacts due to change request. On the other hand, their automated
7 requirements management tool (MARS) also provides richer traceability support.
8 In short, Company B implements traceability to ensure that two conditions are
9 ful¯lled. First, that high level requirements known as MRS are decomposed into low
10 level requirements called Feature Speci¯cations. Second, to guarantee that a feature
11 is complete. In the practices followed by Company B, a feature is said to be complete
12 when there is a traceable link between MRS and test cases. Albeit these conditions
13 are not enough by themselves, we can conclude that Company A has a fairly well
14 de¯ned traceability processes. In the future Company A is planning to shift to
15 another requirements management tool (Rational RequisitePro), a tool which is
16 brie°y discussed in Sec. 3.3. One question is if a new tool will solve the problems with
17 high maintenance identi¯ed as a major challenge for Company A?
18 In Company B traceability practices are not fully implemented. Although they
19 are using tools for managing requirements (RMS) and tools for managing test cases
20 (TMS), these tools provide traceability only for requirements and test cases, i.e. there
21 is no traceability support between RMS and TMS. Similarly no traceability infor-
22 mation is maintained for traceability between the design and development phase.
23 High-end traceability users like Company A view traceability as one of the core
24 requirements engineering practice. They believe that traceability delivers more than
25 it costs. They can develop traceability tools in-house or purchase them. On the other
26 hand, low-end traceability users like Company B, while acknowledging the impor-
27 tance of traceability, treat traceability more as an expensive overhead.
28 In the systematic review the authors identi¯ed the challenges related to trace-
29 ability (reported in Tables 5 and 6). The views of Companies A and B are sum-
30 marized in Table 15 connected to the challenges identi¯ed by the systematic review.
31 Table 15 contains some very interesting results. For instance both companies
32 believe that traceability delivers more than it costs. But in case of Company B there
33 is no traceability in design and code phase and, in addition, Company B uses manual
34 RT for maintaining traceability links.
35 The challenges related to RT mapped to both the systematic review and the case
36 studies can be seen in Table 16.
37 It is evident from Table 16 that some of the challenges identi¯ed by the systematic
38 review also hold in practice.
39
40 6. Threats to Validity
41
42 An analysis of validity threats enhance the accuracy of a research design by iden-
43 tifying factors which can a®ect the results. In this section four di®erent kinds of
validity threats and their implications are discussed [82].
May 19, 2012 6:13:58pm WSPC/117-IJSEKE 00584
1stReading
40 R. Torkar et al.
1 Table 15. Views of Company A and B concerning challenges identi¯ed in the systematic review.
2 Ref. Issues identi¯ed in systematic review View of Company A and B
3
[5, 9] Requirement traceability does not o®er Both companies disagree with it. Require-
4 immediate bene¯t to development process. ments traceability o®ers bene¯ts for both
5 the companies.
6 [9, 22] Lack of coordination between team members, Both companies believe that traceability
7 developers think that traceability costs delivers more than it cost. Both companies
more then it delivers, excessive use of do not consider lack of coordination
8 traceability generate more links that are between team members and generation of
9 not easy to manage. traceability links as factors a®ecting
10 traceability.
11 [9, 63] Organizational, environmental and technical These factors are observed in both companies.
factors. Based on these factors Company A is
12 classi¯ed as being a `high-end traceability
13 user' and Company B is classi¯ed as `low-
14 end traceability user'.
15 [16] Informal development methods, insu±cient Among these challenges the cost for trace-
resources, time and cost for traceability, ability is a signi¯cant factor for both
16
lack of coordination between people, companies.
17 failure to follow standards.
18 [14] Failure to trace non-functional requirements These factors are important for both compa-
19 (NFR) like security, performance and us- nies. The NFRs like security, performance
20 ability. and usability are of interest to Company B
as they are developing applications for
21 mobile phones. However Company B lacks
22 support for traceability of NFR.
23 [21] Manual construction of requirements trace- Manual construction of RTM is costly for
24 ability matrix (RTM) is costly. Company A, therefore they are focusing
on using automated tools. Manual con-
25 struction of RTM is cheap for Company
26 B, therefore they are using MS Excel
27 sheets for manual construction of RTM.
28 [38] Some problematic questions are identi¯ed as These challenges are not important for both
challenges e.g. who identify a requirement companies. Company A is using the
29
and how, who was responsible for the re- MARS tool and Company B is using RMS
30 quirement to begin with, who is currently that takes care of these challenges.
31 responsible, who will take care of change
32 (s) in requirements, what will be the e®ect
on project in terms of knowledge loss, if
33 some individual or the whole group leave
34 from company?
35 [39] Requirements change by user, less appropri- Requirements change by user is always there
36 ate information is available for making but both companies have maintained
decision with requirements. enough information to make decision
37 about requirements.
38 [47] Cost and e®ort related to RT. Cost related to RT is important issue for both
39 companies.
40 [66] Problems associated with tracing back to This issue is resolved in Company A by using
their sources. the MARS tool. This problem is important
41
but unsolved in the case of Company B.
42
43
May 19, 2012 6:13:59pm WSPC/117-IJSEKE 00584
1stReading
42 R. Torkar et al.
1 The questionnaires used for the interviews in industry were validated to rectify
2 poor questions and °ow in the layout of questionnaire. The heterogeneity of subjects
3 is another threat to conclusion validity [82]. Heterogeneity of subjects means that the
4 subjects belong to varied group with respect to background, education and experi-
5 ence. While homogeneity of subjects means that the subjects belong to the same
6 group based on education, background and experience. In our case, person x worked
7 as a discipline driver in the process area of RT for ¯ve years whereas person y worked
8 as a requirements manager and had experience 1.5 years of experience in RT. Person
9 z had been working as a software quality engineer for two years. In short, the subjects
10 of our interviews would be hard to categorize as heterogeneous or homogeneous.
11
12
13 6.2. Construct validity
14
Evaluation apprehension is the main threat to construct validity. According to [82],
15
evaluation apprehension means that humans have the tendency to look better when
16
they are evaluated. On the other hand some people are afraid of being evaluated. In
17
order to eliminate this threat the interviewees in both companies were ensured that
18
they and their companies would be anonymous.
19
Mono-operation bias is another threat to construct validity. Mono-operation bias
20
means that there is a single independent variable, case, subject or treatment in a
21
research study [82]. We have decreased this threat by conducting interviews at two
22
software companies. In Company A we interviewed two requirements engineers and
23
in Company B we interviewed one quality engineer.
24
25
26
6.3. Internal validity
27
28 In our case, the interviews were recorded and it is a common observation that people
29 may not feel comfortable if the interview is audio taped. In order to eliminate this
30 potential threat to internal validity the interviewees were assured that the recordings
31 would only be used by us.
32 The internal validity threat related to the systematic review is mainly the pub-
33 lication bias. Publication bias refers to the fact that positive results are more likely to
34 be published than the negative facts [55]. In order to eliminate this threat the se-
35 lected conferences and proceedings mentioned in Table 1 were manually searched.
36 Similarly the `study quality assessment and procedures', as mentioned in the review
37 protocol, decreased the factor of publication bias according to our opinion.
38 Selection of subjects from a population also a®ects internal validity [82]. In
39 our case we interviewed people working in the process area of requirements engi-
40 neering. However, one of our subjects (person z) did not work directly with RT.
41 Nevertheless, person z had adequate educational background, experience and
42 knowledge about the practices related to RT and requirements engineering carried
43 out in Company B.
May 19, 2012 6:13:59pm WSPC/117-IJSEKE 00584
1stReading
44 R. Torkar et al.
1 Among all traceability techniques only the pre-RS technique emphasizes pre-RS
2 traceability aspects, the rest of the techniques focus on post-RS traceability aspects.
3 This might be an indication of that development projects are the focus, and not what
4 happens before or after a project. The possible implications of this is evident as the
5 bene¯ts of RT is only partly associated with the project. This is especially true
6 looking at product development going beyond the `one shot' bespoke development
7 projects.
8 The realization that RT reaches over the project perspective and into the product
9 life-cycle perspective might be an important realization for both industry and
10 researchers. It will enable techniques that o®er a complete RT solution, which might
11 include, and take inspiration from, the good-examples presented in this systematic
12 review, and extending them to cover a life-cycle perspective. These new techniques
13 would also give industry the possibility to see RT as something more than a project
14 problem as the cost and e®ort can additionally be spread over the pre-RT (e.g.
15 product management) and post-project (next release or maintenance) perspectives.
16
17
18 References
19 1. A. Abran, J. W. Moore, P. Bourque and R. Dupuis (editors), Guide to the Software
20 Engineering Body of Knowledge (SWEBOK ) (IEEE Computer Society, Los Alamitos,
21 2004).
22 2. W. Afzal, R. Torkar and R. Feldt, A systematic review of search-based testing for non-
23 functional system properties, Information and Software Technology 51(6) (2009)
957976.
24
3. S. Ahn and K. Chong, A feature-oriented requirements tracing method: A study of cost-
25 bene¯t analysis, in Proc. of 2006 International Conference on Hybrid Information
26 Technology, Washington, DC, USA, 2006, IEEE Computer Society, pp. 611616.
27 4. G. Antoniol, G. Canfora, G. Casazza, A. De Lucia and E. Merlo, Recovering traceability
28 links between code and documentation, IEEE Transactions on Software Engineering
28(10) (2002) 970983.
29
5. P. Arkley and S. Riddle, Overcoming the traceability bene¯t problem, in Proc. of the 13th
30 IEEE International Conference on Requirements Engineering, Washington, DC, 2005,
31 pp. 385389.
32 6. A. Aurum and C. Wohlin, Engineering and Managing Software Requirements (Springer-
33 Verlag, New York, 2005).
34 7. S. C. Bailin, A. M. Davis, M. Dorfman, R. E. Fairley, S. R. Faulk, K. Forsberg, L. M.
Ippolito, H. Mooz, J. D. Palmer, J. Reilly, H. Saiedian, R. H. Thayer and D. R. Wallace,
35
Software Requirements Engineering, 2nd edition (IEEE Computer Society Press,
36 Los Alamitos, 1997).
37 8. M. F. Bashir and M. A. Qadir, Traceability techniques: A critical study, in IEEE
38 Multitopic Conference, Washington, DC, December 2006, pp. 265268.
39 9. F. Blaauboer, K. Sikkel and M. N. Aydin, Deciding to adopt requirements traceability in
practice, in J. Krogstie, A. L. Opdahl and G. Sindre (eds.), CAiSE, LNCS (Springer,
40
2007), pp. 294308.
41 10. F. Bouquet, E. Ja®uel, B. Legeard, F. Peureux and M. Utting, Requirements traceability
42 in automated test generation: Application to smart card software validation, SIGSOFT
43 Software Engineering Notes 30(4) (2005) 17.
May 19, 2012 6:13:59pm WSPC/117-IJSEKE 00584
1stReading
1 11. F. Bubl and M. Balser, Tracing cross-cutting requirements via context-based constraints,
2 in Proceedings of the 9th European Conference on Software Maintenance and Reengi-
neering, Washington, DC, USA, 2005, pp. 8090.
3
12. P. Carlshamre, K. Sandahl, M. Lindvall, B. Regnell and J. Natt och Dag, An industrial
4 survey of requirements interdependencies in software product release planning, in
5 Proceedings of the Fifth IEEE International Symposium on Requirements Engineering,
6 Washington, DC, 2001, p. 84.
7 13. L. Chung, B. A. Nixon, E. Yu and J. Mylopoulos, Non-Functional Requirements in Soft-
8 ware Engineering (The Kluwer International Series in Software Engineering, Volume 5)
(Kluwer Academic Publisher, 1999).
9
14. J. Cleland-Huang, Toward improved traceability of non-functional requirements, in
10 Proceedings of the 3rd International Workshop on Traceability in Emerging Forms of
11 Software Engineering (ACM, New York, NY, USA, 2005), pp. 1419.
12 15. J. Cleland-Huang, B. Berenbach, S. Clark, R. Settimi and E. Romanova, Best practices
13 for automated traceability, IEEE Computer 40(6) (2007) 2735.
16. J. Cleland-Huang, C. K. Chang and M. Christensen, Event-based traceability for man-
14
aging evolutionary change, IEEE Transactions on Software Engineering 29(9) (2003)
15 796810.
16 17. J. Cleland-Huang, C. K. Chang and Y. Ge, Supporting event based traceability through
17 high-level recognition of change events, in Proceedings of the 26th International
18 Computer Software and Applications Conference on Prolonging Software Life: Develop-
19 ment and Redevelopment, Washington, DC, 2002, IEEE Computer Society, pp. 595602.
18. J. Cleland-Huang, C. K. Chang, G. Sethi, K. Javvaji, H. Hu and J. Xia. Automating
20
speculative queries through event-based requirements traceability, in Proceedings of the
21 10th Anniversary IEEE Joint International Conference on Requirements Engineering,
22 Washington, DC, 2002, IEEE Computer Society, pp. 289298.
23 19. J. Cleland-Huang, C. K. Chang and J. C. Wise, Automating performance-related impact
24 analysis through event based traceability, Requirements Engineering 8(3) (2003)
171182.
25
20. J. Cleland-Huang, R. Settimi, O. BenKhadra, E. Berezhanskaya and S. Christina, Goal-
26 centric traceability for managing non-functional requirements, in Proceedings of the 27th
27 International Conference on Software Engineering, ACM, New York, NY, USA, 2005,
28 pp. 362371.
29 21. J. Cleland-Huang, R. Settimi, C. Duan and X. Zou, Utilizing supporting evidence to
30 improve dynamic requirements traceability, in Proceedings of the 13th IEEE Interna-
tional Conference on Requirements Engineering, Washington, DC, USA, 2005, IEEE
31
Computer Society, pp. 135144.
32 22. J. Cleland-Huang, G. Zemont and W. Lukasik, A heterogeneous solution for improving
33 the return on investment of requirements traceability, in Proceedings of the 12th IEEE
34 International Requirements Engineering Conference, Washington, DC, 2004, IEEE
35 Computer Society, pp. 230239.
23. Software Engineering Institute j Carnegie Mellon, http://www.sei.cmu.edu/cmmi/ Oc-
36
tober 2008.
37 24. L. M. Cysneiros and J. C. S. do Prado Leite, Nonfunctional requirements: From elicita-
38 tion to conceptual models, IEEE Transactions on Software Engineering 30(5) (2004)
39 328350.
40 25. A. Dekhtyar, J. H. Hayes and S. K. Sundaram, Advancing candidate link generation for
41 requirements tracing: The study of methods, IEEE Transactions on Software Engineering
32(1) (2006) 419.
42
43
May 19, 2012 6:13:59pm WSPC/117-IJSEKE 00584
1stReading
46 R. Torkar et al.
1 26. Department of Defence, US. Military standard: Defense system software development
2 (DOD-STD-2167A), Technical report, Space and Naval Warfare Systems Command,
Washington, DC, February 1988.
3
27. J. Dick, Design traceability, IEEE Software 22(6) (2005) 1416.
4 28. J. C. S. do Prado Leite and K. K. Breitman, Experiences using scenarios to enhance
5 traceability, in 2nd International Workshop on Traceability in Emerging Forms of Soft-
6 ware Engineering in Conjunction with the 18th IEEE International Conference on
7 Automated Software Engineering, Montreal, Canada, October 2003, pp. 6370.
8 29. T. Dybå, V. B. Kampenes and D. I. Sjøberg, A systematic review of statistical power in
software engineering experiments, Information and Software Technology 48(8) (2006)
9
745755.
10 30. M. Edwards and S. L. Howell, A Methodology for system requirements speci¯cation and
11 traceability for large real-time complex systems, Technical report, Naval Surface Warfare
12 Center, 1991.
13 31. A. Egyed and P. Grünbacher, Automating requirements traceability: Beyond the record
and replay paradigm, in Proceedings of the 17th IEEE International Conference on
14
Automated Software Engineering, Washington, DC, USA, 2002, p. 163.
15 32. E. Engstr€om, P. Runeson and M. Skoglund, A systematic review on regression test se-
16 lection techniques, Information and Software Technology 52(1) (2010) 1430.
17 33. R. Feldt, R. Torkar, T. Gorschek and W. Afzal, Searching for cognitively diverse tests:
18 Towards universal test diversity metrics, in Proceedings of 1st Search-Based Software
19 Testing Workshop (SBST'08 ), 2008, pp. 178186.
34. T. Gorschek, Requirements Engineering Supporting Technical Product Management,
20
PhD thesis, Department of Systems and Software Engineering, Blekinge Institute of
21 Technology, May 2006.
22 35. T. Gorschek, P. Garre, S. Larsson and C. Wohlin, A model for technology transfer in
23 practice, IEEE Software 23(6) (2006) 8895.
24 36. T. Gorschek and C. Wohlin, Requirements abstraction model, Requirements Engineering
11(1) (2005) 79101.
25
37. O. Gotel and A. Finkelstein, An analysis of the requirements traceability problem, in
26 International Conference on Requirements Engineering, 1994, pp. 94101.
27 38. O. Gotel and A. Finkelstein, Extended requirements traceability: Results of an industrial
28 case study, in Proceedings of the 3rd IEEE International Symposium on Requirements
29 Engineering, Washington, DC, 1997, p. 169.
30 39. O. Gotel and S. J. Morris, Crafting the requirements record with the informed use of
media, in Proceedings of the First International Workshop on Multimedia Requirements
31
Engineering, Washington, DC, 2006, p. 5.
32 40. D. Gross and E. Yu, From non-functional requirements to design through patterns,
33 Requirements Engineering 6(1) (2001) 1836.
34 41. V. L. Hamilton and M. L. Beeby, Issues of traceability in integrating tools, in Proceedings
35 of the IEE Colloquium Tools and Techniques for Maintaining Traceability during Design,
Piscataway, NJ, USA, 1991, pp. 4/14/3.
36
42. J. Han, TRAM: A tool for requirements and architecture management, Australian
37 Computer Science Communincations 23(1) (2001) 6068.
38 43. Ø. Hauge, C. Ayala and R. Conradi, Adoption of open source software in software-
39 intensive organizations
A systematic literature review, Information and Software
40 Technology 52(11) (2010) 11331154.
41 44. J. H. Hayes, A. Dekhtyar and J. Osborne, Improving requirements tracing via informa-
tion retrieval, in Proceedings of the 11th IEEE International Conference on Requirements
42
Engineering, Washington, DC, 2003, p. 138.
43
May 19, 2012 6:13:59pm WSPC/117-IJSEKE 00584
1stReading
48 R. Torkar et al.