Download as pdf or txt
Download as pdf or txt
You are on page 1of 6

2019 IEEE 27th International Requirements Engineering Conference (RE)

An Agile Method of Representing, Organizing, and (Re)prioritizing


Requirements in a Large Enterprise

Woogon Shim
Software Engineering Lab., LG Electronics
Dept. of Software & Computer Engineering, Ajou University
Seoul, South Korea
woogon.shim@lge.com

Abstract— Agile and lean are now the mainstream tenets experimentation such as A/B testing and the data-driven deci-
of software development. This development methodology is sions to deliver value to the customer rapidly. Lean Software
attracting attention because it can develop software that op- Development [4], [5] provides comprehensive guidance for the
erates quickly despite uncertain situations and limited in- combination of design, development, and validation built as a
formation (requirements), and it can adaptively develop single feedback loop focused on the discovery and delivery of
while checking feedback both frequently and regularly. The value. Lean Startup [6] provides mechanisms to ensure that what
key is to welcome change without resistance, which is not customers want gets adequately addressed by the development.
limited to development but also requires changes in require- It proposes applying the scientific method and thinking to startup
ment management. However, there is a lack of practical businesses in the form of learning experiments.
guidance on how to manage changes in agile requirements Moreover, to handle uncertainty, agile fixes the date and re-
engineering. Although we need to consider all areas of re- sources and varies the scope. The key to agile or lean is selecting
quirements engineering, the author has seen three key and focusing the minimum viable product (MVP) and adopting
themes: (1) How to focus on customer value and represent feedback from the customer in each iteration. Behind this, we
it clearly for smooth communication between teams in a should prioritize requirements by considering the value in paral-
large enterprise? (2) What are the requirements for organ- lel or regularly [7]. Major studies such as the Standish Group’s
izing elements that make it easy to accept changes and assess CHAOS report (www.standishgroup.com) reveal that value-ori-
impacts? (3) What reasonable algorithm is easy to use and ented shortfalls cause the most software project failures [8].
reliable to prioritize or reorder requirements based on val-
ues regularly? The author wants to suggest practical ways II.MOTIVATION
to address these themes and propose a way to deal with Many companies are attempting to introduce the value of
three themes without compromising on the essence of agile. agile, but they are often more interested in the fruits rather
The author is going to do case studies with some organiza- than fundamental changes and following certain practices or
tions that are already using or first adopting agile software processes. Agile development is an empirical and evolutionary
development. The author could get the strengths and weak-
developmental approach that can be applied and adapted in
nesses in terms of collaboration promotion and change both
the acceptance and qualitative feedback on application re- any environment. However, studies by [9], [10], and [11] have
sults. found that the cultural characteristics of a nation also affect
how it applies agile. These studies analyzed certain countries
Index Terms—Agile, Lean, Just-in-time, VBSE, based on the Hofstede model, which is a model for comparing
Requirement representation, Requirement organization, and analyzing the country’s cultural characteristics and show-
Requirement (re)prioritization ing that there is a correlation between the six-dimension in-
dexes and the agile manifesto [10] and agile practices [11],
I.INTRODUCTION with some limitations.
Change is an intrinsic characteristic of the software engineer- East Asian countries, including South Korea, tend to be
ing discipline compared to other engineering disciplines [1]. organized homogeneously, although some companies are dis-
This characteristic means that agile software development (ASD) tributed around the world. In other words, there is a high prob-
emphasizes the embrace of changes rather than resistance. On ability that the country’s cultural characteristics may be re-
the same horizon, Value-Based Software Engineering (VBSE) flected rather than the conflict of multiple cultures. According
[2] highlights that it is not sufficient to merely meet schedules,
to [12], South Korea is a slightly hierarchical society, consid-
budgets, and quality objectives; ultimately, software must create
ered a collectivistic society, one of the world’s most
value. Continuous Experimentation [3] emphasizes

2332-6441/19/$31.00 ©2019 IEEE 464


DOI 10.1109/RE.2019.00063
uncertainty-avoiding countries, and shown to be one of the re-
straints (the green bar in Figure 1). That is, it is difficult to in-
troduce and apply agility because the Power Distance Index
(PDI) is high, and Individualism (IDV) is low. For example,
employees tend to act conservatively, depending on their or-
ganization’s hierarchy, role, and accountability, rather than
freely expressing and discussing their opinions. This may
cause daily scrum meetings to fall into a daily reporting form
or interfere with cooperation among related departments. In Fig. 2. Various requirement channels
other words, South Korea's representative "ppalli ppalli" cul-
ture, which means fast-paced lifecycle [13] can nicely suit Second, Asian companies are generally more interested in
with agile, but it should be introduced as tailored agile rather asking How rather than Why or What. As a fast follower, they
than pure agile in consideration of national cultural character- have developed quickly over a relatively short period, so they
istics. do not need to care about why or what. There is a severe lack
of interest in value, so the activity of deriving and evaluating
requirements on a value basis is not well established.
Third, only SW development organizations work in agile.
In other words, the company develops a product in a water-ag-
ile-fall manner. As the nature of the embedded products, the
whole development schedule is driven by HW. Since the com-
pany has to deliver the end product to its customers, it has to
be equipped with the perfect quality function before launch. In
addition, all requirements are often specified as MUST
HAVE.
Although my company mainly applies agile to the devel-
opment of new products or services, we still face the same
problem because the overall organizational and collaborative
Fig. 1. Country comparison between East Asia and the United States nature of the features remains the same.
(Hofstede Insights)
III.PROBLEM
In particular, the company to which the author belongs is The author believes that successfully implementing agile
a large-scale consumer electronics manufacturing company development will require that the preceding step, Requirement
with both scale-driven and manufacturer-specific characteris- Engineering activities, is done first in an agile manner. Natu-
tics. First, as the organization grows in size, more detailed rally, activities should be done to regularly (re)evaluate priori-
functional organizations are created, and the interests become ties, take the value and urgency of the collected requirements
more complex. Therefore, requirements must be created and into account, define the requirements by considering who the
managed so that all related departments can easily understand target customer is precisely, and identify the value to deliver
each other and communicate [14]. The formal specification or to that customer.
model is not very attractive for agile development because it is This will first require that the author has a practical guide
costly for all of the organization's practitioners to learn. As can for agile REs, and the author needs to propose tailoring activi-
be seen in Figure 2, the requirements were gathered from vari- ties based on cultural characteristics and the properties of large
ous functional departments. The level of content created by organizations. The important thing is that the proposed method
each department varies widely; preserving the current organi- should not hurt the essence of agile.
zational form; there is no individual or organization has the
role and responsibility of Scrum's Product Owner. Therefore,
it is difficult to evaluate, prioritize, and make choices that
must go through the consensus of relevant organizations.

Fig. 3. Objective and relationship among RQs

465
Our research questions are: RQ2. Organize requirements effectively and support dual-
track agile [23]
RQ1. How can we focus on customer value and represent it
clearly to ensure smooth communication among various teams 1) Academic Community
in a large enterprise? How should we use this to evaluate prior- In academia, similar to traditional organizations, they know
ity? that agile RE is carried out in projects’ early stages [24] or before
the start of each iteration [25]. However, in agile development,
RQ2. How should we handle the problem of User Story Hell? RE works in parallel with development throughout the project
[15] How can we organize well-mixed requirements with hy- period [26], and development starts even when the requirements
potheses and validated facts? Are there procedures to facilitate are imperfect. In the goal sketching technique [27], allowing
the impact assessment of changes and requirements updates? TBD in goal modeling means that it can provide a level of ab-
How can we support dual-track agile? straction to Goal-oriented RE (only as needed when necessary)
and give a few notations that various stakeholders can use easily.
RQ3. What is a natural, efficient, and reliable way to re-eval- However, RE-KOMBINE [28] tolerates the presence of incon-
uate/prioritize requirements on a value-based basis? Is it possi- sistencies and conflicts in requirements that are incurred when
ble to utilize crowd knowledge rather than a few specific indi- undertaking development with incomplete requirements. How-
viduals for valuation? ever, it also provides paraconsistent logic to detect and resolve
inconsistencies early on. It is quite flexible but challenging to
Figure 3 depicts that the three research questions are strictly learn and use because it is a formal method.
interrelated and an example of the classic chicken-and-egg prob- Olsson and Bosch noted that a few decision-makers tend to
lem, so the author has to look at all three together rather than determine features because getting feedback from customers
individually. takes too long, and collecting and analyzing data is inefficient.
To overcome this, they proposed a Hypothesis Experiment Data-
IV.RELATED WORKS
Driven Development (HYPEX) [29] or Qualitative/quantitative
Agile software development can be summarized in one sen- Customer-driven Development (QCD) [30] model that can
tence: "Deliver customer highest value with working quality check the customer's value and product features by receiving
product frequently and continuously." However, agile require- prompt and accurate feedback data from customers. This model
ment engineering can be summarized on this basis as "Discover considers requirements as a hypothesis and validates them pro-
and manage customer highest value frequently and continu- gressively based on the results of the experiment.
ously."
RQ1. Customer value-based requirement representation 2) Business Analyst Community
Discover to Deliver [31] presents agile product planning and
It is pointed out that the format of the user story itself is sim- analysis methods. The discovery phase, which uncovers new re-
ple and not faithful as a document [16]. However, agile advo- quirements, and the delivery phase, which develops and delivers
cates claim that the user story format was initially intended to the needs of the identified customers, are repeated continuously.
consider customer-focused value without containing all levels of This provides seven predefined dimensions for stakeholders to
its details. All of the hidden and implicit information should be guide collaboration. Structural communication is made for each
revealed by face-to-face communication [17], [18], which is dimension, and after pouring out various ideas (divergence) and
why agile emphasizes cross-functional teams in the same loca- screening the items with the highest ROI (convergence), the
tion. Meanwhile, Lucassen created a format to satisfy 14 quality product requirements of the next iteration are determined. Alt-
attributes of requirement documentation [19]; however, this will hough this approach is concerned with iterative development, it
likely hinder the agility of product development. does not deal with how to manage the change of requirements
There have been many attempts to solve this problem. Since based on what we have learned.
customers feel that their values help them carry out their tasks Value Proposition Design [32] is a follow-up study of the
and find solutions, Klement proposed another format, the Job Business Model Canvas [33]; it is a separate extension of “Value
Story [16]. Martin Fowler often failed to represent all cases in a Proposition” and “Customer Segment” among the nine blocks.
single User Story, which suggests that one User Story should be In-depth analysis of customers can help us identify needs and
extended to at least three categories (basic, uncaught exceptions, pains and separately consider what features the company will
and bug fixes). To link test-driven development (TDD) with the offer to provide value to customers. During each iteration, we
requirements, we need to specify Specification by Example [20] can review whether the product prepared (“value proposition”)
and use examples to make requirements that can then be used to by the company matches the customer analysis (“customer seg-
create test cases through specific scenarios instantly. There is ment”) and visualize it with a whiteboard and post-its.
Behavior-Driven Development (BDD), which records behavior
based on the style of the Given-When-Then; these things are not
3) Agile and Lean Community
new. RE researchers had already presented the similar concepts
Scenario-based RE [21] and Goal-Scenario [22].

466
Impact Mapping [34] presented a simple and effective re- identified 62 universal and comprehensive value propositions
quirements management system to escape the "user story card (called components). Value is assessed subjectively, which
hell” [15]. Impact mapping is a strategic planning technique that means that there is not only a difference between individuals but
prevents organizations from getting lost while building products also between companies. In the VALUE project [41], [42], they
and delivering projects by clearly communicating assumptions, saw a different value proposition (or value factor) for each com-
helping teams align their activities with overall business objec- pany. Therefore, they interviewed the key stakeholders involved
tives, and make better roadmap decisions. It restructures the typ- in valuation and analyzed the results to identify 36 value propo-
ical user story format "As a <type of actor>, I want <delivera- sitions that were classified into six dimensions: customer value,
ble> so that <impact>" into a mind-map form. It rearranges user market competitiveness, economic value/profitability, cost effi-
stories into the form Why–Who–How–What. It acts as a goal- and ciency, technology & architecture, and company strategy.
customer-centered map that guides us not to be lost during de-
velopment. V.PROPOSED METHOD (IN PROGRESS)
Moreover, the connection line between each item is the as-
RQ1. Customer value-based requirement representation
sumption that needs to be validated. However, there is a limita-
tion in that despite showing the entire requirement map. It is dif- Using specific scenarios can help with goal modeling [43].
ficult to see how features are connected and how they flow from In addition, if the development team uses TDD, it can be used as
a customer's perspective. a test case (i.e., an executable specification) to check the imple-
Instead, User Story Mapping [18] creates narrative end-to- mentation results [20]. Besides, large companies often have re-
end backbone scenarios that deliver value to the customer and quirements that must be communicated across multiple channels,
then attach specific work and exception handling tasks below so multiple functional departments often communicate and eval-
each stage of the skeleton scenario according to priority and then uate their priorities. Here, the authors prefer intuitive represen-
groups them so that can provide the outcome of the customer's tation rather than the formal model. (1) When the UI flow show-
perspective and establish a release plan. It is easy to understand ing the screen switching is the most natural to understand, and
the overall functional flow and natural to plan releases, but it is there is no UI; (2) it was adequate to use the Given–When–Then
difficult to grasp the product context from an individual cus- style of BDD; (3) JTBD style (When <situation> I want to <mo-
tomer perspective. tivation> So I can <outcome>) [16], which makes the custom-
er's situation easier to understand instead of the classic User
Story format. This study aimed to establish the criteria based on
RQ3. Value-based requirement (re)prioritization the actual application result, which is suitable to use in certain
There are several ways to calculate priority. Generally, if the situations. Meanwhile, there is a concern about coverage based
Return on Investment (ROI) is high, the priority is set to high. on the scenarios [44], but it is not a problem unless it is either an
The ROI is the result of the potential return (value) divided by embedded product or a mission-critical system.
the effort (the cost to implement) a feature. This approach as-
sumes that high ROI requirements must be developed first for
low ROI requirements [35]. However, if the ROI is low but
needs to enable other requirements, [7] should be developed first.
For example, if requirement A depends on requirement B, then
the priorities of these requirements should be consistent, and re-
quirement B should have at least as high a priority as A (e.g., B
should be implemented either before A or during the same build
as A) [36].
To address this problem, the Scaled Agile Framework (SAFe)
has adopted the Weight Shortest Job First (WSJF) method that
uses Reinersten's Cost of Delay (CoD) concept, as a method of
prioritizing [37], [38]. Cost of Delay is calculated as the sum of
user-business value, time criticality, and risk reduction and/or
opportunity enablement. WSJF makes a relative estimate by
concentrating on one item at a time, evaluating the item with the
smallest value as "one," and assigning the remainder to the rela-
tive value. Relative values are the Fibonacci numbers used in
planning poker. Finally, the WSJF value can calculate the CoD Fig. 4. Process of our approach in the dual-track context
as the job size (job duration), and the item with the highest WSJF
value has the highest priority [38]. RQ2. Organize requirements effectively and support dual-
Meanwhile, there have been attempts to evaluate and apply track agile
17 different priority algorithms for value-based requirement pri- The author will consider how to manage the three candidate
oritization (VBRP) [39]. There are in-depth studies of items and requirement representations for RQ1. In addition, the author
criteria that assess value. The Software Value Map [40] wants to provide what prioritization information is necessary for

467
RQ3 to quickly reflect updated information and expand previous that the algorithm’s constraints do not result in productive and
research [23] on how to express and reveal the dependencies be- positive results.
tween requirements.
REFERENCES
RQ3. Value-based requirement (re)prioritization
[1] S. Jayatilleke and R. Lai, “A systematic review of require-
(Re)prioritization activities that should be performed regu- ments change management,” pp. 1–23, Oct. 2017.
larly are based on three alternatives: (1) In the early stage of de- [2] B. Boehm, “Value-based software engineering: reinvent-
velopment. Key stakeholders were interviewed in a similar man- ing,” SIGSOFT Software Engineering Notes, vol. 28, no.
ner to [41] to derive value factors. AHP analysis identifies the 2, p. 3, Mar. 2003.
hierarchy between value factors and calculates the weights. [3] A. Fabijan, P. Dmitriev, H. H. Olsson, and J. Bosch, “The
Then, individual requirements are used to prioritize using TOP- Evolution of Continuous Experimentation in Software
SIS, such as [39]. (2) The author will expand the WSJF method Product Development: From Data to a Data-Driven Or-
and adopt the method proposed by [45]. The proposal is to cor- ganization at Scale,” presented at the 2017 IEEE/ACM
rect the CoD calculation by separating the orthogonal Urgency 39th International Conference on Software Engineering
from the Value. In addition, (3) whether KANO Analysis can be (ICSE), pp. 770–780.
applied practically is confirmed through a case study. [4] M. Poppendieck and M. A. Cusumano, “Lean software
development: A tutorial,” SOFTWARE, vol. 29, no. 5, pp.
VI.PLAN TO VERIFY
26–32, 2012.
The proposed approach was developed experimentally and [5] M. Poppendieck and T. Poppendieck, Implementing lean
promoted at the in-house SW college, and training was con- software development: From concept to cash. Pearson Ed-
ducted twice in total with personnel who wanted to learn and get ucation, 2007.
the opportunity to practice with various organizations. In this [6] E. Ries, The Lean Startup. Crown Business, 2011.
process, the approach was refined to be easily understood and [7] D. Leffingwell, Agile software requirements: lean re-
applied, and actual cases were collected while conducting semi- quirements practices for teams, programs, and the enter-
nars for organizations managing requirements within the com- prise. Addison-Wesley Professional, 2010.
pany. [8] B. Boehm and L. G. Huang, “Value-based software engi-
The author would like to apply the pilot to a few projects at neering: a case study,” Computer, vol. 36, no. 3, pp. 33–
the organization that manages the requirements to verify that the 41, 2003.
proposed approach can solve problems at the company to which [9] S. Pahuja, “Questioning if Agile Works in Asia,” Jun.
the author belongs. Meanwhile, the author will recruit agile prac- 2016.
titioners who belong to medium- and large-scale organizations [10]B. Vodde, “Scrum doesn't work in China!?,” presented at
with the same cultural environment in South Korea and confirm the Scrum Gathering, Shanghai, 2010, pp. 1–66.
that the proposed approach is both practical and helpful. [11]H. Ayed, B. Vanderose, and N. Habra, “Agile cultural
challenges in Europe and Asia: insights from practition-
VII.TECHNICAL CHALLENGES AND DISCUSSIONS
ers,” presented at the 2017 IEEE/ACM 39th International
The author determined that none of the three RQs should be Conference on Software Engineering: Software Engineer-
handled separately. There will be meaningful results when it is ing in Practice Track (ICSE-SEIP), pp. 153–162.
finished but could each be considered too big a topic and thus [12] “Country Comparison of East Asia and the United
the focus should be on just one of them? This is a concern. Mean- States,” Hofstede Insights. [Online]. Available:
while, business analysts and startup entrepreneurs also need lean https://www.hofstede-insights.com/country-compari-
and agile philosophies, and similar approaches have recently son/china,japan,south-korea,the-usa/.
emerged in terms of concepts such as Value Proposition Design [13]H. L. Yi, “[Korean Culture 101] ‘Ppalli Ppalli Culture’ –
and Lean UX. It is difficult to determine the meaning of a doc- Korean’s Disposition to Like Fast-paced Lifestyle,” Ko-
toral dissertation by simply acquiring cases and providing spe- rea Daily, 26-Feb-2016.
cific procedures for the software development domain. [14]R. Kasauli, G. Liebel, E. Knauss, S. Gopakumar, and B.
Regarding priority evaluation, the most important thing is Kanagwa, “Requirements Engineering Challenges in
that priority evaluation and rebalancing should occur at least at Large-Scale Agile System Development,” presented at the
every iteration. Therefore, the cost of prioritization should be 2017 IEEE 25th International Requirements Engineering
small, and there should be consistent and realistic decision mak- Conference (RE), 2017, pp. 352–361.
ing so that the priority evaluation results do not overturn the re- [15]J. Shore, “Beyond Story Cards: Agile Requirements Col-
sults per metric. For the MCDM algorithm, there may be a limit laboration,” 25-Mar-2005. [Online]. Available:
to the number of items to be evaluated or the scale, or sometimes http://www.jamesshore.com/Presentations/Be-
a reverse ranking problem. In addition, since the MCDM is the yond%20Story%20Cards.html.
optimal choice among equivalent levels of mutually competitive [16]A. Klement, “Replacing The User Story With The Job
alternatives, it is questionable whether the same algorithm can Story – Jobs to be Done,” 13-Nov-2013. [Online]. Availa-
be used to assess mutually dependent requirements; therefore, ble: https://jtbd.info/replacing-the-user-story-with-the-
we intend to concentrate on WSJF. In this case, there is a concern job-story-af7cdee10c27#.i6750x1yu.

468
[17]M. Fowler, “Conversational Stories,” 04-Feb-2010. [31]E. Gottesdiener and M. Gorman, Discover to Deliver.
[Online]. Available: https://martinfowler.com/bliki/Con- EBG Consulting, Inc., 2012.
versationalStories.html. [32]A. Osterwalder, Y. Pigneur, G. Bernarda, and A. Smith,
[18]J. Patton and P. Economy, User Story Mapping. “O'Reilly Value Proposition Design. John Wiley & Sons, 2014.
Media, Inc.,” 2014. [33]A. Osterwalder and Y. Pigneur, Business Model Genera-
[19]G. Lucassen, F. Dalpiaz, J. M. E. M. van der Werf, and S. tion. John Wiley & Sons, 2010.
Brinkkemper, “Forging high-quality User Stories: To- [34]G. Adzic, Impact Mapping. Provoking Thoughts, 2012.
wards a discipline for Agile Requirements.,” RE, pp. 126– [35]J. Karlsson and K. Ryan, “A cost-value approach for pri-
135, 2015. oritizing requirements,” SOFTWARE, vol. 14, no. 5, pp.
[20]G. Adzic, Specification by Example. Manning Publica- 67–74, 1997.
tions, 2011. [36]D. Firesmith, “Prioritizing Requirements,” JOT, vol. 3,
[21]A. Sutcliffe, “Scenario-based requirements engineering,” no. 8, pp. 35–14, 2004.
presented at the Requirements Engineering Conference, [37]D. G. Reinertsten, The principles of product development
2003. Proceedings. 11th IEEE International, 2003, pp. flow: second generation lean product development. Celer-
320–329. itas, 2009.
[22]C. Rolland, G. Grosz, and R. Kla, Experience with goal- [38]D. Leffingwell, SAFe 4.5 Reference Guide: Scaled Agile
scenario coupling in requirements engineering. IEEE, Framework for Lean Enterprises. Addison-Wesley Pro-
1999, pp. 74–81. fessional, 2018.
[23]W. Shim and S. W. Lee, “An agile approach for managing [39]N. Kukreja, S. S. Payyavula, B. Boehm, and S. Pad-
requirements change to improve learning and adaptabil- manabhuni, “Value-Based Requirements Prioritization:
ity,” Journal of Industrial Information Integration, vol. Usage Experiences,” Procedia - Procedia Computer Sci-
14, pp. 16–23, 2019. ence, vol. 16, pp. 806–813, 2013.
[24]Ben Linders, “Delivering Software with Water-Scrum- [40]M. Khurum, T. Gorschek, and M. Wilson, “The software
Fall,” Nov. 2015. value map—an exhaustive collection of value aspects for
[25]Z. Racheva, M. Daneva, and A. Herrmann, “A conceptual the development of software intensive products,” Journal
model of client-driven agile requirements prioritization: of software: Evolution and Process, vol. 25, no. 7, pp.
Results of a case study,” presented at the Proceedings of 711–741, 2013.
the 2010 acm-ieee international symposium on empirical [41]P. Rodríguez, E. Mendes, and B. Turhan, “Key Stake-
software engineering and measurement, 2010, p. 39. holders' Value Propositions for Feature Selection in Soft-
[26]S. Lerén and I. Domingues, “Impact-Driven Scrum Deliv- ware-intensive Products: An Industrial Case Study,” IEEE
ery,” presented at the Global SCRUM Gathering, Phoe- Transactions on Software Engineering Y1 -, pp. 1–1,
nix, 2015. 2018.
[27]K. Boness and R. Harrison, “Goal Sketching: Towards [42]V. Freitas, M. Perkusich, E. Mendes, P. Rodríguez, and
Agile Requirements Engineering,” presented at the Inter- M. Oivo, “Value-Based Decision-Making Using a Web-
national Conference on Software Engineering Advances Based Tool: A Multiple Case Study,” presented at the
(ICSEA), 2007, pp. 71–71. 2017 24th Asia-Pacific Software Engineering Conference
[28]N. A. Ernst, A. Borgida, I. J. Jureta, and J. Mylopoulos, (APSEC), Nanjing, China, 2017, pp. 279–288.
“Agile requirements engineering via paraconsistent rea- [43]C. Rolland, C. Souveyet, and C. Ben Achour, “Guiding
soning,” Information Systems, vol. 43, no. C, pp. 100– Goal Modeling Using Scenarios,” IEEE Transactions on
116, Jul. 2014. Software Engineering, vol. 24, no. 12, pp. 1055–1071,
[29]H. H. Olsson and J. Bosch, “From Opinions to Data- Dec. 1998.
Driven Software R&D,” presented at the Software Engi- [44]S. Misra, V. Kumar, and U. Kumar, “Goal-oriented or
neering and Advanced Applications (SEAA), 2014 40th scenario-based requirements engineering technique - what
EUROMICRO Conference on, 2014, pp. 9–16. should a practitioner select?,” … and Computer Engineer-
[30]H. H. Olsson and J. Bosch, “Towards Continuous Cus- ing, pp. 2288–2292, 2005.
tomer Validation - A Conceptual Model for Combining [45]J. Arnold, “SAFe & Cost of Delay: a suggested improve-
Qualitative Customer Feedback with Quantitative Cus- ment,” Jun-2016. [Online]. Available: http://blackswan-
tomer Observation.,” ICSOB, vol. 210, no. 13, pp. 154– farming.com/safes-cost-of-delay-a-suggested-improve-
166, 2015. ment/.

469

You might also like