Solution Evaluation

Chapter Study Group

Learning Materials

1
2015, International Institute of Business Analysis™
(IIBA®).

Permission is granted to IIBA Chapters to use and

modify this content to support chapter activities.

All other rights reserved.

2
Study Session Schedule
Session Date Chapters Topics
1 Jan 25 1&2 Introduction and BA Key Concepts

2 Feb 15 3 Business Analysis Planning and Monitoring

3 Mar 15 4 Elicitation and Collaboration

4 Apr 12 5 Requirements Life Cycle Management

5 May 17 6 Strategy Analysis

6 Jun 14 7 Requirements Analysis and Design Definition

7 Jul 12 8 Solution Evaluation

8 Sep 13 9 Underlying Competencies & Perspectives

9 Oct 11 10 Techniques

10 Nov 1 11 Jeopardy Game and Mock Exam

3
AGENDA
• (6:00 – 6:20) Review of Chapter 7 RADD (20 minutes)

• (6:20 – 6:45) introduction and 8.1 Measure Solution Performance (25

minutes)

• (6:45 – 7:10) 8.2 Analyze Performance Measures (25 minutes)

• (7:10 – 7:35) 8.3 Assess Solution Limitations (25 minutes)
• (7:35 – 7:45) Break (10 minutes)

• (7:45 – 8:10) 8.4 Assess Enterprise Limitations (25 minutes)

• (8:10 – 8:35) 8.5 Recommend Actions to Increase Solution Value (25
minutes)

• (8:35 – 8:50) Group practice (15 minutes)

Solution Evaluation 4
INTRODUCTION
• Chapter 8 of BABOK v3

• Consists of tasks performed to:

• Determine how the performance of a solution will be assessed
• Assess solution performance to understand value delivered
• Investigate solution issues that may be leading to poor performance
• Investigate issues external to the solution that may be leading to poor
performance
• Identify actions that can be taken to increase the value delivered by a
solution

Solution Evaluation 5
INTRODUCTION

Solution Evaluation 6
INTRODUCTION

Solution Evaluation 7
INTRODUCTION - APPLYING THE BUSINESS
ANALYSIS CORE CONCEPT MODEL
Core Concept Usage in Solution Evaluation
Change Recommend a change to either a solution or the enterprise
in order to realize the potential value of a solution
Need Evaluate how a solution or solution component is fulfilling
the need
Solution Assess the performance of the solution, examine if it is
delivering the potential value, if not determine why
Stakeholder Elicit information from the stakeholders about solution
performance and value delivery
Value Determine if the solution is delivering the potential value and
examine why value may not be being realized
Context Consider the context in determining solution performance
measures and any limitations within the context that may
prohibit value from being realized
Solution Evaluation 8
8.1 Measure Solution
Performance

9
8.1 MEASURE SOLUTION PERFORMANCE

• Purpose
• Define performance measures.
• Evaluate the effectiveness of a solution in relation to the value the
solution brings.

• Description
• Determines the value of a newly deployed or existing solution.
• Measurement depends on the solution: context and how the
organization defines value.
• Solutions are evaluated based on:
• Exist: built-in performance measures
• Not exist: BA works with stakeholders to determine and collect the measures
that will best reflect the performance of a solution.

Solution Evaluation 10
8.1 MEASURE SOLUTION PERFORMANCE

• Description
• Performance may be assessed through the following:
• key performance indicators (KPI’s)
• Project goals and objectives
• Process performance targets
• Tests for a software application

Solution Evaluation 11
8.1 MEASURE SOLUTION PERFORMANCE

Solution Evaluation 12
8.1 MEASURE SOLUTION PERFORMANCE
• Business Objectives
• Measureable results that the enterprise wants to achieve.
• Provides a benchmark against which solution performance
can be assessed.

• Implemented Solution (external)

• A solution (or component of a solution) that exists in some
1. form:
Inputs • Operating solution
• Prototype
• Pilot or Beta

Solution Evaluation 13
8.1 MEASURE SOLUTION PERFORMANCE
• Define Solution Performance Measures
• Business analysts determine if current measures exist,
or if methods for capturing them are in place.
• Business analysts ensure that any existing performance
measures are accurate, relevant and elicit any
additional performance measures identified by
stakeholders.
2. • Business goals, objectives, and business processes are
Elements common sources of measures.
• Performance measures may be influenced or imposed
by third parties such as solution vendors, government
bodies, or other regulatory organizations.

Solution Evaluation 14
8.1 MEASURE SOLUTION PERFORMANCE
• Define Solution Performance Measures
• Type and nature of the measurements are considered
when choosing the elicitation method.
• Type of solution performance measures depend on the
value being measured:
• Quantitative Measures: Numerical, countable, or finite,
usually involving amounts, quantities, or rates
2. • Qualitative Measures: subjective and can include attitudes,
Elements perceptions, and any other subjective response
• Can be both depending on the value being measured

Solution Evaluation 15
8.1 MEASURE SOLUTION PERFORMANCE
• Validate Performance Measures
• Helps to ensure that the assessment of solution
performance is useful.
• Business Analysts validate the performance measures
and any influencing criteria with stakeholders.
• Performance measures should align with any higher-
level measures that exist within the context affecting
2. the solution.
Elements • The sponsor, or any stakeholder with decision-making
authority, decides which measures are used to
evaluate solution performance.

Solution Evaluation 16
8.1 MEASURE SOLUTION PERFORMANCE
• Collect Performance Measures
• Business Analysts may facilitate discussions to estimate
the qualitative measures/value realized by a solution by
consulting with stakeholders knowledgeable about the
operation and use of the solution.
• Business Analysts may consider the following when
collecting quantitative performance measures:
2. • Volume or Sample Size: an appropriate sample size is
selected. Too small may lead to inaccurate conclusions.
Elements Larger samples may not be practical to obtain.
• Frequency and Timing: frequency and timing with which
measurements are taken may have an effect on the outcome
• Currency: measurements taken more recently tend to be
representative than older data

Solution Evaluation 17
8.1 MEASURE SOLUTION PERFORMANCE
• Change Strategy
• Strategy used or in use to implement the potential value.

• Future State Description

• Boundaries of the proposed new, removed, or modified
components of the enterprise, and the potential value
expected from the future state.
3.
• Requirements (validated)
Guidelines
• A set of requirements that have been analyzed and
and Tools appraised to determine their value.

• Solution Scope
• Solution boundaries to measure and evaluate.

Solution Evaluation 18
8.1 MEASURE SOLUTION PERFORMANCE
• Acceptance and Evaluation Criteria
• Used to define acceptable solution performance.

• Benchmarking and Market Analysis

• Used to define measures and their acceptable levels.

• Business Case
4.
• Used to define business objectives and performance
Techniques measures for a proposed solution.

• Data Mining
• Used to collect and analyze large amounts of data
regarding solution performance.

Solution Evaluation 19
8.1 MEASURE SOLUTION PERFORMANCE
• Decision Analysis
• Used to assist stakeholders in deciding on suitable ways
to measure solution performance and acceptable levels
of performance.

• Focus Group
• Used to provide subjective assessments, insights, and
4. impressions of a solution’s performance.
Techniques
• Metrics and Key Performance Indicators (KPIs)
• Used to measure solution performance.

Solution Evaluation 20
8.1 MEASURE SOLUTION PERFORMANCE
• Non-functional Requirements Analysis
• Used to define expected characteristics of a solution.

• Observation
• Used either to provide feedback on perceptions of
solution performance or to reconcile contradictory
results.
4.
Techniques • Prototyping
• Used to simulate a new solution so that performance
measures can be determined and collected.

Solution Evaluation 21
8.1 MEASURE SOLUTION PERFORMANCE
• Survey or Questionnaire
• Used to gather opinions and attitudes about solution
performance.

• Use Cases and Scenarios

• Used to define the expected outcomes of a solution.

4. • Vendor Assessment
Techniques • Used to assess which of the vendor’s performance
measures should be included in the solution’s
performance assessment.

Solution Evaluation 22
8.1 MEASURE SOLUTION PERFORMANCE
• Customer
• May be consulted to provide feedback on solution
performance.

• Domain Subject Matter Expert

• Person familiar with the domain who can be
consulted to provide potential measurements.
5.
Stakeholders • End User
• Contributes to the actual value realized by the
solution in terms of solution performance.
• May be consulted to provide reviews and feedback
on areas such as workload and job satisfaction.

Solution Evaluation 23
8.1 MEASURE SOLUTION PERFORMANCE
• Project Manager
• Responsible for managing the schedule and tasks to
perform the solution measurement.
• This role may not be required for operational solutions.

• Sponsor
5. • Responsible for approving the measurements used to
determine solution performance.
Stakeholders
• May provide performance expectations.

• Regulator
• Internal or external group.
• Dictate or prescribe constraints and guidelines in
solution performance measures.

Solution Evaluation 24
8.1 MEASURE SOLUTION PERFORMANCE
• Solution Performance Measures
• Measures that provide information on how well the
solution is performing or potentially could perform.

6.
Outputs

Solution Evaluation 25
8.2 Analyze Performance
Measures

28
8.2 ANALYZE PERFORMANCE MEASURES
• Purpose
• Provide insights into the performance of a solution in relation to the
value it brings.

• Description
• Interpret and derive meaning on measures collected from 8.1 Measure
Solution Performance.
• Business Analysts require a thorough understanding of the potential
value that stakeholders hope to achieve with the solution.
• Analyze by considering the variables:
• Goals and objectives of the enterprise
• Key performance indicators (KPIs)
• Level of risk of the solution
• Risk tolerance of both stakeholders and enterprise
• Other stated targets
Solution Evaluation 29
8.2 ANALYZE PERFORMANCE MEASURES

Solution Evaluation 30
8.2 ANALYZE PERFORMANCE MEASURES
• Potential Value
• Describe the value that may be realized by
implementing the proposed future state.
• Can be used as a benchmark against which solution
performance can be evaluated.

• Solution Performance Measures

1. • Measures and provides information on how well the
Inputs solution is performing or potentially could perform.

Solution Evaluation 31
8.2 ANALYZE PERFORMANCE MEASURES
• Solution Performance vs. Desired Value
• Business Analysts examine the measures previously
collected to assess the solutions ability to help
stakeholders understand the solution’s value:
• High performing solutions may contribute a lower value
than expected
• Low performing, but potentially valuable, solutions may
2. be enhanced to increase its performance

Elements • If the measures are not sufficient to determine

solution value, the business analysts can either
collect more measurements or treat the lack of
measures as a solution risk.

Solution Evaluation 32
8.2 ANALYZE PERFORMANCE MEASURES
• Risks
• Performance measures may uncover new risks to
solution performance and to enterprise.
• Identified and managed like any other risks.

• Trends
• A large enough sample size or collected data over a
2. sufficient time period.
Elements • Business analysts consider the time period when
the data was collected to guard against anomalies
and skewed trends.
• Pronounced and repeated trends are noted.

Solution Evaluation 33
8.2 ANALYZE PERFORMANCE MEASURES
• Accuracy
• Essential to the validity of the analysis.
• Business analysts test and analyze the data collected by
the performance measures to ensure their accuracy.
• Results of performance measures should be reproducible
and repeatable.

2. • Performance Variances
Elements • Difference between expected and actual performance.
• Considered when analyzing solution performance.
• Root cause analysis may be necessary to analyze the
causes of the significant variances within a solution.

Solution Evaluation 34
8.2 ANALYZE PERFORMANCE MEASURES
• Change Strategy
• Strategy used or is in use to implement the potential value.

• Future State Description

• Boundaries of the proposed new, removed, or modified
components of the enterprise, and the potential value
expected from the future state.
3.
Guidelines • Risk Analysis Results
and Tools • Overall level of risk and the planned approach to modifying
the individual risks.

• Solution Scope
• Solution boundaries to measure and evaluate.

Solution Evaluation 35
8.2 ANALYZE PERFORMANCE MEASURES
• Acceptance and Evaluation Criteria
• Used to define acceptable solution performance.
• Degree of variance from these criteria will guide the
analysis of that performance.

• Benchmarking and Market Analysis

• Used to observe the results of other organizations
4. employing similar solutions when assessing risks,
Techniques trends, and variances.

• Data Mining
• Used to collect data regarding performance, trends,
common issues, and variances from expected
performance levels and understand patterns and
meaning in that data.

Solution Evaluation 36
8.2 ANALYZE PERFORMANCE MEASURES
• Interviews
• Used to determine expected value of a solution and
its perceived performance from an individual or
small group’s perspective.

• Metrics and KPIs

• Used to analyze solution performance, especially
4. when judging how well a solution contributes to
Techniques achieving goals.

• Observation
• Used to observe a solution in action if the data
collected does not provide definitive conclusions.

Solution Evaluation 37
8.2 ANALYZE PERFORMANCE MEASURES
• Risk Analysis and Management
• Used to identify, analyze, develop plans to modify
the risks, and to manage the risks on an ongoing
basis.

• Root Cause Analysis

• Used to determine the underlying cause of
4. performance variance.
Techniques
• Survey or Questionnaire
• Used to determine expected value of a solutions and
its perceived performance.

Solution Evaluation 38
8.2 ANALYZE PERFORMANCE MEASURES
• Domain Subject Matter Expert
• Identifies risks and provide insights into data for
analyzing solution performance.

• Project Manager
• Responsible for overall risk management and may
participate in risk analysis for new or changed
5. solutions in a project.
Stakeholders
• Sponsor
• Identifies risks, provide insights into data and the
potential value of a solution.
• Make decisions about the significance of expected
versus actual solution performance.

Solution Evaluation 39
8.2 ANALYZE PERFORMANCE MEASURES
• Solution Performance Analysis
• Results of the analysis of measurements collected.
• Recommendations to solve performance gaps.
• Leverage opportunities to improve value.

6.
Outputs

Solution Evaluation 40
8.3 Assess Solution
Limitations

43
8.3 ASSESS SOLUTION LIMITATIONS

• Purpose
• Determine the factors internal to the solution that restrict the full
realization of value.

• Description
• Identifies the root causes for under-performing and ineffective
solutions and solution components.
• Closely linked to the task 8.4 Assess Enterprise Limitations.
• Focuses on the assessment of the internal factors that is limiting the
solution’s value or potential value.

Solution Evaluation 44
8.3 ASSESS SOLUTION LIMITATIONS

Solution Evaluation 45
8.3 ASSESS SOLUTION LIMITATIONS
• Implemented Solution (external)
• An existing solution.
• May or may not be operational (i.e. prototype).
• Must be in use in some form to be evaluated.

• Solution Performance Analysis

1. • Results of the analysis of measurements collected.
Inputs • Recommendations to solve for performance gaps.
• Leverage opportunities to improve value.

Solution Evaluation 46
8.3 ASSESS SOLUTION LIMITATIONS
• Identify Internal Solution Component
Dependencies
• Business analysts identify solution components which
have dependencies on other solution components, and
determine if any of those limit solution performance and
value realization.

2. • Investigate Solution Problems

Elements • Performed to identify the source of the problem.
• Conducted when the solution is consistently or
repeatedly producing ineffective outputs.
• Business analysts identify problems by examining
instances where the outputs from the solution are below
an acceptable level of quality or where the potential
value is not being realized.

Solution Evaluation 47
8.3 ASSESS SOLUTION LIMITATIONS
• Impact Assessment
• Business analysts review identified problems to assess
the effect they may have on the operation of the
organization or the ability of the solution to deliver its
potential value.
• Determine the severity of the problem, the probability of
the reoccurrence of the problem, the impact on the
2. business operations, and the capacity of the business to
absorb the impact.
Elements
• Business analysts identify which problems must be
resolved, mitigated, or which accepted.
• Include additional quality control measures, new or
adjusted business processes, or additional support for
exceptions to the desired outcome.
• Risk assessment is specific to the solution and its
limitations.
Solution Evaluation 48
8.3 ASSESS SOLUTION LIMITATIONS
• Change Strategy
• Strategy used or in use to implement the potential
value.

• Risk Analysis Results

• Overall level of risk and the planned approach to
modifying the individual risks.
3.
Guidelines • Solution Scope
and Tools • Solution boundaries to measure and evaluate.

Solution Evaluation 49
8.3 ASSESS SOLUTION LIMITATIONS
• Acceptance and Evaluation Criteria
• Used to indicate the level at which acceptance criteria
are met or anticipated to be met by the solution.
• To identify criteria that are not met by the solution.

• Benchmarking and Market Analysis

• Used to assess if other organizations are experiencing
4. the same solution challenges and, if possible,
Techniques determine how they are addressing it.

• Business Rules Analysis

• Used to illustrate the current business rules and the
changes required to achieve the potential value of the
change.

Solution Evaluation 50
8.3 ASSESS SOLUTION LIMITATIONS
• Data Mining
• Used to identify factors constraining performance of
the solution.

• Decision Analysis
• Used to illustrate the current business decisions and
the changes required to achieve the potential value of
4. the change.
Techniques • Interviews
• Used to help perform problem analysis.

• Item Tracking
• Used to record and manage stakeholder issues related
to why the solution is not meeting the potential value.

Solution Evaluation 51
8.3 ASSESS SOLUTION LIMITATIONS
• Lessons Learned
• Used to determine what can be learned from the
inception, definition, and construction of the solution
to have potentially impacted its ability to deliver
value.

• Risk Analysis and Management

4. • Used to identify, analyze, and manage risks, as they
relate to the solution and its potential limitations, that
Techniques may impede the realization of potential value.

• Root Cause Analysis

• Used to identify and understand the combination of
factors and their underlying causes that led to the
solution being unable to deliver its potential value.

Solution Evaluation 52
8.3 ASSESS SOLUTION LIMITATIONS
• Survey or Questionnaire
• Used to help perform problem analysis.

4.
Techniques

Solution Evaluation 53
8.3 ASSESS SOLUTION LIMITATIONS
• Customer
• Ultimately affected by the solution.
• Has an important perspective on the solution’s value.
• May be consulted to provide reviews and feedback.

• Domain Subject Matter Expert

5. • Provides input into how the solution should perform
Stakeholders and identifies potential limitations to value realization.

Solution Evaluation 54
8.3 ASSESS SOLUTION LIMITATIONS
• End User
• Uses the solution or a component of the solution.
• Contributes to the actual value realized by the solution
in terms of solution performance.
• May be consulted to provide reviews and feedback on
areas such as workload and job satisfaction.
5.
• Regulator
Stakeholders
• A person whose organization needs to be consulted
about planned and potential value of a solution.
• This person’s organization may constrain the solution,
the degree to which actual value is realized, or when
actual value is realized.

Solution Evaluation 55
8.3 ASSESS SOLUTION LIMITATIONS
• Sponsor
• Responsible for approving the potential value of the
solution.
• Responsible for providing resources to develop,
implement and support the solution.
• Responsible for directing enterprise resources to use
the solution.
5.
Stakeholders • Responsible for approving a change to potential value.

• Tester
• Responsible for identifying solution problems during
construction and implementation; not often used in
assessing an existing solution outside of a change.

Solution Evaluation 56
8.3 ASSESS SOLUTION LIMITATIONS
• Solution Limitation
• Description of the current limitations of the solution
including constraints and defects.

6.
Outputs

Solution Evaluation 57
8.4 Assess Enterprise
Limitations

60
8.4 ASSESS ENTERPRISE LIMITATIONS

• Purpose
• Determine how factors external to the solution are restricting value realization.

• Description
• Identifies root causes and describes how enterprise factors limit value
realization.
• Enterprise limitations may include factors such as culture, operations, technical
components, stakeholder interests, or reporting structures.
• Assessment may be performed at any point during the solution cycle and the
assessment activities are similar and require the same skills.
• Assessment may occur on a solution component during its development or on
a completed solution prior to full implementation.
• Assessment may occur on an existing solution that is currently working within
an organization.

Solution Evaluation 61
8.4 ASSESS ENTERPRISE LIMITATIONS

• Description
• Note that solutions operate across various organizations within an enterprise,
and therefore have many interactions and interdependencies.
• Solutions may depend on environmental factors that are external to the
enterprise.

Solution Evaluation 62
8.4 ASSESS ENTERPRISE LIMITATIONS

Solution Evaluation 63
8.4 ASSESS ENTERPRISE LIMITATIONS
• Current State Description
• Current internal environment of the solution including the
environmental, cultural, and internal factors influencing
the solution limitations.

• Implemented (or Constructed) Solution (external)

• An existing solution.
1. • May or may not be operational (i.e. prototype).
Inputs • Must be in use in some form to be evaluated.

• Solution Performance Analysis

• Results of the analysis of measurements collected.
• Recommendations to solve for performance gaps.
• Leverage opportunities to improve value.

Solution Evaluation 64
8.4 ASSESS ENTERPRISE LIMITATIONS
• Enterprise Culture Assessment
• Enterprise culture is defined as the deeply rooted
beliefs, values, and norms shared by the members
of an enterprise. While these beliefs and values may
not be directly visible, they drive the actions taken
by an enterprise.
• Business analysts perform cultural assessments to:
2. • Identify whether or not stakeholders understand the
reasons why a solution exists
Elements
• Ascertain whether or not the stakeholders view the
solution as something beneficial and are supportive of
the change
• Determine if and what cultural changes are required to
better realize value from a solution.

Solution Evaluation 65
8.4 ASSESS ENTERPRISE LIMITATIONS
• Enterprise Culture Assessment
• Involves evaluating the extent to which the
enterprise culture can accept a solution:
• Cultural adjustments are needed to support the solution
• Assessment is used to judge the enterprise's ability and
willingness to adapt to these cultural changes
• Business analysts evaluate internal and external
2. stakeholders to:
Elements • Gauge understanding and acceptance of the solution
• Assess perception of value and benefit from the solution
• Determine what communication activities are needed to
ensure awareness and understanding of the solution

Solution Evaluation 66
8.4 ASSESS ENTERPRISE LIMITATIONS
• Stakeholder Impact Analysis
• Provides insight into how the solution affects a particular
stakeholder group.
• Business analysts consider: Functions, Locations, and Concerns
• Functions: the processes in which the stakeholder uses the
solution, which include inputs a stakeholder provides into the
process, how the stakeholder uses the solution to execute the
2. process, and what outputs the stakeholder receives from the
process
Elements • Locations: geographic locations of the stakeholders interacting
with the solution. If the stakeholders are in disparate locations, it
may impact their use of the solution and the ability to realize the
value of the solution
• Concerns: the issues, risks, and overall concerns the
stakeholders have with the solution. This may include the use of
the solution, the perceptions of the value of the solution, and the
impact the solution has on a stakeholder’s ability to perform
necessary functions.
Solution Evaluation 67
8.4 ASSESS ENTERPRISE LIMITATIONS
• Organizational Structure Changes
• Assess how the organization’s structure is impacted by
a solution.
• Business analysts would consider the following:
• Use of a solution and the ability to adopt a change can be
enabled or blocked by formal and informal relationships
among stakeholders
2. • Reporting structure may be too complex or too simple to
allow a solution to perform effectively
Elements
• The organizational hierarchy supporting the solution is a key
activity.
• Informal relationships within an organization, whether
alliances, friendships, or matrix-reporting impact the ability
of a solution to deliver potential value

Solution Evaluation 68
8.4 ASSESS ENTERPRISE LIMITATIONS
• Operational Assessment
• Determines if an enterprise is able to adapt to or
effectively use a solution.
• This identifies which processes and tools within the
enterprise are adequately equipped to benefit from the
solution, and if sufficient and appropriate assets are in
place to support it.
2. • Business analysts consider the following:
Elements • Policies and procedures
• Capabilities and processes that enable other capabilities
• Skill and training needs
• Human resources practices
• Risk tolerance and management approaches
• Tools and technology that support a solution

Solution Evaluation 69
8.4 ASSESS ENTERPRISE LIMITATIONS
• Business Objectives
• Considered when measuring and determining
solution performance.

• Change Strategy
• Used or in use to implement the potential value.

3. • Future State Description

Guidelines • Boundaries of the proposed new, removed, or
modified components of the enterprise, as well as
and Tools the potential value expected from the future state.

• Risk Analysis Results

• The overall level of risk and the planned approach to
modifying the individual risks.

Solution Evaluation 70
8.4 ASSESS ENTERPRISE LIMITATIONS
• Solution Scope
• The solution boundaries to measure and evaluate.

3.
Guidelines
and Tools

Solution Evaluation 71
8.4 ASSESS ENTERPRISE LIMITATIONS
• Benchmarking and Market Analysis
• Used to identify existing solutions and enterprise
interactions.

• Brainstorming
• Used to identify organizational gaps or stakeholder
concerns.
4.
Techniques • Data Mining
• Used to identify factors constraining performance of
the solution.

Solution Evaluation 72
8.4 ASSESS ENTERPRISE LIMITATIONS
• Decision Analysis
• Used to assist in making an optimal decision under
conditions of uncertainty.
• Used in the assessment to make decisions about functional,
technical, or procedural gaps.

• Document Analysis
4. • Used to gain an understanding of the culture, operations,
Techniques and structure of the organization.

• Interviews
• Used to identify organizational gaps or stakeholder
concerns.

Solution Evaluation 73
8.4 ASSESS ENTERPRISE LIMITATIONS
• Item Tracking
• Used to ensure that issues are not neglected or lost
and that issues identified by assessment are resolved.

• Lessons Learned
• Used to analyze previous initiatives and the enterprise
interactions with the solutions.
4.
Techniques • Observation
• Used to witness the enterprise and solution
interactions to identify impact.

Solution Evaluation 74
8.4 ASSESS ENTERPRISE LIMITATIONS
• Organizational Modelling
• Used to ensure the identification of any required
changes to the organizational structure that may
have to be addressed.

• Process Analysis
• Used to identify possible opportunities to improve
4. performance.
Techniques
• Process Modelling
• Used to illustrate the current business processes
and/or changes that must be made in order to
achieve the potential value of the solution.

Solution Evaluation 75
8.4 ASSESS ENTERPRISE LIMITATIONS
• Risk Analysis and Management
• Used to consider risk in the areas of technology, finance,
and business.
• Technology: if the selected technological resources
provide required functionality.
• Finance: if costs could exceed levels that make the
change salvageable.
4. • Business: if the organization will be able to make the
Techniques changes necessary to attain potential value from the
solution.

• Roles and Permissions Matrix

• Used to determine roles and associated permissions for
stakeholders, as well as stability of end users.

Solution Evaluation 76
8.4 ASSESS ENTERPRISE LIMITATIONS
• Root Cause Analysis
• Used to determine if the underlying cause may be
related to enterprise limitations.

• Survey or Questionnaire
• Used to identify organizational gaps or stakeholder
concerns.
4.
Techniques • SWOT Analysis
• Used to demonstrate how a change will help the
organization maximize strengths and minimize
weaknesses.
• Used to assess strategies developed to respond to
identified issues.

Solution Evaluation 77
8.4 ASSESS ENTERPRISE LIMITATIONS
• Workshops
• Used to identify organizational gaps or stakeholder
concerns.

4.
Techniques

Solution Evaluation 78
8.4 ASSESS ENTERPRISE LIMITATIONS
• Customer
• Directly purchasing or consuming the solution.
• May interact with the organization in the use of the
solution.

• Domain Subject Matter Expert

5. • Provides input into how the organization interacts
with the solution and identifies potential limitations.
Stakeholders
• End User
• Uses the solution or a component of the solution.
• Users could be customers or people who work
within the organization.

Solution Evaluation 79
8.4 ASSESS ENTERPRISE LIMITATIONS
• Regulator
• One or many governmental or professional entities
that ensure adherence to laws, regulations, or rules;
may have unique input to the organizational
assessment, as relevant regulations must be
included in the requirements.
• There may be laws and regulations that must be
5. complied with prior to (or as a result of) a planned or
Stakeholders implemented change.

• Sponsor
• Authorizes and ensures funding for a solution
delivery.
• Champions action to resolve problems identified in
the organizational assessment.

Solution Evaluation 80
8.4 ASSESS ENTERPRISE LIMITATIONS
• Enterprise Limitation
• Description of the current limitations of the
enterprise including how the solution performance is
impacting the enterprise.

6.
Outputs

Solution Evaluation 81
8.5 Recommend Actions
to Increase Solution
Value

84
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Purpose
• Understand the factors that create differences between potential value and
actual value, and to recommend a course of action to align them.

• Description
• Focuses on understanding the aggregate of the performed assessments
(i.e. measure, analyze, determine causes of unacceptable solution
performance) and identifying alternatives and actions to improve solution
performance and increase value realization.
• Recommendations generally identify how a solution should be replaced,
retired, or enhanced.
• Consider long-term effects and contributions of the solution to stakeholders.
• May include recommendations to adjust the organization to allow for
maximum solution performance and value realization.

Solution Evaluation 85
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE

Solution Evaluation 86
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Enterprise Limitation
• Description of the current limitations of the
enterprise including how the solution performance is
impacting the enterprise.

• Solution Limitation
• Description of the current limitations of the solution
1. including constraints and defect.
Inputs

Solution Evaluation 87
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Adjust Solution Performance Measures
• Analysis effort to identify and define more
appropriate measures may be required.
• Applicable for cases where the performance of a
solution is considered acceptable but may not
support the fulfillment of business goals and
objectives.
2.
• Recommendations
Elements
• Recommendations often describe ways to increase
solution performance, this is not always the case.
• Depending on the reason for lower than expected
performance, it may be reasonable to take no action,
adjust factors that are external to the solution, or
reset expectations for the solution.

Solution Evaluation 88
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Recommendations
• Examples of recommendations that a business
analyst may make:
• Do Nothing: usually recommended when the value of a
change is low relative to the effort required to make the
change, or when the risks of change significantly
outweigh the risks of remaining in the current state. It
may also be impossible to make a change with the
2. resources available or in the allotted time frame
Elements

Solution Evaluation 89
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Recommendations
• Additional examples of recommendations that a business
analyst may make:
• Organizational Change: a process and set of tools for
managing attitudes about, perceptions of, and participation in
the change related to the solution. Possible recommendations
that relate to organizational change include:
• Automating or simplifying the work people perform. Relatively simple
2. tasks are prime candidates for automation. Additionally, work
Elements activities and business rules can be reviewed and analyzed to
determine opportunities for re-engineering, changes in
responsibilities, and outsourcing.
• Improving access to information. Change may provide greater
amounts of information and better quality of information to staff and
decision makers.

Solution Evaluation 90
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Recommendations
• Additional examples of recommendations that a
business analyst may make:
• Reduce Complexity of Interfaces: interfaces are
needed whenever work is transferred between systems
or between people to improve understanding
• Eliminate Redundancy: different stakeholder groups
may have common needs that can be met with a single
2. solution, reducing the cost of implementation
Elements • Avoid Waste: the aim of avoiding waste is to completely
remove those activities that do not add value and
minimize those activities that do not contribute to the final
product directly

Solution Evaluation 91
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Recommendations
• Additional examples of recommendations that a
business analyst may make:
• Identify Additional Capabilities: solution options may offer
capabilities to the organization above and beyond those
identified in the requirements. In many cases, these
capabilities are not of immediate value to the organization but
have the potential to provide future value, as the solution may
2. support the rapid development or implementation of those
Elements capabilities if they are required (for example, a software
application may have features that the organization
anticipates using in the future)

Solution Evaluation 92
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Recommendations
• Additional examples of recommendations that a
business analyst may make:
• Retire the Solution: consider to replace a solution or
solution component: technology has reached the end of its
life, services are being insourced or outsourced, or the
solution is not fulfilling the goals for which it was created.
Some additional factors that may impact the decision
2. regarding the replacement or retirement of a solution include:
Elements • Ongoing cost versus initial investment: it is common for the
existing solution to have increasing costs over time, while
alternatives have a higher investment cost upfront but lower
maintenance costs.
• Opportunity cost: represents the potential value that could be
realized by pursuing alternative courses of action

Solution Evaluation 93
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Recommendations
• Additional examples of recommendations that a business
analyst may make:
• Retire the Solution: Some additional factors that may impact
the decision regarding the replacement or retirement of a
solution include:
• Necessity: most solution components have a limited lifespan (due to
obsolescence, changing market conditions, and other causes). After
2. a certain point in the life cycle it will become impractical or impossible
to maintain the existing component
Elements
• Sunk cost: describes the money and effort already committed to an
initiative. This investment cannot be recovered, it is effectively
irrelevant when considering future action. Decisions should be based
on the future investment required and the future benefits that can be
gained.

Solution Evaluation 94
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Business Objectives
• Considered in evaluating, measuring, and determining
solution performance.

• Current State Description

• Provides the context within which the work needs to be
completed.
3. • Can be used to assess:
Guidelines • Alternatives
and Tools • Better understand the potential increased value that could
be delivered
• Help highlight unintended consequences of alternatives that
may otherwise remain undetected

Solution Evaluation 95
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Solution Scope
• Solution boundaries to measure and evaluate.

3.
Guidelines
and Tools

Solution Evaluation 96
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Data Mining
• Used to generate predictive estimates of solution
performance.

• Decision Analysis
• Used to determine the impact of acting on any of the
potential value or performance issues.
4.
Techniques • Financial Analysis
• Used to assess the potential costs and benefits of a
change.

Solution Evaluation 97
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Focus Groups
• Used to determine if solution performance measures
need to be adjusted and used to identify potential
opportunities to improve performance.

• Organizational Modelling
• Used to demonstrate potential change within the
4. organization's structure.
Techniques
• Prioritization
• Used to identify relative value of different actions to
improve solution performance.

Solution Evaluation 98
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Process Analysis
• Used to identify opportunities within related processes.

• Risk Analysis and Management

• Used to evaluate different outcomes under specific
conditions.

4. • Survey or Questionnaire
Techniques • Used to gather feedback from a wide variety of
stakeholders to determine:
• If value has been met or exceeded
• If the metrics are still valid or relevant in the current context
• What actions might be taken to improve the solution

Solution Evaluation 99
8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Customer
• Directly purchasing or consuming the solution.
• May interact with the organization in the use of the
solution.

• Domain Subject Matter Expert

5. • Provides input into how to change the solution
and/or the organization in order to increase value.
Stakeholders
• End User
• Uses the solution or who are a component of the
solution.
• Users could be customers or people who work
within the organization.

Solution Evaluation 100

8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Regulator
• One or many governmental or professional entities
that ensure adherence to laws, regulations, or rules.
Relevant regulations must be included in the
requirements.

• Sponsor
5. • Authorizes and ensures funding for implementation
Stakeholders of any recommended actions.

Solution Evaluation 101

8.5 RECOMMEND ACTIONS TO INCREASE
SOLUTION VALUE
• Recommended Actions
• Recommendation of what should be done to improve
the value of the solution within the enterprise.

6.
Outputs

Solution Evaluation 102

HOMEWORK

• Think of performance measures used at your job, how

do they align to the business objectives of the
organization? If a measure does not align to the
business objectives, how effective is the measure?

• How do you make sure that measures used are

aligned with the objectives?

• How does your company collect qualitative

measures? E.g., customer satisfaction, quality of
service… Please give an example.

103