CBAP® Exam Preparation Course

Lesson 8 – Solution Evaluation

CBAP ®, CCBA ® and BABOK ® are registered certification marks owned by International Institute of Business Analysis.
 WHAT’S IN IT FOR ME

Measure solution performance

Analyze performance measures

Assess the limitations of the solution

Assess the limitations of an enterprise

Recommend actions to increase solution

value
 INTRODUCTION

The Solution Evaluation Knowledge area

Assesses the performance of an
enterprise solution

Assesses the value delivered by the

Business Analyst
$$ enterprise solution

 ACTION Recommends action for improving

the value
 SOLUTION EVALUATION
BUSINESS ANALYSIS VALUE SPECTRUM

Solution Evaluation
Requirements Analysis
Strategy Analysis and Design Definition

Potential Actual
Value Need Solution Requirements Design Proof of Concept or Pilot release Operating
Value
Scope Prototype or Beta Solution
 SOLUTION EVALUATION
OVERVIEW

Recommend a change to be made either to the solution

or the enterprise in order to realize the potential value of
Change a solution

Need Solution
Evaluate how the solution or solution
component is fulfiling the need Assess the performance of the solution

Elicit information from the Consider the context in determining

stakeholders about solution solution performance measures and
Stakeholder Context
performance and value delivery any limitations within the context

Value
Determine if the solution is delivering the
potential value and examine why value may
not be realized
 SOLUTION EVALUATION
INPUTS, TASKS, AND OUTPUT

1 2 3 4 5
Measure Analyze Assess Assess Recommend
Solution Performance Solution Enterprise Actions to
Performance Measures Limitations Limitations Increase Value
 SOLUTION EVALUATION (contd.)
INPUTS, TASKS, AND OUTPUT

• Business Objectives
• Implemented
solution (External)
I
N
P
U
T
S

4 5
1 2 3
Assess Recommend
Measure Analyze Assess
O Enterprise Actions to
Solution Performance Solution
U Limitations Increase
Performance Measures Limitations
T Solution
P Value
U
T
Solution performance
measure
 SOLUTION EVALUATION (contd.)
INPUTS, TASKS, AND OUTPUT
• Potential value
• Solution
performance
I measures
N
P
U
T
S

1 4 5
2 3
Assess Recommend
Measure Analyze Assess
O Enterprise Actions to
Solution Performance Solution
U Limitations Increase
Performance Measures Limitations
T Solution
P Value
U
T
Solution performance
analysis
 SOLUTION EVALUATION (contd.)
INPUTS, TASKS, AND OUTPUT

• Implemented solution
• Solution performance
I analysis
N
P
U
T
S

1 4 5
2 3
Assess Recommend
Measure Analyze Assess
O Enterprise Actions to
Solution Performance Solution
U Limitations Increase
Performance Measures Limitations
T Solution
P Value
U
T

Solution limitation
 SOLUTION EVALUATION (contd.)
INPUTS, TASKS, AND OUTPUT
• Implemented solution
• Solution performance
analysis
I • Current state
N description
P
U
T
S

1 4 5
2
Recommend
Measure Analyze 3 Assess
O Enterprise Actions to
Solution Performance Assess
U Limitations Increase
Performance Measures Solution
T Solution
Limitations
P Value
U
T

Enterprise limitation
 SOLUTION EVALUATION (contd.)
INPUTS, TASKS, AND OUTPUT

• Solution limitation
• Enterprise limitation
I
N
P
U
T
S

1 5
2
Measure Analyze 3 4 Recommend
Assess O Actions to
Solution Performance Assess
Enterprise U Increase
Performance Measures Solution
Limitations T Solution
Limitations P Value
U
T

Recommend actions
Lesson 8: Solution Evaluation
Topic 8.1: Measure Solution Performance

✓ Purpose
✓ Elements
✓ Guidelines
Defect per and
Unit Tools
✓ Techniques
✓ Stakeholders
 MEASURE SOLUTION PERFORMANCE
PURPOSE

Define performance measures

 Use the data collected to evaluate the effectiveness of a

solution

1
Measure
Solution
Performance Select a measure based on the solution and the context

Assess solution performance through KPIs

 MEASURE SOLUTION PERFORMANCE (contd.)
PURPOSE

• Business Objectives Business objectives provide measureable results that the enterprise wants to
achieve.
• Implemented
solution (External)
I The implemented solution or solution components may be in any form.
N
P
U
T
S

1
Measure
O Solution
U Performance
T
P
U
T
Solution performance
measure
 MEASURE SOLUTION PERFORMANCE
ELEMENTS

1
Measure
Solution
Performance
 ✓
DEFINE VALIDATE COLLECT

Solution performance measures

• Include:
o Business goals
o Objectives
o Processes
• Can be:
o Quantitative – Numerical
or countable
o Qualitative - Subjective
o Both
 MEASURE SOLUTION PERFORMANCE (contd.)
ELEMENTS

1
Measure
Solution
Performance
 ✓
DEFINE VALIDATE COLLECT

Validate performance measures:

• The Solution Performance

measures are validated with
the stakeholders.

• The decision on the measure

to be used resides with the
sponsor.
 MEASURE SOLUTION PERFORMANCE (contd.)
ELEMENTS

1
Measure
Solution
Performance
 ✓
DEFINE VALIDATE COLLECT

To collect performance measures:

• Consider volume or sample

size

• Consider frequency and timing

 MEASURE SOLUTION PERFORMANCE
GUIDELINES AND TOOLS

The business analyst may use the following guidelines and tools.

Help determine value of the

Requirements (Validated) solution

Sets the boundaries to

1 Solution Scope measure and evaluate
Measure
Solution
Performance
Provides expected potential
Future State Description value

 Change Strategy
Used to implement potential
value
 MEASURE SOLUTION PERFORMANCE
TECHNIQUES

There are 13 techniques for measuring solution performance.

 Used to define measures, acceptance levels, and outcome of the solution

1. Acceptance and Evaluation Criteria

1
Measure
Solution
Performance 2. Benchmarking and Market Analysis

3. Non-Functional Requirements Analysis

4. Use Cases and Scenarios

 MEASURE SOLUTION PERFORMANCE (contd.)
TECHNIQUES

There are 13 techniques for measuring solution performance.

Used to define performance measures for a proposed solution

5. Business Cases
1
Measure
Solution
Performance
 MEASURE SOLUTION PERFORMANCE (contd.)
TECHNIQUES

There are 13 techniques for measuring solution performance.

Used to collect large amounts of data about solution performance

6. Data Mining
1
Measure
Solution
Performance
 MEASURE SOLUTION PERFORMANCE (contd.)
TECHNIQUES

There are 13 techniques for measuring solution performance.

Used to help stakeholders take decisions on ways to measure solution

performance

7. Decision Analysis
1
Measure
Solution
Performance
 MEASURE SOLUTION PERFORMANCE (contd.)
TECHNIQUES

There are 13 techniques for measuring solution performance.

Used to gather opinions, feedback, and attitudes toward solution performance

8. Observations
1
Measure
Solution
Performance
9. Surveys and Questionnaires

10. Focus Groups

 MEASURE SOLUTION PERFORMANCE (contd.)
TECHNIQUES

There are 13 techniques for measuring solution performance.

Used to simulate the solution in order to determine solution performance

11. Prototyping
1
Measure
Solution
Performance
 MEASURE SOLUTION PERFORMANCE (contd.)
TECHNIQUES

There are 13 techniques for measuring solution performance.

Used to measure solution performance

12. Metrics and Key Performance Indicators

1
Measure
Solution
Performance
 MEASURE SOLUTION PERFORMANCE (contd.)
TECHNIQUES

There are 13 techniques for measuring solution performance.

Used to evaluate and select performance measures of vendors that must be

included in the solution performance assessment

13. Vendor Assessment

1
Measure
Solution
Performance
 MEASURE SOLUTION PERFORMANCE
STAKEHOLDERS

Project Manager
Domain SME

End User Sponsor

1
Measure
Solution
Performance

Business Analyst
Customer Regulator
 MEASURE SOLUTION PERFORMANCE (contd.)
STAKEHOLDERS

Project Manager
Domain SME

End User Sponsor

Provide
feedback

Business Analyst
Customer Regulator
 MEASURE SOLUTION PERFORMANCE (contd.)
Provides
STAKEHOLDERS
potential
measures

Project Manager
Domain SME

End User Sponsor

Business Analyst
Customer Regulator
 MEASURE SOLUTION PERFORMANCE (contd.)
STAKEHOLDERS
Maintains schedules
and executes
performance tasks

Domain SME Project Manager

End User Sponsor

Business Analyst
Customer Regulator
 MEASURE SOLUTION PERFORMANCE (contd.)
STAKEHOLDERS

Project Manager
Domain SME

Approves
End User Sponsor measures

Business Analyst
Customer Regulator
 MEASURE SOLUTION PERFORMANCE (contd.)
STAKEHOLDERS

Project Manager
Domain SME

End User Sponsor

Prescribes
constraints
Business Analyst and guidelines
Customer Regulator
Lesson 8: Solution Evaluation
Topic 8.2: Analyze Performance Measures

✓ Purpose
✓ Elements
✓ Guidelines and Tools
Defect per Unit
✓ Techniques
✓ Stakeholders
 ANALYZE PERFORMANCE MEASURES
PURPOSE
Analyze
Provide collected
insights into measures to
the derive
performance actionable
of the items
solution Consider
enterprise
goals and
objectives

2
Analyze
Performance
Measures
Consider the
risk
associated
with the
Consider Consider the solution
other stated risk tolerance
targets of the
enterprise
and
stakeholders
 ANALYZE PERFORMANCE MEASURES
PURPOSE
• Potential value
• Solution
performance
I
measures
N
P
U
T
S

2 Potential value can be used as a benchmark against which

Analyze solution performance can be evaluated.
O Performance
U Measures
T
P
U
T
Solution performance
analysis
 ANALYZE PERFORMANCE MEASURES
ELEMENTS

2
Analyze
Performance
    
Measures
SOLUTION PERFORMANCE PERFORMANCE
RISKS TRENDS ACCURACY
Vs VARIANCES
DESIRED VALUE

The business analyst assesses

the performance of the
solution and helps
stakeholders understand its
value.
 ANALYZE PERFORMANCE MEASURES (contd.)
ELEMENTS

2
Analyze
Performance
    
Measures
SOLUTION PERFORMANCE PERFORMANCE
RISKS TRENDS ACCURACY
Vs. VARIANCES
DESIRED VALUE

New risks may be uncovered

while assessing solution
performance and must be
managed.
 ANALYZE PERFORMANCE MEASURES (contd.)
ELEMENTS

2
Analyze
Performance
    
Measures
SOLUTION PERFORMANCE PERFORMANCE
RISKS TRENDS ACCURACY
Vs. VARIANCES
DESIRED VALUE

The data collection period is

an important factor to be
considered when analyzing
performance data.
 ANALYZE PERFORMANCE MEASURES (contd.)
ELEMENTS

2
Analyze
Performance
    
Measures
SOLUTION PERFORMANCE PERFORMANCE
RISKS TRENDS ACCURACY
Vs. VARIANCES
DESIRED VALUE

The results of performance

measures should be
reproducible and repeatable
to ensure accuracy.
 ANALYZE PERFORMANCE MEASURES (contd.)
ELEMENTS

2
Analyze
Performance
    
Measures
SOLUTION PERFORMANCE PERFORMANCE
RISKS TRENDS ACCURACY
Vs. VARIANCES
DESIRED VALUE

Variance is the difference

between expected and actual
performance.
To understand the causes of
variance, root cause analysis
must be performed.
 ANALYZE PERFORMANCE MEASURES
GUIDELINES AND TOOLS

The business analyst may use the following guidelines and tools.

Sets the boundaries to

Solution Scope measure and evaluate

2 Provides expected potential

Analyze Future State Description value
Performance
Measures
Used to implement potential
Change Strategy value

 Risk Analysis Results

Provide overall level of risk
and planned risk responses
 ANALYZE PERFORMANCE MEASURES
TECHNIQUES

There are 9 techniques for analyzing performance measures.

 Used to determine the expected value of a solution

1. Interviews
2
Analyze
Performance
Measures 2. Surveys and Questionnaires
 ANALYZE PERFORMANCE MEASURES (contd.)
TECHNIQUES

There are 9 techniques for analyzing performance measures.

Used to determine risks, trends, and variance

2 3. Benchmarking and Market Analysis

Analyze
Performance
Measures
4. Data Mining
 ANALYZE PERFORMANCE MEASURES (contd.)
TECHNIQUES

There are 9 techniques for analyzing performance measures.

 Used to identify, analyze, and develop a risk response plan

2 5. Risk Analysis and Management

Analyze
Performance
Measures
 ANALYZE PERFORMANCE MEASURES (contd.)
TECHNIQUES

There are 9 techniques for analyzing performance measures.

Used to define acceptable solution performance through acceptance criteria

2 6. Acceptance and Evaluation Criteria

Analyze
Performance
Measures
 ANALYZE PERFORMANCE MEASURES (contd.)
TECHNIQUES

There are 9 techniques for analyzing performance measures.

 Used to determine how well a solution contributes to goals

2 7. Metrics and Key Performance Indicators

Analyze
Performance
Measures
 ANALYZE PERFORMANCE MEASURES (contd.)
TECHNIQUES

There are 9 techniques for analyzing performance measures.

To observe a solution in action

2 8. Observations
Analyze
Performance
Measures
 ANALYZE PERFORMANCE MEASURES (contd.)
TECHNIQUES

There are 9 techniques for analyzing performance measures.

Used to determine causes of performance variance

2 9. Root Cause Analysis

Analyze
Performance
Measures
 ANALYZE PERFORMANCE MEASURES
DATA MINING

Data Mining:
• Is an analytical process of examining large amounts of data from different perspectives.
• Results describe the underlying patterns and relationships in the data.
• Is used to improve decision making by finding insights from data.

Descriptive Enables finding patterns in a set of data

 Diagnostic Shows why a pattern exists in a set of data

DATA MINING

Enables calculating the probability of a

Predictive
particular scenario occurring in the future
 ANALYZE PERFORMANCE MEASURES
DATA MINING – STRENGTHS AND LIMITATIONS

Strengths Limitations


• Requires specialized
DATA MINING Can reveal hidden patterns knowledge
and can provide insights • Applying techniques without
during analysis understanding them may
result in erroneous
judgements during analysis
 ANALYZE PERFORMANCE MEASURES
DATA MINING - PROCESS

 Requirements
Elicitation
Data
Preparation
Data
Analysis
Deployment

DATA MINING
PROCESS
• Data mining is performed for identified
business decisions or in a functional area.

• A decision modeling technique is used as

a top-down data mining strategy.

• A pattern discovery exercise is used as a

bottom-up data mining strategy.
 ANALYZE PERFORMANCE MEASURES (contd.)
DATA MINING - PROCESS

 Requirements
Elicitation
Data
Preparation
Data
Analysis
Deployment

DATA MINING
PROCESS
• Data mining tools work on analytical data
sets.

• Therefore, data sets must contain data of

appropriate types, volume, and quality.
 ANALYZE PERFORMANCE MEASURES (contd.)
DATA MINING - PROCESS

 Requirements
Elicitation
Data
Preparation
Data
Analysis
Deployment

DATA MINING
PROCESS
• A wide variety of statistical measures are
applied and visualization tools are used.

• Examples of tools used:

o Classification and Regression Trees
o C5
o Linear and Logistic Regression
o Neural Networks
o Support sector Machines
o Predictive Scorecards
 ANALYZE PERFORMANCE MEASURES (contd.)
DATA MINING - PROCESS

 Requirements
Elicitation
Data
Preparation
Data
Analysis
Deployment

DATA MINING
PROCESS
• Models must be deployed to be useful.

• Models support human decision-making

processes or automated decision-making
systems.
 ANALYZE PERFORMANCE MEASURES
STAKEHOLDERS

Project Manager

2
Analyze
Performance Domain SME
Measures
Business Analyst

Sponsor
 ANALYZE PERFORMANCE MEASURES (contd.)
STAKEHOLDERS

Project Manager

2
Analyze
Performance Domain SME
Measures Identify risks
Business Analyst and provide
insights into
the data

Sponsor
 ANALYZE PERFORMANCE MEASURES (contd.)
STAKEHOLDERS

Responsible
for risk
management
and may
Project Manager
participate in
risk analysis

2
Analyze
Performance Domain SME
Measures
Business Analyst

Sponsor
Lesson 8: Solution Evaluation
Topic 8.3: Assess Solution Limitations

✓ Purpose
✓ Elements
✓ Guidelines and Tools
Defect per Unit
✓ Techniques
✓ Stakeholders
 ASSESS SOLUTION LIMITATIONS
PURPOSE

• Implemented solution
• Solution performance
I analysis
N
P
U
T The purpose of the task is to determine factors internal to the
S solution that restrict the full realization of its value.

3
O
U
T
Assess
Solution
Limitations
 
P
U
T

Solution limitation
 ASSESS SOLUTION LIMITATIONS
ELEMENTS

3
Assess

Solution INVESTIGATE
ASSESS
Limitations IDENTIFY INTERNAL SOLUTION SOLUTION
IMPACT
COMPONENT DEPENDENCIES PROBLEMS

A solution may have internal

dependencies that limit the
performance of the entire
solution.
 ASSESS SOLUTION LIMITATIONS (contd.)
ELEMENTS

3
Assess

Solution
INVESTIGATE ASSESS
Limitations IDENTIFY INTERNAL SOLUTION IMPACT
SOLUTION
COMPONENT DEPENDENCIES PROBLEMS

Problems must be analyzed to

identify their source.
 ASSESS SOLUTION LIMITATIONS (contd.)
ELEMENTS

3
Assess

Solution
Limitations IDENTIFY INTERNAL SOLUTION INVESTIGATE ASSESS
COMPONENT DEPENDENCIES SOLUTION IMPACT
PROBLEMS

Identified problems must be

assessed to evaluate their
effect on the solution’s
capability to deliver potential
value.
 ASSESS SOLUTION LIMITATIONS
GUIDELINES AND TOOLS

The business analyst may use the following guidelines and tools.

Sets the boundaries to

Solution Scope measure and evaluate

Used to implement potential

3 Change Strategy value
Assess
Solution
Limitations
Provide overall level of risk
Risk Analysis Results and planned risk responses


 ASSESS SOLUTION LIMITATIONS
TECHNIQUES

There are 11 techniques for assessing solution limitations.

Used to see up to what level the solution meets the defined acceptance criteria

3 1. Acceptance and Evaluation Criteria

Assess
Solution
Limitations
 ASSESS SOLUTION LIMITATIONS (contd.)
TECHNIQUES

There are 11 techniques for assessing solution limitations.

 Used to see if other organizations are experiencing similar solution challenges

3 2. Benchmarking and Market Analysis

Assess
Solution
Limitations
 ASSESS SOLUTION LIMITATIONS (contd.)
TECHNIQUES

There are 11 techniques for assessing solution limitations.

 Used to illustrate current business rules and decisions, and the changes required
to achieve the potential value of the change

3 3. Business rules Analysis

Assess
Solution
Limitations

4. Decision Analysis
 ASSESS SOLUTION LIMITATIONS (contd.)
TECHNIQUES

There are 11 techniques for assessing solution limitations.

Used to identify the factors constraining the solution

3 5. Data Mining
Assess
Solution
Limitations
 ASSESS SOLUTION LIMITATIONS (contd.)
TECHNIQUES

There are 11 techniques for assessing solution limitations.

 Help in performing problem analysis

3 6. Interviews
Assess
Solution
Limitations

7. Surveys and Questionnaires

 ASSESS SOLUTION LIMITATIONS (contd.)
TECHNIQUES

There are 11 techniques for assessing solution limitations.

Used to identify underlying causes of the solution problems

3 8. Root Cause Analysis

Assess
Solution
Limitations
 ASSESS SOLUTION LIMITATIONS (contd.)
TECHNIQUES

There are 11 techniques for assessing solution limitations.

 Used to record stakeholder concerns on why the solution is not meeting potential
value

3 9. Item Tracking
Assess
Solution
Limitations
 ASSESS SOLUTION LIMITATIONS (contd.)
TECHNIQUES

There are 11 techniques for assessing solution limitations.

 Used to identify what can be learned from the previous stages of solution
development

3 10. Lessons Learned

Assess
Solution
Limitations
 ASSESS SOLUTION LIMITATIONS (contd.)
TECHNIQUES

There are 11 techniques for assessing solution limitations.

 Used to identify, analyze, and manage the risks related to potential limitations of
the solution.

3 11. Risk Analysis and Management

Assess
Solution
Limitations
 ASSESS SOLUTION LIMITATIONS
STAKEHOLDERS

Regulator
Domain SME

End User Sponsor

3
Assess
Solution
Limitations

Business Analyst
Customer Tester
 ASSESS SOLUTION LIMITATIONS (contd.)
STAKEHOLDERS

Regulator
Domain SME

End Users Sponsor

Review and
provide
feedback

Business Analyst
Customer Tester
 ASSESS SOLUTION LIMITATIONS (contd.)
STAKEHOLDERS
Identifies the potential
limitations to realization
of the solution’s value

Regulator
Domain SME

End Users Sponsor

Business Analyst
Customer Tester
 ASSESS SOLUTION LIMITATIONS (contd.)
STAKEHOLDERS
Is consulted about
the planning and
potential value of a
solution

Regulator
Domain SME

End Users Sponsor

Business Analyst
Customer Tester
 ASSESS SOLUTION LIMITATIONS (contd.)
STAKEHOLDERS

Regulator
Domain SME Approves the
potential value
and the change
End Users Sponsor to the potential
value

Business Analyst
Customer Tester
 ASSESS SOLUTION LIMITATIONS (contd.)
STAKEHOLDERS

Regulator
Domain SME

End Users Sponsor

Identifies
solution
Business Analyst problems during
Customer Tester construction and
implementation
Lesson 8: Solution Evaluation
Topic 8.4: Assess Enterprise Limitations

✓ Purpose
✓ Elements
✓ Guidelines and Tools
Defect per Unit
✓ Techniques
✓ Stakeholders
 ASSESS ENTERPRISE LIMITATIONS
PURPOSE
• Current state description
• Implemented or constructed
solution (external)
I
• Solution performance analysis
N
P
U
T The purpose of the task is to determine how factors external to the
S solution are restricting value realization.
4
Assess

 
Enterprise
O Limitations
U
T
P
U
T

Enterprise limitation
 ASSESS ENTERPRISE LIMITATIONS
ELEMENTS

4
Assess
Enterprise
✓
Limitations ORGANIZATION
ENTERPRISE CULTURE STAKEHOLDER OPERATIONAL
STRUCTURE
ASSESSMENT IMPACT ANALYSIS ASSESSMENT
CHANGES

This element is used to

determine the cultural changes
required to better realize value
from a solution.
 ASSESS ENTERPRISE LIMITATIONS (contd.)
ELEMENTS

4
Assess
Enterprise
✓
Limitations ORGANIZATION
ENTERPRISE CULTURE STAKEHOLDER OPERATIONAL
STRUCTURE
ASSESSMENT IMPACT ANALYSIS ASSESSMENT
CHANGES

This element provides

information on how the
solution will affect a particular
stakeholder group.
 ASSESS ENTERPRISE LIMITATIONS (contd.)
ELEMENTS

4
Assess
Enterprise
✓
Limitations ORGANIZATION
ENTERPRISE CULTURE STAKEHOLDER OPERATIONAL
STRUCTURE
ASSESSMENT IMPACT ANALYSIS ASSESSMENT
CHANGES

This element helps a business

analyst assess how the
organization structure will be
impacted by a solution.
 ASSESS ENTERPRISE LIMITATIONS (contd.)
ELEMENTS

4
Assess
Enterprise
✓
Limitations ORGANIZATION
ENTERPRISE CULTURE STAKEHOLDER OPERATIONAL
STRUCTURE
ASSESSMENT IMPACT ANALYSIS ASSESSMENT
CHANGES

This element helps to

determine if the enterprise will
be able to adapt or effectively
use a solution.
 ASSESS ENTERPRISE LIMITATIONS
GUIDELINES AND TOOLS
The business analyst may use the following guidelines and tools.

Help determine solution

Business Objectives performance

Sets the boundaries to

Solution Scope measure and evaluate

4 Provides expected potential

Future State Description value
Assess
Enterprise
Limitations
Used to implement potential
Change Strategy value

 Risk Analysis Results

Provide overall level of risk
and planned risk responses
 ASSESS ENTERPRISE LIMITATIONS
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

 Help in identifying organizational gaps or stakeholder concerns

1. Brainstorming

4
Assess
Enterprise 2. Interviews
Limitations

3. Surveys or Questionnaires

4. Workshops
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

 Used to understand the organization structure, its culture, and its operations

4 5. Document Analysis
Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

1 2
Used to identify possible opportunities to improve solution performance
5 3
4

4 6. Process Analysis
Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

Used to illustrate current or changed business processes to achieve the potential

value of the solution

4 7. Process Modeling
Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

 Used to assist in making decisions under uncertain conditions

4 8. Decision Analysis
Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

Used to identify the underlying causes of enterprise limitations

4 9. Root Cause Analysis

Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

Used to ensure that issues identified during assessments are tracked and
resolved

4 10. Item Tracking

Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

 Used to identify previous initiatives and enterprise interactions with the solution

4 11. Lessons Learned

Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

Used to identify, analyze, and manage the risks related to potential limitations of
! the enterprise

4 12. Risk Analysis and Management

Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

Used to identify existing enterprise interactions

4 13. Benchmarking and Market Analysis

Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

Used to identify the factors constraining the performance of the solution

4 14. Data Mining

Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

Used to observe the interactions between the enterprise and the solution

4 15. Observations
Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

Used to identify the required changes to the organization’s structure

4 16. Organizational Modeling

Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

Used to determine stakeholder roles and permissions

4 17. Roles and Permissions Matrix

Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS (contd.)
TECHNIQUES

There are 18 techniques for assessing enterprise limitations.

S W Used to demonstrate how a change will help the organization maximize its
O T strengths and minimize its weaknesses

4 18. SWOT Analysis

Assess
Enterprise
Limitations
 ASSESS ENTERPRISE LIMITATIONS
ROLES AND PERMISSIONS MATRIX
Roles

Purchase

Purchase

Accounts

Accounts
Manager

Manager
Roles and

Officer

Officer

Officer
Stores
Permissions Matrix

Create Account
X X
Modify Account
X
Create Organization
Activities X
Create Order
X X
Edit Order
X X
View Order
X X X X X

• Is used to ensure that all the activities are covered by delineating

responsibilities and identifying roles
• Provides procedural checks and balances and data security
 ASSESS ENTERPRISE LIMITATIONS
ROLES AND PERMISSIONS MATRIX - ELEMENTS

To create the Roles and Permissions Matrix, the business analyst needs to perform the following:

Identify Roles Identify Authorities

• Organizational models • Identify the level of

• Job descriptions authority for each role
• User guides • Delegations
• Procedural manuals • Inheritance

Identify Activities

• Functional decomposition
• Process modeling
• RACI Matrix
• CRUD Matrix
 ASSESS ENTERPISE LIMITATIONS
STAKEHOLDERS

Domain SME

End Users Regulator

4
Assess
Enterprise
Limitations
Business Analyst
Customer
Sponsor
 ASSESS ENTERPISE LIMITATIONS (contd.)
STAKEHOLDERS

Domain SME

End Users Regulator

May interact with

the organization
in the use of the Business Analyst
Customer
solution
Sponsor
 ASSESS ENTERPISE LIMITATIONS (contd.)
STAKEHOLDERS

May be
Domain SME
customers or
employees
End Users Regulator

Business Analyst
Customer
Sponsor
 ASSESS ENTERPISE LIMITATIONS (contd.)
STAKEHOLDERS
May provide inputs on
how the organization
interacts with the
solution

Domain SME

End Users Regulator

Business Analyst
Customer
Sponsor
 ASSESS ENTERPISE LIMITATIONS (contd.)
STAKEHOLDERS

Domain SME May have unique

inputs to
organizational
End Users Regulator assessment

Business Analyst
Customer
Sponsor
 ASSESS ENTERPISE LIMITATIONS (contd.)
STAKEHOLDERS

Domain SME

End Users Regulator

Authorizes and
Business Analyst ensures funding
Customer for solution
Sponsor delivery
Lesson 8: Solution Evaluation
Topic 8.5: Recommend Actions to increase Solution Value

✓ Purpose
✓ Elements
✓ Guidelines and tools
Defect per Unit
✓ Techniques
✓ Stakeholders
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE
PURPOSE

• Solution limitations
• Enterprise limitations
I
N
P
The purpose of the task is to:
U
T
• enable identifying factors causing the gap between potential
S
value and actual value.
5 • identify alternatives to improve solution performance and value
Recommend realization.
Actions to
O
Increase
U
Solution
T
P
U
T
Value

 
Recommended actions
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE
ELEMENTS

5
Recommend
Actions to

Increase
Solution PROVIDE FACTORS IMPACT
ADJUST SOLUTION PERFORMANCE
Value MEASURES RECOMMENDATIONS DECISIONS

The business analyst defines

appropriate measures when
the solution performance is not
supporting business goals and
objectives.
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
ELEMENTS

5
Recommend
Actions to

Increase
Solution PROVIDE FACTORS IMPACT
ADJUST SOLUTION PERFORMANCE
Value MEASURES RECOMMENDATIONS DECISIONS

To increase solution performance,

recommendations may include:
• Do nothing
• Make an organizational change
• Reduce complexity of interfaces
• Eliminate redundancy
• Avoid waste
• Identify additional capabilities
• Retire the solution
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
ELEMENTS

5
Recommend
Actions to

Increase
Solution PROVIDE FACTORS IMPACT
ADJUST SOLUTION PERFORMANCE
Value MEASURES RECOMMENDATIONS DECISIONS

Factors that impact the decision:

• Ongoing cost versus initial investment

• Opportunity cost

• Necessity

• Sunk cost
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE
GUIDELINES AND TOOLS

The business analyst may use the following guidelines and tools.

Help determine solution

Business Objectives performance

5
Recommend Sets the boundaries to
Actions to Solution Scope measure and evaluate
Increase
Solution
Value
Provides the context in which
Current State Description work needs to be completed


 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE
TECHNIQUES

There are 9 techniques for recommending actions to increase solution value.

Used to identify the factors constraining the performance of the solution

5
Recommend 1. Data Mining
Actions to
Increase
Solution
Value
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
TECHNIQUES

There are 9 techniques for recommending actions to increase solution value.

Used to make decisions under conditions of uncertainty

5
Recommend 2. Decision Analysis
Actions to
Increase
Solution
Value
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
TECHNIQUES

There are 9 techniques for recommending actions to increase solution value.

1 2
Used to identify possible opportunities to improve performance
5 3
4

5
Recommend 3. Process Analysis
Actions to
Increase
Solution
Value
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
TECHNIQUES

There are 9 techniques for recommending actions to increase solution value.

 Used to assess the potential costs and benefits of a change

5
Recommend 4. Financial Analysis
Actions to
Increase
Solution
Value
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
TECHNIQUES

There are 9 techniques for recommending actions to increase solution value.

Used to gather opinions, feedback, and attitudes to determine if value has been
met or exceeded

5
Recommend 5. Focus Groups
Actions to
Increase
Solution
Value
6. Surveys and Questionnaires
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
TECHNIQUES

There are 9 techniques for recommending actions to increase solution value.

Used to ensure identification of the required changes to the organization

structure

5
Recommend 7. Organizational Modeling
Actions to
Increase
Solution
Value
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
TECHNIQUES

There are 9 techniques for recommending actions to increase solution value.

Used to identify the relative value of different actions to improve solution

performance

5
Recommend 8. Prioritization
Actions to
Increase
Solution
Value
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
TECHNIQUES

There are 9 techniques for recommending actions to increase solution value.

Used to identify, analyze, and manage the risks related to potential limitations of
the solution

5
Recommend 9. Risk Analysis and Management
Actions to
Increase
Solution
Value
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE
STAKEHOLDERS

Domain SME

End Users Regulator

5
Recommend
Actions to
Increase
Solution
Value
Business Analyst
Customer
Sponsor
 RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
STAKEHOLDERS

Domain SME

End Users Regulator

May interact with

the organization
in the use of the Business Analyst
Customer
solution
Sponsor
RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
STAKEHOLDERS

May be
Domain SME
customers or
employees
End Users Regulator

Business Analyst
Customer
Sponsor
RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
STAKEHOLDERS
May provide inputs on
how to change the
solution in order to
increase value

Domain SME

End Users Regulator

Business Analyst
Customer
Sponsor
RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
STAKEHOLDERS

Domain SME May have unique

inputs to
organizational
End Users Regulator assessment

Business Analyst
Customer
Sponsor
RECOMMEND ACTIONS TO INCREASE SOLUTION VALUE (contd.)
STAKEHOLDERS

Domain SME

End Users Regulator

Authorizes and
ensures funding
for
Business Analyst implementation of
Customer
Sponsor recommended
actions
 KEY TAKEAWAYS
Solution evaluation is the process of assessing performance and
value delivered by a solution in use by the enterprise, and to
recommend removal of barriers or constraints that prevent the full
Solution or a solution component may be in any form - prototype realization of the value.
or proof of concept, pilot or beta release, or operational release.

Measure solution performance is to define performance measures

and use the data collected to evaluate effectiveness of a solution.

The collected measures are analyzed to derive meaning and

actionable items.

The business analysts shall determine the factors internal and

external to the solution that restrict the full realization of value.
The business analyst shall recommend appropriate actions to
increase solution value, keeping in mind solution and enterprise
limitations.

It’s not necessary that recommendations are always made for

increasing solution performance. Depending on the reason, it may
be reasonable to take no action, adjust factors external to the
solution, or reset expectations from the solution.
Lesson 8: SOLUTION EVALUATION
CASE STUDY EXERCISE

✓ Defect per Unit

CASE STUDY
OVERVIEW

Quick facts:
✓ Established in 1990
✓ Leading Mutual Funds
business
✓ Has one of the largest
teams of research
analysts
✓ Diversified and sector
specific equity schemes
✓ Services through
distributors
 CASE STUDY
OVERVIEW – CURRENT STATE

Investors need to submit

transaction slips for redemption
and additional purchases.

Current analytics models are

based on limited parameters
and data from limited sources.

Current State
 CASE STUDY
OVERVIEW – FUTURE STATE

Investors are able to submit Fund Managers proactively

request online for redemption manage funds using analytics
and additional purchases. model based on multiple
parameters.

Future State Distributor services to existing

and new investors improves
using CRM capability and digital
marketing tools.
 CASE STUDY
BUSINESS ANALYSIS ACTIVITIES
Activities in Elicitation, Analysis, and Design:

 
Business analyst identifies Requirements are elicited, Implementation team is
solution options, analyzes analyzed, specified, modeled, develops the solution
potential value, and verified, validated, and incrementally to enhance
recommends a solution. allocated to solution investor and distributor
components. services.

Vendors provide analytics

solutions to the fund
managers.
The in-house team integrates the analytics
solution with the existing solution. 
 CASE STUDY
BUSINESS ANALYSIS ACTIVITIES
Activities in Requirements Management and Communication and Solution Evaluation:


The business analyst analyzes
The business analyst prioritizes performance measures and assesses
the requirements and the solution and enterprise limitations.
communicates them iteratively.

The business analyst is also involved

in validating the incremental solution
to ensure value delivery.
 CASE STUDY
EXERCISE
Questions Response
1 In the case study, what is the type of solution Create
approach used? Purchase
A combination of both
None of the above

2 Which of the following should a business analyst Solution limitation

know to recommend actions to increase solution Current state description
value? Enterprise limitation
All of the above

3 Which one of the following is most likely a factor Existing system interfaces
limiting the vendor solution performance? Fund manager skills
Distributor skills
None of the above

4 Which one of the following is not a likely to be Skill and Training Needs
considered while performing operational Tools and Technology that support a solution
assessment in the task ‘Assess Enterprise Policies and Procedures
Limitations’? Organizational Culture
 CASE STUDY
ANSWERS
Questions Answers
1 In the case study, what is the type of solution approach used? A combination of both

2 Which of the following should a business analyst know to All of the above
recommend actions to increase solution value?

3 Which one of the following is most likely a factor limiting the vendor Existing system interfaces
solution performance?

4 Which one of the following is not a likely to be considered while Organizational Culture
performing operational assessment in the task ‘Assess Enterprise
Limitations’?
Quiz
QUIZ
Which one of the following should the business analyst not consider when
collecting performance measures?
1

a. Sample Size

b. Objective

c. Frequency

d. Timing
 QUIZ
Which one of the following should the business analyst not consider when
collecting performance measures?
1

a. Sample Size

b. Objective

c. Frequency

d. Timing

The correct answer is b.

Explanation: The business analyst need not consider the objective when collecting performance measures. The
business analyst shall consider Volume or Sample Size, Frequency, and Timing when collecting performance measures.
QUIZ
What does CRUD stand for?
2

a. Create, Restore, Update, Delete

b. Create, Read, Update, Delete

c. Create, Restore, Upload, Download

d. Customer, Regulator, User, Developer

 QUIZ
What does CRUD stand for?
2

a. Create, Restore, Update, Delete

b. Create, Read, Update, Delete

c. Create, Restore, Upload, Download

d. Customer, Regulator, User, Developer

The correct answer is b.

Explanation: CRUD Stands for Create, Read, Update, Delete

QUIZ
Which one of the following should not be considered while taking decisions regarding
replacing a solution?
3

a. Necessity

b. Sunk Cost

c. Ongoing Cost

d. All of the above

 QUIZ
Which one of the following should not be considered while taking decisions regarding
replacing a solution?
3

a. Necessity

b. Sunk Cost

c. Ongoing Cost

d. All of the above

The correct answer is b.

Explanation: Sunk Cost should not be considered while taking decisions regarding replacing a solution.
Sunk Cost is the money and effort already committed to an initiative, which cannot be recovered. So, the business
analyst should not consider sunk cost when considering future action.
QUIZ
Which one of the following is a common input to the tasks solution limitations and
enterprise limitations?
4

a. Current State Description

b. Future State Description

c. Potential Value

d. Solution Performance Measure

 QUIZ
Which one of the following is a common input to the tasks solution limitations and
enterprise limitations?
4

a. Current State Description

b. Future State Description

c. Potential Value

d. Solution Performance Measure

The correct answer is d.

Explanation: Solution Performance Measure is an input to both the tasks, solution limitations and enterprise
limitations.
QUIZ
Which one of the following is not a type of data mining techniques?
5

a. Prescriptive

b. Descriptive

c. Predictive

d. Diagnostic
 QUIZ
Which one of the following is not a type of data mining techniques?
5

a. Prescriptive

b. Descriptive

c. Predictive

d. Diagnostic

The correct answer is a.

Explanation: Data mining is a general term that covers three types of techniques - Descriptive, Diagnostic, and
Predictive .
This concludes “Solution Evaluation.”

The next lesson is “The Agile Perspective.”

THANK YOU