Professional Documents
Culture Documents
CMMI V2 0 Performance Summary Results - FINAL
CMMI V2 0 Performance Summary Results - FINAL
CMMI V2 0 Performance Summary Results - FINAL
CMMI V2.0
Performance
Report Summary
How Early-Adopters
Leveraged CMMI V2.0
to Consistently Improve
Their Performance
October 2020
® 2020 ISACA. All Rights Reserved.
Executive Summary
These appraised organizations reported their “before and after” improvement objectives (a total
of 735 objectives across the 95 organizations appraised) in the required CMMI V2.0
Performance Report template with an astounding 70% achievement success rate for their
accomplished improvement objectives and another 20% soon to be achieved for a total of 90%
across key areas such as quality, cost and schedule performance, productivity and more
Each of these results were identified and achieved by the organizations being appraised against
CMMI V2,0, with the resulting performance improvements independently verified by Certified
CMMI Appraisals teams
This report is organized by sections for multi-use purposes with marketing, messaging, social
media and similar communications channels
2
® 2020 ISACA. All Rights Reserved..
Report Roadmap
Report Purpose V2.0 Key Messaging
Context and reason for publishing the Additional context for communicating
first year of CMMI V2.0 performance the initial results and analysis of the
outcomes performance data
What’s Next
Actions for additional information and
near-term plans for the CMMI V2.0
3 ® 2020 ISACA. All Rights Reserved. Performance Data
Purpose
This report will help you and your
clients make a shift and get a better
understanding on how to embrace
the value of CMMI V2.0.
• While it cannot be used alone to determine a maturity or capability level rating, the
Performance Report is used to verify if the appraisal findings and performance
improvement results are consistent with the Performance Report
ü Initial CMMI V2.0 adoption (2019) shows that clear and demonstrable performance
outcomes and results are consistently being achieved or exceeded
ü The CMMI V2.0 integrated product suite has successfully shifted the framework
and product suite from a process compliance model to a business performance
improvement model.
ü Flexible content architecture and online/digital experience allows for fitting content
to customer pain points, customizable views and integration with other
frameworks, such as CMMC, ISO and COBIT.
ü Initial value propositions for V2.0 have all been achieved or exceeded, including a reduced
impact/effort for V2.0 appraisals (up to a reported 50% reduction in some cases) and reduction
in overall adoption life cycle cost
ü The model overall footprint of content was reduced by 64% while still retaining key best
practices and content
ü Flesch-Kincaid-measured model content readability and grade levels were both significantly
improved to aide in easier adoption and understanding
ü Still the best of breed in terms of consistency, fidelity and benchmarking, the CMMI Appraisal
method was also simplified and streamlined; the V2.0 Method Definition Document was reduced
by 46%, streamlining and simplifying content and concepts, removing redundancies, etc.
Cost
Management
Productivity Staff
Development
Organizations that
Organizational
implement CMMI V2.0 Financials
are consistently realizing
tangible performance
improvements in multiple
Customer
aspects and areas of
their business.
Satisfaction
Process
Quality
Adherence
Process Functionality
Cycle Time
Schedule
Management
INTENT Intent
Estimate the size, effort, duration, and cost of the
The Intent statement provides insight into why the Practice work and resources needed to develop, acquire,
Area is important to take into consideration in your or deliver the solution.
process. Value
Estimation provides a basis for making
VALUE commitments, planning, and reducing uncertainty,
The Value statement explains the business performance which allows for early corrective actions and
increases the likelihood of meeting objectives.
improvement impact achieved when the intent of the
Practice Area and individual practices are met.
No other framework or model has built-in
and integrated performance improvement
requirements or provides a clear
13 ® 2020 ISACA. All Rights Reserved..
evolutionary path to performance.
Common Questions
Organizational Size: Most organizations were with 1-100 staff, with the
The following slides (18-31) include
vast majority small businesses (50 FTEs or less)
data from the initial 2019 CMMI V2.0
appraisals reported/registered with
Types of Businesses – Financial, Transportation, Commercial – not just ISACA, verified independently by
Defense and Government Contracting CMMI Certified Appraisal Teams.
Remarkable Results: 70% of reported objectives were met or Based on 95 organizations targeting
exceeded; another 20% have the capabilities to meet in the future. All a total of 735 performance objectives
initially appraised to CMMI V2.0, this
categories of business performance objectives showed significant
is an excerpt from the soon-to-be-
performance improvement results
released report:
“CMMI V2.0 2019 Performance Report
Performance Results in Several Categories: Quality, productivity, Summary”.
organizational financials and schedule improvements were the biggest
categories of improvement
500
of target objectives achieved!
For the 20% not achieved, data shows
400 these were mostly “not yet” but were on
track to be achieved after the appraisal
300
Quality Productivity Cost Management Schedule Management
19.46%
100
5.44% 5.58%
0
Achieved Not Achieved WIP Unknown
200
183
180
Number of Objective Achieved
152
160
140
80% of the targeted objectives showed
significant improvement for:
120
Quality | Productivity | Cost Performance
100 Schedule Performance | Functionality
Process Adherence
80
67 59 56
44
60
41 40 37 36
40
20
20
0
Quality Productivity Cost Schedule Process Customer Staff Process Cycle Org Financials Functionality Unclear
Ma nagement Ma nagement Adherence Satisfaction Development Time
Organizational Financials The bottom line! How well and consistently are revenue and profitability targets being met?
What is the solution, product, or service supposed to do? Did you build, deliver or operate the
Functionality
right thing and did you build, delivery or operate the thing correctly?
Unclear The objectives as written were not clear in terms of tangible targeted and achieved results
60
59 These categories of quality performance objectives are critical
for methods like DevOps, and agile, which can propagate a
defect as rapidly as it does a new version of software!
Number of Quality-related Performance
50
40
30
Objectives Achieved
20
20 20
8 6 4 4
10
3 2 2
0
Delivered Product Work Product Defect Testing Review Rework Defect Resolution Peer Review Service Quality Supplier Quality
Quality Quality Conta inment Effectiveness Effectiveness Ratio Effic iency
Defect Containment
improvement mean of 23%
with an upper range of 30%.
Top 3 Quality Performance Objectives Achieved
Standard Deviation of
Improvement 26.71% 32.88% 18.98%
15
11
10 8
6 5 4 3
5
2 2 2 2 2
0
Development Delivery Effort Rework Utiliza tion Test Review Rate Test Bug fix Rate In-hous e Safety Test Service Rate
Velocity Effic iency Proportion Execution Development Ratio Coverage
Rate Rate
Average of
13%
improvement on
development velocity
with a maximum of
almost 40%
25 ® 2020 ISACA. All Rights Reserved. Quality Improvements Box & Whisker
Productivity Improvements
Category Definitions Description
Development Velocity Amount of product or work product produced per unit effort or time
Effort Proportion The proportion of total time or effort a task or phase consumes
Utilization The proportion of total time or effort that a resource is gainfully employed
Test Execution Rate The number of tests or test cases executed per unit effort or time
Review Rate Amount of product or work product reviewed per unit effort or time
Test Development Rate The number of tests or test cases produced per unit effort or time
In-house Ratio Proportion of work done internally relative to that done by external sources.
20
20 42 of the objectives include various types of cost management
performance improvements, including reduced cost and effort
Performance Objectives Achieved
Effort Variance Difference in planned and actual effort as a percentage of planned effort
Cost Variance Difference in planned and actual costs as a percentage of planned costs
Indirect Costs Costs that are not directly accountable to the project or service work
23
25
Average of
95%
20
What organization
Number of Schedule Performance
10 milestones!
5
5
3 2 2
0
Me eting Milestones Schedule Variance Schedule Performance Process Timing Schedule Status Schedule Delay
Index
ü V1.3 Total Number of Specific Practices – 500 - V2.0 39.2% practice reduction (64.5% including Generic
Practices)
ü Aligns the CMMI content with simpler architectural elements to make it easier to understand, integrate with
other adopted methodologies like ISO, CMMC, etc.
Shifting from engineering “hands-on” and “excellence” Implementation infrastructure and governance practices areas
1 to “insight/oversight.” Lack of ownership. for project performance clarity and transparency
NASA Report: NASA/CR—2012–217471. Elements of Engineering Excellence J.C. Blair, R.S. Ryan, and L.A. Schutzenhofer Huntsville, Alabama
Wipro
Tailored organizational processes and the steps needed to embrace
and institutionalize the improvements to make them sustainable
Improved Critical to Quality metric and business objective performance
Identified root causes and implemented processes to sustain revenues,
retain customers, and increase growth, customer satisfaction, and
operating margins
Strengthened processes through a focus on Governance
Dynanet
92% decrease in the average number of high-priority defects
found within 30 days after delivery of a quarterly software release
52% decrease in the rate of all software defects found from beta
testing phase through warranty period
CeloNova
BioSciences, Inc.
World’s first Capability Level Benchmark appraisal!
27% reduction in time period - significant reduction in
complaint time cycle.
10K+ 12
ORGANIZATIONS NATIONAL GOVERNMENTS
106
COUNTRIES
10LANGUAGES
403
PARTNERS
3,377
APPRAISALS IN 2019
37
40
70%
of appraisals were conducted
35 on organizations with 100 FTEs or less
Full-time Equivalent (FTES) - Staff
17%
30
were organizations with
greater than 500 FTEs.
25
19
16
20
15
10
9
2 2 3 3
1 1 1
5
0
0 - 50 51-100 101-150 151-200 201-250 251-300 301-350 351-400 401-450 451-500 >500
39
Fi
Units per Domain
na
nc
0
5
10
15
20
25
30
ia
lS
er
vic
es
G Tr
24
ov an
er sp
nm or
ta
en tio
t /P n
ub
lic
Se
rv
H
ea
lth
ca
re
13 13 12
D
ef
en
se
11
Ae
ro
sp
ac
e
8
U
tili
t ie
s
7
Ed
R uc
et at
io
ai
l& n
5
C E-
on co
st m
m
Domain Type Reported
ru
ct er
io ce
n
5
&
En
gi
ne
er
En in
e g
CMMI V2.0 Adaptable to Multiple Industries
H
4
ot r gy
el ,O
s,
Re il &
st G
au as
ra
4
nt
s
&
have initially adopted
A broadly diverse set of industries
Le
is
ur
e
3
Li
fe
Sc
ie
nc
es
2
R
ea
lE
Em st
at
V2.0
er e
2
ge
nc
y
Se
rv
ice
s
1
M
ar
in
e
1
2019 V2.0 Appraised Organizations - Country Breakdown
For
over 25 years
CMMI has helped thousands
of globally recognized
companies assess their
organizational capability and
drive higher performance –
many of which are Fortune
500 organizations.
1% 1%
1%
1% 1% 1%
1% 1%
1% India USA
1%
2% China Mexico
2% 26%
2% Germany Vietnam
2%
Spain Argentina
4%
Italy Brazil
5%
Multiple Locations Taiwan
Switzerland UK
22%
Japan Canada
13%
Singapore Saudi Arabia
* According to McKinsey & Company. Source: CMMI Institute, “Assess Your Organizational Capability Study,“ 2015.
Defects
Rework
Cost
Risk
Speed
Quality
Customer Satisfaction
Productivity
43 ® 2020 ISACA. All Rights Reserved. Source: CMMI Institute, “Assess Your Organizational Capability Study,“ 2015.
How Capable is Your Organization?
CMMI V2.0 Performance Data: Case Example #1 CMMI V2.0 Performance Data: Case Example #2
Success Success
Measure Results Measure Results
Factor Factor
More examples of CMMI case studies can be found on the CMMI website.
https://cmmiinstitute.com/products/cmmi/cmmi-v2-
products
https://cmmiinstitute.com/learning/appraisals
https://cmmiinstitute.com/learning/training
CMMI -
-
1
-
2
GOAL
-
MET!
3
-
4
5
DEVELOP ACTION PLAN DEPLOY IMPROVEMENTS
04 Develop, keep updated and 05 Deploy and follow 06
ASSESS CAPABILITY
Assess
follow an improvement plan updated organizational
organizational
to get from the current state processes, and measure
capabilities
to the desired state performance
Current and Legacy programs in Development, Total end-to-end lifecycle best practices for new and legacy
Production and Sustainment programs, including Operational Transition and Sustainment
Derived from: Delorise, Ambrose. “Managing Complex Change.” The Enterprise Group, Ltd.
ü The data is very clear – adopting CMMI V2.0 yields tangible and consistent performance
results across multiple types of industries, organizations and geographies
ü CMMI V2.0’s focus on persistent and habitual performance improvement sets it apart
from any other standard or model
ü Our next major content release in 2021 includes: new Managing Security and Safety
Capability Area with 3 new Practice Areas, covering areas such as cybersecurity and CMMC
context specific information, and a brand-new Practice Area on Enabling Virtual Solution
Delivery Practice area (already released) and certification role that contains best practices on
virtual delivery for remote workforces
• Fundamental focus shift to continual performance improvement from process compliance are still being
recognized/realized by appraisal teams and organizations adopting V2.0.
• Well in advance of the CMMI Appraisal - Conduct Appraisal Phase (2), the Lead Appraiser must have a
detailed and thorough understanding of the critical importance of the nexus between the processes and work
flow, process roles, organizational context, and Practice Area and Practice Intent and Value Statements
• The Performance Report is NOT just an appraisal artifact. It is a planning tool, data collection tool and cross-
checking performance improvement tool to verifying business performance improvement before, during, and
after the appraisal
• While it cannot be used alone to determine a rating, the Performance Report SHOULD be used to verify if the
appraisal findings and performance improvement results are consistent with the Performance Report
1999 Undersecretary of Defense (J. Gansler) Memo – SW-CMM ML3 Required for ACAT 1 programs
Appendix
Thank you to the CMMI community around the world for making this report possible. These are YOUR
accomplishments, and we cannot wait to see what the data shows in 2020 and beyond! This report has been made
possible through the creation and rapid improvements made to the CMMI V2.0 Performance Report template based on
community and adoption feedback data. The new template has proven to be a powerful tool in the CMMI results
arsenal.
This 2019 report represents a new baseline for real-world, global performance results from adopting CMMI V2.0. There
is more to come, including an upcoming series of performance-focused webinars based on the information contained in
YOU
this report, and we welcome your feedback and use of this report and valuable information with your clients and students.
A special thank you to our current illustrious Partner Advisory Board Chairman, Mr. Kieran Doyle, for his invaluable help,
support and keen analysis and reviews for this effort.
Sincerely,
Ron Lear
CMMI Chief Architect, ISACA Director of CMMI Product Development and Services
and on behalf of the entire ISACA staff and CMMI V2.0 development teams and working groups
CMMI Categories and Capability Areas
Process implementation:
That meets the intents of practices of the Practice Area up to and
including the target capability level in the set of processes used for the
work. (Some PAs have fewer than 5 levels.)