SCAMPI Class A Method

Overview

SMCMM Integration, SCAMPI, and SCAMPI Lead Appraiser are service marks of Carnegie Mellon University.
® Capability Maturity Model, Capability Maturity Modeling, Carnegie Mellon, CMM, and CMMI are registered
in the US Patent and Trademark Office by Carnegie Mellon University.
For more information on CMU/SEI trademark use, please visit
http://www.sei.cmu.edu/about/legal-trademarks.html

Sponsored by the U.S. Department of Defense

© 2008 by Carnegie Mellon University
This material is approved for public release. Distribution is limited by the Software Engineering Institute to attendees.

SATT 0208, Module C

SATT 0208, Module C

Module Objectives

Provide a high-level overview (30,000 foot or 9,144 meter level) of

• sequence of activities performed during an appraisal

• terminology used during an appraisal

• roles and responsibilities of appraisal team

© 2008 Carnegie Mellon University 2

 SATT 0208, Module C

Module Topics

> What is a SCAMPI Class A?

Terminology
SCAMPI Class A activities overview
Appraisal roles and responsibilities

© 2008 Carnegie Mellon University 3

SATT 0208, Module C

Standard CMMI Appraisal Method for Process

Improvement (SCAMPI) – Class A
Full comprehensive appraisal method
ARC class A method
Uses CMMI models as reference models

© 2008 Carnegie Mellon University 4

SATT 0208, Module C

SCAMPI Class A Objectives

Provide an accurate picture of the organization’s processes relative to

CMMI:
• Understand the current implemented process.
• Identify process weaknesses (and strengths) in the
organizational unit.
• Determine degree of satisfaction of CMMI Process Area goals
investigated.
• Assign ratings, if requested by appraisal sponsor.

© 2008 Carnegie Mellon University 5

SATT 0208, Module C

Appraisal Principles

Start with a process reference model.

Use a defined appraisal method.
Involve senior management as an appraisal sponsor.
Observe strict confidentiality and non-attribution.
Approach the appraisal collaboratively.*
Focus on action.

* When SCAMPI is used for Supplier Selection or Process Monitoring modes, it may not be
possible to use a collaborative appraisal approach.

© 2008 Carnegie Mellon University 6

 SATT 0208, Module C

Module Topics

What is SCAMPI Class A?

> Terminology
SCAMPI Class A activities overview
Appraisal roles and responsibilities

© 2008 Carnegie Mellon University 7

SATT 0208, Module C

Terms with Special Meaning

Organizational Unit (OU)

Organizational Scope
Objective Evidence
Instantiation
Practice Implementation Indicators
• Direct Artifact
• Indirect Artifact
• Affirmation
Characterization (Practice Implementation)

© 2008 Carnegie Mellon University 8

SATT 0208, Module C

Organizational Unit

That part of an organization that is the subject of an appraisal.

• Deploys one or more processes that have a coherent process
context and operates within a coherent set of business
objectives.
• Typically part of a larger organization.
— In a small organization, the organizational unit may be the
whole organization.

[SCAMPI MDD glossary, v1.2]

© 2008 Carnegie Mellon University 9

SATT 0208, Module C

Organizational Unit (OU) Examples

Specific project or set of (related) projects

OR
A unit within an organization focused on a specific life-cycle phase (or
phases)
• e.g., acquisition, development, maintenance, or support

OR
Part of an organization responsible for all aspects of a particular
product or product set

© 2008 Carnegie Mellon University 10

SATT 0208, Module C

Organizational Scope

The collection of projects and support functions that provides

instantiations of practices used within, and representative of, an
organizational unit.

[SCAMPI MDD glossary, v1.2]

© 2008 Carnegie Mellon University 11

SATT 0208, Module C

Objective Evidence (OE)

Documents or interview results used as indicators of the

implementation or institutionalization of model practices. Sources of
objective evidence can include instruments, presentations, documents,
and interviews.

[SCAMPI MDD glossary, v1.2]

© 2008 Carnegie Mellon University 12

SATT 0208, Module C

Instantiation

The implementation of a model practice used in the appropriate context

within the boundaries of an organizational unit.

[SCAMPI MDD glossary, v1.2]

© 2008 Carnegie Mellon University 13

SATT 0208, Module C

Instantiation Example

Example – Instantiations of the abstract term “automobile”:

Example – A potential alternate instantiation of the abstract term

“automobile”:

© 2008 Carnegie Mellon University 14

SATT 0208, Module C

Instantiation – Use Within CMMI

CMMI practices are abstractions

• Implemented and made real in their application by projects and
support groups.
The context within which the practice is applied “drives” the
implementation.
Instantiation includes:
• Details of implementation

AND
• Context within which the practice is implemented

May be either at organizational unit or project level

© 2008 Carnegie Mellon University 15

SATT 0208, Module C

Sampling and Instantiation in Appraisals

A sample of practice instantiations (within the organizational unit) is

identified.

Organizational Unit Inst Inst

Inst Sample
Inst Inst Inst Inst
Inst Inst Inst

Implementation of practice is appraised for each member of the sample

instantiations.
Extent to which the practice is implemented across the sample is used
to indicate the extent to which the practice is implemented in the
organizational unit.

© 2008 Carnegie Mellon University 16

SATT 0208, Module C

Practice Implementation Indicators (PIIs)

Practice implementation indicators are

“footprints” which are evidence of the conduct
or implementation of a practice.
SCAMPI appraisals use practice
implementation indicators as the focus to
verify practice implementation.
Verifying practice implementation is the review
of Objective Evidence to determine whether a
practice is implemented within a project and/or
organization.

© 2008 Carnegie Mellon University 17

SATT 0208, Module C

Practice Implementation Indicators and the

Appraisal
PIIs include documents as well as information gathered from interviews
with managers and practitioners.
Indicators provide a useful and reliable way of predicting that something
is present or true.
Example: Automobile fuel gauge
Pros: Cons:
• can highly simplify repetitive • can be misleading
and costly operations
• can be wrong
• can be great time savers

© 2008 Carnegie Mellon University 18

SATT 0208, Module C

Practice Implementation Indicator Types

Direct Artifacts
Indirect Artifacts
Affirmations

© 2008 Carnegie Mellon University 19

SATT 0208, Module C

Direct Artifacts

Direct Artifacts are

• the tangible outputs resulting directly from implementation of a
specific or generic practice
• an integral part of verifying practice implementation and may be
explicitly stated or implied by the practice statement or
associated informative material
• the primary intended result of performing a practice
Examples:
• typical work products listed in CMMI practices

• target products of an “establish and maintain” specific practice

• documents, deliverable products, training materials, etc.

© 2008 Carnegie Mellon University 20

SATT 0208, Module C

Special Cases of Direct Artifacts

For some practices, documents are accepted as direct artifacts even if

they are not the primary intended result of performing the practice. For
example:
• CM SP1.2 Establish a configuration management system (this
could be represented by a schematic or a description of the
library system from a CM plan)
• PI SP1.2 Establish the product integration environment (this
could be represented by a schematic, a description from an
engineering plan, or a picture)
• GP 2.5 Train people (this could be represented by training
records showing that specific individuals have completed specific
training)

© 2008 Carnegie Mellon University 21

SATT 0208, Module C

Indirect Artifacts

Indirect artifacts are

• a consequence of performing a specific or generic practice or
that substantiate its implementation, but which are not the
purpose for which the practice is performed
• an indication where the direct artifact came from, who worked to
develop it, or how it is used
Examples:
• meeting minutes
• review results

• status reports
• performance measures

© 2008 Carnegie Mellon University 22

SATT 0208, Module C

Affirmations

Affirmation are
• oral or written statements confirming or supporting
implementation (or lack of implementation) of a specific or
generic practice
• usually provided by the practice implementers or other
stakeholders
Examples:
• face-to-face interview
• video conference or teleconference
• questionnaire responses
• email responses

© 2008 Carnegie Mellon University 23

SATT 0208, Module C

Example – Indicators of Practice

Implementation

PP SP1.1:
Establish a top-level work breakdown structure (WBS) to
estimate the scope of the project.

Direct artifacts: Indirect artifacts:

- top-level WBS - meeting minutes
- task descriptions - team charter
- work package descriptions - WBS development notes

Affirmations:
- “I worked on the WBS team.”
- “We used the WBS to generate the estimates.”

© 2008 Carnegie Mellon University 24

SATT 0208, Module C

Characterizing Practice Implementation

Fully Implemented (FI) • One or more direct artifacts are present and judged to be adequate,
• at least one indirect artifact and/or affirmation exists to confirm the
implementation, and
• no weaknesses are noted.
Largely Implemented (LI) • One or more direct artifacts are present and judged to be adequate,
• at least one indirect artifact and/or affirmation exists to confirm the
implementation, and
• one or more weaknesses are noted.
Partially Implemented (PI) • Direct artifacts absent or judged to be inadequate
• one or more indirect artifacts or affirmations suggest that some
aspects of the practice are implemented, and
• one or more weaknesses noted.
OR
• One or more direct artifacts are present and judged to be adequate,
• no other evidence supports (indirect artifacts, affirmations) supports
the direct artifact(s), and
• one or more weaknesses are noted.

Not Implemented (NI) • Direct artifacts are absent or judged to be inadequate,

• no other evidence (indirect artifacts, affirmations) supports the
practice implementation, and
• one or more weaknesses are noted.
Not Yet (NY) • The project or support group has not yet reached the stage in the
lifecycle to have implemented the practice.
© 2008 Carnegie Mellon University 25
SATT 0208, Module C

Exercise 2 – Recognizing Direct Artifacts

Team Solution

© 2008 Carnegie Mellon University 26

 SATT 0208, Module C

Module Topics

What is SCAMPI Class A?

Terminology
> SCAMPI Class A activities overview
Appraisal roles and responsibilities

© 2008 Carnegie Mellon University 27

SATT 0208, Module C

SCAMPI Class A Phase Structure

Phase I - Plan and Prepare for Appraisal

Phase II – Conduct Appraisal
Phase III – Report Results

© 2008 Carnegie Mellon University 28

SATT 0208, Module C

Plan and Prepare for Appraisal Activities

Develop Appraisal
Appraisal Readiness
On-site
Goals and Review
Objectives

Plan Plan for

Train Team Process PIIs
Appraisal On-site

What mechanism to gather

Objective Evidence
information needed?
Review
1. Interviews
Weaknesses
Organizational Appraisal 2. New Documents
Information Needed
Unit Develops Participants 3. Demos (optional)
PIIs / Gathers Briefing 4. Presentations (optional)
Objective (In Pre On-site
Evidence or On-site)

© 2008 Carnegie Mellon University 29

SATT 0208, Module C

Conduct Appraisal Activities

Hold
Opening
Briefing

Confirming Develop and Characterize Prepare &

Present Organizational Consolidate Present
Practice Preliminary Unit and Rate Final
Implementation Findings Implementation Findings

Conduct
Executive
Session
(Optional)

Wrap-Up

Note: The Report Results phase is included in this graphic.

© 2008 Carnegie Mellon University 30
 SATT 0208, Module C

Module Topics

What is SCAMPI Class A?

Terminology
SCAMPI Class A activities overview
> Appraisal roles and responsibilities

© 2008 Carnegie Mellon University 31

SATT 0208, Module C

Appraisal Participants - Roles

Appraisal sponsor - Sponsors appraisal

- Owns appraisal results
- Signs ADS

Middle managers - From line or staff management positions

- Interviewee and data provider
- *Review preliminary findings

Project leaders - Leadership responsibilities for a project

- Interviewee and data provider
- *Review preliminary findings

Functional Area - Practitioner

Representatives - Interviewee and data provider
(FARs) - Review preliminary findings

* If participant
© 2008 Carnegie Mellon University 32
SATT 0208, Module C

Appraisal Team – Key Roles

Team Leader - Overall responsibility for appraisal

- Authorized by the SEI to lead appraisal and
has appropriate experience and required
training
- Signs Final Findings and ADS

Team members - Satisfy requirements for team members

relative to experience and training
- Assume one or more specific roles
- Signs Final Findings

© 2008 Carnegie Mellon University 33

SATT 0208, Module C
Team Leader Responsibilities
Overall responsibility for the
appraisal
Ensures planning activities are
complete
Assigns team roles
Ensures SCAMPI appraisal
process followed
Monitors schedule and
performance
Facilitates team resolution of
conflicts and impasses
Interfaces with sponsor
Reports results to CMMI Steward
(SEI)
© 2008 Carnegie Mellon University 34
SATT 0208, Module C

Team Member Responsibilities

Site coordinator - Handles on-site logistics; may need more than

one for a multi-site appraisal (might not be on
team)
Librarian - Manages appraisal documents and returns
documents at end of appraisal
Timekeeper - Tracks team’s time and schedule constraints
during interviews and other activities (might
have one per mini-team)
Note takers - Take notes during data gathering sessions for
all PAs
Appraisal team - Reviews all the work
Mini-teams - (see next slides)

© 2008 Carnegie Mellon University 35

SATT 0208, Module C

Why Mini-teams?

Mini-teams are typically formed to

• Balance CMMI experience level of mini-team members.
• Balance workload of mini-teams.
• Mini-team composition may be changed by team leader to improve
performance.
• Enable the team to work in parallel in many time-consuming tasks
May be organized around
• Process areas (PAs)
• Projects
• Generic vs. specific practices
• PA categories
• Maturity levels
• Others . . .

© 2008 Carnegie Mellon University 36

SATT 0208, Module C

Mini-Team Responsibilities

• Verify implementation of CMMI practices by reviewing objective

evidence provided.
• Identify weaknesses in implementation and characterize extent of
implementation of practices at instantiation levels.
• Request additional information as needed.

© 2008 Carnegie Mellon University 37

SATT 0208, Module C

Example Mini-Team PA Assignments

Mini- Process Area

team Categories
Process Area Assignments
A Project 5 process areas
Management
(PP, PMC, SAM, IPM, RSKM)
B Engineering 6 process areas
(REQM, RD, TS, PI, VER, VAL)
C Process 7 process areas
Management
(OPF, OPD, OT, MA, PPQA, CM, DAR)
and Support

Mini-teams allow the appraisal team to work more efficiently by

performing many tasks in parallel.

© 2008 Carnegie Mellon University 38

SATT 0208, Module C

Team Responsibilities

Entire team acts as a review group for mini-team work.

• supplements data gathering

• approves data consolidation

• agrees on wording of preliminary and final findings

• achieves consensus on organizational unit characterizations
• achieves consensus on ratings

© 2008 Carnegie Mellon University 39

SATT 0208, Module C

Summary

You have just seen SCAMPI from the 30,000 feet or 9,144 meter level.
Remainder of course provides the detailed view.

© 2008 Carnegie Mellon University 40