Pete Cotterill ASQ CQIA Certification Chair

ASQ Certification Process

An Overview of the ASQ Certification Program: From Concept; through Development; to Execution.

What is Certification?

Certification is formal recognition by ASQ that an individual has demonstrated a proficiency within, and a comprehension of, a specified body of knowledge. It is peer recognition, not registration or licensure.
- Source: ASQ Website

ASQ Certifications: Some are lifetime while others require renewal every three years.
Quality Engineer CQE Reliability Engineer CRE Six Sigma Black Belt SSBB Manager of Quality/Organizational Excellence CMQ/QE Biomedical Auditor CBA Certified HACCP Auditor CHA Calibration Technician CCT Quality Auditor CQA Software Quality Engineer CSQE

Six Sigma Green Belt SSGB Quality Process Analyst CQPA Quality Technician CQT Quality Inspector CQI Quality Improvement Associate CQIA

Life-time

Qualification to Sit for Exams Education and Work Experience

CMQ/QE: Ten years of Education and/or Work Experience (5 years of Work Exp must be in a decision making role) CRE/CQE/CSQE/CQA: Eight years of Education and/or Work Experience (3 years of W.E. must be in a decision making role) CBA/CHA: Five years of Education and/or Work Experience (1 year of Work Exp must be in a decision making role) SSBB: 2 completed projects or one completed project and 3 years Work Exp

SSGB: Three years of Work Exp CQI : Two years of Work Exp and a degree or 5 Years Work Exp CCT: Five years of Education and/or Work Exp CQT: Four years of Education and/or Work Exp CQIA/CQPA: Two years Work Exp or a degree

How Are Tests Developed?

STEPS IN TEST DEVELOPMENT

New Cert. approved by ASQ BOD 5-year Review of Existing Program

Job Analysis Workshop

Job Analysis Survey

Test Specifications Workshop

Item-Writing Workshop

No

Enough Items in Pool?

Yes

Item-Review Workshop
IPM Workshop

Item Pool Maintenance Needed?

No

Yes

Review Camera-Ready Version Administer Exam

Exam-Review Workshop
New BOK?

Yes

Cut-Score Study

Item Analysis & Scoring Changes

No

Score Exams and Notify Candidates

STEPS IN TEST DEVELOPMENT

New Cert. approved by ASQ BOD 5-year Review of Existing Program

Job Analysis Workshop

Job Analysis Survey

Test Specifications Workshop

Item-Writing Workshop

No

Enough Items in Pool?

Yes
Workshops conducted with between 12 and 20 ASQ volunteers

Item-Review Workshop
IPM Workshop

Item Pool Maintenance Needed?

No

Yes

Review Camera-Ready Version Administer Exam

Exam-Review Workshop
New BOK?

Yes

Cut-Score Study

Item Analysis & Scoring Changes

No

Score Exams and Notify Candidates

What makes a good test?

Tests and the component Questions, need to be: Valid Reliable Fair

ASSESSMENT CONCERNS: VALIDITY

VALIDITY:
Evidence that the test is measuring what it says it will measure. Processes that provide test validity evidence include:
Job Analysis Test Specifications Body of Knowledge (BOK) Test Question Classification Reference Text & Rationale

ASSESSMENT CONCERNS: RELIABILITY

Is the behavior being measured consistently?

Consistency is based on Test Length Consistency in Test Administration Clarity of Questions * Appropriateness of Questions *
* Major Focus of ASQ volunteer Workshops

EQUATERS

Equating =

Fairness to Candidates

Equaters defined: They are a set of 25-30 items selected from a previous administration of the exam and placed in the newly created exam in same position as previously used. This set of items represent a Mini-test in terms of BOK coverage and difficulty.

9. What is the percent yield for a Which of the following items is normally NOT part of inspection planning? distributed process in which the item length specification is 5.750 0.005, is 5.754, and the (A) Characteristics to inspect standard deviation is 0.002 ? (B) Sampling plans (A) 19.15% (C) 30.85% Cost of inspection (B) (D) Calibration status (C) 50.00% (D) 69.15% 10. In preparation for construction of a The extent to which an instrument replicates result when measurements are cause andits effect diagram, it is important to taken repeatedly on the same unit is called (A) charts for each source (A) plot real separate bias (B) only on what makes things go (B) focus precision (C) wrong accuracy (C) record everything people suggest (D) true value (D) validate possible root causes 11. Inspection accuracy increases with (A) reductions in defect rates (B) repeated inspections (C) additional product complexity (D) order of sample selection 12. Which of the following describes tools would the be best for measuring projected surface intersections? Deming method for continuous improvement? (A) Cost Optical (A) of comparator quality analysis (B) Thread micrometer (B) Process map (C) Tree Profilometer (C) diagram (D) Protractor (D) Plan-do-check-act cycle

13. The accuracy of a profilometer is verified with (A) fingernail comparators (B) surface plates (C) calibration patches (D) optical flats

OCTOBER 2003 EXAM

cylindrical part? (A) Median (B) (A) Mid-range Blade (C) Mean (B) Three-point (D) (C) Mode V-anvil (D) Depth

14. The ability of a material to resist What is the recommended minimum number of mechanical penetration a test ofthe limits for a subgroups necessary tois calculate control chart? (A) surface finish (B) (A) hardness 10 (C) tensile strength (B) 15 (D) (C) porosity 25 (D) 35

15. Which of 2003 the following measures is a types of micrometers is Equated to MARCH sufficient statistic for the parameter ? best for detecting external out-of-roundness on a

16. Which of the following tools can only be used to collect attributes data? (A) Dial indicator (B) Ring gage (C) Micrometer (D) Steel scale

Mar 2003 Exam 1. 9999.1254 2. 9999.5412 3. 9999.6578 4. 9999.0324 5. 9999.0154 October 2003 6. 9999.0111 Equated to 7. 9999.4521 March 2003 8. 9999.3256 9. 9999.4514 10. 9999.0547 11. 9999.0978 12. 9999.0877 13. 9999.0364 14. 9999.0254 15. 9999.0574 16. 9999.6574

Oct 2003 Exam 1. 9999.4567 2. 9999.1571 3. 9999.6578 4. 9999.1865 5. 9999.1954 6. 9999.0998 7. 9999.4521 8. 9999.1244 9. 9999.4514 10. 9999.9145 11. 9999.8748 12. 9999.1754 13. 9999.0364 14. 9999.1654 15. 9999.7457 16. 9999.4214

October 2003 Equated to March 2003

Oct 2003 Mar 2003Exam Exam 1. 9999.4567 9999.1254 2. 9999.1571 9999.5412 3. 9999.6578 4. 9999.1865 9999.0324 5. 9999.1954 9999.0154 6. 9999.0998 9999.0111 7. 9999.4521 8. 9999.1244 9999.3256 9. 9999.4514 10. 9999.9145 9999.0547 11. 9999.8748 9999.0978 12. 9999.1754 9999.0877 13. 9999.0364 14. 9999.1654 9999.0254 15. 9999.7457 9999.0574 16. 9999.4214 9999.6574

EQUATERS Continued
100 ITEM TEST WITH 25 EQUATERS

MARCH 2003 ADMIN Total Exam Mean score = 80

OCT 2003 ADMIN Total Exam Mean score = 70

Is there a difference in the tests or a difference in the groups?

Mini Exam (equaters) Mini Exam (equaters)

Mean score = 20

Mean score = 20

EQUATERS
It is VERY important that Equaters not be revised or moved within the exam. Think of it as an experiment in which this is the one variable that can be truly controlled.

ITEM INFORMATION
NUMBER 8000.0554 REV: 001 ANS: A ST: T-tried, A-accept, H-hold, R-raw, D-delete CLASSIFICATION: 1 A 1

TYPE: NG RN MT RO
EXAM: 0/0/00 EDIT:1/8/00 EQUATER: 2 or 1-Equater, 0-non-Equater AUTHOR: GRIFFITH, GARY K. REF: QUALITY HANDBOOK ED. 5 1998 PP. 2.1-2.2

All Items are Coded to the lowest level in the BOK

IV. AUDIT REPORTING (10 questions) A. Review and Finalize Audit Results B. Written Report Format and Content 1. Audit details 2. Compliance 3. System effectiveness 4. Conclusions to be reported 5. Request for corrective actions Items are classified to the lowest sub-topic level, e.g., 4B3 If there is no sub-topic level, a zero is used, e.g., 4A0

Tested Item and its Stats

NUMBER: 8000.0273 REV: 001 ACC: OVLP: 0000 ANS: D ST: T L4X: 2 CLASSIFICATION: 2 C 1 TYPE: EXAM: 12/04/04 EDIT: 01/20/04 AUTHOR: Scholtes, Joiner and Streibel REF: The Team Handbook ed 2 p 4-37

The Test Question (The Stem) a) b) c) d) Distracter Distracter Distracter Key

PERCENT RESPONDING CORRECT BY QUINTILE MATRIX OF RESPONSES BY QUINTILE 12/04/04 [001] A B C D E BLANK 5TH: * 5TH: 5 9 0 90 0 0 4TH: * 4TH: 3 13 2 76 0 0 3RD: * 3RD: 5 20 2 77 0 0 2ND: * 2ND: 22 26 3 42 0 0 1ST: * 1ST: 31 29 10 22 0 3 +----+----+----+----+----+----+----+----+----+----+ PROB: 0.13 0.20 0.03 [0.63] 0.00 0.01 0 10 20 30 40 50 60 70 80 90 100 DSCR: 0.65

Stats on an Easy question

PERCENT RESPONDING CORRECT BY QUINTILE MATRIX OF RESPONSES BY QUINTILE 12/06/03 [000] A B C D E BLANK 5TH: * 5TH: 81 0 0 0 0 0 4TH: * 4TH: 79 1 2 0 0 0 3RD: * 3RD: 72 1 0 1 0 1 2ND: * 2ND: 73 3 3 0 0 0 1ST: * 1ST: 68 5 4 1 0 0 +----+----+----+----+----+----+----+----+----+----+ PROB: [0.94] 0.03 0.02 0.01 0.00 0.00 0 10 20 30 40 50 60 70 80 90 100 DSCR: 0.13

Stats on an Hard question

PERCENT RESPONDING CORRECT BY QUINTILE 06/05/99 [000] 5TH: * 5TH: 4TH: * 4TH: 3RD: * 3RD: 2ND: * 2ND: 1ST: * 1ST: +----+----+----+----+----+----+----+----+----+----+ PROB: 0 10 20 30 40 50 60 70 80 90 100 DSCR:

MATRIX OF RESPONSES BY QUINTILE A B C D E BLANK 79 97 16 218 0 1 99 116 22 104 0 1 123 126 30 118 0 1 118 87 32 100 0 5 103 115 39 110 0 3 0.28 0.29 0.07 [0.35] 0.00 0.01 0.23

Stats on an question with Issues

PERCENT RESPONDING CORRECT BY QUINTILE MATRIX OF RESPONSES BY QUINTILE 12/04/04 [000] A B C D E BLANK 5TH: * 5TH: 46 41 17 0 0 0 4TH: * 4TH: 33 44 16 1 0 0 3RD: * 3RD: 36 41 23 4 0 0 2ND: * 2ND: 41 36 13 3 0 0 1ST: * 1ST: 43 23 22 6 0 1 +----+----+----+----+----+----+----+----+----+----+ PROB: [0.41] 0.38 0.19 0.03 0.00 0.00 0 10 20 30 40 50 60 70 80 90 100 DSCR: -0.02

Questions?

Contact Information Pete Cotterill

petecotterill@hotmail.com