Establish Mechanisms to Ensure

Consistency and Reliability Description

BY: GWYNETH NICOLE A. MARAŇA

In competency-based systems, students advance
upon demonstrated mastery of learning. In order
to do so, those learning objectives must be clearly
articulated and reliably understood by all.
Moderation builds shared understanding of
proficiency, and calibration creates consistency of
grading practices to improve consistency in
credentialing learning. Creating cross-district and
cross-school clarity and consistency reduces
variability in expectations. Systems of
assessments are aligned with appropriate level of
depth of knowledge as defined by the learning
objectives.
 Key Characteristics
• Valid and reliable. Districts and schools have accurate, standards-based
definitions of proficiency. These definitions are transparent and available to
all educators and students. Rubrics, examples of proficient student work and
other tools are used to communicate proficiency.
• Authentic assessment. Systems of assessment are valid and reliable, and
produce data that accurately assesses student mastery of standards.
Assessment is also meaningful and valuable to the learning process by
supporting reflection and guiding further instruction.
• Aligned to learning objectives. Systems of assessment are aligned to
competencies and standards at the appropriate depth of knowledge.
• Assessment literacy.  Teachers are supported in using different types of
assessments and providing productive feedback to students. Teachers build
capacity in assessing building blocks of learning, transferable skills and
performance-based assessments.
• Moderated. Districts and schools have systems and processes to ensure
consistency in the way that proficiency is understood across schools.
• Calibrated. Educators work together to ensure inter-rater reliability of
grading of student work and assessments.
 How Is Establishing Mechanisms to Ensure Consistency and
Reliability Important to Quality?
Traditional education systems demonstrate high degrees of variability: they permit
different understandings of what it means to be proficient between schools (higher-
income communities often have higher expectations than lower-income communities),
between educators (different definitions in every classroom or school) and between
students (different definitions being applied to students, often based on their race,
class and perceived ability). Many factors contribute to this variability, including
educators working in isolation, A-F grading systems based on student behaviors,
assignments and summative tests, biased educator perception and different
expectations for students within and across schools. In these contexts, inequities are
produced. Students are told they are proficient when they are not resulting in
widening learning gaps. Neither students nor educators can access accurate
information about what students know and can do to inform instructional decision-
making. The results are many: each year teachers are challenged by the number of
students with gaps in their knowledge from the previous year. Students without
prerequisite knowledge and no avenue to build it become less engaged and motivation
decreases. Students with high GPAs go off to college only to discover they need
remediation, and parents and communities lose trust in the educational system.
Balanced System of Assessments
Competency-based systems emphasize a balanced approach to assessment that drives
powerful learning that leads toward common outcomes. Elements of a balanced
system of assessment includes: strong emphasis on formative assessment for learning
including productive feedback, multiple opportunities for students to reach
proficiency, multiple measures used to determine proficiency, assessment aligned with
depth of knowledge of learning targets including performance-based assessment and
opportunities for students to pursue personalized strategies to provide evidence of
learning.
Assessment Literacy
Given the critical role assessment plays in the cycle of learning, competency-based
systems invest in building assessment literacy throughout the districts and schools.
Assessment literacy—the knowledge and skills to use the full range of types of
assessment which are developmentally appropriate on behalf of helping students to
learn—becomes a priority after the first stage of implementation. As districts and
schools advance in implementation, attention to the system of and knowledge about
appropriate assessments increases. Professional learning about assessment often
includes attention to formative assessment including the use of learning progressions
to better understand how students are solving problems. Student knowledge around
self-assessment gains in importance. Districts and schools frequently invest in building
the capacity and professional learning around assessment literacy, especially around
performance-based assessment, if they do not yet have it integrated into their
ongoing pre-service and in-service professional learning.
Moderation and Calibration
Two processes are critical for creating the consistency need for a high-quality, equitable
competency-based system: moderation and calibration. Moderation is a process used to
evaluate and improve comparability. The process involves having teachers (or others) work
to develop a common understanding of varying levels of quality of student work. Calibration
describes the process of creating consistent, shared understanding of what proficiency means
for learning targets for specific levels of performance (or grade levels) and requires teachers to
look at student work together. Moderation processes must take place within and across schools,
and even across districts, to ensure that students are all held to high standards. Then, there is a
need to calibrate the grading practices so that teachers can consistently determine proficiency
and identify what students need to learn to reach proficiency. Calibration, like moderation,
builds professional knowledge while also operating as a formal mechanism that ensures
students are advancing upon mastery.

Proficiency-Based Diploma
Proficiency-based diplomas are being developed to create consistency in what students know
and can do upon graduation. Essentially, the graduate profile drives alignment and also the
requirements for graduation. When used as a high leverage policy, the introduction of a
proficiency-based diploma can catalyze districts and schools to become more responsive to
students so that they are fully supported in their learning starting in elementary school.
However, if districts don’t make the necessary adjustments to ensure students are building
mastery for all the critical learning objectives in the younger years, pressure builds at the high
school level about how to respond to students with gaps in their learning within the four years,
so that they can demonstrate mastery of all the graduation competencies.
 Policies and Practices to Look For

•Structures and processes are in place to ensure that the instruction and assessments are fully
aligned with the learning objectives and offer rich and frequent opportunities for students to
perform at the highest possible depth of knowledge.
•Teachers engage in calibration or joint scoring of student work to ensure inter-rater reliability.
•Teacher-generated performance assessments are strengthened by engaging in task validation
protocols.
•States, districts and schools establish moderation processes to ensure that levels of proficiency
and mastery (application of the skills and knowledge) are aligned to state standards and shared
among teachers.
•Professional learning communities seek to create consistency in determining learning. Teachers
provide feedback to their colleagues if they credential students as reaching proficiency when
they haven’t.
•Transparency in the learning cycle and grading provides feedback on student progress and is
designed to recognize and monitor growth with improved consistency and reliability. Students
are able to see examples of proficiency work on the walls of classrooms or in other resources.
•Districts and schools have mechanisms in place for quality assurance to ensure that variation is
not creating situation of lower expectations for some students or students advancing without
the opportunity to fully master skills.
Examples of Red Flag

Teachers are spending substantial time on unpacking

standards and writing rubrics, without looking at student
work to moderate their understanding. 

•Standards-based grading is introduced too early

without the structures for consistency in place.

•Students can tell you who are the “easy” educators and
the “hard” educators in which the hard educators have
expectations for students to master the knowledge and
skills