magazine for managers of change in educationTurning Up the H.E.A.T. on Student Learning Assessing 21st Century Skills in Classrooms Today Christopher Moersch Best practices involving 21st Century Skills have been analyzed and debated in Educational Technology and in other publications; yet, the fidelity of implementa- tion in the overwhelming majority of school systems nationwide has been fragmented at best. This article proposes strategies for assessing 21st Century Skills or HEAT. (Higher-order thinking, Engaged learning Authentic connections, Technology use) as an integral part of the daily curriculum in schools today. Turning up the HEAT. is no longer a cliché, but a driving force to prepare students for the challenges of a digital society and to transition low-periorming schools into high-performing schools. Twenty-first Century Skills have been referenced repeatedly during the past decade in national commis- sion reports (e.g., Partnership for 21st Century Skills; New Commission on the Skills of the American Workforce; SCANS), business/education partnerships (New Technology Foundation), state content and per- formance standards, and school technology plans. The very mention of 21st Century Skills conjures images of students purposefully using technology to collaborate seamlessly with their classmates, conduct information searches, or create sophisticated multimedia products that align with grade level technology standards. One acknowledgment of the need for 21st Century Skills was Jamie McKenzie’s plea for schools to make changes “...a8 if our future depended on it...” in light of the impending Age of Information (McKenzie, 1987). Yet, today, most school districts still relegate 21st Century Skills or digital-age learning to second- tier status, behind priorities such as test scores, school reform initiatives (e.g,, differentiated instruction, rigor, Christopher Moersch is the Executive Director of LoT Connection, Inc., in Carlsbad, California (e-mail: chris@ loticonnection.com), EDUCATIONAL TECHNOLOGY/May-June 2011 and relevance), and teaming structures (e.g., profes- sional learning communities, instructional teams, school community councils) Why have 21st Century Skills been given such a lukewarm reception by public schools nationwide? Part of the hesitancy to promote such skills rests with (1)a lack of clarity about what constitutes 21st Century Skills, (2) the absence of resources for measuring suc- cess, and (3) a dearth of evidence linking effective technology use with student academic achievement as measured by standardized tests 2Ast Century Skills According to the Partnership for 21st Century Skills, the skill-sets needed by today’s digital natives fall into three distinct yet interrelated categories: Learning and Innovation Skills; Information, Media, and Technology Skills; and Life and Career Skills. Learning and Innovation Skills * Creativity and Innovation * Critical Thinking and Problem Solving * Communication and Collaboration Information, Media, and Technology Skills * Information Literacy + Media Literacy * ICT Literacy Life and Career Skills + Flexibility and Adaptability * Initiative and Self-Direction * Social and Cross-Cultural Skills * Productivity and Accountability * Leadership and Responsibility When asked the question, however, about what constitutes 27st Century Skills, the first word mentioned by many K-12 stakeholders, including parents, stu- dents, teachers, administrators, and school board members, is technology. ‘An overemphasis on technology without an equal focus on integrating the other skill-sets (ie., Learning and Innovation Skills, Life and Career Skills) has left the majority of classrooms nationwide with a disjoi ced and over-simplified understanding of 21st Century Skills. The result has manifested itself in classroom learning experiences void of higher-order thinking and real-world connections to the content as well as the isolated use of technology for technology's sake. As Pearlman (2009) states, “How many of us have visited 110-1 classrooms, schools, or colleges, in this country and others, only to see teachers lecturing and students taking notes?” A recent evaluation of approximately 1,500 K-12 lesson plans in a large suburban school system corrob- 43orates this finding. Using the research-based LoTi (formerly Levels of Technology Implementation) Framework (Moersch, 2001, 2002), 85% of these “technology-infused” lesson submissions were docu- mented at a LoTi Level 2. At this level, digital tools and resources are used by students for extension activities, ‘enrichment exercises, or information-gathering assign- ments that generally reinforce lower-level cognitive skill development (e.g., knowledge, comprehension) relating to the content under investigation The issue of isolated technology use without regard to the other critical attributes of 21st Century Skills (eg,, creative thinking, collaborative problem-solving) was further captured in the results of a national self-assessment using the empirically-validated. LoTi Digital-Age Survey. This survey, administered national ly to approximately 75,000 teachers annually, revealed that the majority of classroom teachers plan, imple- ment, and evaluate their classroom learning experi- ences at a LoTi 2 level, where the emphasis is on isolated technology use (e.g., multimedia presenta- tions, blogs, podcasts, digital-responders), coupled with traditional teaching methods and lower-level student cognition tied to the content, The H.E.A.T. Rubric One way to help educators encapsulate the true meaning of 21st Century Skills in the classroom and raise the bar for effective digital learning is focusing on the key components of these skills as they apply to the learner; namely, Higher-order thinking, Engaged learning, Authentic connections, and Technology use, or HEAT, Collectively, H.E.A.T. represents the amount of 21st Century Skills applied by stuclents within any learning environment (see Figure 7). The manner in which classroom teachers plan, implement, and evaluate their lessons determines the amount of student-gener- ated H.E.A.T. in the classroom. A brief explanation of the elements comprising H.E.A.T. follows. Higher-order Thinking Higher-order thinking references the level of student cognition generated by students from the learning experience based on Bloom's taxonomy. For example, in a math classroom, are students operating at the knowledge level (e.g., defining, repeating, showing) about their experiences with ratio and proportion based on student artifacts (e.g., work samples, type Cf questions) or at the synthesis level (e.g., predicting, hypothesizing, designing) with these same math concepts? Engaged Learning Engaged learning represents (1) the amount of complex thinking (e.g., problem-solving, decision- 44 ‘Higher-order Thinking ‘Students take notes only; no questions asked. + Student learning/questioning is at knowledge level + Student learning/questioning is at comprehension level. + Student learning/questioning is at application level + Student learning/questioning is at analysis level. + Student learning/questioning is at syntesis/evaluation levels. Engaged Learning + Students report what they have learned only. + Students report what they have learned only; collaborate with others. + Students given options to solve a problem. + Students given options to solve a problem; collaborate with others. + Students help define the task, the process, and the solution + Students help define the task, the process, and the solution; collaboration extends beyond the classroom. Authentic Connections + The leaming experience is missing or too vague to determine relevance. + The learning experience represents a group of connected activities, but provides no real-world application, +The learning experience provides limited real-world relevance, but does not apply the learning to a real-world situation. + The learning experience provides extensive real-world relevance, but does not apply the learning to a real-world situation. +The learning experience provides real-world relevance: and opportunity for students to apply their learning to a real-world situation + The learning experience is directly relevant to students and involves creating a product that has a purpose beyond the classroom that directly impacts the students, Technology Use No technology use is evident + Technology use is unrelated to the task. + Technology use appears to be an add-on and is not needed for task completion, + Technology use is somewhat connected to task completion involving one or more applications. + Technology use is directly connected to task completion involving one or more applications. + Technology use is directly connected and needed for task completion, and students determine which application(s) would best address their needs. Figure 1. The H.£.A.T. Rubric. making, experimental inquiry, inductive/deductive reasoning) and (2) the degree of self-directed learning, occurring by students. In a science classroom, are students merely reporting back what they learned EDUCATIONAL TECHNOLOGY/May-June 2011about the water cycle to their teacher, or are they collaborating with other students on a self-selected problem about the water cycle (e.g., water runoff and non-source pollution) that requires a resolution? Authentic Connections Authentic connections relate to students applying their learning to real-world situations consistent with one or more 21st Century Skills themes (ie., Global Awareness; Financial, Economic, Business, and Entrepreneurial Literacy; Health Literacy). Civic students in a government classroom participating in a series of isolated learning experiences (eg,, complete chapter review questions in a textbook or online course}, or are they applying thi learning to a real-world situation (e.g,, voter apathy) that extends beyond the classroom? Technology Use Technology use involves the critical use of digital tools and resources to extend or expand the effective- ness and efficiency of student learning. Is technology use viewed as a mere “add-on” to instruction, or is it directly connected to student task completion? If tech- nology use is not elevating the amount of higher-order thinking, engaged learning, and authentic connections by students, then what is its purpose in the classroom beyond satisfying a set of state or district technology standards? Technology use alone does not constitute 21st Century Skills. Measuring H.E.A.T. The H.E.A.T. rubric (Figure 1) provides K-12 educa- tors with a viable roadmap in which to gauge the amount of 21st Century Skills applied to student learning. Its multitude of uses may range from a self-assessment checklist for teachers to the foundation for a classroom walkthrough protocol involving mentors, coaches, and administrators. Provided below are two lesson plan summaries that address the same ‘math content using similar digital tools. Is there a difference in terms of the projected amount of student H.E.A.T. generated from these two math lessons? Math Sample 1 In this lesson, the teacher uses an interactive white board to present a formal lesson on probability, Students observe the teacher explaining the underlying math concepts using the interactive whiteboard. Afterwards, students use their digital responders. to answer specific questions about probability based on the teacher's presentation and modeling. The teacher next assigns an online tutorial for students experienc- ing difficulty with the concept of probability, while other students are assigned one or more enrichment activities designed to gauge their proficiency level with these math concepts. The final assessment for all EDUCATIONAL TECHNOLOGY/May-June 2011 Math Sample | Math Sample 2: Higher-order thinking ‘Student learing/questioning | Student learning/questioning is at comprehension level. | is at synthesis/evaluation levels. Engaged learning Students report what they | Students are given options have leamed only. to solve a problem; collaborate with others, Authentic connections ‘The learning experience | The leaming experience represents a group of provides real-world connected activities, but | relevance and opportunity provides no real-world for students to apply their application leaming to a real-world situation, Technology use Technology use appears to be an add-on and is not needed for task completion, Technology use is directly ‘connected to task ‘completion involving one or more applications, Figure 2. Assessment of two sample math lessons. students is an open-ended problem about probability aligned to the state assessment. Math Sample 2 In this lesson, students are first exposed to a real- world problem involving probability (e.g., the school cafeteria needs to determine how many hamburgers will be purchased by students next Friday). Students collaborate in small groups to brainstorm possible solutions for the cafeteria staif’s dilemma. The teacher next uses the interactive whiteboard to present a small- group mini-lesson on probability to targeted students needing assistance, while other students collaborate with one another using online data supplied by the cafeteria staff that addresses the cafeteria’s current problem. The activity culminates with teams of stu- dents creating probability estimates for the cafeteria staff based on prior monthly data logs showcasing the number of students consuming hamburgers over time. The final assessment is the same open-ended problem about probability aligned to the state assessment. Which sample math lesson was bringing the H.EA.T. to student leaming? A quick assessment of the two sample ‘math lessons reveals the observations shown in Figure 2.The LoTi Organization LoTi Connection, inc. is an educational consulting company specializing in the integration of 21st century skills into K-12 classrooms. Its founder, Chris Moersch, is the author Of the internationally-recognized and research-based LoT| (Levels of Teaching Innovation) Framework, which has been used in over 100 research studies and dissertations ‘worldwide. For the past four years, Dr. Moersch has been the principal investigator of the LoTi Digital-Age Schoo! movement—a school-improvement initiative focused on inoreasing student academic achievement in low-perform- ing schools nationally through the strategic integration of 2ist Century Skills in the core content areas. Based on the H.E.A.T. assessments, which approach, if nurtured over time, would give students the best opportunity to contextualize and apply their under- standing of probability in another real-life situation or even ona high-stakes test, Math Sample 1 or 22 Many state assessments today intentionally pose multi-step questions that require students to apply fundamental content or process understanding to new situations in both reading and mathematics. With all of the conversation nationally about achieving peak performance from students on high-stakes measures in the core content areas (e.g., reading, math, science, social studies), the need to provide students with learning opportunities that challenge them to think critically, apply their learning, and solve complex problems is critical H.E.A.T. and Student Achievement What does the research say about the impact of HEAL. or 21st Century Skills on student academic growth in the classroom? In the current era of high stakes testing, educators and district stakeholders are looking for proven, research-based methods that have demonstrably been shown to impact student achieve- ment ‘A cursory review of major research studies over the past 20 years that have focused primarily on tech- nology use have delivered mixed results in terms of overall impact on student achievement (ETS Policy Information Center, 1998; Kulik, 1994); however, the literature is confirming that technology use in conjunc- tion with the other elements of 21st Century Skills (eg., collaborative learning methods, critical-thinking, and. problem-solving skills) are realizing higher achievement gains than classrooms focusing exclu- sively on technology (e€MINTS National Center, 2009; Papanastasiou, Zemblyas, & Vrasidas, 2003). Similar findings were noted by Roschelle, Pea, Hoadley, Gordin, and Means (2000), that technology can enhance both what and how children learn when used in conjunction with: (1) active engagement, (2) participation in groups, (3) frequent interaction and feedback, and (4) connections to real-world contexts. What is significant about these latter studies is that the elements of H.E.A.T. or 21st Century Skills are now providing a growing foundation of empirical support as to their collective impact on student aca- demic achievement. Yet, more research is needed— especially in the areas of social networking, online collaboration, and Web 2.0 tools—to provide demon- strable proof as to the efficacy of 21st Century Skills on student academic achievement among diverse groups of students. Without a proven track record of raising test scores, the role of 21st Century Skills or H.E.A.T. in this heyday of high-stakes testing will continue to take a backseat to other instructional and non-instructional priorities on campus References Educational Testing Service Policy Information Center. (1998). The relationship between educational technology and student achievement in mathematics (Ist ed.). Princeton, NJ: ETS eMINTS National Center. (2009, November). eMINTS fact sheet: Experience unlimited possbilties for learning. Columbia, MO: eMINTS National Center. Kulik, J. (1994). Meta-analytic studies of findings on ‘computer-based instruction. In E. L. Baker & H. F. O'Neil, It (Fds.), Technology assessment in education and training (pp. 9-33) Hillsdale, Ni: Lawrence Erlbaum Associates. McKenzie, J. (1987). Making change in education: Preparing your schools for the future. Westbury, NY: J. L. Wilkerson, Moersch, C. (2001, November). Next steps: Using LoTi as a research tool. Leaming and Leading with Technology, 2263), 2-27. Moersch, C. (2002). Beyond hardware: Using existing tech- nology t0 promote higher level thinking. Eugene, OR International Society for Technology in Education (ISTE). Papanastasiou, E. C., Zemblyas, M., & Vrasidas, C. (2003) Can computer use hurt science achievement? The USA results from PISA. Journal of Science Education and Technology, 1213), 325-332. Partnership for 21st Century Skills. (2010). Framework for 2ist_ century learning; http://www.p2 |.org/index.php? ‘option=com_content&task=viewsid=254<ernid=120 Peariman, B. (2009, September-October). Making 21st cen~ tury schools: Creating learner-centered. schoolplaces/ workplaces for a new culture of students at work. Educational Technology, 495), 14-19, Roschelle, |, Pea, R., Hoadley, C., Gordin, D., & Means, B (2000), Changing how and what children learn in school ‘with computer-based technologies. The Future of Children, 1002), 76-101 EDUCATIONAL TECHNOLOGY/May-June 2011