= AMERICAN PSYCHOLOGICAL ASSOCIATION Psychological Science Agenda (/science/about/psa/ | June 2016 (/science/about/psa/2016/06/) SCIENCE BRIEF A powerful way to improve learning and memory Practicing retrieval enhances long-term, meaningful learning, By Jeffrey D. Karpicke, PhD (https:/ww.apa.org/search?query=&fa=ContributorFilt%22Karpicke, Jeffrey D22&sort-ContentDateSort desc) Jeffrey Karpicke, PhD, is the James V. Bradley associate professor of psychological sciences at Purdue University. He received a BA in psychology from Indiana University and a PhD in psychology from Washington University in St. Louis. Karpicke's research sits at the interface between cognitive science and education, with the goal of identifying effective strategies that promote long-term learning and comprehension. Karpicke's research has been funded by the National Science Foundation and the Institute of Education Sciences at the U.S. Department of Education. Karpicke was the recipient of the Janet Taylor Spence Award for Transformative Early Career Contributions from Association for Psychological Science, a National Science Foundation CAREER award, and the Presidential Early Career Award for Scientists and Engineers. Author website http://learninglab.psych.purdue.edu/) . The movie "Inside Out" (2015) takes us inside the mind of its young protagonist, an 11- year-old girl named Riley, and depicts memory in a way that is sure to resonate with many people. In Riley's mind, her memories are objects — globes colored with emotions — that are stored in a mental space, just as physical objects are stored in a physical space. When Riley experiences an event and creates a new memory, a new globe is produced in her mind, rolling down a ramp like a ball returning in a bowling alley. When Riley re-experiences a past event, a globe is placed in a projector and events arereplayed, projected on a screen in her mind. Cognitive psychologists refer to the mental processes involved in the creation of new memories and the recovery of past memories as encoding and retrieval, respectively. The depiction of the mind in "Inside Out" follows centuries of thought on how mind and memory work. Throughout history, scholars have used a common metaphor to talk about memory: The mind is a vast storehouse or space; memories are objects stored in that space; and retrieving a memory is akin to searching for and finding an object in a physical space (Roediger, 1980). To learn something new, according to this view, the challenge lies in getting knowledge "in" one's mental space. Getting it back "out" when needed is important, too, but learning is usually identified with the encoding of new knowledge in memory. Retrieval is assumed to be neutral for learning; retrieval is needed to assess what a person has learned, but retrieval processes themselves are not thought to produce learning. Recent advances in the science of learning and memory have challenged common assumptions about how learning happens. Specifically, recent work has shown that retrieval is critical for robust, durable, long-term learning. Every time a memory is retrieved, that memory becomes more accessible in the future. Retrieval also helps people create coherent and integrated mental representations of complex concepts, the kind of deep learning necessary to solve new problems and draw new inferences. Perhaps most surprisingly, practicing retrieval has been shown to produce more learning than engaging in other effective encoding techniques (Karpicke & Blunt, 2011). This approach, which recognizes the central role of retrieval processes in learning and aims to develop new learning strategies based on retrieval practice, is referred to as retrieval- based learning. Retrieval Creates Learning Research dating back a century has shown that retrieval contributes to learning (for a historical review, see Roediger & Karpicke, 2006a), but the past decade has seen a renewed, intense focus on exploring the benefits of retrieval for learning. This recent research has established that repeated retrieval enhances learning with a wide range of materials, in a variety of settings and contexts, and with learners ranging from preschoolages into later adulthood (Balota, Duchek, Sergent-Marshall & Roediger, 2006; Fritz, Morris, Nolan & Singleton, 2007). Aword-learing experiment illustrates some key points about retrieval-based learning. In the experiment (Karpicke & Bauernschmidt, 2011), students learned a list of foreign language words (e.g., Swahili vocabulary words like "mashua — boat") across cycles of study and recall trials. In study trials, the students saw a vocabulary word and its translation on the computer screen, and in recall trials, they saw a vocabulary word and had to recall and type its translation. The students studied a list of vocabulary words, then attempted to retrieve the whole list, studied it again, retrieved it again, and so on across alternating study and retrieval practice blocks, There were several different conditions in the experiment. In one condition, students simply studied the words once, without trying to recall them at all. In a second condition, students continued studying and recalling the words until they had recalled all of them once. After a word was successfully retrieved once, it was "dropped! from further practice — the students did not see it again in the learning session Other conditions in the experiment examined the effects of repeated retrieval practice. Once a word was recalled, the computer program required the students to practice retrieving the items three more times. One repeated retrieval condition had the three recall trials happen immediately, three times in a row. This condition, referred to as massed retrieval practice, is akin to repeating a new piece of information over and over in your head right after you experience it. Finally, in the last condition highlighted here, the students also practiced retrieving the words three times, but the repeated retrievals were spaced throughout the learning session. For instance, once a student correctly recalled the translation for mashua, the program moved on to other vocabulary words, but prompts to practice retrieval of the translation for mashua would pop up later on in the program. In this way, the retrieval opportunities were spaced throughout the learning session. The key question in this research was, how well would students remember the vocabulary word translations in the long term? Figure 1 shows the proportion of translations that students remembered one week after the initial learning session Merely studying the words once without ever recalling them produced extremely poorperformance (average recall was 1 percent, barely visible on the figure). Practicing until each translation was recalled once was much better. But what about the effects of repeated retrieval practice? Massed retrieval — repeating the translations three times immediately — produced no additional gain in learning. Repeated retrieval enhanced learning only when the repetitions were spaced, and indeed, the effects of repeated spaced retrieval were very large. In a single experiment, simple changes that incorporated spaced retrieval practice took performance from nearly total forgetting to extremely good retention (about 80 percent correct) one week after an initial learning experience (see also Karpicke & Roediger, 2008; Pyc & Rawson, 2010). a6. as. Proportion Final Recall a 0. Study Recall Repeated Repeated Once Once. Massed ‘Spaced Figure 1. Data from Karpicke & Bauernschmidt (2011). Retrieval Practice is Underappreciated as a Learning Strategy If retrieval practice is such a potent learning strategy, one would hope that many learners would practice retrieval to learn many different things in many situations. However, as noted earlier, retrieval is not typically considered an important part of the learning process, and unfortunately, many learners do not practice retrieval as often or as effectively as they could. An emphasis on getting knowledge in memory shows up on surveys of students’ learning strategies. In one survey (Karpicke, Butler & Roediger, 2009), college students were asked to list the strategies they use while studying and to rank-order thestrategies. The results, shown in Figure 2, indicate that students' most frequent study strategy, by far, is repetitive reading of notes or textbooks. Active retrieval practice lagged far behind repetitive reading and other strategies (for a review of several learning strategies, see Dunlosky, Rawson, Marsh, Nathan & Willingham, 2013). A wealth of research has shown that passive repetitive reading produces little or no benefit for learning (Callender & McDaniel, 2009). Yet not only was repetitive reading the most frequently listed strategy, it was also the strategy most often listed as students’ number one choice, by a large margin. ie = LUMLEE LELLEE & oe Figure 2. Student study strategy usage. Survey data from Karpicke, Butler, & Roediger (2009), Why don't learners use repeated retrieval practice more frequently? Many students view retrieval as a "knowledge check"; they test themselves to see if they know something, rather than out of the belief that practicing retrieval itself will help them learn. This means that many students will use a "one-and-done" strategy: If they can recall something once, they believe they have learned it, so they remove it from further practice. Many students study this way when they regulate their own learning (Karpicke, 2009), even though their long-term learning will not benefit from repeated retrieval practice. Instead, a one-and-done strategy will produce long-term performance similar to the recall-once condition in Figure 1. There is some evidence that instructing learners about the benefits of retrieval leads students to report using retrieval practice morefrequently when they study on their own (Einstein, Mullet & Harrison, 2012), but the best ways to influence students to practice retrieval remain to be discovered. Practicing Retrieval Promotes Meaningful Learning Perhaps another reason retrieval practice is not used more widely is because repeated retrieval may seem like "rote learning.” Rote learning — simple memorization based on repetition — is short-lived, poorly organized and does not support the ability to transfer knowledge, make inferences or solve new problems. The outcome of rote learning is obviously not what students and educators aim for. Meaningful learning is essentially the opposite of rote learning: It is long-lasting and durable, coherent and well organized, and supports transfer, inferencing and problem solving. In fact, the past decade of research on retrieval-based learning has firmly established that retrieval practice promotes meaningful learning. Retrieval-based learning may be a more effective means of achieving meaningful learning than other popular active learning strategies. In one example of this (Karpicke & Blunt, 2011), students studied educational texts about science topics using one of two strategies. In a retrieval practice condition, students read a text, then set it aside and spent time recalling and writing down as much as they could remember from it (Roediger & Karpicke, 2006b). They then reread the text and recalled it a second time. In a second condition, students created concept maps while they read the texts. Concept maps are node-and-link diagrams that require learners to think about the relational and organizational structure of materials (Novak, 2013). The students spent the same amount of time studying in the two conditions; the difference was whether they created concept maps or practiced actively retrieving while learning. Figure 3 shows the results of two different final assessments given one week after the learning session. On one assessment, students answered two types of short-answer questions aimed at measuring meaningful learning: verbatim questions, which assessed concepts stated directly in the texts, and inference questions, which required students to make new connections across concepts. On another assessment, the final assessment involved creating a concept map, because concept mapping is often usedas an assessment of the coherence and integration of students’ knowledge. On the final verbatim and inference questions and on the final concept map assessment, practicing retrieval during learning produced the best performance, even better than studying the material by making concept maps. Final Short Answer Test Final Concept Map Test 3 ¥ é é 3 06 8 03 5 5 be Be Concept etival concept etieval Mop Practice Mop Practice Figure 3. Data adapted from Karpicke & Blunt (2011) (from Figure 2, panels A and C). Several additional studies have established that retrieval practice promotes meaningful learning. Retrieval practice enhances the learning of educationally relevant materials, including educational texts, multimedia presentations, material explained in classroom lectures and a variety of other complex concepts Jensen, McDaniel, Woodard & Kummer, 2014; Johnson & Mayer, 2009; Larsen, Butler, Lawson & Roediger, 2013; Lyle & Crawford, 2011; Roediger, Agarwal, McDaniel & McDermott, 2011). Retrieval practice also supports students in making inferences, solving new problems and transferring knowledge (Butler, 2010; Chan, 2009; Hinze & Wiley, 2011; McDaniel, Howard & Einstein, 2009; Smith & Karpicke, 2014). Retrieval-based learning is an effective method for improving meaningful learning. Creating Retrieval-based Learning Activities Perhaps the best aspect of retrieval-based learning is that it is free. Although there are sophisticated tools that can be used to implement retrieval practice, like classroom clicker systems (Roediger et al., 2011) and other computer-based learning systems(Grimaldi & Karpicke, 2014; Lindsey, Shroyer, Pashler, & Mozer, 2014), retrieval practice does not require special equipment or technology. The essence of retrieval-based learning is taking material you are trying to learn, setting it aside, and spending time actively retrieving the information. Existing educational activities can be converted into retrieval-based learning activities. For instance, answering questions and taking quizzes are effective ways to practice retrieval. In some circumstances, students might answer questions on quizzes or practice worksheets by looking up answers in their notes or books, rather than by attempting to retrieve the answers. One study directly compared this type of open-book questioning to closed-book conditions in which students were required to retrieve the answers to questions rather than looking them up (Agarwal, Karpicke, Kang, Roediger & McDermott, 2008). Answering questions in open-book conditions led to more forgetting over one week than did attempting to retrieve the answers, closed-book and then studying the answers. In other words, closed-book quizzes, which required retrieval practice, were more effective than open-book quizzes, which did not require learners to engage in retrieval. Another study (Blunt & Karpicke, 2014) examined the effectiveness of using concept mapping as a retrieval practice activity. In the experiment, students read texts about science topics and then created concept maps either with or without viewing the texts. In other words, some students created maps while studying the texts whereas other students had to engage in retrieval to create their maps. On a short-answer assessment one week after the learning session, students did better when they had learned by creating concept maps without viewing the texts, as a retrieval practice activity, than by creating maps while studying the texts. Thus, educational activities can be enhanced when they involve retrieval-based learning, Take Home Points About Retrieval-based Learning This article has made the case for four take-home points about retrieval-based learning:= Retrieval is a learning event. Practicing retrieval is a simple and effective way to enhance long-term, meaningful learning. = Some effective learning strategies, like retrieval practice, are underutilized. Conversely, the most popular learning strategy among college students — repetitive reading — leads to very limited levels of learning. = When practicing retrieval, retrieve more than once and space your retrievals, rather than massing them all together. Self-testing as a knowledge check is a good idea, but don't stop at just one successful retrieval (one-and-done). Two or three additional spaced retrievals will bolster long-term learning. = Retrieval can happen in a variety of ways, and many existing activities may be converted into retrieval-based learning activities. The key ingredient is to spend time actively retrieving when trying to learn something new. References Agarwal, P.K., Karpicke, J. D., Kang, S.H.K., Roediger, H.L., & McDermott, K.B. (2008). Examining the testing effect with open- and closed-book tests. Applied Cognitive Psychology, 22(7), 861- 876. doi:10.1002/acp.1391. Balota, D.A., Duchek, J.M., Sergent-Marshall, S.D., & Roediger, H.L. (2006). Does expanded retrieval produce benefits over equal-interval spacing? Explorations of spacing effects in healthy aging and early stage Alzheimer's disease. Psychology and Aging, 21(1), 19-31 doi:10.1037/0882-7974.21.1.19, Blunt, J.R., & Karpicke, J.D. (2014), Learning with retrieval-based concept mapping. Journal of Educational Psychology, 106(3), 849-858. doi:10.1037/a0035934. Butler, A.C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(5), 1118- 1133, doi:10.1037/a0019902, 10.1037/a0019902.supp. Callender, A.A., & McDaniel, M.A. (2009). The limited benefits of rereading educational texts. Contemporary Educational Psychology, 24(1), 30-41. doi:10.1016/j.cedpsych.2008.07.001. Chan, J.C.K. (2009). When does retrieval induce forgetting and when does it induce facilitation? Implications for retrieval inhibition, testing effect, and text processing, Journal of Memory and Language, 61(2), 153-170. doi:10.1016/}iml.2009.04.004.Dunlosky, J., Rawson, K.A., Marsh, €.J., Nathan, M.J., & Willingham, D.T. (2013). Improving students! learning with effective learning techniques: Promising directions from cognitive and ‘educational psychology. Psychological Science in the Public Interest, 14(1), 4-58. doi:10.1177/1529100612453266. Einstein, G.O., Mullet, H.G., & Hartison, T.L. (2012). The testing effect: Illustrating a fundamental concept and changing study strategies. Teaching of Psychology, 39(3), 190-193. doi:10.1177/0098628312450432. Fritz, C.O., Mortis, P.E., Nolan, D., & Singleton, J. (2007). Expanding retrieval practice: An effective aid to preschool children’s learning. The Quarterly Journal of Experimental Psychology, 60(7), 9911004. doi:10.1080/17470210600823595 Grimaldi, P.J., & Karpicke, J.D. (2074). Guided retrieval practice of educational materials using automated scoring. Journal of Educational Psychology, 106(1), 58-68. doi:10.1037/a0033208. Hinze, S.R., & Wiley, J. (2011). Testing the limits of testing effects using completion tests. Memory, 19(8), 290-304. doi:10.1080/09658211.2011.560121 Jensen, J. L., McDaniel, M.A., Woodard, S.M., & Kummer, T.A. (2014). Teaching to the test ... or testing to teach: Exams requiring higher order thinking skills encourage greater conceptual understanding, Educational Psychology Review, 26(2), 307-329. doi:10.1007/s10648-013-9248- 9. Johnson, Cl, & Mayer, RE. (2009). A testing effect with multimedia learning. Journal of Educational Psychology, 1013), 621-629. doi:10.1037/a0015183, Karpicke, J.D. (2009). Metacognitive control and strategy selection: Deciding to practice retrieval during learning. Journal of Experimental Psychology: General, 138(4), 469-486. doi:10.1037/a0017341 Karpicke, J.D., & Bauernschmict, A. (2011). Spaced retrieval: Absolute spacing enhances learning regardless of relative spacing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1250-1257. doi:10.1037/a0023436. Karpicke, J.D., & Blunt, J.R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping, Science, 3316018), 772-775. dois10.1126/science.1199327.Karpicke, J.D., Butler, A.C., & Roediger, H.L. (2009). Metacognitive strategies in student learning: Do students practise retrieval when they study on their own? Memory, 17(4), 471-479. doi:10.1080/09658210802647009, Karpicke, J.D., & Roediger, H.L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966-968. doi:10.1126/science.152408. Larsen, D.P., Butler, A.C., Lawson, ALL, & Roediger, H.L,, Ill. (2013). The importance of seeing the patient: Test-enhanced learning with standardized patients and written tests improves clinical application of knowledge. Advances in Health Sciences Education, 18(3), 409-425. doi:10.1007/s10459-012-9379-7. Lindsey, R.V., Shroyer, J.D., Pashler, H., & Mozer, M.C. (2014). Improving students’ long-term knowledge retention through personalized review. Psychological Science, 25(3), 639-647. doi:10.1177/0956797613504302. Lyle, K.B., & Crawford, N.A. (2011), Retrieving essential material at the end of lectures improves performance on statistics exams. Teaching of Psychology, 38(2), 94-97. doi:10.1177/0098628311401587. McDaniel, M.A., Howard, D.C., & Einstein, G.O. (2009). The read-recite-review study strategy: Effective and portable, Psychological Science, 20(4), 516-522. doi:10.1111/).1467- '9280.2009.02325.x. Novak, J.D. (2013). Concept mapping. In J. Hattie & E.M. Anderman (Eds), International Guide to Student Achievement (pp. 362-365). New York, NY: Routledge/Taylor & Francis Group. Pyc, M.A., & Rawson, K.A. (2010). Why testing improves memory: Mediator effectiveness hypothesis. Science, 330(6002), 335. doi:10.1126/science.1191465, Roediger, H.L. (1980). Memory metaphors in cognitive psychology. Memory & Cognition, 8(3), 231- 246. doi:10.3758/b{03197611 Roediger, H.L., Agarwal, P.K., McDaniel, M.A., & McDermott, K.B. (2011). Test-enhanced learning in the classroom: Long-term improvements from quizzing. Journal of Experimental Psychology: Applied, 17(4), 382-395. doi:10.1037/a0026252. Roediger, H.L., & Karpicke, J.D. (2006a). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181-210.doi:10.1111,.1745-6916,2006.00012.x. Roediger, H.L., & Karpicke, J.D. (2006b). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249-255. doi:10.1111/}.1467- 9280.2006.01693.x. Smith, MA., & Karpicke, J.D. (2014). Retrieval practice with short-answer, multiple-choice, and hybrid tests. Memory, 22(7), 784-802. doi:10.1080/09658211.2013.831454, The views expressed in this article are those of the author and do not reflect the opinions or policies of APA. © The content | just read: A e IS HELPFUL Is NOT HELPFUL, Find this article at: https:/mww.apa.org/science/about/psa/2016/06/learning-memory