Profile 39 Metacognitive Awareness Listening Questionnaire (MALQ) (Vandergrift, Goh, Mareschal, & Tafaghodtari, 2006) Profiled by: Christine C.M. Goh, PhD" Novena astute of Education Nanyang Fechelogical Unversity, Singopore Construct ‘The Metacognitive Awareness of Listening Questionnaire (MAIQ) is an instrument for cliciting second language (L2) learners’ metacognitive awareness about listening (Vandergrift, Goh, Mareschal, & Tafaghodtari, 2006), The construct consists of learners’ reported use of strategies in a listening event or task as well as metacognitive knowledge about themselves as L2 listeners, the nature of listening, and listening strategies (Goh, 1997; Vandergrift & Goh, 2012), The MALQ examines four areas of strategy use problem-solving, planning-evaluation, mental translation, and directed attention—as well as learners’ person knowledge. Instrument Type Self-report Description ‘The MALQ consists of 21 items that represent five areas of metacognitive awareness about 12 listening. These comprise four groups of strategies—problem-solving strate gies (making and monitoring inferences), planning and evaluation strategies (preparing for listening and evaluating results of listening efforts), mental translation strategies 1 In memory of Larry Vandergrift (1946-2015), a great scholar and a wonderful colleague. ‘The Sourcebook of Listening Research: Methodology and Measures, itt Edition. Edited by Debra L, Worthington and Graham D. Bodie. © 2018 John Wiley & Sons, Inc. Published 2018 by John Wiley & Sons, Inc.-Metacognitive Awareness Listening Questionnaire (MALQ) (finding equivalents in the learners’ own language), and directed attention strategies (concentrating and staying on task)—and one dimension of metacognitive knowledge labeled person knowledge (perceptions of challenges, anxiety, and self-efficacy). The five factors consist of different numbers of items each: three items (mental translation and person knowledge), four items (directed attention), five items (planning and evalua tion), and six items (problem-solving). All items are listed at the end of this profile and are organized by factor. Participants respond to the items along 6-point Likert scaling, The MALQ was designed to be used in various language instructional settings for three purposes: assessing L2 learners’ metacognitive awareness at any point in time of their language development, tracking their metacognitive development in L2 listening at different points of their language learning process, and guiding learners to reflect on their own strategy use and person knowledge. Researchers also can use the MALQ to identify relations between learners’ metacognitive awareness and their listening com. prehension performance. The MALQ has been translated from English into a number of other languages (c.g., Mandarin, Persian, and Turkish). Administration ‘The MALQ is administered after language learners have completed a listening task. An information cover page, which is provided to participants, states that there are no right or wrong answers and that confidentiality will be assured. Administrators of the ques tionnaire repeat this information and also iterate that the researchers/instructors are only interested in the students’ own honest appraisal of how they have attempted to understand the oral texts. The cover page also includes instructions with a sample response item illustrating how to use the answer scales. The MALQ takes about 15-20 minutes to complete. Scoring After reverse scoring responses to six items (see Scale section), items are averaged within each subscale to generate five scores per participant. Scoring can be done by the researcher, or participants can be given a guided scoring sheet for self-evaluation. Higher scores for the factors of problem solving, planning and evaluation, and directed attention are desirable as they indicate use of facilitative listening strategies. Lower scores for mental translation and person knowledge (after reverse coding) would sug- gest less frequent use of translation, which may take up more processing time, and lower listening anxiety. A more nuanced understanding of the interpretation of the scores for the five factors is offered by Goh and Hu (2014). Development 1.2 strategy researchers in the past three decades have argued that learners’ use of strategies and awareness of other variables in learning can positively influence learning development (eg. Cohen & Macaro, 2007; Wenden, 1998). Attention in L2 listening research also has 431432 | Christine. M.Goh increasingly been directed at learners’ self-reports of their understanding and awareness of listening comprehension processes (Goh, 1997; Mendelsohn & Rubin, 1995). At the same time, several questionnaires have been developed to elicit learners’ self-reports about L2 listening (see Chamot, Barnhardt, El Dinary, & Robbins, 1999; Goh, 2002; Hasan, 2000; Vandergrift, 2005; Vogely, 1995). ‘The MALQ is founded on research from the areas of metacognition, listening comprehension, and self-regulation. After reviewing existing instruments assessing L2 listening and reading comprehension processes, an initial list of 88 items was generated. Further assessment (e.g., content validity, clarity, readability, and redun- dancy) resulted in a preliminary set of 51 items. This draft version of the MALQ was eventually field-tested with a large sample (N=966) in three countries: Canada (n=725), Singapore (n= 193), and the Netherlands (n=48). The collected data were subjected to an exploratory factor analysis (EFA) and a principal component analysis (PCA) to find a parsimonious set of items. An initial 13-factor solution resulted, but for reasons of parsimony and meaningfulness, additional analyses were conducted including principal factor analyses (PAs) with Promax rotation with Kaiser normali zation, resulting in a final five-factor solution. The questionnaire was next examined for the presence of unsatisfactory items due to low factor loadings, complex loadings, and reduced internal consistency estimates—12 items were deleted. Using an iterative process, the remaining 39 items were submitted to additional PEAs with Promax rotation, confirming the five-factor model (informa- tion on the additional PFAs was not provided).” After each analysis, the items were reevaluated for unsatisfactory items (i., low factor loadings, complex loadings, and reduced reliability), and this resulted in the retention of 21 items. Another round of data were collected from a second sample (N=512) using the revised MALQ; these data were submitted to a confirmatory factor analysis (CFA). The hypothesized five-factor model of metacognitive awareness generated from the EFA results was compared with two potential rival models comprising four factors and six factors. The five-factor model comprising 21 items was finally accepted and selected as the model for the MALQ (CFI=.91, RMSEA =.07, TL1=.90). Reliability Vandergrift et al. (2006) reported internal consistency estimates of sample data ranging from .68 to .78 (Cronbach's alphas: problem solving = .74; planning and evaluation =.75; translation =.78; person knowledge =.74; and directed attention = 68). It is more com- mon for studies to present overall reliability estimates of the MALQ. For example, Cronbach's alpha values of .86 (Rahimia & Katala, 2012) and .76 and .74 (Alamdari & Fahim, 2015) have been reported, Of course, only reporting overall reliability estimates may mask potential issues with subscales, and computing an overall score assumes a second-order latent construct that seems questionable in the case of the MALQ. It is recommended that researchers both confirm the established factor structure as well as assess score reliability (and report the results). 2 The original article does not specify the number of PFAs performed.Metacogntve Awareness Listening Questionnaire (MALQ) Validity In support of construct validity, the MALQ was submitted to both EFA and CFA during the entire process of its development (as described in this profile) to ensure strong psy- chometric properties. The procedure for administering the MALQ further aims to address a criticism of learner self-reports in L2 strategy research—that learners do not report accurately when they are asked about the strategies they use, thereby threatening the validity of the self-reports. Students complete the MALQ immediately following an authentic listening activity in class, basing their responses on this activity. This form of immediate verbal reporting, which taps into information still fresh in memory, strengthens the validity of self-reports (Ericsson & Simon, 1987). Al-Alwan, Asassfeh, and Al-Shboul (2013) found that three dimensions of the MALQ, problem solving (24%), planning and evaluation (17%), and directed attention (15%), explained 56% of the total variance in students’ performance on a listening comprehension test developed by the authors (the Cronbach's alpha for their listening comprehension scale was .77).. Availability ‘The original version of the MALQ was first published in Language Learning (Vandergrift et al, 2006), and has since been republished with permission in Goh (2008) and Vandergrift and Goh (2012). The measure has been adapted depending on the language and context under study. The MALQ is provided, with permission, at the end of this profile and is free to use for research purposes. An unpublished guide for scoring and interpreting MALQ scores is available from the author of this profile. Sample Studies The MALQ has been used to elicit L2 listeners’ metacognitive awareness as well as to examine the relationships between metacognitive awareness and L2 listening compre- hension. Using a sample of 341 Canadian and Iranian participants, Vandergrift et al. (2006) reported a significant association (r=.36) between metacognitive awareness of listening processes and comprehension ability. A regression analysis further suggested that metacognition significantly predicted participants’ listening scores, indicating that 13% of the variance in listening performance could be explained by metacognition. Zeng (2012) reported that metacognition accounted for about 15% of the variance in the listening performance of his EFL Chinese undergraduates, and an even higher per- centage of variance of 22% was reported by Goh and Hu (2014) in their study of 113 ESL learners in Singapore. Their study was also the first to examine intrapersonal variation in different aspects of metacognitive awareness using the MALQ. The MALQ also has been used for comparing metacognitive awareness development before and after metacognitive instruction in listening. In Vandergrift and Tafaghodtari (2010), L2 learners demonstrated gains in listening performance when they were taught listening strategies; the learners also experienced an increase in metacognitive know!- edge at the end of the metacognitive instruction. In another intervention study that assessed changes through the MALQ, O'Bryan and Hegelheimer (2009) found some 433434 Christine. M.Goh development in their participants’ metacognitive awareness but not in all factors. Greater metacognitive developments were found in the participants in the respective studies by Mareschal (2007) and Zeng (2012) and this was likely due to the longer period of metacognitive intervention in L2 listening, ‘The MALQ has been adapted to the L1 context (see Metacognitive Listening Instrument, Profile 40). Critique Since its publication, Vandergrift et al’s (2006) article on the development of the MALQ has been cited in over 250 studies and scholarly discussions about L2 listening. The questionnaire, which has undergone rigorous validation procedures to ensure robust psychometric properties, has been used in a number of published studies as well as ‘unpublished master’s and doctoral dissertations involving L2 learners of various lan- guages, such as French, English, and Arabic, from different parts of the world. Further applications of MALQ can include intrapersonal variations in perceived strategy use and person knowledge for different kinds of listening tasks. One criticism may be that the MALQ may not be fully comprehensive (i.e, that it does not address some strategies that are listed in the 2 listening literature). Although a questionnaire that includes an “exhaustive” list of items may offer a greater degree of face validity, the validation pro- cedures have indicated that many of the original 88 items were problematic as explained in this profile and in the article. Nevertheless, the small number of items for some fac- tors in the MALQ may merit further consideration, and one possible enhancement to the MALQ that is being considered is adding further items to these factors and submit- ting data to further CFAs. This is especially true given that decisions on inclusion and exclusion of items and factors in the original development studies were made on statis tical bases rather than strictly theoretical ones. Other items might tap into additional factors. Perhaps these five factors are the most important, and perhaps they are not; only additional research can determine this. Finally, some work has treated metacogni- tive listening strategies as a single score (e.g,, in reports of internal consistency). The implication is that one can average across all strategies to generate a total “master strat- egy” This assumption seems questionable on theoretical and empirical grounds. Researchers are encouraged to submit their own data to CFA to test whether a second- order latent factor explains associations among the first-order constructs. References AL-Alwan, A. Asassfeh, S,, & Al-Shboul, ¥. (2013). EFL learners’ listening comprehension and awareness of metacognitive strategies: How are they related? International Education Studies, 6, 31-39. doi:10.5539/ies.v6n9p31 Alamdari, E. F, & Fahim, M. (2015). The effect of metacognitive instruction and dialogic interactions on Iranian EFL learners’ metacognitive awareness in listening. Indian Journal of Fundamental and Applied Life Sciences, 5(S2), 1368-1376. Chamot, A. U, Barnhardt, S,, E-Dinaty, P.B., & Robbins, J (1999). The learning strategies handbook. White Plains, NJ: Longman.‘Metacognitive Awareness Listening Questionnaire (MALQ) | 435 Cohen, A. D., & Macaro, E. (2007). Language learner strategies: 30 years of research and practice. Oxford: Oxford University Press. Ericsson, K. A., & Simon, H. (1987). Verbal reports on thinking. In C. Frerch & G. Kasper (Eds.), Introspection in second language research (pp. 24~53). Clevedon, UK: Multilingual Matters, Goh, C. C. M, (1997). Metacognitive awareness and second language listeners. ELT Journal, 51, 361-369, doi:10.1093/elt/51.4.361 Goh, C. C. M, (2002). Learners’ self-reports on comprehension and learning strategies for listening. Asian Journal of English Language Teaching, 12, 46-68. Goh, C. C. M, (2008). Metacognitive instruction for second language listening development: Theory, practice and research implications. RELC Journal, 39, 188-213, doi:10.1177/0033688208092184 Goh, C.CM,, & Hu, G. (2014). Exploring the relationship between metacognitive awareness and listening performance with questionnaire data. Language Awareness, 23, 255-274, doi:10,1080/09658416.2013.769558 Hasan, A. (2000), Learners’ perceptions of listening comprehension problems. Language, Culture and Curriculum, 13, 137-153. doi:10.1080/07908310008666595 Mareschal, C. (2007). Student perceptions of a self-regulatory approach to second language listening comprehension development. Unpublished doctoral dissertation. University of Ottawa, Canada. Mendelsohn, D. J, & J. Rubin (1995). A guide for the teaching of second language listening. San Diego, CA: Dominie Press. ‘O'Bryan, A., & Hegelheimer, V. (2009). Using a mixed methods approach to explore strategies, metacognitive awareness and the effects of task design on listening development. Canadian Journal of Applied Linguistics, 12, 9-37. Rahimi, M., & Katal, M. (2012). Metacognitive listening strategies awareness in learning English as a foreign language: a comparison between university and high-school. Procedia—Social and Behavioral Sciences, 31, 82-89. doi:10.1016/}.sbspro.2011.12.020 Vandergrift, L. (2005). Relationships among motivation orientations, metacognitive awareness and proficiency in L2 listening. Applied Linguistics, 26, 70-89. doi:10.1093/ applin/amh039 Vandergtitt, L. & Goh, C. C. M. (2012). Teaching and learning second language listening: ‘Metacognition in action. New York: Routledge. Vandergrift,L., Goh, C. C. M,, Mareschal, C, & Tafaghodtari, M. H. (2006). The metacognitive awareness listening questionnaire (MALQ): Development and validation. Language Learning, 56,431~462. doi:10.1111/}.1467-992.2006.00373.x Vandergrift, L, & Tafaghodtari, M. H. (2010). Teaching 1.2 learners how to listen does make a difference: An empirical study. Language Learning, 60, 470-497. dois10.1111/ j.1467-9922.2009.00559.x Vogely, A. (1995). Perceived strategy use during performance on three authentic listening comprehension tasks, The modern language journal, 79, 41-56. doi:10.1111/.1540-4781,1995.tb05414x ‘Wenden, A. (1998). Metacognitive knowledge and language learning. Applied Linguistics, 19, §15~37. doi:10.1093/applin/19.4.515 Zeng, Y. (2012). Metacognition and self-regulated learning (SRL) for Chinese EFL listening development. Unpublished doctoral dissertation. Nanyang Technological University, Singapore,436 | Christine CM. Goh Scale Metacognitive Awareness Listening Questionnaire (MALQ) (Vandergrift et al, 2006) Source: Vandergrift etal. (2006). Reproduced with permission of John Wiley & Sons. The statements below describe some strategies for listening comprehension and how you feel about listening in the language you are learning. Do you agree with them? This is not a test, so there are no “right” or “wrong” answers. By responding to these statements, you can help yourself and your teacher understand your progress in learning to listen. Please indicate your opinion after each statement. Circle the number which best shows your level of agreement with the statement. For example: Strongly Disagree Slightly Partly Agree Strongly disagree disagree agree agree Like learning 1 2 3 4 5 6 another language Please circle only ONE number for each statement. Strategy Groups Problem Solving 1) I use the words I understand to guess the meaning of the words I don't understand, 2) As listen, I compare what I understand with what I know about the topic. 3) Luse my experience and knowledge to help me understand. 4) As listen, I quickly adjust my interpretation if I realize that it is not correct. 5) Tse the general idea of the text to help me guess the meaning of the words that Idon't understand. 6) When I guess the meaning of a word, I think back to everything else that I have heard, to see if my guess makes sense. Planning-Evaluation 7) Before I start to listen, I have a plan in my head for how lam going to listen. 8) Before listening, | think of similar texts that I may have listened to. 9) After listening, I think back to how I listened, and about what I might do differently next time, 10) As I listen, I periodically ask myself if I am satistied with my level of comprehension. 11) Ihave a goal in mind as | listen. Mental Translation 12) I translate in my head as [ listen.” 13) I translate key words as I listen.” 14) I translate word by word, as I listen.*‘Metacognitive Awareness Listening Questionnaire (MALQ) | 437 Directed Attention 15) I focus harder on the text when I have trouble understanding. 16) When my mind wanders, I recover my concentration right away. 17) try to get back on track when I lose concentration, 18) When I have difficulty understanding what I hear, I give up and stop listening.* Metacognitive Knowledge Person Knowl 19) I find that listening in is more difficult than reading, speaking, or writing 20) | feel that listening comprehension in isa challenge for me.* 21) I don't feel nervous when I listen to Note: The blank spaces in items 19-21 should be completed by the researcher and refer- ence the respondents’ first language. Labels should be removed and Items randomized prior to administration. Item marked with an asterisk (*) should be reverse-coded prior to scoring. Participants can be given a scoring sheet for self-scoring purposes.