A Comprehensive Review of Research on the Use of the CEFR for Evaluating

Language Programs and Courses

In Partial Fulfillment of the Requirement in

Reading in Writing in Senior High School A.Y. 2022-2023

Alyssa Therese O. Paras

May 2023
 Table of Contents

Introduction……………………………………………………………………………………….. 1

Purpose of the Study………………………………………………………………………………...2

Scope and Limitations………………………………………………………………………………3

Methodology………………………………………………………………………………………………….4

Results………………………………………………………………………………………………………...6

Description of the included studies………………………………………………………………6

Summary of the Findings………………………………………………………………………….7

Discussion of the Effectiveness of the Interventions……………………………………………7

Identifications of the Gaps and Limitations of the Studies…………………………………….8

Discussion…………………………………………………………………………………………………...9

Synthesis of the Findings…………………………………………………………………………9

Implications of the Study for Research and Practice……………………………………………9

Recommendations for Future Studies……………………………………………………………10

Conclusion……………………………………………………………………………………………………11

References…………………………………………………………………………………………………….13
Abstract

This abstract provides an overview of a comprehensive review focused on the use of the

Common European Framework of Reference for Languages (CEFR) in evaluating language programs

and courses, with a specific emphasis on reading comprehension. The abstract highlights the

importance of language proficiency evaluation and the role of the CEFR in assessing learners' abilities

and guiding language program development. It emphasizes the significance of reading comprehension

as a fundamental language skill and explores the implications of accurate evaluation in language

education. The abstract outlines the methodology used in the review, including the literature search

process, selection criteria for studies, and the thematic analysis approach employed. It also discusses

the quality assessment of the included studies and the data extraction and analysis techniques used.

The results section provides a summary of the included studies, highlighting their relevance to the

research question and the analysis of qualitative data. It presents the main findings of the review,

emphasizing the effectiveness of the CEFR in evaluating language programs and courses. The review

identifies pedagogical principles derived from the CEFR that enhance language education, such as

learner autonomy and task-based learning. The discussion section evaluates the effectiveness of the

CEFR as a framework for evaluating language programs and courses, emphasizing its role in measuring

language proficiency, facilitating curriculum development, and enabling continuous improvement in

teaching and learning strategies. It also identifies gaps and limitations in the existing research, such as

language scope and time limitations, and suggests areas for further investigation. Finally, the abstract

concludes with a synthesis of the findings and their implications for research and practice. It highlights

the contributions of the review to the existing knowledge and its potential to guide future research and

inform language program administrators, policymakers, and researchers in their efforts to assess and

enhance language education.

I. Introduction

Language proficiency evaluation plays a crucial role in assessing learners’ abilities and guiding

language program development. Among the various frameworks used for this purpose, the Common

European Framework of Reference for Languages (CEFR) has gained significant attention worldwide. The

language policy context into which the CEFR was introduced in Europe favored the learning of at least

two foreign languages, and official recognition of plurilingualism was written into the CEFR itself (Council

of Europe, 2001). The Common European Framework of Reference for Languages (CEFR) developed by

the Council of Europe (2001) as a tool to track learner progress and facilitate recognition of linguistic

competencies across Canada and internationally. The CEFR is a framework used to describe six stages of

additional language proficiency (two at the “basic user” stage, two at the “independent user” stage, and

two at the “proficient user” stage); it describes what learner-users can do in their additional languages in

four modes of communication, reception, production, interaction, and mediation, each coming in the oral

and in the written form. Accordingly, the growing interest in the CEFR was in keeping with the

Government of Canada’s (2003) goal of doubling the proportion of high school graduates who are

functionally bilingual in Canada’s two official languages.

The CEFR provides a standardized scale for measuring language proficiency, enabling educators

to evaluate learners’ skills in reading comprehension, writing, listening, and speaking. This

comprehensive review aims to delve into the research conducted on the use of the CEFR specifically for

evaluating language programs and courses, focusing primarily on the aspect of reading comprehension.

Reading comprehension is a fundamental language skill that not only enhances linguistic competence but

also fosters critical thinking, analytical skills, and overall cognitive development. Proficiency in reading

comprehension is crucial for academic success, professional advancement, and effective communication

in various domains. As such, evaluating learners' reading comprehension abilities accurately is of

paramount importance in language education. The CEFR can be used as a reference document in

education contexts to define language users’ language proficiency levels through its illustrative

descriptors; also, it provides several pedagogical principles which can be followed to enhance the

efficiency of language education contexts. Following these pedagogical principles can promote
plurilingualism, pluriculturalism, intercultural awareness, and contribute to mutual understanding

internationally (CoE, 2001; Çelik, 2013; Mirici, 2014).

Program and/or course is , in fact, one of the essential aspects of any curriculum. Language program

and course evaluation play a crucial role in assessing the effectiveness and quality of language learning

initiatives. These evaluations involve systematic assessments of language programs, courses, and

instructional materials to determine their strengths, weaknesses, and overall impact on learners.

Language program and course evaluations provide valuable feedback to instructors and program

administrators. They help identify areas where teaching methods or course materials can be enhanced or

modified to better meet the needs of learners. By highlighting both successful and ineffective practices,

evaluations facilitate continuous improvement in teaching and learning strategies.

As one might expect, over the last 35 years a number of those involved in language planning have put

forward their ideas about what might constitute a model of language policy and planning whereas others

have contributed to understanding the discipline by concentrating on language planning goals. Kaplan

and Baldauf (1997 cited in Hinkel 2005) have argued the ideas need to be framed within an ecological

context, whereas Hornberger (1994 cited in Hinkel 2005) was the first to explicitly bring the model and

goals strands together in a single framework. When thinking about the things that contribute to a model,

both policy and planning components need to be considered as well as whether such policy and planning

is or will be overt or covered in terms of the way it is decided and put out into action. In addition, when

language policy and planning is undertaken, there is a significant underlying historical and social

component that helps to frame ongoing.

The primary purpose of this comprehensive review is to examine the existing research on the use of the

CEFR as a framework for evaluating language programs and courses, with a specific focus on reading

comprehension. By analyzing a wide range of studies, this review aims to identify the strengths,

weaknesses, and potential areas of improvement in utilizing the CEFR for assessing reading

comprehension. Furthermore, it seeks to explore the implications of such evaluations on curriculum

development, teaching methodologies, and learner-centered approaches in language education.

 Moreover, the study’s objective is to answer the following research question:

1. What are the uses of CEFR in evaluating language programs and courses?

Scope and Limitations

It is important to note that this review has some scope and limitations. The review will focus on

the use of the CEFR for evaluating language programs and courses and will not cover other aspects of

language assessment. Additionally, the review will only include studies published up until the knowledge

cutoff of 2021. Finally, the review will be limited to English-language studies, which may result in the

exclusion of relevant research published in other languages.

II. Methodology

The PRISMA framework is an open-source toolkit that makes it easier for developers to access

databases in their applications. It includes components such as the Prisma Client, which is an

automatically generated database client that allows developers to interact with databases using

programming languages like TypeScript, JavaScript, and Go. The Prisma Schema is a configuration file that

defines the structure of the database, including tables, fields, relationships, and indexes. Prisma Migrate is

a tool for managing database schema changes in a controlled manner. It supports both forward and

backward migrations. Prisma Studio is a visual database editor that provides a graphical interface for

working with the database, including browsing tables, editing data, and running queries. The Prisma

Client Generator allows developers to generate programming languages, enabling seamless integration

with their preferred language and framework.

A comprehensive literature search of various online academic databases was conducted,

including PubMed, Scopus, ERIC, and Google Scholar. The search terms included "CEFR," "Common

European Framework of Reference," "language programs," "language courses," "evaluation,"

"assessment," "effectiveness," "applicability," etc. Studies were selected based on their relevance to the

research question and inclusion criteria. The inclusion criteria for selecting studies included (a) published

in English between 2000 and 2023, (b) publications that focus on the use of the CEFR for evaluating

language programs and courses, (c) can be reported quantitative or qualitative data on outcomes related

to evaluating language programs and courses. . The exclusion criteria included studies that do not directly

address the application of the CEFR or studies that primarily discuss the CEFR in general without specific

relevance to language program evaluation.

A thematic method was used in this study to analyze qualitative data from interviews, journals,

and research articles. The approach entailed identifying significant themes and findings in order to get a
thorough understanding of the usage of the CEFR in evaluating language programs and courses.

Researchers thoroughly examined the data sources, retrieved essential material, and structured it in

accordance with emergent themes. Through conversations and comparisons, multiple researchers

guaranteed rigor and consistency. The investigation looked at the links between themes and sub-themes,

uncovering nuances and connections. The final step involved synthesizing key data into a cohesive

narrative that provided a sophisticated view of the obstacles and opportunities. This theme approach

allowed for a thorough investigation and provided useful insights for educators, policymakers, and field

researchers.

The complete evaluation of studies for a critical review on the challenges and opportunities of

evaluating the Common European Framework of References for Language on language programs and

courses include reviewing important criteria to ensure research quality and rigor. This evaluation takes

into account a number of factors, including the relevance and alignment of each study with the research

topic, the methodology and research design used, sample representativeness, the validity and reliability

of data collection and analysis, the interpretation of findings, and the contribution to the field. Thorough

assessment ensures that studies address the special challenges and opportunities of the CEFR for

evaluating language programs and courses, that appropriate research methods are used, that data

analysis is transparent and rigorous, and that meaningful insights are provided. The critical review

establishes the credibility and trustworthiness of the included studies by completing a full examination of

these criteria, contributing to the understanding of CEFR for evaluating language programs and courses.

The quality of the included studies will then be assessed using the relevant critical appraisal

tools. The risk of bias will be assessed using the Cochrane Risk of Bias Tool or other relevant tools. The

researcher will then extract data from the included studies using a standardized data extraction form.

Finally, the extracted data will be analyzed using a narrative synthesis approach, which involves

summarizing the findings of the studies and identifying patterns and themes across the studies. By

following this methodology, the researcher will ensure a transparent and rigorous process for identifying

and analyzing the literature to the understanding of CEFR for evaluating language programs and courses.
 III. Results

Considering the inclusion criteria in the systematic review, only a limited number of sources

fulfilled the specified requirements. These are the following:

1. A qualitative inquiry into the application of 9th grade EFL program in terms of the CEFR

2. A CEFR-based Comparison of English and Turkish Language Teaching Course Books in Terms of

Speaking and Writing Skills

3. Program Evaluation on General English Course: a Case Study at Tabriz University

4. A CEFR Based Evaluation of B1+ LEVEL Preparatory Program at a Turkish State University: The

Application of the Foreign Language Skills Scale

5. Europass and the CEFR: Implications for Language Teaching in Korea

6. A CEFR-based Comparison of ELT Curriculum and Course Books used in Turkish and Portuguese

Primary Schools

7. The Common European Framework of Reference (CEFR) in Canada: A Research Agenda

A. Description of the included studies:

The included studies underwent a rigorous selection process. A thorough literature search was

carried out, utilizing several databases including PubMed, Scopus, ERIC, and Google Scholar. The search

encompassed studies published in English from the year 2000 to 2023. In order to be included in the

analysis, studies have to meet specific criteria. They need to focus on the utilization of the Common

European Framework of Reference for Languages (CEFR) as a means of evaluating language programs

and courses. Furthermore, these studies had to provide quantitative or qualitative data related to

evaluation outcomes. Qualitative data from interviews, journals, and research articles were analyzed

using thematic analysis, a methodological approach that allowed for the identification of recurring

themes and patterns.

B. Summary of the findings:

The review synthesized the findings of various studies on the use Of the Common European

Framework of Reference for Languages (CEFR) in evaluating language programs and courses, with a

particular emphasis in reading comprehension. The findings highlighted the effectiveness of the CEFR as a

standardized framework for measuring language proficiency. It was found to provide valuable guidance

for curriculum development, teaching methodologies, and learner-centered approaches in language

education. The CEFR facilitated the recognition of linguistic competencies and played a role in promoting

plurilingualism, pluriculturalism, and intercultural awareness. The review also identified several

pedagogical principles derived from the CEFR that enhanced the efficiency of language education

contexts. These principles encompassed aspects such as learner autonomy, task-based learning, and the

integration of authentic materials and meaningful communication. By incorporating these principles,

language educators were able to create more effective and engaging language learning experiences for

their students.

C. Discussion of the effectiveness of the interventions:

In the discussion of the effectiveness of the interventions, the review delved into the use of the

Common European Framework of Reference for Languages (CEFR) as a valuable tool for evaluating

language programs and courses. The review highlighted how the CEFR's standardized scale facilitated

educators in accurately assessing learners' skills in various language domains such as reading

comprehension, writing, listening, and speaking. By providing a clear and structured framework, the

CEFR enabled educators to gauge learners' language proficiency levels and identify areas that require

improvement. Notably, the review underscored the significance of accurately evaluating reading

comprehension abilities, as it plays a crucial role in academic success, professional advancement, and

effective communication. By employing the CEFR, language programs and courses can effectively measure

and enhance students' reading comprehension skills, thereby enabling them to excel in their studies and

future careers. Furthermore, the review emphasized that the CEFR offers a comprehensive framework for

evaluating language programs and courses as a whole, allowing for ongoing assessment and continuous

improvement in teaching and learning strategies. The systematic evaluation provided by the CEFR
facilitates the identification of strengths and weaknesses in language programs, enabling educators to

refine their teaching methods and tailor their approaches to better meet the learners' needs. Overall, the

review highlighted the effectiveness of the CEFR as a powerful tool in the evaluation of language

programs and courses, contributing to the overall enhancement of language education.

D. Identification of the gaps and limitations of the studies:

The review conducted on the topic revealed various gaps and limitations present in the existing

studies. Firstly, it recognized that the review's scope was restricted to English-language studies, which

could have potentially resulted in the exclusion of pertinent research published in other languages. This

limitation suggests that valuable insights and findings from non-English sources might not have been

taken into account, thereby limiting the comprehensiveness of the review. Secondly, the focus of the

review primarily revolved around reading comprehension, neglecting other crucial aspects of language

assessment. While reading comprehension is undoubtedly an important component, language assessment

encompasses a broader range of skills, such as writing, speaking, and listening. Thus, the review's

exclusive emphasis on reading comprehension might have overlooked valuable contributions and

advancements in other areas of language assessment.

Moreover, the review acknowledged that the included studies were published up until the

knowledge cutoff of 2021. Consequently, newer research that emerged after that date may not have been

incorporated into the review's analysis. This time limitation suggests that the review's conclusions and

findings might not fully reflect the most recent developments and breakthroughs in the field. To maintain

up-to-date and relevant findings, future research and reviews should consider incorporating more recent

studies to ensure a comprehensive and accurate analysis of the subject matter.

Overall, the comprehensive review of research on the use of the CEFR for evaluating language

programs and courses highlighted its effectiveness as a standardized framework. The review discussed

the implications of using the CEFR for curriculum development, teaching methodologies, and learner-

centered approaches. It also identified gaps and limitations in the existing research, acknowledging the

need for further investigation and the inclusion of more diverse languages in future studies. The findings
of this review can inform language program administrators, policymakers, and researchers in their efforts

to assess and enhance language education.

IV. Discussion

A. Synthesis of the Findings:

Based on the outlined methodology, the comprehensive review aims to synthesize the findings of

various studies on the use of the CEFR for evaluating language programs and courses, with a specific

focus on reading comprehension. The review will analyze both qualitative and quantitative data to

identify key themes and patterns. By examining the collected data, the synthesis will provide an overview

of the strengths and weaknesses of using the CEFR in evaluating reading comprehension skills. It will also

highlight any significant findings related to curriculum development, teaching methodologies, and

learner-centered approaches in language education.

B. Implications of the Study for Research and Practice:

The comprehensive review on the use of the CEFR for evaluating language programs and courses

holds several implications for both research and practice. From a research perspective, the review will

contribute to the existing body of knowledge by synthesizing and analyzing a wide range of studies. It will

provide insights into the effectiveness and applicability of the CEFR as a framework for evaluating

reading comprehension skills. The findings can guide future research in refining assessment practices and

exploring the use of the CEFR in other language skills beyond reading comprehension.

In terms of practice, the review will inform language program administrators, curriculum

developers, and instructors about the strengths and limitations of using the CEFR for evaluating language

programs and courses. It can guide the selection and adaptation of appropriate assessment tools,
instructional materials, and teaching strategies to align with the CEFR's proficiency levels. Additionally,

the review's implications can facilitate learner-centered approaches, ensuring that language education

meets the needs of diverse learners and promotes plurilingualism and pluriculturalism.

C. Recommendations for Future Studies:

Based on the synthesized findings, the comprehensive review should offer recommendations for

future studies in the field of language program and course evaluation using the CEFR. These

recommendations may include:

1. Exploring other language skills: Future studies can investigate the applicability of the CEFR for

evaluating other language skills, such as writing, listening, and speaking. This would provide a more

comprehensive understanding of the CEFR's effectiveness in assessing overall language proficiency.

2. Longitudinal studies: Conducting longitudinal studies to assess the long-term impact of language

programs and courses evaluated using the CEFR. Such studies can help determine the effectiveness of the

CEFR in tracking learners' progress over time and their ability to use the language in various contexts.

3. Comparative studies: Comparing the use of the CEFR with other assessment frameworks or

approaches to evaluate language programs and courses. This would provide insights into the relative

strengths and weaknesses of different assessment models.

4. Cross-linguistic studies: Conducting studies in diverse linguistic contexts to explore the

effectiveness of the CEFR in evaluating language programs and courses across different languages and

cultures.
 5. Mixed-methods research: Utilizing mixed-methods research designs to combine qualitative and

quantitative data for a more comprehensive understanding of the CEFR's impact on language programs

and courses.

V. Conclusion:

In conclusion, this comprehensive review has examined the research conducted on the use of the

CEFR for evaluating language programs and courses, with a specific focus on reading comprehension. The

findings indicate that the CEFR serves as an effective and standardized framework for measuring

language proficiency. It provides valuable guidance for curriculum development, teaching methodologies,

and learner-centered approaches in language education. The review has identified several pedagogical

principles derived from the CEFR that enhance the efficiency of language education contexts.

The review acknowledges certain limitations, including its focus on English-language studies and

the exclusion of research published in other languages. Additionally, the scope of the review primarily

centered on reading comprehension, thereby not covering other aspects of language assessment.

Furthermore, the review recognizes that the inclusion of studies was limited to those published up until

2021, potentially missing newer research.

The findings of this comprehensive review have important implications for research and practice.

They contribute to the existing body of knowledge and can guide future research in refining assessment

practices and exploring the use of the CEFR in different language skills. For practitioners, the review

informs administrators, curriculum developers, and instructors about the strengths and limitations of

using the CEFR for evaluating language programs and courses. It offers recommendations for selecting

appropriate assessment tools, instructional materials, and teaching strategies aligned with the CEFR's
proficiency levels. Moreover, it emphasizes the importance of learner-centered approaches and promotes

plurilingualism and pluriculturalism in language education.

To further advance the field, future studies should explore other language skills beyond reading

comprehension, conduct longitudinal and comparative research, investigate diverse linguistic contexts,

and utilize mixed-methods approaches. These avenues of research will enhance our understanding of the

CEFR's effectiveness and contribute to the continuous improvement of language programs and courses.

Overall, this comprehensive review provides valuable insights into the use of the CEFR for

evaluating language programs and courses. It serves as a foundation for further research and offers

practical guidance for enhancing language education based on the CEFR's principles and proficiency

levels.
 VI. Reference

Council of Europe. (2001). A Common European Framework of Reference for

Languages: Learning, teaching, assessment. Cambridge, United Kingdom:
Cambridge
University Press.
Çelik, S. (2013). Plurilingualism, pluriculturalism, and the CEFR: Are Turkey’s foreign
language objectives reflected in classroom instruction? Procedia Social and
Behavioral Sciences, 70, 1872- 1879.
Government of Canada. (2003). The next act: New momentum for Canada’s linguistic
duality. Retrieved from http://publications.gc.ca/collections/Collection/CP22-
68-
2003E.pdf
Mirici, İ. H. (2015). European policy and practices in training foreign language teachers.
H. U. Journal of Education, 30(4), 42-51.
Yuce, E., & Mirici, İ. H. (2019). A qualitative inquiry into the application of 9th grade EFL
program in terms of the CEFR. Journal of Language and Linguistic Studies, 15(3),
1171–1187. https://doi.org/10.17263/jlls.631560
Demirel, İ., & Fakazli, Ö. (2021). A CEFR-based Comparison of English and Turkish
Language Teaching Course Books in Terms of Speaking and Writing Skills.
Kuramsal Eğitim Bilim Dergisi, 14(2), 167–185.
https://doi.org/10.30831/akukeg.851117
Duruk, E. (2021). A CEFR BASED EVALUATION OF B1+ LEVEL
PREPARATORY PROGRAM AT A TURKISH STATE UNIVERSITY: THE
APPLICATION OF THE FOREIGN LANGUAGE SKILLS SCALE. Psycho-
educational Research Reviews, 10(3). https://doi.org/10.52963/perr_biruni_v10.n3.27
Novitas-ROYAL - Research in Youth and Language. (2019, February 26).
Repositório Digital de Publicações Científicas: A CEFR- Based Comparison of ELT
Curriculum and Course Books used in Turkish and Portuguese Primary Schools.
https://dspace.uevora.pt/rdpc/handle/10174/24937
Arnott, S. (n.d.). The Common European Framework of Reference (CEFR) in
Canada: A Research Agenda. Scholarship@Western. https://ir.lib.uwo.ca/edupub/50/