Running head: RESEARCH ANALYSIS AND CRITIQUE 1

Assignment 2 - Research Analysis & Critique

Effects of a classroom-based program on physical activity and on-task behavior

Stephen Scott (​27727635)

University of British Columbia

ETEC 500 (65A) - Dr. Kisha McPherson

April 2020
RESEARCH ANALYSIS AND CRITIQUE 2

Assignment 2: A Research Analysis and Critique

Research Study to be Analyzed and Critiqued:

Mahar, M. T., Murphy, S. K., Rowe, D. A., Golden, J., Shields, A. T., & Raedeke, T. D. (2006).
Effects of a classroom-based program on physical activity and on-task behavior. ​Medicine &
Science in Sports & Exercise,​ ​38(​ 12), 2086-2094.

Research Analysis

Purpose of this Study:

The purpose of this study was to evaluate the effects of a classroom based physical activity

program on the physical activity levels and on-task behaviour during instructional time for

elementary school aged children. The researchers in this study describe a need for more data in

order to promote and document the importance of physical activity in schools during

instructional time.

Connection With Existing Literature/Research:

The researchers indicated that there have been few studies connected directly to the area of

connecting physical activity to on-task behaviour. A study by Stewart et al. (2004) provided

evidence that classroom teachers can successfully implement intense physical activity in their

elementary classrooms throughout the school day. This is important information moving forward

in this study. Literature prepared by Biddle (1995) explores the positive impacts of physical

activity on concentration, stress, cognitive abilities, and academic performance. Research

performed by Pellegrini and Davis (1993) has found that when elementary aged children

participate in extended periods of academic instruction their concentration levels decrease and
RESEARCH ANALYSIS AND CRITIQUE 3

they become significantly less productive. The researchers felt there was a need for exploration

in the specific area of on-task behaviour as it relates to varying levels of physical activity.

Significant Constructs:

The most significant construct in this study was on-task behaviour. Mahar et al (2006) defined

on-task behaviour as “verbal or motor behaviour that followed the class rules and was

appropriate to the learning situation.” (p. 2088). In order to identify observed off-task behavior

they utilized three operational definitions to assist in data collection and tracking. These

included motor off-task, noise off-task, and passive/other off-task. These three off-task

behaviours were grouped together as one and then used to measure the level of on-task behaviour

for the students.

Another construct to consider in this study was physical activity. The researchers used

pedometers to measure the level of physical activity of the students involved in the study.

Students would put on their pedometer at the start of the school day and record the number of

steps taken at various times throughout the day. Control groups and intervention groups had

different guidelines around times to record their steps. Total steps as measured by pedometers

was the operation used to define physical activity.

Research Approach:

The research in this study was quantitative. Outcome data is presented numerically for a variety

of areas. Mean daily steps taken for both the control and intervention groups are recorded and are

broken down to grade levels of the students participating. On-task behaviours are presented by

the researchers as percentages of total observations for both the control and intervention groups.
RESEARCH ANALYSIS AND CRITIQUE 4

Quantitative data is used to explain the findings and develop conclusions connected to the

purpose of the study.

This study is problem-based. The researchers are interested in exploring a method that could

potentially improve student behaviour and thus performance in the classroom. Suter (2014)

describes problem based research as “improving educational practice or evaluating a specific

program in order to make sound decisions” (Chapter 6, P. 20). The use of physical activity as a

tool to increase on-task behaviours is evaluated in this study and the results can be beneficial to

schools and teachers in developing their programs.

Variables:

The independent variable in this study is the assignment of classes into intervention

(experimental) and control groups. Intervention group classes participated in the class physical

activity program through the “Energizer” activities. The control group classes did not participate

in the “Energizer” activities. This is an example of a quasi-independent variable as the

intervention and control groups were formed by using pre-established classes and teachers

(Suter, 2014). The groups were already formed (as classes) for the researchers who then

determined which set of students would be intervention and control groups.

One dependent variable in this study is the amount of physical activity that students are

participating in during the school day. This was measured by steps recorded on a pedometer by

each individual student in all classes involved. Another dependent variable is the number of

off-task behaviours by students that are observed and recorded by the researchers. A system that

will be described later was used to identify these behaviours and ensure reliability in assessment.
RESEARCH ANALYSIS AND CRITIQUE 5

A number of extraneous variables were also evident in this study but were not described by the

researchers. This will be addressed in the Critique section.

Research Design:

This study is an example of quasi-experimental research. The researchers actively intervened by

creating differences between the control groups and the intervention groups. The intervention

group received extensive physical activity through “Energizers” presented by their teacher while

the control group did not. Students were not randomly divided into groups but stayed in their

original classes and worked with their own teacher for the duration of this study. This lack of

true randomness of participants is what makes this study quasi-experimental (Suter, 2014).

Control Procedures:

The researchers used a number of control procedures. All classes involved in this study were

from the same school but 2 classes from each grade were randomly selected to serve as the

intervention classes. They also ensured that all teachers were trained at the same time in using

the “Energizer” activities and were provided with a booklet. All intervention classes participated

in the “Energizer” activities at the same time in the school day. Blinding was also used as

students and classes were not informed of their specific role in the study. Also, students and

teachers did not know which individuals were being observed at any given time.

Sampling Procedures:

A total of 243 students in Grades 3 and 4 had their physical activity levels assessed. The

intervention group had 135 students while the control group had 108 students. On-task behaviour

was assessed for 62 (37 Grade 3 and 25 Grade 4) of the intervention group students.
RESEARCH ANALYSIS AND CRITIQUE 6

Reliability and Validity:

All observers (for on and off-task behaviour) participated in several training sessions where they

practiced observing and recording through videos of classrooms. They were given detailed

definitions of on and off task behaviours. These sessions continued to be held until observers

reached a level of 80% reliability. During the study observers reached a level of 94% of

agreement. The researchers stressed the importance of interrater reliability for this study (Suter,

2014).

Alternative Hypotheses:

An alternative hypothesis identified by the researchers is the possibility that an alternative

classroom physical activity program does not have an influence on the level of on-task classroom

behavior of students.

Data Analysis:

Significant data presented by the researchers in this study included daily mean in school steps

taken by the students, mean percentages of on-task behaviour of students before and after

“Energizers”, and mean differences in on-task behaviour among the various observation periods.

Intervention classes averaged approximately 800 more steps per day than control classes. The

mean on-task behaviour among all intervention students improved by 8% from before the

“Energizers” to after. A more significant (20%) improvement was observed and recorded for

students identified by teachers as normally being the least on-task.

RESEARCH ANALYSIS AND CRITIQUE 7

Major Findings and Key Conclusions:

The researchers concluded that a classroom based physical activity program was effective in

significantly increasing physical activity for students while also increasing the amount of on-task

behaviour during instructional time. They describe a need for more research in this area in order

to provide a rationale for schools to make policy changes regarding physical activity programs.

Critique

Contribution of the study:

The researchers involved in this study were clear in their explanation about the purpose of their

work. They point to a variety of previous studies that explore the effects of physical activity on

student learning and achievement as well as overall health and wellness. However, there is very

little previous research that relates directly to classroom physical activity programs and how they

impact behaviour. Improved behaviour will usually result in improved academic performance.

This study contributes significantly to making a stronger case for teachers and schools to

consider implementing extra physical activity during instructional time.

Significance and Implications:

The findings and data collected in this study definitely point to a benefit related to in class

teacher driven physical activity programs. It is obvious to state that physical activity levels

increased for students when they participated in the “Energizer” activities compared to students

who did not. The more difficult area for the researchers to measure was the level of on-task

behaviours. The overall result of an 8% improvement in mean percentage of on-task behaviors

was moderately significant (Effect size = 0.60). However, more significant findings are evident
RESEARCH ANALYSIS AND CRITIQUE 8

when considering those students who were considered to be the most off-task by classroom

teachers. These students displayed an improvement of 20% in mean on-task behaviours after

“Energizer” activities (Effect size 2.20). Thus, the most significant results and implications of

this study may be directed to working with the most challenging students for teachers.

Methodological Issues:

Suter (2014) explains that the way researchers deal with methodology challenges is crucial in

determining whether or not it is a strong or weak study. Mahar et al. (2006) describe most of

their methods and procedures in detail. Processes such as research design, grouping, control,

reliability, and measurement are all addressed in various levels of detail. The researchers

undoubtedly put an emphasis on ensuring and maintaining the validity of this study. Details on

these methodological processes are grouped into the Strength and Weaknesses section that

follows.

Strengths and Weaknesses:

One weakness in this study would be that only one school was involved. Classes were randomly

selected for intervention and control groups within the primary classes of the school. The

findings and results from this study could have more validity and reliability if an expanded group

was involved. Multiple school involvement could have allowed the researchers to gain a better

understanding of the overall effect of their interventions for primary aged students. Researchers

could have then randomized the population more by selecting groups of students from each

school and focussing on their results.

Another weakness was the absence of attention to extraneous variables. There are several

extraneous variables related to this study that the researchers needed to consider. These include
RESEARCH ANALYSIS AND CRITIQUE 9

the teacher attitude and experience with physical activity in the classroom, teacher response to

training for “Energizers”, student motivation, classroom management variation and time of day

that “Energizer” activities occur. There was not a lot of information in this study that addressed

how these extraneous variables were to be controlled. It is worth noting that any of these could

influence the dependent variables and result in a confounding within the intervention and control

groups.

Students wore pedometers throughout the duration of this study to measure their physical activity

each day. This could have led to potential problems related to the Hawthorne effect (Suter,

2014). It is worth considering if the children involved changed their behaviour and actions

throughout this study because they knew that some form of special attention was being directed

at them. Did some students make a change from their natural activity levels in order to show

more steps on their pedometer? This was not something that the researchers addressed; however,

their options would have been limited for measuring activity levels.

An area of strength in this study was the attention that the researchers directed at observer or

rater reliability. All observers involved were required to attend a series of training sessions on

observing and recording on-task and off-task behaviours. This involved the observation of video

recordings of real classrooms. These training sessions continued to be held until 80% reliability

was achieved. Eventually observers interpreted the behaviours similarly 94% of the time. This

contributed significantly to giving reliability to the data collected.

Another strength was that all teachers were trained in the delivery of the “Energizer” activities.

They were not simply handed a resource to view and interpret on their own. Teachers

participated in activities themselves first. After the training they were provided with a booklet
RESEARCH ANALYSIS AND CRITIQUE 10

that included a wide variety of activities that they could use. This undoubtedly contributed to

control in this study as all teachers were given the same message, experience, strategies, and

activities related to administering an in class physical activity program.

Another strength involved the researchers using a multiple-baseline across-classrooms approach

(Mahar et al., 2006). Intervention classes were staggered in their start dates for participation in

“Energizer” activities. Two classes started after 4 weeks of baseline observations and two classes

started after 8 weeks of baseline observations. This method helps to eliminate some of the

extraneous variables by having classes start under a variety of baseline conditions.

Conclusion and Overall Recommendation:

This study is useful to educators in that it provides information related to physical activity and

the specific area of on task behaviour for elementary students. As the researchers state, it has the

potential to influence schools and school districts in their decision making around the

implementation of in class physical activity initiatives. This study has been published in a

reputable journal and obviously meets the requirements of sound educational research.

PEER REVIEW PROCESS:

I found this process useful from both sides of the reviews. As a reviewer I found it a valuable
process in reflecting upon my own paper as I critiqued and reviewed the paper of a peer. The
feedback that I received was a good combination of praise and constructive feedback. I was
given several recommendations for improvement and I saw the value of implementing most of
them. These involved language clarification, more detailed explanations, and areas where theory
and course materials could more effectively be highlighted.
RESEARCH ANALYSIS AND CRITIQUE 11

References:

Biddle, S. (1995). Exercise and psychosocial health. ​Research quarterly for exercise and sport​,

66​(4), 292-297.

Mahar, M. T., Murphy, S. K., Rowe, D. A., Golden, J., Shields, A. T., & Raedeke, T. D. (2006).

Effects of a classroom-based program on physical activity and on-task behavior.

Medicine & Science in Sports & Exercise,​ ​38(​ 12), 2086-2094.

Pellegrini, A. D., & Davis, P. D. (1993). Relations between children's playground and classroom

behaviour. ​British journal of educational psychology,​ ​63(​ 1), 88-95.

Stewart, J. A., Dennison, D. A., Kohl III, H. W., & Doyle, J. A. (2004). Exercise level and

energy expenditure in the TAKE 10!® in-class physical activity program. ​Journal of

School Health,​ ​74(​ 10), 397-400.

Suter, W. N. (2012). ​Introduction to educational research: A critical thinking approach (​ 2nd

ed.). Thousand Oaks, CA: SAGE Publications.