Professional Documents
Culture Documents
Research Analysis and Critique Final Draft
Research Analysis and Critique Final Draft
1
Research Analysis & Critique
Introduction
The study I have chosen to analyze and critique is titled “The Effects of Adult Learning on Self-
Efficacy”, and was published in the London Review of Education in 2005, by Cathie Hammond
Analysis
Purpose
The purpose of the study is, as Hammond and Feinstein state, to “investigate the links between
participation in adult learning and self-efficacy, particularly for the subgroup of adults who had
low levels of achievement at school” (Hammond and Feinstein, 2005, pg. 265). The researchers
hypothesize that “adults with relatively high levels of self-efficacy are more likely to engage in
learning, and also that the experience of adult learning raises levels of self-efficacy” (Hammond
& Feinstein, 2005, pg. 266). The study focuses on a subgroup of women with what Feinstein
and Hammond classify as “poor school attainment” (Hammond and Feinstein, pg. 266, 2005).
Three corollary research questions addressed are as follows: “Is participation in adult learning
related to transformed or sustained efficacy? Are relationships found for men and women with
poor and good school achievement? What are the causal pathways that underlie any
relationships found?” (Hammond and Feinstein, 2005, pg. 269). The knowledge gap is related
to a lack of research regarding which processes might underlie an association between adult
learning and self-efficacy, and whether participation in learning contributes to transformed self-
efficacy (Hammond and Feinstein, 2005). The study conducts a qualitative and quantitative
A notable prior study was conducted by Schuller et al., 2002, 2004. A compelling idea emerging
from Schuller et al. 2002, 2004 is that adult learners show increased confidence, and even
though confidence differs from self-efficacy, there are many wider benefits that flow from
greater confidence, and several of these could be described as the manifestations of increased
self-efficacy (as cited in Hammond and Feinstein, 2005). Ultimately, the implications are that
increased self-efficacy can benefit adult learners in a wide variety of ways. Another notable
study was conducted by Bandura 1997. An important idea from this study is that self-efficacy
plays important roles in protection from the development of anxiety disorders, depression and
performance, the enactment and fulfillment of occupational roles, creative productivity and
Constructs
The most significant constructs examined in the study are sustained and transformed self-
that stays the same between measurements taken at 33 and then at 43. Transformed self-
efficacy refers to changes in self-efficacy during this period. These self-efficacy constructs are
representing low self-efficacy. The researchers create this operational definition (binary
variable) by having cohort members select from three binary positive/negative statement pairs.
Those who chose all three positive statements are assigned a value of 1, indicating high self-
efficacy. Those who chose at least one negative statement were assigned a value of 0. These
Research Analysis & Critique
constructs were further analyzed and operationally defined as an odds ratio in Table 2.
Participation in Adult Education was operationally defined as whether cohort members took
any classes between ages 33 and 43. Another construct in the study is school attainment,
which researchers operationally define through a binary variable based on whether the
member attained an O level by age 16; they are given a value of 1 and are considered to have
had school attainment if they have achieved an O level by 16, and are given a value of 0 if they
haven’t.
Research Type
The study can be defined as theory based, as it draws on past theoretical research regarding
self-efficacy and school attainment, and creates a hypothesis out of this. The opening line of
the study is “we use quantitative and qualitative methods to investigate…” which indicates a
clear mixed methods approach (Hammond and Feinstein, 2005, pg. 265). Researchers use a
independent variables, and a qualitative approach to further explore the association on an in-
depth macro level. The study does not use an experimental design, instead focusing on a non-
intervention design that examines a potential causal relationship between school attainment
and self-efficacy. The study is considered causal-comparative as it compares two groups (those
with school attainment and those without) in an important dimension (presence of self-
efficacy). The researchers also examine qualitative and quantitative data from a longitudinal
Variables
The study’s dependent variables are sustained or transformed self-efficacy. The independent
whether cohort member took any taught-courses between the ages of 33 and 42 between 1991
and 2000 (Hammond and Feinstein, 2005). An attribute variable, male versus female, was also
used to determine whether the relationship between participation in adult education may be
different for men and for women (Hammond and Feinstein, 2005).
Research Design
gathered from a previous longitudinal study, and compares two groups (those with school
Control Procedures
Researchers attempt to control for social, family, cognitive and biological factors through
several methods (Hammond and Feinstein, 2005). First, “the childhood controls include
measures derived from an instrument devised by Rutter et al. (1970) to measure social and
assessed using the Bristol Social Adjustment Guide (BSAG)” (Hammond and Feinstein, 2005, pg.
272). Second, researchers “control for socio-economic status based on occupation and the
Registrar-General’s schema, in five categories with manual semi-skilled and unskilled combined
(Hammond and Feinstein, 2005, pg. 272). Third, researchers control for age self-efficacy as
Research Analysis & Critique
by creating dummy variables that describe sixteen groups of individuals depending on these
criteria (Hammond and Feinstein, 2005). Unfortunately, control procedures for the original
Sampling
The sample of research participants varies throughout the study. The quantitative data is
derived from a longitudinal study of surveys conducted between 1958 and 2000, of all UK
children born between March 3rd and March 9th, 1958. Hammond and Feinstein focus on
participants at ages 7, 11, 16, 23, 33 and 42. The first data collected by the NCDS study
featured 17, 415 participants, and the final sweep of data in 1999 featured 11, 419 participants.
For the qualitative portion of the study, the researchers sampled 15 women with poor school
attainment (no O level equivalents at age 16) from the original NCDS study (Hammond and
Feinstein, 2005). These women were all aged 46 when they were interviewed in 2004.
The fact that the binary odds ratios used to measure the constructs were estimates, validity is
pointing out that the participant interviewee cross-referenced the interview data with the
quantitative survey data, and found no contradictions in results. Another potential reliability
concern is the fact that for the qualitative portion, there are only three sets of binary questions,
which may not be enough to adequately measure self-efficacy. Though the researchers do not
provide specific reliability indicators, the cross referencing does lend itself to reliability.
Ultimately, however, the researchers do not present clear evidence of either validity or
reliability.
Research Analysis & Critique
Alternative Hypothesis
The researchers suggest the alternative hypothesis that it is possible that life experience
participant’s self-efficacy measure, as well as his or her school attainment measure. They seek
to eliminate this confounding bias by enacting a series of statistical controls (as detailed in
section 7 above), but ultimately admit that their controls cannot eliminate all potential bias,
and as such does not completely explain away this alternate hypothesis. They also conducted
their analyses for men and women separately because they believed that participation and self-
efficacy would be different for men and women (Hammond and Feinstein, 2005).
For the quantitative section of the study, Hammond and Feinstein used estimate odds ratios to
analyze the data, and logit regression analysis (to deal with the binary variables). The major
findings for the quantitative portion of the study suggest that those who took courses during
the periods measured were far more likely to experience transformed self-efficacy. However,
there is no evidence to suggest that sustained self-efficacy is more likely for those who took
courses during the periods measured. For the qualitative portion of the study, interviewers
collected data, completed a 2000 word reflection on the responses, and then used analytic and
inductive software NVivo 1.2 to analyze the response data. Several major findings emerged
from this portion of the study. First, an adult’s perception of achievement in adult education
can increase self-efficacy. Second, as adults increase their self-efficacy, there is less resistance
to participation in adult learning. Third, those who find success in adult education may find
Research Analysis & Critique
more challenging jobs and have higher self-efficacy. Finally, challenging backgrounds and life-
Conclusions
Hammond and Feinstein found that adults, particularly women with low school attainment, and
at specific periods of their lives, may engage in adult learning and see increased-self efficacy.
They identify a positive feedback loop in which self-efficacy increases, along with participation
such do not suggest adult learning as being directly causal of self-efficacy and increased self-
confidence.
Critique
In their study, Hammond and Feinstein seek to investigate potential links between participation
in adult learning and self-efficacy. They believe that adult learning can contribute to self-
increased efficacy, and hypothesize that “adults with relatively high levels of self-efficacy are
more likely to engage in learning, and also that the experience of adult learning raises levels of
self-efficacy” (Hammond & Feinstein, 2005, pg. 266). The study is situated in the context of
previous studies that suggest increased self-efficacy can protect adults from depression and
anxiety, two factors that can also contribute to low school attainment or a lack of participation
in adult learning. As such, the study examines the role of adult education as a potential aid for
One major strength of the study is its methodology. That is, a mixed methods approach allows
the researchers to supplement a quantitative data analysis with an in-depth qualitative study.
Further, the quantitative portion has the benefit of accessing data from a previous large scale
longitudinal national study. Though the quantitative data was collected from a previous study,
the researcher do a commendable job incorporating the data into their study through the
creation of a summary ratio formula, thereby making the data relevant and useful to their
examination. They further complete a logit regression analysis to compensate for the
shortcomings of the estimation ratio. Another strength of the study is in the sample size of the
Though Hammond and Feinstein do their best to utilize this data to address their hypothesis,
several weaknesses are apparent in the study. One such weakness is the inability of the
researchers to adequately control for confounding bias and reverse causality. This is a
consequence of the chosen methodology of the study. That is, because the quantitative
portion of the study relies on previously collected data from the NCDS, the researchers are
unable to account for several factors. Most notable, is their inability to access information
about the timing of changes in self-efficacy of the participant cohort. Hammond and Feinstein
note several times in the study that their inability to control for certain factors, mostly biases,
makes them cautious about identifying causal relationships from the quantitative data alone.
Their inability to do so, suggests a potential inability of them to generalize from the findings.
Even though the qualitative portion of the study is beneficial when synthesized with the
qualitative data, it too has its weaknesses. Though the researchers suggest that the subjectivity
Research Analysis & Critique
of both the participants and the interviewers is a potential weakness of the study, this
subjectivity is a hallmark of qualitative research and allows for personal, macro level insights
and should be viewed as a strength. Another potential threat to validity is the fact that the
odds ratios used to measure the constructs of self-efficacy are estimates, and as such construct
validity is questionable. However, Hammond and Feinstein argue that the findings are still
Ultimately, though this study does uncover some interesting findings, it does not do enough to
account for biases. Though it does result in findings that are general in nature, it is not easily
generalizable. A study of this nature is dependent on longitudinal data, and although the
researchers access this data via the NCDS, they did not collect the data themselves. As such,
they are not able to control for the necessary variables that allow their findings to go much
further beyond the previous research they reference in their work. Though I find the study’s
convinced that this study fills the knowledge gap or controls for bias enough to justify its
publication.
Research Analysis & Critique
References
Hammond, C., & Feinstein, L. (2005). The effects of adult learning on self-efficacy. London
Bandura, A. (1997) Self-efficacy: the exercise of control (New York, W. H. Freeman & Company).
Schuller, T., Brassett-Grundy, A., Green, A., Hammond, C. & Preston, J. (2002) Learning,
continuity and change in adult life, wider benefits of learning research report no. 3
Schuller, T., Preston, J., Hammond, C., Brassett-Grundy, A. & Bynner, J. (2004) The benefits of
learning: the impacts of formal and informal education on social capital, health and