Professional Documents
Culture Documents
Chapter 5 - Theory and Methodology in Learning Design Research
Chapter 5 - Theory and Methodology in Learning Design Research
Chapter 5 - Theory and Methodology in Learning Design Research
design research
Introduction 1
Definitions 1
Theoretical perspectives 1
Communities of Practice 1
Methodological approaches 1
Content analysis 1
Ethnography 1
Case studies 1
Action research 1
Evaluation 1
Conclusion 1
Introduction
This chapter will provide an overview of the theoretical perspectives and associated
methodologies that und2erpin learning design. It will articulate how learning design is
distinct from, but relates to, related research such as Instructional Design and Learning
Sciences. Discussing learning technology as a field, Oliver et al. argue that the object of
investigation is the knowledge-technology-society nexus. (Oliver, Roberts, et al., 2007).
They describe how the study of each of these, knowledge, technology and society, draws
on a rich range of research fields across the Social Sciences; including Instructional
Design, Education, Philosophy and Sociology. Clearly which research fields are drawn on
has implications for both the methodologies used and the theoretical perspectives chosen
by the researchers. It will locate learning design within the broader field of e-learning,
drawing on the findings of a study which looked at the nature of interdisciplinarity in
Technology-Enhanced Learning (TEL) (Conole, Scanlon, Mundin, & Farrow, 2010) and
a Networked Learning conference hotseat on theory and methodology in Networked
Learning (Conole, 2010).
Definitions
Research into the use of technology in an educational context had a long history with
changing labels over the years, each indicating evolving trends in the field and
emphasising different types of foci of inquiry. Commonly used terms include:
Educational Technology, Learning Technology, E-learning, Computer Supported
Collaborative Learning (CSCL) Networked Learning and more recently Technology-
enhanced Learning (TEL). The focus of this chapter is on theories and methodologies
used across these fields. Many books have been written on research methods in Social
Science. Cohen et al. is one of the standard texts for educational research (Cohen,
Manion, & Morrison, 2007). The Research Methods Knowledge Base1 provides covers
the entire research process including: formulating research questions; sampling;
measurement; research design; data analysis; and, writing the research paper. It also
addresses the major theoretical and philosophical underpinnings of research including:
the idea of validity in research; reliability of measures and ethics. The ESRC National
Centre for Research Methods2 provides a comprehensive site for collating research
methods activities across the Social Sciences, along with the latest in innovations in
research methods. Early work carried out by the centre included a review of research
methods and the generation of a typology of research methods (Beissel-Durrant, 2004),
which illustrates the rich variety of research methods being used reflecting the breadth of
different epistemological perspectives in the field.
Oliver et al. (2007) argue that there are a range of different epistemological positions
adopted by researchers in the field and that these have implications for how the field is
researched. They argue that this is often explained in terms of the paradigm debate, and
framed as a contrast between qualitative and quantitative methods; although they go on to
qualify that this is a rather crude distinction; i.e. qualitative data can be interpreted in a
1 http://www.socialresearchmethods.net/kb/
2 http://www.ncrm.ac.uk/
positivist way and quantitative data can be used to yield understandings beyond the
specific numerical data. They argue that:
We need to consider how different philosophical positions would interpret the
kinds of data generated by particular empirical methods. Methodology describes
this relationship, and must be understood separately from methods, which are
the techniques used to collect and analyse data (This will include things like
interviews, questionnaires, observation etc.) Methodology determines whether the
implementation of particular methods is successful or credible. Indeed, according
to Agger (2004, p. 77), methodologies cant solve intellectual problems but are
simply ways of making arguments for what we already know or suspect to be
true.
To do this, methodology codifies beliefs about the world, reflecting out there or
in here positions.
The view that knowledge is hard, objective and tangible will demand of
researchers an observer role, together with an allegiance to methods of natural
science; to see knowledge as personal, subjective and unique, however, imposes
on researchers an involvement with their subjects and a rejection of the ways of
the natural scientist. To subscribe to the former is to be positivist; to the latter,
anti-positivist (Cohen, Manion and Morrison, 2000: 6).
Such commitments and interests arise from historical, cultural and political
influences, which collectively shape traditions of research that provide the context
for current work (Conole, 2003). These have profound implications for the topics
that people study and the kinds of conclusions they are willing to draw (Oliver,
Roberts, et al., 2007, p. 9).
Therefore methods are the techniques used to collect and analyse data, whereas
methodology align with different epistemological beliefs and views of the world. The
term theory is contested and is used in a variety of different ways; here are some
definitions that are the closest to how it is used in an e-learning research context:
Theory, in the scientific sense of the word, is an analytic structure designed to
explain a set of empirical observations. A scientific theory does two things: i) it
identifies this set of distinct observations as a class of phenomena, and ii) makes
assertions about the underlying reality that brings about or affects this class. In the
scientific or empirical tradition, the term theory is reserved for ideas which meet
baseline requirements about the kinds of empirical observations made, the
methods of classification used, and the consistency of the theory in its application
among members of the class to which it pertains. These requirements vary across
different scientific fields of knowledge, but in general theories are expected to be
functional and parsimonious: i.e. a theory should be the simplest possible tool that
can be used to effectively address the given class of phenomena.3
3 http://en.wikipedia.org/wiki/Theory
A set of statements or principles devised to explain a group of facts or
phenomena, especially one that has been repeatedly tested or is widely accepted
and can be used to make predictions about natural phenomena.4
The relationship between theory and empirical data can be defined as follows:
Social research is theoretical, meaning that much of it is concerned with
developing, exploring or testing the theories or ideas that social researchers have
about how the world operates. But it is also empirical, meaning that it is based on
observations and measurements of reality -- on what we perceive of the world
around us. You can even think of most research as a blending of these two terms --
a comparison of our theories about how the world operates with our observations
of its operation.5
4 http://www.thefreedictionary.com/theory
5 http://www.socialresearchmethods.net/kb/naturres.php
6 http://cloudworks.ac.uk/cloud/view/2806
offer? What are the issues? This pragmatic stance is coupled with a desire to
understand and describe emergent theoretical perspectives.
Irrespective of the theoretical and methodological lenses used to study technological
phenomena, it is important to take account of the contextual and in particular the
human dimension where e-learning takes place.
Theoretical perspectives
This section articulates some of the main theoretical perspectives that are evident in e-
learning research. Reviewing the research literature, the following range of theoretical
perspectives are evident: Social Constructivism, Actor Network Theory, Constructivism,
Critical Theory, Action Research, Communities of Practice, Scientific Enquiry,
Conversational Framework, Philosophy of Technology, Anthropological views on tools
artefacts and technology, and Activity Theory. A sample of these is discussed below.
Communities of Practice
Wengers notion of Communities of Practice (Wenger, 1998) has been picked up and used
extensively in the field of e-learning (Breuleux, Laferrire, & Bracewell, 1998; Cousin &
Deepwell, 2005; Guldberg & Pilkington, 2006). Its appeal is probably a combination of
the fact that it is relatively easy concept to grasp and that it offered a means of explaining
some of the more socially situated interactions arising in networked learning.
Methodological approaches
This section describes some of the key methodological approaches used in e-learning.
The choice of methodology tends to reflect both the individuals epistemological stance
and their focus of inquiry. Oliver et al. (2007) argue that the kinds of data that are
available to e-learning researchers may suggest particular kinds of interpretation. This
hints at the suggestion there is a complex inter-relationship between research in the field
and the affordances of the technologies themselves.
It is not possible to provide a comprehensive review of all the different methodological
approaches used in e-learning. Methodologies are predominantly interpretive in nature;
although experimental approaches are still used extensively in North America. In terms of
methods a range are evident interviews, focus groups, observation, surveys, student
journals, video and audio diaries, document analysis, and web tracking. In-depth case
studies are popular, as are large-scale surveys. The use of learning analytics and web
tracking as a means of data collection is still in its infancy but is a growing area of
research; indeed the first international conference on learning analytics was held this year
in Banff.7
Content analysis
Early research in the field was dominated by analysis of asynchronous discussion forums
(Mason and Kay, 1989). Coding schemes such as those developed by Henri and
Gunawadena et al. were used extensively (Gunawardena, Lowe, & Anderson, 1997;
Henri, 1992). Henri identified following five dimensions, which can be used to evaluate
Computer-Mediated Conferences: participative, social, interactive, cognitive and
metacognitive. Garrison et al. (2000) developed a 'community of learning' model, which
assumes that learning occurs through the interaction of three core components: cognitive
presence, teaching presence, and social presence (Arbaugh, et al., 2008; D. R. Garrison,
Anderson, T., and Archer, W. , 2000). Gunawadena et al. divided content into the types of
cognitive activities the participant engaged with (questioning, clarifying, negotiating,
synthesising, etc), the types of arguments they put forward, the resources used and any
evidence of changes in understanding (Gunawardena, et al., 1997). In this early work
arguably there was a nave assumption that focusing on the content in the treaded
messages was enough to capture the whole event. Whereas in reality the level of
detail/object of focus will naturally have a significant impact on results and it was soon
realised that taking account of the broader context within which discussion forums were
7 https://tekri.athabascau.ca/analytics/
taken place was important. Jones for example reports students simulating collaboration
online whilst actually being co-present seated around four computers (Jones, 1999). A
number of approaches have been used to take account of the broader perspective. For
example, De Laat et al. use a multi-method approach using Social Network Analysis with
content analysis and critical event recall (De Laat, Lally, Lipponen, & Simons, 2007; M.
F. De Laat, 2006; M. F. De Laat, Lally, V., Lipponen, L., and Simons, P.R.J., 2006). In
this work, they used Social Network Analysis (Hawthornthwaite, 2002) to visualise the
social structures and dynamics of the course, content analysis is used to identify the
learning and teaching processes and critical event recall is used to elicit teachers
experiences and perceptions.
Ethnography
Ethnography has been used extensively in e-learning (Hodgson & Watland, 2004; Kruger,
2006; Rice-Lively, 1994). The approach is qualitative based on systematic description
of human behaviour and organisational culture based on first-hand observation (Howard,
2002).
Case studies
Rich, situated case studies are a very popular and common form of studying e-learning. A
case study is an in-depth investigation/study of a single individual, group, incident, or
community (Yin, 2009). The nature and scope of the cases can vary significantly and the
approach often overlaps with other methodological approaches (such as action research,
evaluation and ethnography). Critics of the case study approach argue that the findings
are not generalisable or transferable. Proponents argue that the case-based approach
enables the researcher to gather a rich, contextual understanding of a situation in context.
Action research
As might be expected, given the educational nature of e-learning as a research field,
action research is often used as a methodological approach, particularly by practitioners
who are trialling out the use of technologies in their classroom and want a framework
within which to study the interventions (Derntl & Motschnig-Pitrik, 2004; D. R.
Garrison, 2003; Sloman, 2001).
Evaluation
The importance of evaluation has grown in recent years; as new learning technologies
emerge there is a need to evaluate how these are used to support an increasingly diverse
student population. The relationship between evaluation and research more generally
remains contested. Both processes may use the same methods and study the same things.
However, one way to distinguish them is to consider how findings are used. If they are
interpreted by an immediate, local audience and used to support decision-making, the
study was probably an evaluation; if findings are interpreted in terms of theories and are
presented as a contribution to knowledge, it was probably research. Oliver et al. contend
that approaches in evaluation range from positivist approaches focussed upon objective
data collection (typically using quantitative methods) to interpretivist ones more rooted in
constructivism (typically using qualitative methodologies) (Oliver, Harvey, Conole, &
Jones, 2007).
Choosing an appropriate methodology
So which methodology should be used when, and are some methodologies better than
others? Oliver et al. considers how five different methodological approaches (action
research, behaviourist, activity theory-based, and a perspective based on power) are used
to tackle the same research problem (Oliver, Roberts, et al., 2007). This provides a nice
illustration of how different theoretical perspectives would explain this situation
differently, and how each can contribute to our understanding of this field.