LR, Methods and

Evaluation in Multimedia
Literature Review
• A literature review is a text of a scholarly paper, which
includes the current knowledge including substantive
findings, as well as theoretical and methodological
contributions to a particular topic.
• Literature reviews are secondary sources, and do not report
new or original experimental work.
 Literature Review
• A literature review has four main objectives:
• It surveys the literature in your chosen area of study
• It synthesizes the information in that literature into a
summary
• It critically analyses the information gathered by
identifying gaps in current knowledge; by showing
limitations of theories and points of view; and by
formulating areas for further research and reviewing
areas of controversy
• It presents the literature in an organised way
The Literature Review
• Literature reviews use secondary sources, and do not report new or
original experimental work.
• Most often associated with academic-oriented literature, such as
a thesis, dissertation or peer-reviewed journal article, a literature review
usually precedes the methodology and results section.
• Literature reviews are also common in a research proposal or prospectus
(the document that is approved before a student formally begins a
dissertation or thesis). Its main goals are to situate the current study
within the body of literature and to provide context for the particular
reader. Literature reviews are a staple for research in nearly every
academic field.
Literature Review
• A literature review is a select analysis of existing research which is relevant
to your topic, showing how it relates to your investigation. It explains and
justifies how your investigation may help answer some of the questions or
gaps in this area of research.
• A literature review is not a straightforward summary of everything you
have read on the topic and it is not a chronological description of what was
discovered in your field.
• A longer literature review may have headings to help group the relevant
research into themes or topics. This gives a focus to your analysis, as you
can group similar studies together and compare and contrast their
approaches, any weaknesses or strengths in their methods, and their
findings.
Literature Review

• One common way to approach a literature review is to start out broad and then
become more specific. Think of it as an inverted triangle.
• First briefly explain the broad issues related to your investigation; you don't need
to write much about this, just demonstrate that you are aware of the breadth of
your subject.
• Then narrow your focus to deal with the studies that overlap with your research.
• Finally, hone in on any research which is directly related to your specific
investigation. Proportionally you spend most time discussing those studies which
have most direct relevance to your research.
Search for Sources
• It's easy to think that the best way to search for texts is to use the Internet - to
'Google it'. There are useful online tools that you may use, like Google Scholar.
However, for most literature reviews you will need to focus on academically
authoritative texts like academic books, journals, research reports,
government publications. Searching Google will give you thousands of hits,
few of them authoritative, and you will waste time sorting through them.

• A better idea is to use databases. These are available through the Library in
paper and electronic (usually online) forms.
Search for Sources
• Use journal articles: They normally have the most up-to-date research and
you will be expected to refer to them in your literature review.
• The Library will have a liaison librarian for your topic (Get ahead with help
from your liaison librarian!) and guides to finding information in your
subject. They also have advice on how to develop your research skills.
• You may find review articles that survey developments in your field. These
are very useful for identifying relevant sources - but do go back to the
original texts and develop your own critical analysis if possible.
• Another good way to find sources is to look at the reference lists in articles
or books already identified as relevant to your topic. You will be expected to
prioritise recent research, but it's also important to acknowledge the
standard texts in your field. An easy way to identify these is to check
reference lists to see which texts are frequently cited.
Literature Review Guidelines
• Step 1: Review APA guidelines
Read through the links provided below on APA guidelines so that you become
familiar with the common core elements of how to write in APA style: in
particular, pay attention to general document guidelines (e.g. font, margins,
spacing), title page, abstract, body, text citations, quotations.
• Step 2: Decide on a topic
It will help you considerably if your topic for your literature review is the one
on which you intend to do your final M.Ed. project, or is in some way related to
the topic of your final project. However, you may pick any scholarly topic.
Literature Review Guidelines
• Step 3: Identify the literature that you will review:
• Familiarize yourself with online databases 
• Using relevant databases
• Import your references into your Referencing Management Software
(EndNote, RefWorks, Mendeley, Zotero…)
• Step 4: Analyze the literature
• Overview the articles
• Group the articles into categories: Decide on the format, Define key terms,
Note key statistics, Select useful quotes that you may want to include in your
review, Note emphases, strengths & weaknesses, Identify major trends or
patterns, Identify gaps in the literature, and reflect on why these might exist
etc..
 Literature Review Guidelines
• Step 5: Summarize the literature in table or concept map format
Paper Title (year) Author(s) Method(s) used Results / Finding Remarks/Notes

Example:        
         
Investigation of anti- Joven Mayer, Owen Design: pre–post design Positive behaviour change Investigation of using
smoking behaviour Smith and Victoria Briss Sample: 927 healthy adults outcomes were observed in SMS to change smoking
(2007)   Setting: Korean public health 13 of the 14 reviewed studies. behaviour.
  clinics Intervention initiation
Duration: 12 weeks (researcher or participant),
  SMS
dialogue initiation, tailoring of
SMS content, and interactivity
were found to be important
features of SMS-delivered
interventions. Therefore -
SMS-delivered interventions
have positive short-term
behavioural outcomes.
Literature Review Guidelines
• Step 6: Synthesize the literature prior to writing your review
• Create a topic outline that traces your argument: first explain to the reader
your line or argument (or thesis); then your narrative that follows should
explain and justify your line of argument
• Step 7: Writing the review
• Identify the broad problem area, but avoid global statements
• Early in the review, indicate why the topic being reviewed is important
• Distinguish between research finding and other sources of information
• Indicate why certain studies are important
Literature Review Guidelines
• Step 8: Developing a coherent essay
• If your review is long, provide an overview near the beginning of the review
• Near the beginning of a review, state explicitly what will and will not be
covered
• Specify your point of view early in the review: this serves as the thesis
statement of the review.
• Aim for a clear and cohesive essay that integrates the key details of the
literature and communicates your point of view
Type of research methodology
Qualitative
This type of research methods involves describing in details specific
situation using research tools like interviews, surveys, and Observations. 

Quantitative
This type of research methods requires quantifiable data involving
numerical and statistical explanations.

Correlation/Regression Analysis
This research method involves determining the strength of the relationship
between two or more variables (e.g. are violent video games correlated
with aggression in children).
Type of research methodology
Quasi-Experimental
This research involves the comparison of two groups, one which is influenced by an
external source and another which is not.

Experimental
Use of random assignment to place participants in two groups: an experimental group
which receives intervention, and another control group without any intervention. It is
using a positive control for you to base it or compare it in your result. 

Meta-Analysis
This research method is useful for finding out the average impact of several different
studies on a hypothesis.
Research Classification
Research may be classified as:

- Basic Research
- Applied Research
Basic Research
• Basic Research also be called
• Fundamental Research
• Pure Research
• primary objectives,
• The advancement of knowledge
• The theoretical understanding of the relation among variables.
• Basic Research is,
• Exploratory
• Driven by researcher
 Curiosity
 Interest
 Intuition
• conducted without any practical end in mind.
• Basic Research may have unexpected practical applications.
• Basic Research may become the foundation for Applied Research.
Applied Research
• Applied Research has the primary aim of:
• Discovering
• Interpreting
• Development of methods and systems for advancement of
human knowledge on wide variety of scientific matters of
the world and the universe.
Formal Research
• Most formal research, either basic or applied, may follow the
steps below:
• Formation of the topic
• Hypothesis
• Conceptual definitions
• Operational definitions
• Gathering of data
• Analysis of data
• Test, revising of hypothesis (if necessary)
• Conclusion or repeat all the steps if necessary
 Common Research Process
Chap 1 - Introduction
Cyclic and dynamic iteration

FORMULATION PHASE
Chap 2 - Literature review

Chap 3 – Methodology DESIGN AND EXECUTION PHASE

Chap 4 - Data analysis

ANALYTICAL PHASE
Chap 5- Discussion/ Conclusion

References

Appendix
Formulation Phase
Confirm need – Review literature - Define the problem – Develop
questions and objectives – specify hypotheses – Develop
theoretical model
Chapter 2 literature review
Chapter 1 introduction 1.1 Introduction
1.1 Introduction 1.2 Definition of the terms (concepts)
1.2 Problem background 1.3 Logical flow of ideas
1.3 Problem statement 1.4 Unbiased and comprehensive view
1.4 Research questions of the previous research on the topic.
1.5 Objective 1.5 Identify areas of controversy in the
1.6 Scope literature
1.7 Significant of the study 1.6 Identify gaps in the literature
1.7 Development of the theoretical framework
The Relationship between Problem
statement, Research Question and
Hypotheses
Problem statement: a clear concise description of the issues
that need to be created addressed and presented by the
researcher.
Lead to formulate
research questions

Research question: a statement that identifies the

phenomenon to be studied. It must define the domain, the
variables, and its relationship.
Lead to formulate
hypotheses

Hypotheses: a specific statement of prediction. It

describes in concrete (rather than theoretical) terms what
you expect will happen in your study.
Research Objective
Can be divided into:
• Main Objectives
• Specific Objectives: Normally stated as
1. To identify ….
2. To develop …
3. To measure ….
4. To determine ….
Hyphothesis
It should be noted that:
• A hypothesis is established to make prediction.
• Example:
• Hypothesis: Use of Learning Objects can improve students’ achievement
in mathematics in schools.
• Objective: To propose Learning Objects to……
• A hypothesis is tested by observing the outcome of experiment or survey.
• If the outcome is not consistent with the hypothesis, the hypothesis is
rejected.
• If the outcome is consistent with the hypothesis, the hypothesis is accepted
(the hypothesis is not proven but is supported by the experiment or survey).
Research Design & execution phase
Research design is based on:
• How the results will be achieved.
• How data will be collected.
• How data will be analyzed.
 Research Design & execution phase
• Quantitative
• Quantitative Data Collection
 Survey (questionnaire)
 Experimental
• Data analysis
 Quantitative data analysis (objective)
 Quantitative data analysis often
contain descriptive statistics and
inferential statistics.
 Descriptive statistics include
measures of central tendency.
 Inferential statistics is a set of
measurements can be regarded as
measurements on a sample of items
from a population Thus, it make
inferences about the population from
the sample.
• Qualitative
• Qualitative Data Collection
 Interview / Documents /
Observation
• Data analysis
 Qualitative data analysis (subjective)
 Qualitative data analysis describes
and summarizes the mass of words
generated by interviews or
observational data. It allows
researchers to seek relationships
between various themes that have
been identified.
 Qualitative data analysis often
contain content analysis that leads to
emerge of themes and patterns.
Analytical Phase
Discussion of findings
 Discussion in relating to literature.
 Discussion in relating to research objectives.
 Discussion in relating to research questions.
Conclusion
 Conclusion is often the most difficult part to write.
 Conclusion is often what a reader remembers best.
Elements of conclusion
 Answer the question "So what?" (important of your thesis).
 Synthesize, don't summarize (Don't simply repeat things that were in
your thesis).
 Create a new meaning (Create a new picture).
Evaluation Methods
• Cognitive Walkthrough
• Formative Evaluation
• Heuristic or Guidelines-Based Expert Evaluation
• Post-hoc Questionnaire
• Interview/Demo
• Summative or Comparative Evaluation