DESIGN & DEVELOPMENT

RESEARCH (DDR)
9th January 2021
By: Sensei
 NOR TUTIAINI BT AB WAHID, PhD
Head of Training and Development
I Pey Resources Group of Companies

Doctor of Philosophy (Curriculum and Instruction, UPM)

Certified Human Resource Development Fund (HRDF) Trainer
Certified NeuroLinguistic-Hypnotherapist (NLHt) Practitioner
Certified ONE Therapy Practitioner
Certified Professional Learning Facilitator (Mastery)
nortutiaini@gmail.com
+60192254136
Nor Tutiaini

www.linkedin.com/in/nortutiaini
thesenseistory.blogspot.com
Table of
contents

01 02 03
Introduction Process Focus DDR Phases

04 05 06
Model ID Techniques & Tools Validity & Reliability
 01
Introduction
 Instructional Design

Instructional Technology
Design and
Specific- problems solving oriented Developmental
Research

Educational research mushroomed

Systematic study

(Richey, 1994; Richey & Klein, 2005; Baker, 1973; Seels & Richey, 1994)
Systematic Study?
Richey & Klein, 2007; Seel &
Richey, 2004; Richey, Klein &
Nielson, 2004; Hasan, 2003

Approaches?
Ellis & Levy, 2010; Sahrir, Alias,
Ismail, & Osman, 2012; Brown &

What is DDR?
Collins, 1992

Research Design?
Ghazali & Suffean, 2016 in
Afifi, 2018

Methodology?
Gunaydin & Karamete, 2016
DDR: systematic study that provide systematic
processes for developmental research represent by
three general phases which are analysis, design and
development and evaluation.
DDR can only be implemented in ISD?

“The systematic study of design, development

and evaluation processes with the aim of
establishing an empirical basis for the creation
of instructional and non-instructional products”
一(Richey & Klein, 2007)
 Common Characteristic for DDR

Generalizable or Should address design process

context-specific conclusion and tools

Problem-solving function To establish new product

 Bridging Artifact

一 (Ellis & Levy, 2010; Nunamaker et al.,1991; Richey & Klein, 2007; Hevner et al., 2004)
 02
Process Focus
Type 1 vs Type 2
 Context specific
have conclusions that are exclusively
directed toward the target product

Follow the tenets of ISD

Type 1: Product
encompassing front-end analysis
through formative or summative

Research
evaluation.

Documentation
The entire design or particular phase
of ISD

Production aspects of ISD

describe specifically the authoring
procedures
 General principles
focus on generic use of
development process and have
generalized conclusions

Implications
Type 2: Model
offering implications for any design
or development project and

Research
proposed an enhancement

Documentation
Concentrate upon a particular phase
of ISD or the entire design

Non-traditional
focus on the use of ISD approach to
non-traditional products and settings
 03
DDR PHASES
Structured based on the phases
The Cloud Dreamer
 DESIGN DEVELOPMENT

2 2

DDR
PHASES

1 3

ANALYSIS EVALUATION

(Richey & Klein, 2004)

 04
MODEL ID
Comprehensive processes for
development
 Kemp Model
Advantages Disadvantages
Circular stage Lengthy process

Flexible Time consuming

Open to ongoing revision Non-linear approach

Addressed some Prevents consistency

stages simultaneously or
some stages are not even required
 ASSURE Model
Advantages Disadvantages
Incorporates media in delivery Limited to its purpose only

Easy logical structure Mainly teacher friendly

Learner centered Less emphasis in analysis

step

Evaluation for the

end only
 Dick and Carey Model
Advantages Disadvantages
Provides guidance Non linear
for instruction

emphasis on Too rigid

sequencing
and organizing
content Many steps/
Can be applied stages
Almost in any content
Dick and Reiser Model
 ADDIE Model
Advantages Disadvantages
Generic and structured Vigorous analysis

Considerable Flexibility Time consuming

Straightforward starting Evaluate Too systematic

point

Extremely versatile Non-linear

Types of Development Types of Model
Classroom ID The Gerlach and Ely Model
The Heinich, Molenda, and Russell Model
The Dick and Reiser Model

Product The Kemp Model

The Van Patten Model
The Leshin, Pollock, and Reigeluth Model
The Bergman and Moore Model

System The IDI Model

The IPISD Model
The Dick and Carey Model
The See Is and Glasgow Model
The Diamond Model
(Gustafon, 1991)
 (Nor Tutiaini, 2019)

(Almomen et al., 2016; Kosucu, 2017; Lee et al., 2017; Gagne, Briggs, & Wagner, 1992; Jonassen et al., 1999)
 Module is
developed based
on storyboard.
1 3 • Expert 5
To analyse the validation
needs and main (FDM, DM,
features for PCM, NGT, Reliability
newly ISM) Field Observation
developed • Pilot study*
product

A Analysis D Design D Development I Implementation E Evaluation

Main features
for module is Pilot Study
design in (formative
storyboard. evaluation)

2 4
 Evaluation could
happen
throughout the
Formative Summative Confirmative
phases
Evaluation Evaluation Evaluation
Focus group discussion Usability. Perception. Retesting. Retention
in the end of each Effectiveness test. different type of
phases. Validity, evaluation after
reliability. Pilot study. summative evaluation.
 05
TECHNIQUES & TOOLS
Different techniques and tools
employed in phases
Experimental design
Classical summative evaluation
studies. Performance test. Learners’
survey

Interview
Needs assessment studies.
Structured/semi-structured/unstructured

Techniques & tools

questions

Survey
Descriptive process studies.
Questionnaire. Open-ended
& close ended.

Expert consensus
statistical analysis, measurement
theory, and establishing internal
and external validity.
Questionnaire. Open-ended or
close ended. Narrative.
Case study
Dominant in Type 1 studies. Multiple
sources of data.

Think aloud protocols

Evaluation for the learners.
Observation list.

Techniques & tools

Document analysis
Needs assessment studies.
Document related to the process

Field observation
Evaluation. Observation list.
Designers, developers,
evaluators

Clients, learners,
users
Participants
Instructors, facilitators

Organisation,
researcher, theorist
Experimental design
Statistical analysis. Thematic
analysis. Content analysis.

Interview
Thematic analysis. Content analysis.
Conversational analysis. Discourse

Data Analysis
analysis. Narrative analysis

Survey
Statistical analysis. Content
analysis. Descriptive analysis

Expert consensus
statistical analysis. PCM. FDM. DM.
CVI. NGT. Narrative analysis.
Case study
Content analysis. Descriptive analysis.
Thematic analysis. Narrative analysis.

Think aloud protocols

Content analysis. Thematic
analysis. Narrative analysis

Data Analysis
Document analysis
Content analysis. Thematic analysis.
Descriptive analysis.

Field observation
Descriptive analysis. Narrative
analysis. Thematic analysis.
 06
Validity and Reliability
To ensure the proposed
product would be appropriate
for the target population.

Changes were made to improve the

syntax, sentence structure, and Validity
nomenclature of the product

able to measure things

they supposed to measure
The Experts
Two types: Professional and Field
Criteria: Experiences, Knowledge,
Recognition, Expertise
At least three experts from each type
Needed (Rubio et al.)
Types of Validation
Content Validation: content, suitability of
Activities, language
Face validation: typing error, questions
Structure, sentence structure, grammar
and tenses
Construct validation: the items used in
questionnaire able to measure skills or abilities
They supposed to measure
PCM

Percentage Calculation Method (PCM) by Tuckman and Waheed (1981) in Sidek and
Jamaluddin (2005). Acceptable value > 0.70 or 70%

(Total expert score (x)/Total maximum score) X (100%) = Content validity level
CVI

Content Validity Index (CVI) Lynn (1986).

Acceptable value > 0.70 or 70%
CVI= n/N
n= numbers of evaluator agreed
N= Sum of evaluator

Divide the ordinal scale into two groups for example:

1 and 2 = not agreed (give value 0)
3 and 4 = agreed (give value 1)

Experts 1 2 3 CVI
Item 1 1 1 0 2/3= 0.67
Item 2 1 0 0 1/3= 0.33
Item 3 1 1 1 3/3= 1.00
To ensure that the product is
able to produce consistency
when applied to a different group

Calculated using Cronbach’s Alpha

statistical analysis. Reliability

Coefficient value should be

at a minimum value of at
least 0.7.
Conclusion
 Integrating
Developmental research methodologies facilitate the study of new models, tools, and procedures so that
researcher can reliably anticipate their effectiveness and efficiency and, at the same time address the pressing
problems of this field.
innovative lines of developmental
research have extended the
boundaries of the more traditional
orientation to developmental
research to current practice
 Thanks!
Does anyone have any questions?

nortutiaini@gmail.com
+60 192254136
thesensei