Instrumentasi PembentuknKesahanKebolehpercayaan

Instrumentasi:
Pembentukan Instrumen,
Kesahan dan
Kebolehpercayaan
Instrumen
Powerpoint Templates Mengilham harapan, mencipta masa depan

Inspiring Futrures, Inspiring Possibilities
Main Steps in Quantitative
Research:
1. Theory
2. Hypothesis
3. Research design
4. Devise measures of concepts
5. Select research site(s)
6. Select research subjects/respondents
7. Administer research instruments/ collect data
8. Process data
9. Analyse data
10. Write up findings and conclusions
Powerpoint Templates
Examples of Quantitative Research Methods:
Experiments
Social surveys
Cross-sectional
Comparative (cross-national)
Longitudinal
Content Analysis
Secondary Statistical Analysis
Official Statistics
Demography
Epidemiology
Field stimulations
Structured Interviews and Observation

Terms survey and questionnaire are
often used interchangeably
Survey refers to a descriptive
research technique or methodology
(also includes interviews, and focus
groups)
Questionnaire refers to a paper and
pencil instrument through which
information is obtained

Useful for surveying groups of any size
Typically designed to determine

opinions, attitudes or present practices
Limitation: rely on self report

information v. observed behavior
Challenges to validity become a

concern

INITIAL CONSIDERATIONS
Is the development of a new questionnaire

...
Necessary?
Feasible?
Select mode of administration
Web Based?
Paper and pencil?
Combination?
QUESTIONNAIRE PLANNING
1. Determine the Objective
What do you want to ask and of whom?
Consider how the data will be analyzed. What information is
needed?
2. Determine the Mode of Administration
Paper and pencil
Cost effective, but low response rate
Web / E-mail based
Cost effective, higher response rate, some suspicion of
the medium
Combination
Can be administratively cumbersome
QUESTIONNAIRE PLANNING
Determine the Sampling Methodology

To whom will you send the questionnaire? Who can supply
the information?
Will everyone be surveyed?
ALL current students? ALL alumni?
If yes, this is a census of the finite population
The selection of a representative sample of the population
may be more effective based on time and cost of
administration
Decision depends on the size of the finite population
CONSTRUCTING the
QUESTIONNAIRE
Consider what objective each question measures
Three to five questions per objective is recommended
How many questions?
Find a balance: long enough to obtain the necessary
information but short enough so respondents wont lose
interest.
Questionnaires that are too long wont be completed
Goal: 25-30 minutes MAX
DESIGNING THE
QUESTIONNAIRE
Question Formats: Open Ended
Questions
Allow respondents to express feelings and expand on ideas.
Question should be phrased to avoid a one word response.
What aspects of the Sociology program best prepared
you for your current position?
Open ended questions at the beginning of the questionnaire
may be discouraging to respondent include one or two at
the end
Analysis of responses can be challenging
QUESTIONNAIRE
DEVELOPMENT
Question Formats: Closed Questions
Ranking
Forces the respondent to rank order responses based on a
value judgment.
Rank the following items with regard to how you spend time at
Bucknell from 1 (spend the most time) to 5 (spend the least
time).
_____ Studying
_____ Socializing
_____ Volunteering
_____ Working at a part time job
_____ Exercising
QUESTIONNAIRE
DEVELOPMENT
Checklist
A number of possible answers are provided and the
respondent is asked to choose one or choose all that apply.
How did you finance your Bucknell University education?
Choose all that apply.
_____ Parental contribution
_____ Student Loan
_____ Work Study
_____ Off campus employment
_____ Other (please specify): _____________
QUESTIONNAIRE
DEVELOPMENT
Scaled items
Very common
Indicate strength of agreement or disagreement with a
statement.
Numbers are assigned to each response on the continuum
The English curriculum at Bucknell University adequately
prepared me for my current position.
1 y
2 e 3 / No 4 5 gly
ngl gre t ral on ree o n
ro
St gree Di s a u
Ne pin i Ag Str ree
a O Ag
Dis
QUESTIONNAIRE
DEVELOPMENT
Suggestions for writing scaled items
Use positively worded or negatively worded items
consistently
Use an odd number of responses along the continuum to
allow for a neutral response
Anchors may be different based on focus of the item
(Strongly Agree Strongly Disagree; Frequently Never;
Very Good Very Poor)
Keep items short
QUESTIONNAIRE
DEVELOPMENT
Question Formats: Categorical
Typical demographic questions:
Yes/No/?, Male/Female, F/S/J/S/G
Answers fall into certain categories respondents pick the one
that applies to them
Analyzed using frequency, proportion, percentages
2 considerations:
Questions on a continuum should stay on a continuum if they
will be directly analyzed
Example: Age allow respondent to write in their age
rather than check off a box for their age range
Categories must be collectively exhaustive & mutually
exclusive (next slide)
QUESTIONNAIRE
DEVELOPMENT
Problematic: Corrected:
Number of hours spent Number of hours spent
studying per week: studying per week:
18 21 18 21
21 24 22 25
24 27 26 29
Categories are not
mutually exclusive
IMPORTANT
CONSIDERATIONS
APPEARANCE AND DESIGN
Have clear directions that are age appropriate & easy to
follow
Format, size, reproduction should facilitate completion
No less than 12 pt
Times, Arial, Courier
PILOT STUDY
Give the questionnaire to a few people to try it out
Have them evaluate readability and clarity of questions
Estimate time required to complete the questionnaire
Suggestions for revisions
SENDING IT OUT
INITIAL CONTACT / COVER LETTER

Short & to the point
Identify person by name
Convince respondent that participation is
important
Ensure confidentiality
Deadline for return of questionnaire
SASE for return (mailed questionnaires only)
FINAL THOUGHTS
FOLLOW UP PROCEDURES
Send to all participants to preserve confidentiality (If you
havent already returned the questionnaire..)
May include a second copy if available
Usually done in 2 week intervals
RESPONSE RATE FOR MAILED QUESTIONNAIRES
Typically low - 30% is considered good
Higher response rate with web-based questionnaires
Good follow-up increases response rate
SUMMARY OF STEPS
1. Determine the Objective (consider analyses)
2. Determine the Mode of Administration
3. Determine the Sampling Methodology
4. Construct the Questionnaire
5. Institutional Approval
6. Conduct the Pilot Study
7. Write the Initial Communication
8. Send the Questionnaire
9. Follow up
10. Analyze the Results
DEVELOPING QUALITATIVE
INSTRUMENT

Main Steps in Qualitative
Research:
1. General research question
2. Select relevant site(s) and subjects
3. Collection of relevant data
4. Interpretation of data
5. Conceptual and theoretical work
6. Tighter specification of the research question
7. Collection of further data
8. Conceptual and theoretical work
9. Write up findings
Examples of Qualitative Research
In-depth Interviews
Focus Groups
Ethnography/Field Research
Historical-Comparative Research
Discourse Analysis
Narrative Analysis
Media Analysis

RELIABILITY & VALIDITY OF
INSTRUMENT - QUANTITATIVE
INSTRUMENT

Criteria of Measurement Quality
How do we judge the relative success (or

failure) in measuring various concepts?
Reliability consistency of measurement
Validity confidence in measures and design
Reliability and Validity
Reliability focuses on measurement

Validity also extends to:
Precision in the design of the study ability to
isolate causal agents while controlling other
factors
(Internal Validity)
Ability to generalized from the unique and
idiosyncratic settings, procedures and participants
to other populations and conditions
(External Validity)
Reliability
Consistency of Measurement
Reproducibility over time
Consistency between different
coders/observers
Consistency among multiple indicators
Estimates of Reliability
Statistical coefficients that tell use how
consistently we measured something
Measurement Validity
Are we really measuring concept we defined?
Is it a valid way to measure the concept?
Many different approaches to validation
Judgmental as well as empirical aspects
Key to Reliability and Validity
Concept explication
Thorough meaning analysis
Conceptual definition:
Defining what a concept means
Operational definition:
Spelling out how we are going to measure
concept
Four Aspects of Reliability:
1. Stability
2. Reproducibility
3. Homogeneity
4. Accuracy
1. Stability
Consistency across time

repeating a measure at a later time to
examine the consistency
Compare time 1 and time 2
2. Reproducibility
Consistency between observers

Equivalent application of measuring
device
Do observers reach the same conclusion?
If we dont get the same results, what are we
measuring?
Lack of reliability can compromise validity
3. Homogeneity
Consistency between different measures

of the same concept
Different items used to tap a given concept
show similar results ex. open-ended and
closed-ended questions
4. Accuracy
Lack of mistakes in measurement

Increased by clear, defined procedures
Reduce complications that lead to errors
Observers must have sufficient:
Training
Motivation
Concentration
Increasing Reliability
General:
Training coders/interviewers/lab personnel
More careful concept explication (definitions)
Specification of procedures/rules
Reduce subjectivity (room for interpretation)
Survey measurement:
Increase the number of items in scale
Weeding out bad items from item pool
Content analysis coding:
Improve definition of content categories
Eliminate bad coders
Indicators of Reliability
Test-retest
Make measurements more than once and see if they
yield the same result
Split-half
If you have multiple measures of a concept, split
items into two scales, which should then be correlated
Cronbachs Alpha or Mean Item-total Correlation
Konsistensi Dalaman
Satu ujian diambil hanya sekali
Mengukur sejauh mana item dalam satu ujian
adalah konsisten antara satu sama lain dan
dengan keseluruhannya
Terdiri daripada tiga jenis:
1. Kebolehpercayaan belah dua (split-half reliability)
2. Kebolehpercayaan Kuder-Richardson20
3. Kebolehpercayaan Cronbachs alpha
Kebolehpercayaan Split-
Half
Melibatkan pembahagian satu ujiankepada dua sub ujian
Setiap ujian mengandungi sebahagian()daripada
bilangan item yang asal
Tujuan membahagikan kepada dua subujian ialah untuk
mendapatkan dua ujian yang selari mungkin
Kaedah ini sesuai untuk satu ujian yang panjang dan
mengambil masa yang lama atau susah untuk diuji
Prosedur
Jalankan keseluruhan ujian terhadap satu kumpulan
Bahagikan ujian tersebut kepada duabentuk yang berbeza
dengan memilih item ganjil atau item genap
Kirakan setiap skor pelajar bagi kedua-dua bahagian
Hitungkan pekali korelasi bagi kedua-dua skor
Aplikasikan formula pembetulan Spearman-Brown
Menilai keputusan Powerpoint Templates
SPLIT-HALF RELIABILITY
(Spearman-Brown)
Reliability = 2r
1 r
r = the actual correlation between the two halves
of the instrument (e.g. 0.85);
2 ( 0.85 ) 1.70
Reliability = = = 0.919
1 0.85 185
Kuder-Richardson 20 (KR
20)
Rumus tersebut dibuat berdasarkan penskoran
secara dikotomi (betul-salah) seperti dalam ujian
objektif pelbagai pilihan.
Formula KR 21 lebih mudah dihitung dan
mengurangkan masa pengiraan tetapi kurang tepat
berbanding KR 20.
Kuder-Richardson 20 (KR 20)
Kebolehpercayaan Cronbachs
alpha
Digunakan apabila ujian mempunyai skor
yang lebih daripada dua pilihan
Menganalisis konsisten dalaman jika
nombor digunakan untuk menunjukkan
pilihan respon
Pekali alpha yang kerap kali digunakan
dalam menganggarkan kebolehpercayaan
ketekalan-dalaman
Kebolehpercayaan Skorer
(Scorer Reliability)
Mengkaji individu yang mendapat skor untuk ujian yang
dijalankan
Interjudge- merujuk kepada dua atau lebih individu
yangmendapat skor yang konsisten
Intrajudge- merujuk kepada individu yang mendapat skor
yang konsisten selepas satu jangka masa
Penting apabila penskoran berpotensi sebagai subjektif
Penting untuk menentukan kebolehpercayaan seseorang
individu yang menskor ujian terbuka (tidak terhad)(open-
ended tests)
Kebolehpercayaan penskor penting untuk ditentukan
apabila perlakuan ke atas ujian mempunyai kesan yang
serius terhadap pengambil ujian
Reliability and Validity
Reliability is a necessary condition for

validity
If it is not reliable it cannot be valid
Reliability is NOT a sufficient condition for
validity
If it is reliable it may not necessarily be valid
Example:
Bathroom scale, old springs
Not Reliable or Valid
Reliable but not Valid
Reliable and Valid
Types of Validity
1. Face validity
2. Content validity
3. Pragmatic (criterion) validity
A. Concurrent validity
B. Predictive validity
4. Construct validity
A. Testing of hypotheses
B. Convergent validity
C. Discriminant validity
Face Validity
Subjective judgment of experts about:

whats there
Do the measures make sense?
Compare each item to conceptual definition
Do it represent the concept in question?
If not, it should be dropped
Is the measure valid on its face
Content Validity
Subjective judgment of experts about:
what is not there
Start with conceptual definition of each
dimension:
Is it represented by indicators at the
operational level?
Are some over or underrepresented?
If current indicators are insufficient:
develop and add more indicators
Example--Civic Participation questions:
Did you vote in the last election?
Do you belong to any civic groups?
Have you ever attended a city council
meeting?
What about protest participation or online
organizing?
Pragmatic Validity
Empirical evidence used to test validity

Compare measure to other indicators
1. Concurrent validity
Does a measure predict simultaneous
criterion?
Validating new measure by comparing to existing measure
E.g., Does new intelligence test correlate with established test
2. Predictive validity
Does a measure predict future criterion?
E.g., SAT scores: Do they predict college GPA?
Construct Validity
Encompasses other elements of validity

Do measurements:
A. Represent all dimensions of the concept
B. Distinguish concept from other similar concepts
Tied to meaning analysis of the concept
Specifies the dimensions and indicators to be
tested
Assessing construct validity
A. Testing hypotheses
B. Convergent validity
C. Discriminant validity
A. Testing Hypotheses
When measurements are put into practice:

Are hypotheses that are theoretically derived,
supported by observations?
If not, there is a problem with:
A. Theory
B. Research design (internal validity)
C. Measurement (construct validity?)
In seeking to examine construct validity:
Examine theoretical linkages of the concept to
others
Must identify antecedent and consequences
What leads to the concept?
What are the effects of the concept?
B. Convergent Validity
Measuring a concept with different methods

If different methods yield the same results:
than convergent validity is supported
E.g., Survey items measuring Participation:
Voting
Donating to money to candidates
Signing petitions
Writing letters to the editor
Civic group memberships
Volunteer activities
C. Discriminant (Divergent) Validity
Measuring a concept to discriminate that

concept from other closely related concepts
E.g., Measuring Maternalism and Paternalism
as distinct concepts
Dimensions of Validity for
Research Design
Internal
Validity of research design
Validity of sampling, measurement,
procedures
External
Given the research design, how valid are
Inferences made from the conclusions
Implications for real world
Internal and External Validity in
Experimental Design
Internal validity:
Did the experimental treatment make a
difference?
Or is there an internal design flaw that invalidates the results?
External validity:
Are the results generalizable?
Generalizable to:
What populations?
What situations?
Without internal validity, there is no external validity

RELIABILITY & VALIDITY
OF
INSTRUMENT -
QUALITATIVE
INSTRUMENT
Core Qualitative Methods
Semi- or Un-structured, Open-Ended:
In-depth Interviews (in the field, face-to-face)

Participant Observation (field/site visits)
Archival Research (document review and analysis)
Guiding Principles
Qualitative research designs consider ways to foster:
Reflexivity (an ongoing process of reflecting on the researchers subjective experience,
ways to broaden and enhance this source of knowing, & examining how it informs research)
Iteration (a spiraling process: sequential and repetitive steps in examining preliminary

findings for the purposes of guiding additional data collection and analysis)
Intersubjectivity (a process of reaching a shared/ objective agreement about how to
assign meaning to a social experience - with insiders and outsiders)
The Iterative Process of Qualitative
Research: A Model
Analysis
Data Collection
Reflection
Qualitative Research Techniques
Instrumentation:
Key Informants (question development and piloting of instrument)

Unstructured to Semi-structured
Probing
Data Processing and Analytic Tools
Qualitative Research Techniques/
Considerations
Sampling
Single v. Multiple Cases (not an individual)

Expert and Key Informants (identification and recruitment of sample)
Roles of the Researcher (identification and recruitment of sample)
Data Collection
Participants as Data Collectors

Field Notes (personal reflections, observations, emerging concepts/theories)
Debriefing (a participant, a participating researcher, a non-participating researcher)
Analysis
Key Informant Feedback
Codebooks (specifies definitions and relationships of concepts and terms)
Memos (emerging patterns, concepts; documentation of analytic pathways)
Case Analysis Meeting (a meeting of a research team for the purposes of reflecting on analytic process,
tools, and findings)
Matrices or Diagrams (to identify and examine time sequencing, the structure of relationships, conditions of
cross case events)
Analysis
Key Informant Feedback
Codebooks (specifies definitions and relationships of concepts and terms)
Memos (emerging patterns, concepts; documentation of analytic pathways)
Case Analysis Meeting (a meeting of a research team for the purposes of reflecting on analytic process,
tools, and findings)
Matrices or Diagrams (to identify and examine time sequencing, the structure of relationships, conditions of
cross case events)
Three Qualitative Methods:
Ethnography
Focus: study human behaviour in the
cultural context in which it is embedded
Ethnography is the work of describing a
culture the way of life of a cultural group
Associated with Cultural Anthropology
Example: David Counts and Dorothy Ayer
Counts: An Ethnography of RVing
Seniors
Grounded Theory
Focus: develop a theory to explain underlying
social processes of a cultural group
Useful in areas where little is known or when a
new perspective is needed
Used for exploratory, descriptive studies
Because the theory emerges from the data, it is
said to be grounded in the data
Foundation in Symbolic Interactionism
Example: Kerry Daly and the social
construction of fatherhood
Phenomenology
Focus: reveal the meaning of the lived experience from
the perspective of participants
Describe the essences of lived experience
Essences: elements related to the true meaning of
something that gives common understanding to the
phenomenon under study
Conveyed with descriptive language
Drawn from Philosophy; used across disciplines
Example: J.E. Solchany: A phenomenological study of
womens preadoptive experiences
Data Collection Methods
in Qualitative Studies
Three data collection strategies
introduced:
1. Participant observation
2. In-depth interviews
3. Focus group interviews
Qualitative researchers may combine

more than one method
Data Collection Methods:
1. Participant Observation
Intensive, usually long term, examination of a
social group, an organization, etc.
Researcher becomes a participant in the
lives of group members
Observes their behaviour and learns meaning
systems (which are tied to language)
Most closely associated with Ethnography, as
developed in Classical Anthropology
Now done in a variety of disciplines
1. Participant Observation (contd)
Many classic participant observations studies

E.g., Asylums (Erving Goffman), Tallys
Corner (Elliot Liebow), Street Corner
Society (William F. Whyte), etc.
Today most ethnographers take an overt role
I.e., their identity as a researcher is known
to the people being studied
Covert participation (i.e., identity concealed
from participants) is fraught with ethical issues
e.g., Humphreys Tearoom Trade
Steps Involved in
Participant Observation Research
A. Gaining entry into the group
B. Developing and maintaining rapport
C. Developing a method for taking field
notes
D. Integrating data collection and data
analysis
Steps in Participant Observation:
A. Gaining Entry into the Group
Take into consideration the type of group
formal organizations require formal entry;
involves letter writing, permission requests,
etc.
Informal groups different strategy needed
Access may be gained through a gatekeeper
(an individual with special status)
Want to involve key informants (those who are
most knowledgeable about the group)
B. Developing/Maintaining Rapport
Researcher must work hard to develop
and maintain good relationships in the
field
E.g., be sure not to become associated
with one faction in a group or organization
Researcher could be blamed for
problems that arise in the setting
C. Strategies for Taking Field Notes
Field notes integral to participant observation
Include descriptions and interpretations of individuals,
interactions, and events
Distinguish descriptions from interpretations
Record time and location of observations, as well as
key information (weather, events happening and their
significance)
Keep theoretical memos which are the tentative
interpretations emerging and being assessed through
further data collection
Field Notes (contd)
May not be possible or advisable to take

notes while in the field
Important that they be done as soon after
field observation as possible
Note-taking is time-consuming because
it is integral to guiding the data
collection and continuing the analysis
E.g., field notes for When Prophecy Failed
were well over 1,000 typed pages
D. Integrating Data Collection and
Analysis
Organizing field notes into different types of files
facilitates data analysis
Master field file complete journal of field notes;
number pages and include entry dates
Background, history file subfile organizing
background material
Key character files subfiles on key players in
the group or organization
Analytic files subfiles for different types of
observations or relationships
2. In-depth Interviews
Some studies cannot employ the participant
observation method
E.g., Desrochess study of bank robbers
In-depth interviews allow participants to describe
their experiences and the meaning of events
taking place in their lives
Verbatim quotes capture the language and
meaning expressed by participants
Interviews are flexible and allow for probing
Interview method is quite diverse, adaptive
2. In-depth Interviews (contd)
Spradley three key elements for the interview

method to be successful
1. Explicit purpose researcher and informant are
aware that the discussion has a purpose
2. Ethnographic explanations researcher tries out
explanations on the participants to see if they
make sense
Encourage the informants to use colloquial
language, and teach the researcher its
meaning
2. In-depth Interviews (contd)
3. Ethnographic questions include:

i. Descriptive questions ask participants to
describe their experiences (e.g., their ideas,
circumstances, viewpoints, dilemmas, etc)
ii. Structural questions ask participants how they
organize their world (e.g., activities)
iii. Contrast questions ask participants what is
meant by specific terminology
Prus & Grills stress the value of the interview method
in a multi-method approach
3. Focus Group Interviews
Interview format, but in a group setting
6-12 participants with common experience
Dates back to the 1940s used to assess
effectiveness of morale-boosting radio shows
1970s onward used by market researchers
1980s onward used by academics
Transcript of discussion is the data
Plus accompanying notes
Use content analysis or grounded theory
approach to analyze the data
3. Focus Group Interviews (contd)
Strengths:
Open-ended question
Spontaneously deal with issues as they arise
Cost-effective method of collecting data
Less time-consuming
Weaknesses:
One or two participants may dominate
Not done in a natural setting, so little observation to
help understand the experience of the participants
Implications for Handling Threats
to Validity and Reliability
In quantitative research, threats to
validity are addressed by prior design
features (such as randomization and
controls)
Implications for Handling Threats
to Validity and Reliability
In qualitative research, such prior elimination of threats to
validity is less possible because:
qualitative research is more inductive, and
it focuses primarily on understanding particulars rather than

generalizing to universals.
Qualitative researchers view threats as an opportunity for

learning
- e.g. researcher effects and bias are part of the story that is told;
they are not controlled for
Validity
In general, validity concerns the degree

to which an account is accurate or
truthful
In qualitative research, validity concerns

the degree to which a finding is judged
to have been interpreted in a correct way
Assessing the Validity of
Qualitative Research
Can another research read your field (and other types of) notes
(i.e., the explication of your logic) and come to the same
understandings of a given phenomenon?
Concern about validity (as well as reliability) is the primary
reason thick description is an essential component of the
qualitative research enterprise
Resources:
Handout: Different Types of Notes
Example: ACY Site Visit Toolkit
Major Types of Validity in
Descriptive Validity
Interpretive Validity
Theoretical Validity
External Validity (i.e., generalizability)
Concerned with the factual accuracy of an account

(that is, making sure one is not making up or
distorting the things one hears and sees)
All subsequent types of validity are dependent on
the existence of this fundamental aspect of validity
Behavior must be attended to, and with

some exactness, because it is through
the flow of behavior or, more
precisely, social action that cultural
forms find articulation.
Geertz 1973:17
Interpretive Validity
Interpretive accounts are grounded in

the language of the people studied and
rely, as much as possible, on their own
words and concepts
At issue, then, is the accuracy of the
concepts as applied to the perspective
of the individuals included in the
account
Interpretive Validity: Design
Consideration
While the relevant consensus about the terms used in
description rests in the research community, the
relevant consensus for the terms used in
interpretation rests, to a substantial extent, in the
community studied
An important design element, for increasing
interpretive validity, therefore, is to employee, at
some level/to some degree, a participatory research
approach (e.g., through member checks, peer to peer
research model, etc.)
Theoretical Validity
Theoretical understanding goes beyond concrete

description and interpretation; its value is derived
based on its ability to explain succinctly the most
amount of data
A theory articulates/formulates a model of
relationships as they are postulated to exist between
salient variables or concepts
Theoretical validity is thus concerned, not only with
the validity of the concepts, but also their postulated
relationships to one another, and thus its goodness
of fit as an explanation
Major Threats to Validity
Type I error: believing a principle to
be true when it is not (i.e., mistakenly
rejecting the null hypothesis)
Type II error: rejecting a principle
when, in fact, it is true
Type III error: asking the wrong
question
Triangulation: An Important
Theoretical Validity Check
Case example: Parable of the blind men
and the elephant
Triangulation: An Important
Theoretical Validity Check
The most fertile search for validity comes from a
combined series of different measures, each with its
own idiosyncratic weaknesses, each pointed to a
single hypothesis. When a hypothesis can survive
the confrontation of a series of complementary
methods of testing, it contains a degree of validity
unattainable by one tested within the more
constricted framework of a single method.
Webb et al. 1966:174

External Validity in Qualitative
Research
There is broad agreement that generalizability (in the
sense of producing laws that apply universally) is not
a useful standard or goal for qualitative research
This is not to say, however, that studies conducted to
examine a particular phenomenon in a unique setting
cannot contribute to the development of a body of
knowledge accumulating about that particular
phenomenon of interest
Consensus appears to be emerging that for
qualitative researchers generalizability is best thought
of as a matter of the fit between the situation
studied and others to which one might be interested
in applying the concepts and conclusions of that
study. Powerpoint Templates
Enhancing External Validity
Thick descriptions are crucial.

Such descriptions of both the site in which the
studies are conducted and of the site to which one
wishes to generalize (or apply ones findings) are
critical in allowing one to search for the similarities
and differences between the situations.
Analysis of these similarities and differences makes it
possible to make a reasoned judgment about the
extent to which we can use the findings from one
study as a working hypothesis about what might
occur in another situation.
Multi-site Studies: Another Way to
Enhance Generalizability
A finding emerging repeatedly in the study of
numerous sites would appear to be more likely to be
a good working hypothesis about some as yet
unstudied site than a finding emerging from just one
or two sites.
A finding emerging from the study of several very
heterogeneous sites would be more robust and, thus,
more likely to be useful in understanding various
other sites than one emerging from the study of
several very similar sites.
Heterogeneity may be obtained by creating a
sampling frame that maximizes the variation inherent
in the sample, specifically in terms of potentially
theoretically important dimensions
Reliability
Reliability concerns the ability of different

researchers to make the same
observations of a given phenomenon if
and when the observation is conducted
using the same method(s) and
procedure(s)
Enhancing the Reliability of
Researchers can enhance the reliability of their
qualitative research by:
Standardizing data collection techniques and protocols
Again, documenting, documenting, documenting (e.g.,

time day and place observations made)
Inter-rater reliability (a consideration during the

analysis phase of the research process)

Instrumentasi PembentuknKesahanKebolehpercayaan

Uploaded by

Copyright:

Available Formats

You might also like

Instrumentasi PembentuknKesahanKebolehpercayaan

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Instrumentasi PembentuknKesahanKebolehpercayaan

Uploaded by

Copyright:

Available Formats

Instrumentasi:

Powerpoint Templates Mengilham harapan, mencipta masa depan

Powerpoint Templates Mengilham harapan, mencipta masa depan

Powerpoint Templates Mengilham harapan, mencipta masa depan

Typically designed to determine

Limitation: rely on self report

Challenges to validity become a

Powerpoint Templates Mengilham harapan, mencipta masa depan

Is the development of a new questionnaire

Determine the Sampling Methodology

INITIAL CONTACT / COVER LETTER

Powerpoint Templates Mengilham harapan, mencipta masa depan

Powerpoint Templates Mengilham harapan, mencipta masa depan

Powerpoint Templates Mengilham harapan, mencipta masa depan

How do we judge the relative success (or

Reliability focuses on measurement

Consistency across time

Consistency between observers

Consistency between different measures

Lack of mistakes in measurement

Reliability is a necessary condition for

Subjective judgment of experts about:

Empirical evidence used to test validity

Encompasses other elements of validity

When measurements are put into practice:

Measuring a concept with different methods

Measuring a concept to discriminate that

Without internal validity, there is no external validity

Semi- or Un-structured, Open-Ended:

In-depth Interviews (in the field, face-to-face)

Iteration (a spiraling process: sequential and repetitive steps in examining preliminary

Key Informants (question development and piloting of instrument)

Single v. Multiple Cases (not an individual)

Participants as Data Collectors

Qualitative researchers may combine

Many classic participant observations studies

May not be possible or advisable to take

Spradley three key elements for the interview

3. Ethnographic questions include:

it focuses primarily on understanding particulars rather than

Qualitative researchers view threats as an opportunity for

In general, validity concerns the degree

In qualitative research, validity concerns

Example: ACY Site Visit Toolkit

External Validity (i.e., generalizability)

Concerned with the factual accuracy of an account

Behavior must be attended to, and with

Interpretive accounts are grounded in

Theoretical understanding goes beyond concrete

Webb et al. 1966:174

Thick descriptions are crucial.

Reliability concerns the ability of different

Again, documenting, documenting, documenting (e.g.,

Inter-rater reliability (a consideration during the

You might also like