Lecture Material for Research Methodology in Science Education

RESEARCH
INSTRUMENT

Didik Setyawarno
Teaching Staff at Universitas Negeri Yogyakarta
2022
 OBJECTIVES
❑ Describe what research instrument and how it is good
❑ Describe the procedures to develop the research
instrument
❑ Describe dimension and indicator in the research
instrument
❑ Describe forms of assessment and instruments and
how to collect data
 RESEARCH INSTRUMENTS
◼ An instrument is a tool that meets academic
requirements so that it can be used as a tool to measure
an object or collect data on a research variable.

◼ Characteristics of a Good Instrument:

1. Valid => validity
2. Reliable => reliability
3. Sensitive
4. High objectivity
5. Good feasibility
HOW TO MAKE A GOOD INSTRUMENT?

Criteria for a good instrument (Sevilla 1988):

1. Reliability
Reliability shows the consistency and stability of a score
from a measuring instrument.

2. Validity
An instrument is said to be valid if it has the ability to
measure what should be measured.
HOW TO MAKE A GOOD INSTRUMENT?

3. Sensitivity
The ability of an instrument to discriminate which it is
necessary for the research problem. (Usually met when the
degree of validity and reliability of the instrument is high)

4. Objectivity
The degree of instrument’s measurement is free from the
opinions of subjective judgments, free from bias, and the
feelings of the people who use the test.

5. Feasibility
Relates to aspects of skills, use of resources, and time.
 TYPES OF VALIDITY IN INSTRUMENTS
Types of Validity
Content validity
Construct validity
a) Convergent validity
b) Discriminant validity
Criterion –related validity
a) Predictive validity
b) Concurrent validity
Definition
The ability of the statement items in the instrument to represent all the
dimensions of the concept being studied
The suitability of the instrument with the theory about the concept
being studied.
a) The degree of correlation between two instruments measuring the
same concept
b) The low level of correlation between the instrument and the
predicted theoretically uncorrelated concept
The ability of the instrument to predict the variable criterion.
a) The ability of the instrument to predict the criterion variable in the
future
b) The ability of the instrument to predict the criterion variable which
is measured at the same time as the variable under study (the
instrument can distinguish between individuals who are known to
be different
 RELIABILITY
How is the consistency of the measuring
instrument when it is used in the measurement
process

How to determine reliability:

✓ Test-retest

✓ Split test (split half)

✓ Equivalent form
 RELIABILITY
◼ The internal consistency of the measuring instrument
can be calculated using the Alpha-Cronbach, Kuder-
Richardson (KR-20 or KR-21) coefficient formulas, and
the Split in half Technique.
◼ This KR-20 is specifically for calculating the reliability
of a test whose data is dichotomous, for example
multiple choice questions with a true score of 1 and a
false 0. While Alpha-Cronbach can be used for essay
question whose data is polytomy, for example essay
question with score 0, 1, 2, and 3.
Reliable Valid Neither Both
Not Valid Not Reliable Reliable Not Reliable and
Valid Valid
 RELATIONSHIP VALIDITY AND
RELIABILITY
Validity
Closely related to the compatibility between the concept
and empirical reality

Reliability
◼ Closely related to the suitability of measurement
results at the empirical level
◼ Therefore => valid must be reliable but not vice versa
(a reliable measure will measure consistently, but not
necessarily measure what it is supposed to measure).
 TYPES OF VARIABLES IN
INSTRUMENTS
◼ Construct variables (performance concepts and typical
concepts) are measured by a scale or test
◼ Content variable (knowledge and/or ability) measured by
test (written, oral, action)
◼ Factual variables (concrete indicators) are measured by
questionnaires
HOW TO DEVELOP A RESEARCH
INSTRUMENT?
 VARIABLE •What is the name? Can it be measured?

THEORY •What's the concept like? From whom?

CONSTRUCT •Compose your own sentences or statements

CONCEPTUAL DEFINITION •Exactly the same as the construct formula

•Same as conceptual definition

OPERATIONAL DEFINITION •Including how to measure

INSTRUMENT DETERMINATION •Questionnaires, attitude scales, tests,

observation sheets

INSTRUMENT MATRIX •Present in matrix form

 WRITING INSTRUMENT ITEMS Making statements/questions

INSTRUMENT TRIAL Tested on an equivalent sample candidate

Known items that meet the requirements &

ANALYSIS OF TRIAL RESULTS items that must be deleted:
validity & reliability test

INSTRUMENT REVISION Revise bad items

INSTRUMENT FINALIZATION Improvements to instruments, formats, etc.

Duplicate as needed
PRINTING & COPYING INSTRUMENTS
 VARIABLE

THEORY A THEORY B THEORY C THEORY D

CONSTRUCTION/CONCEPTUAL DEFINITION

OPERATIONAL DEFINITION

INDICATORS

ITEMS
DIMENSIONS AND INDICATORS
◼ A variable is a concept that has a variety of values
◼ Dimensions are sub-concepts, while indicators are
markers of concepts
◼ For variables that have a single concept,
dimensions are sub-concepts, while indicators are
sub-dimensions
◼ For variables that have multiple concepts, the
indicator is the multiplication of the dimensions
and components
EXAMPLES OF VARIABLES FOR
ONE AND TWO CONCEPTS
◼ Attitude towards the teaching profession (variable
with two concepts, namely the concept of attitude
and the concept of the teaching profession)
◼ Leadership (one concept)
◼ Education (one concept)
◼ Leadership style (variable with one concept)
◼ Education level (factual variable)
◼ Learning motivation (variable with one concept)
EXPLANATION OF INDICATORS

◼ Indicator is the result of multiplication of

dimensions and components
◼ The indicator is a description as well as a marker of
the dimensions
◼ Indicators must be measurable
◼ Indicators are not parameters but can be
parameterized
◼ Indicators must be complete to measure one variable
 Achievement motivation

Maslow Theory Muray Theory Hezberg Theory Mc Cleland Theory

Achievement motivation is the seriousness or driving

force for someone to do better than what has ever been
made or achieved before or made or achieved by others

1. Trying to excel in the group

2. Complete the task well
3. Rational in achieving success
4. Loves challenges
5. Accept personal responsibility for success,
6. Prefer work situations with personal responsibility, feedback, and moderate risk

Items/ Questions
 TYPES OF INSTRUMENTS
Test
◼ Written test/ Essay
◼ Oral test- Action/Practice/Performance Test

Non-Test
❑ Questionnaire
❑ Attitude Scale
❑ Scoring scale
❑ Sociometry
 FORMS OF ASSESSMENT AND INSTRUMENTS
Form of Assessment Example of instrument
Written test 1. Choice test: multiple choice, true-false,
matchmaking etc.
2. Fill-in test: short entry and description
Oral test a list of questions
Practice test (performance test) 1. Identification test
2. Simulation test
3. Performance test
Individual or group assignments 1. Homework
2. Project
Portfolio assessment Portfolio sheets

Journal Journal notebook

Self-assessment Questionnaire/self-assessment sheet

Peer-assessment Peer-assessment sheet

 DATA COLLECTION
METHODS/TECHNIQUES

Types of data seen from how to obtain it:

❑ Primary data
❑ Secondary Data

Data collection techniques/methods that can be

carried out by researchers:
❑ Tests (Pretest and Posttest)
❑ Interview (interview)
❑ Questionnaire (questionnaire)
❑ Documentation
❑ Observation (observation)
No. Research Variables Dimensions Indictors Collection Techniques

No. Research Variables Instrument Format No. Item Item Total

 TASK 9 – INDIVIDUAL
ASSIGNMENT
Analyze your under thesis (independent variable and
dependent variable) including:
1. Research Variables
2. Dimensions to be measured
3. Indicators that reflect the dimensions of the variables
being measured
4. Data collection techniques
5. The form of the instrument used
 TASK 9 – INDIVIDUAL
ASSIGNMENT
No. Research Variables Dimensions Indictors Collection Techniques

No. Research Variables Instrument Format No. Item Item Total

TERIMA KASIH