PracRe1 Lec9

Defining and Measuring variables

Constructs and Operational Definitions

 The first step in the research process is to find an unanswered question that will serve as a
research idea.
 The second step involves forming a hypothesis, a tentative answer to the question.
 The next steps in the research process involve using the hypothesis to develop an empirical
research study that will either support or refute the hypothesis.
 We begin by specifying how each of the variables will be measured.
 Usually, researchers are interested in how variables are affected by different conditions or how
variables differ from one group of individuals to another.
 The next step in the research process (Step 3) is determining a method for defining and measuring
the variables that are being studied. Occasionally, a research study involves variables that are well
defined, easily observed, and easily measured.
 For example, a study of physical development might involve the variables of height and weight.
Both of these variables are tangible, concrete attributes that can be observed and measured
directly.
 On the other hand, some studies involve intangible, abstract attributes such as motivation or self-
esteem. Such variables are not directly observable, and the process of measuring them is more
complicated.

Theories and Constructs

  In attempting to explain and predict behavior, scientists and philosophers often develop theories
that contain hypothetical mechanisms and intangible elements.
 Although these mechanisms and elements cannot be seen and are only assumed to exist, we
accept them as real because they seem to describe and explain behaviors that we see.
 Many research variables, particularly variables of interest to behavioral scientists, are in fact
hypothetical entities created from theory and speculation. Such variables are called constructs or
hypothetical constructs.
 Although constructs are hypothetical and intangible, they play very important roles in behavioral
theories. In many theories, constructs can be influenced by external stimuli and, in turn, can
influence external behaviors.
Although researchers may not be able to observe and measure a construct directly, it is possible to
examine the factors that influence a construct and the behaviors that are influenced by the
construct.
Operational Definitions
 Although a construct itself cannot be directly observed or measured, it is possible to observe and
measure the external factors and the behaviors that are associated with the construct.
 Researchers can measure these external, observable events as an indirect method of measuring the
construct itself.
 Typically, researchers identify a behavior or a cluster of behaviors associated with a construct;
the behavior is then measured, and the resulting measurements are used as a definition and a
measure of the construct.
 This method of defining and measuring a construct is called an operational definition.
Researchers often refer to the process of using an operational definition as operationalizing a
construct.
Limitations of Operational Definitions
 Although operational definitions are necessary to convert an abstract variable into a concrete
entity that can be observed and studied, you should keep in mind that an operational definition is
not the same as the construct itself.
 The primary limitation of an operational definition is that there is not a one-to-one relationship
between the variable that is being measured and the actual measurements produced by the
operational definition.
Using Operational Definitions
 Whenever the variables in a research study are hypothetical constructs, you must use operational
definitions to define and measure the variables.
 Usually, however, this does not mean creating your own operational definition.
 The best method of determining how a variable should be measured is to consult previous
research involving the same variable. Research reports typically describe in detail how each
variable is defined and measured.
 By reading several research reports concerning the same variable, you typically can discover that
a standard, generally accepted measurement procedure has already been developed.
 When you plan your own research, the best advice is to use the conventional method of defining
and measuring your variables. In this way, your results will be directly comparable to the results
obtained in past research.
 However, keep in mind that any measurement procedure, particularly an operational definition, is
simply an attempt to classify the variable being considered. Other measurement procedures are
always possible and may provide a better way to define and measure the variable.
Validity and Reliability of Measurement
 Whenever the variable is a hypothetical construct, a researcher must use an operational definition
as a measurement procedure.
 In essence, an operational definition is an indirect method of measuring something that cannot be
measured directly. How can we be sure that the measurements obtained from an operational
definition actually represent the intangible construct?
 In general, we are asking how good a measurement procedure, or a measurement, is. Researchers
have developed two general criteria for evaluating the quality of any measurement procedure:
validity and reliability.
 As you will see, validity and reliability are often defined and measured by the consistency of the
relationship between two sets of measurements.
Consistency of a relationship
 Often, the validity and reliability of measurements are established by demonstrating the
consistency of a relationship between two different measurements.
Validity of Measurement
 The first criterion for evaluating a measurement procedure is validity.
 To establish validity, you must demonstrate that the measurement procedure is actually
measuring what it claims to be measuring.
 Although the notion of validity may appear to be self-evident, there are circumstances in which
legitimate questions can be asked about what really is being measured when a particular
measurement procedure is used.
Commonly Used Definitions of Validity
1. Face Validity
 Face validity is the simplest and least scientific definition of validity.
 Face validity concerns the superficial appearance, or face value, of a measurement procedure.
 Does the measurement technique look like it measures the variable that it claims to measure?
2. Concurrent Validity
 Often, the validity of a new measurement is established by demonstrating that the scores obtained
from the new measurement technique are directly related to the scores obtained from another,
better-established procedure for measuring the same variable.
3. Predictive Validity
 Most theories make predictions about the constructs they contain.
 Specifically, theories predict how different values of a construct affect behavior.
 When the measurements of a construct accurately predict behavior (according to the theory), the
measurement procedure is said to have predictive validity.
4. Convergent and Divergent Validity
 Another technique for establishing the validity of a measurement procedure is to demonstrate a
combination of convergent and divergent validity (Campbell & Fiske, 1959).
 In general terms, convergent validity involves creating two different methods for measuring the
same construct, and then showing that the two methods produce strongly related scores.
 The goal is to demonstrate that different measurement procedures “converge”—or join— on the
same construct.
 Divergent validity, on the other hand, involves demonstrating that we are measuring one specific
construct and not combining two different constructs in the same measurement process.
  The goal is to differentiate between two conceptually distinct constructs by measuring both
constructs and then showing that there is little or no relationship between the two measurements.
Reliability of Measurement
 The second criterion for evaluating the quality of a measurement procedure is called reliability.
 A measurement procedure is said to have reliability if it produces identical (or nearly identical)
results when it is used repeatedly to measure the same individual under the same conditions.
 The concept of reliability is based on the assumption that the variable being measured is stable or
constant. For example, your intelligence does not change dramatically from one day to another
but rather stays at a fairly constant level.
 However, when we measure a variable such as intelligence, the measurement procedure
introduces an element of error.
 The inconsistency in a measurement comes from error. Error can come from a variety of sources.
 The more common sources of error are as follows:
1. Observer Error. The individual who makes the measurements can introduce simple human error into
the measurement process, especially when the measurement involves a degree of human judgment.
2. Environmental Changes. Although the goal is to measure the same individual under identical
circumstances, this ideal is difficult to attain. Often, there are small changes in the environment from one
measurement to another, and these small changes can influence the measurements. There are so many
environmental variables (such as time of day, temperature, weather conditions, and lighting) that it is
essentially impossible to obtain two identical environmental conditions.
3. Participant Changes. The participant can change between measurements. As noted earlier, a person’s
degree of focus and attention can change quickly and can have a dramatic effect on measures of reaction
time. Such changes may cause the obtained measurements to differ, producing what appear to be
inconsistent or unreliable measurements.

Types and Measures of Reliability

 We have defined reliability in terms of the consistency between two or more separate
measurements.
 Thus far, the discussion has concentrated on situations involving successive measurements.
Although this is one common example of reliability, it is also possible to measure reliability for
simultaneous measurements and to measure reliability in terms of the internal consistency among
the many items that make up a test or questionnaire.
Successive Measurements
The reliability estimate obtained by comparing the scores obtained from two successive measurements is
commonly called test-retest reliability.
A researcher may use exactly the same measurement procedure for the same group of individuals at two
different times.
Or a researcher may use modified versions of the measurement instrument (such as alternative versions
of an IQ test) to obtain two different measurements for the same group of participants.

When different versions of the instrument are used for the test and the retest, the reliability measure is
often called parallel-forms reliability.
Simultaneous Measurements
 When measurements are obtained by direct observation of behaviors, it is common to use two or
more separate observers who simultaneously record measurements.
  For example, two psychologists may watch a group of preschool children and observe social
behaviors. Each individual records (measures) what he or she observes, and the degree of
agreement between the two observers is called inter-rater reliability.
Internal Consistency
 Often, a complex construct such as intelligence or personality is measured using a test or
questionnaire consisting of multiple items.
 The idea is that no single item or question is sufficient to provide a complete measure of the
construct.
 A common example is the use of exams that consist of multiple items (questions or problems) to
measure performance in an academic course.
 The final measurement for each individual is then determined by adding or averaging the
responses across the full set of items.
 A basic assumption in this process is that each item (or group of items) measures a part of the
total construct.
 If this is true, then there should be some consistency between the scores for different items or
different groups of items.
 To measure the degree of consistency, researchers commonly split the set of items in half and
compute a separate score for each half.
Relationship Between Reliability and Validity
 Although reliability and validity are both criteria for evaluating the quality of a measurement
procedure, these two factors are partially related and partially independent.
 They are related to each other in that reliability is a prerequisite for validity; that is, a
measurement procedure cannot be valid unless it is reliable.
 It is not necessary for a measurement to be valid for it to be reliable.
Scales of Measurement
1. Nominal Scale
 The categories that make up a nominal scale simply represent qualitative (not quantitative)
differences in the variable measured.
 The categories have different names but are not related to each other in any systematic way.
 For example, if you were measuring academic majors for a group of college students, the
categories would be art, chemistry, English, history, psychology, and so on. Each student would
be placed in a category according to his or her major.
 Measurements from a nominal scale allow us to determine whether two individuals are the same
or different, but they do not permit any quantitative comparison.
2. Ordinal Scale
 The categories that make up an ordinal scale have different names and are organized in an
ordered series.
 Often, an ordinal scale consists of a series of ranks (first, second, third, and so on) like the order
of finish in a horse race.
 Occasionally, the categories are identified by verbal labels such as small, medium, and large
drink sizes at a fast-food restaurant.
 In either case, the fact that the categories form an ordered sequence means that there is a
directional relationship between the categories.
 With measurements from an ordinal scale, we can determine whether two individuals are
different, and we can determine the direction of difference.
  However, ordinal measurements do not allow us to determine the magnitude of the difference
between the two individuals.
3. Interval and ratio Scales
The categories on interval and ratio scales are organized sequentially, and all categories are the same size.
Thus, the scale of measurement consists of a series of equal intervals like the inches on a ruler. Other
common examples of interval or ratio scales are the measures of time in seconds, weight in pounds, and
temperature in degrees Fahrenheit.
Notice that in each case, one interval (1 inch, 1 second, 1 pound, and 1 degree) is the same size, no matter
where it is located on the scale.
The fact that the categories are all the same size makes it possible to determine the distance between two
points on the scale. For example, you know that a measurement of 70 degrees Fahrenheit is higher than a
measurement of 55 degrees, and you know that it is exactly 15 degrees higher.
The characteristic that differentiates interval and ratio scales is the zero point. The distinguishing
characteristic of an interval scale is that it has an arbitrary zero point.
That is, the value 0 is assigned to a particular location on the scale simply as a matter of convenience or
reference. Specifically, a value of 0 does not indicate the total absence of the variable being measured.
A ratio scale, on the other hand, is characterized by a zero point that is not an arbitrary location.
Instead, the value 0 on a ratio scale is a meaningful point representing none (a complete absence) of the
variable being measured.
The existence of an absolute, non-arbitrary zero point means that we can measure the absolute amount of
the variable; that is, we can measure the distance from 0.
This makes it possible to compare measurements in terms of ratios.

Modalities of Measurement
Self-report Measures
 One option for measuring, or operationalizing, the fear of flying is to ask participants to describe
or to quantify their own fear. The researcher could simply ask, “Are you afraid to fly?” Or
participants could be asked to rate the amount of fear they are experiencing on a scale from 1 to
10. Or they could be given a comprehensive questionnaire about airline travel, and the researcher
could use the set of responses to obtain an overall score measuring fear of flying. The primary
advantage of a self-report measure is that it is probably the most direct way to assess a construct.
Each individual is in a unique position of self-knowledge and self-awareness; presumably, no one
knows more about the individual’s fear than the individual. Also, a direct question and its answer
have more face validity than measuring some other response that theoretically is influenced by
fear. On the negative side, however, it is very easy for participants to distort self-report measures.
A participant may deliberately lie to create a better self-image, or a response may be influenced
subtly by the presence of a researcher, the wording of the questions, or other aspects of the
research situation. When a participant distorts self-report responses, the validity of the
measurement is undermined.
Physiological Measures
 A second option for measuring a construct is to look at the physiological manifestations of the
underlying construct. Fear, for example, reveals itself by increased heart rate and perspiration
(measured by galvanic skin response, GSR). A researcher measuring “fear of flying” could attach
electrodes to participants and monitor heart rates as they board a plane and during the flight. Or a
researcher could ask participants to imagine a flight experience while GSR and heart rate are
monitored in a laboratory setting. Other physiological measures involve brain-imaging techniques
such as positron emission tomography (PET) scanning and magnetic resonance imaging (MRI).
 These techniques allow researchers to monitor activity levels in specific areas of the brain during
different kinds of activity.
Behavioral Measures
 Constructs often reveal themselves in overt behaviors that can be observed and measured. The
behaviors may be completely natural events such as laughing, playing, eating, sleeping, arguing,
or speaking. Or the behaviors may be structured, as when a researcher measures performance on a
designated task.
Other Aspects of Measurement
Multiple Measures
 One method of obtaining a more complete measure of a construct is to use two (or more) different
procedures to measure the same variable.
Sensitivity and Range Effects
 Typically, a researcher begins a study with some expectation of how the variables will behave,
specifically the direction and magnitude of changes that are likely to be observed.
 An important concern for any measurement procedure is that the measurements are sensitive
enough to respond to the type and magnitude of the changes that are expected.
Experimenter Bias
 Typically, a researcher knows the predicted outcome of a research study and is in a position to
influence the results, either intentionally or unintentionally.
 Rosenthal and Fode (1963) identified a variety of ways that an experimenter can influence a
participant’s behavior:

 By paralinguistic cues (variations in tone of voice) that influence the participants to give the
expected or desired responses
 By kinesthetic cues (body posture or facial expressions)
 By verbal reinforcement of expected or desired responses
 By misjudgment of participants’ responses in the direction of the expected results
 By not recording participants’ responses accurately (errors in recording of data) in the direction
of the expected or desired results.

 One option for limiting experimenter bias is to standardize or automate the experiment.
 Another strategy for reducing experimenter bias is to use a “blind” experiment. If the research
study is conducted by an experimenter (assistant) who does not know the expected results, the
experimenter should not be able to influence the participants.
 This technique is called single-blind research. An alternative is to set up a study in which neither
the experimenter nor the participant knows the expected result.
Demand Characteristics and Participant Reactivity
 The fact that research studies involve living organisms, particularly humans, introduces another
factor that can affect the validity and reliability of the measurements.
 Specifically, living organisms are active and responsive, and their actions and responses can
distort the results.
 If we observe or measure an inanimate object such as a table or a block of wood, we do not
expect the object to have any response, such as, “Whoa! I’m being watched. I had better be on my
best behavior.”
 Unfortunately this kind of reactivity can happen with human participants. Participants who are
aware they are being observed and measured may react in unpredictable ways.
  In addition, the research setting often creates a set of cues or demand characteristics that suggests
what kinds of behavior are appropriate or expected.
 The combination of demand characteristics and participant reactivity can change participants’
normal behavior and thereby influence the measurements they produce.
Selecting a Measurement Procedure
 As seen in the preceding sections, the choice of a measurement procedure involves several
decisions.
 Because each decision has implications for the results of the study, it is important to consider all
the options before deciding on a scheme for measurement for your own study or when critically
reading a report of results from another research study.
 The best starting point for selecting a measurement procedure is to review past research reports
involving the variables or constructs to be examined.
 Most commonly used procedures have been evaluated for reliability and validity. In addition,
using an established measurement procedure means that results can be compared directly to the
previous literature in the area.
 If more than one procedure exists for defining and measuring a particular variable, examine the
options and determine which method is best suited for the specific research question.
 In particular, consider which measure has a level of sensitivity appropriate for detecting the
individual differences and group differences that you expect to observe. Also decide whether the
scale of measurement (nominal, ordinal, interval, or ratio) is appropriate for the kind of
conclusion you would like to make.

Assignment:
Choose 5 questions.

1. Hypothetical concepts, such as honesty, are variables that cannot be observed or measured directly and,
therefore, require operational definitions. a. Describe one procedure that might be used to measure
honesty. b. Use the procedure you described in (a) to explain why there may not be a one-to-one
relationship between the actual variable and the measurement produced by the operational definition of
the variable.
2. Briefly explain what an operational definition is and why operational definitions are sometimes
necessary.
3. A researcher evaluates a new cholesterol medication by measuring cholesterol levels for a group of
patients before they begin taking the medication and after they have been taking the medication for 8
weeks. A second researcher measures quality of life for a group of 40-year-old men who have been
married for at least 5 years and a group of 40-year-old men who are still single. Explain why the first
researcher is probably not concerned about the validity of measurement, whereas the second researcher
probably is. (Hint: What variable is each researcher measuring and how will it be measured?)
4. A clinical researcher has developed a new test for measuring impulsiveness and would like to
determine the validity of the test. The new test and an established measure of impulsiveness are both
administered to a sample of participants. Describe the pattern of results that would establish concurrent
validity for the new test.
5. Suppose that a social scientist has developed a questionnaire intended to measure the quality of
romantic relationships. Describe how you could evaluate the reliability of the questionnaire.
6. Explain how inter-rater reliability is established.
7. A researcher claims that intelligence can be measured by measuring the length of a person’s right-hand
ring finger. Explain why this procedure is very reliable but probably not valid.
8. For each of the following operational definitions, decide whether you consider it to be a valid measure.
Explain why or why not. Decide whether you consider it to be a reliable measure. Explain why or why
not. a. A researcher defines social anxiety in terms of the number of minutes before a child begins to
interact with adults other than his or her parents. b. A professor classifies students as either introverted or
extroverted based on the number of questions each individual asks during one week of class. c. A sports
psychologist measures physical fitness by measuring how high each person can jump. d. Reasoning that
bigger brains require bigger heads, a researcher measures intelligence by measuring the circumference of
each person’s head (just above the ears).
9. In this chapter we identified four scales of measurement: nominal, ordinal, interval, and ratio. a. What
additional information is obtained from measurements on an ordinal scale compared to measurements
from a nominal scale? b. What additional information is obtained from measurements on an interval scale
compared to measurements from an ordinal scale? c. What additional information is obtained from
measurements on a ratio scale compared to measurements from an interval scale?
10. Select one construct from the following list: happiness, hunger, exhaustion, motivation, creativity and
fear. Briefly describe how it might be measured using: a. an operational definition based on self-report
(e.g., a questionnaire). b. an operational definition based on behavior (e.g., what kinds of behavior would
you expect to see from an individual with high self-esteem?)
11. Describe the relative strengths and weaknesses of self-report measures compared to behavioral
measures
12. What is a ceiling effect, and how can it be a problem?
13. Explain how an artifact can limit the validity and reliability of a measurement.
14. What are demand characteristics, and how do they limit the validity of the measurements obtained in
a research study?
15. Describe how the concept of participant reactivity might explain why a person’s behavior in a group
of strangers is different from a person’s behavior with friends.

References

Books:
Arboleda, C. R. (1998). Writing A Thesis Proposal (1st ed.). Manila: Rex Book Store.
Bertrand, I & Hguhes, P. (2005). Media Research Methods. New York: MacMillan.
Breakwell, G. M., Smith, J.A., & Wright D.B. (2012). Research Methods in Psychology (4th ed.).
London: Sage Publication Ltd.
Garcia, A. M., Bongola, V.T., Nuevo, J.J., Profeta, M.M., Malaborbor, P.B., Gamboa, D.L.,
Lupdag-Padaga, E.A. & Hilario, D.S. (2011). Philippines: Booklore Publishing Corporation.
Gravetter, F. J. & Forzano, L.A. (2018). Research methods for behavioral sciences. Cengage
Learning, Inc.Rubin, R. B., Rubin, A.M., & Piele, L.J. (1996). Communication Research:
Strategies and Sources. 4th Ed. USA:Wadsworth Publishing Company.
Schumaker, J. (2000). Statistics: An Intuitive Approach. Belmont California: Wadsworth
Publishing, Inc.
Smith, T. R. (2008). The Psychology Thesis: Research and Coursework. NY: Palgrave
Macmillan.
Qualitative Research Information Systems. (2008) http://w.w.wqual.auckland.ac.anz/
Haslam, S. A. (2012). Research methods and statistics in psychology. Los Angeles: SAGE,
2014.Riley, Sarah, Doing your qualitative psychology project. London: SAGE.
Cozby, P. C. (2012). Methods in behavioral research. Boston: McGraw- Hill.
Bordens, K. S. (2011). Research design and methods: a process approach. New York: McGraw-
Hill.
Paler, C. L. (2010). Research and statistics with computer. Mandaluyong City: National Book
Store.
Shaughnessy, J.J. (2008). Research methods in psychology. Boston: McGraw-Hill, c2009.
Bordens, K. S. (2008). Research Design and methods: a process approach. Boston: McGraw Hill.
Smyth, T. R. (2008). The psychology thesis: research and coursework. Houndmills, Basingstoke,
Hampshire; Palgrave Macmillan.
Gavin, H. (2008). Understanding research methods and statistics in psychology. Los Angeles,
CA: SAGE.
Spatz, C. (2008). Research methods in psychology: ideas, techniques and reports. Boston:
McGraw-Hill Higher Education.
Spats, C. (2008). Research methods in psychology: ideas, techniques, and reports. New York:
McGraw-Hill.
Smith, J. A. (2008). Qualitative psychology: a practical guide to research methods. Los Angeles,
Calif.: SAGE Publications.
Martin, D. W. (2008). Doing psychology experiments. Australia: Thomson/Wadsworth.
Martin, P. (2007). Measuring behavior: an introductory guide. New York: Cambridge University
Press.
Cozby, P. C. (2007). Methods in behavioral research. New York, NY: McGraw-Hill Higher
Education.
Salkind, N. J. (2007). Exploring research. New Jersey: Pearson Education, Inc.
Bootzin, R. R. (2006). Strengthening research methodology: psychological measurement and
evaluation. Washington, DC: American Psychological Association.
Thompson, B. (2006). Foundations of behavioral statistics: an insight-based approach. New York:
Guilford Press.
Davis, S. F. (2005). Handbook of research methods in experimental psychology. Malden, MA:
Blackwell Pub.
Mertens, D. M. (2005). Research and evaluation in education and psychology: integrating
diversity with quantitative, qualitative, and mixed methods. Thousand Oaks, Calif. : Sage
Publications.
Langston, W. (2005). Research methods laboratory manual for psychology. Belmont, CA:
Thomson/Wadsworth.
Parker, I. (2005). Qualitative psychology: introducing radical research. Maidenhead, England:
Open University Press.
Roberts, M. C. (2005). Handbook of research methods in clinical psychology. Malden, MA:
Blackwell Publishing.