Nee​d for a Mixed-Method Study

Early in the process the authors argued that their study employed a mixed- method approach
because each method provided ''distinct strengths'' to broaden the study and afforded ''deeper
insights'' (Russek & Weinberg, 1993, p. 131) into the nature and extent of technology
implementation. The au- thors mentioned the advantage of triangulation ''to achieve a more
complete picture of empirical reality'' (p. 131). Even though they begin with qualitative data
collection followed by quantitative collection, their analysis suggested some attempt to blend or
converge the results from the methods.

Feasibility of the Study

This issue is not addressed explicitly, although the authors devote several pages to elucidating
the key characteristics of qualitative research and ethno- graphic research. With federal funding,
they embarked on elaborate qualita- tive and quantitative methods of data collection. Thus, lack
of skills, money, time, and effort did not seem to be a problem.

Research Questions
Although they did not specify detailed research questions, they posed one open-ended qualitative
question, ''What is going on here?'' and one quantitative question, ''What are the explanations for
what we see happen- ing?'' (Russek & Weinberg, 1993, p. 131) in the introduction to the study.
The quantitative question clearly sought to specify the variables that explain the implementation
of technology.

Types of Data Collection

In a section on data collection, the authors mentioned that they gathered both qualitative and
quantitative data, which can be enumerated as follows:
1. Qualitative data
a. informal in-depth interviews of teachers and administrators
b. informal classroom observations
c. school documents
d. teacher-written responses to open-ended questions or questionnaires
2. Quantitative data
a. classroom observation checklists
b. lesson evaluation forms
c. workshop evaluation forms
d. a self-evaluation questionnaire
e. stages of concern questionnaire

Relative Weight and Implementation Strategy

Although many pages in this study present the qualitative and ethnographic approach to research,
in the analysis and interpretation phase of this study, the authors provide equal weight to
qualitative and quantitative methods. They suggest that they began with the qualitative phase of
data collection by interviewing and observing and then followed this segment by administering
the instruments. Because the quantitative phase was decided before the study began, their study
might be characterized as a sequential one in which the qualitative results led to quantitative
measures and instruments. Alter- natively, one might view the study as two methods
implemented concur- rently, an approach consistent with the convergence model of
mixed-method design.

Visual of the Design

This project might have been enhanced if the authors had presented a visual model of their
procedure. Such a model was not available to the reader, but the authors might have presented
one model that they began with and a second that they ended with. A speculation about how
such a visual might have looked is shown in Figure 3. The rst would be a sequential qualita-
tive followed by quantitative approach; whereas, the second would be the actual model, in
which the researchers took quantitative data in the form of an index of implementation and
compared it with qualitative data that was quantied into categories (high and low) on three
factors found quali- tatively to impact implementation (teachers expressions, volunteer atti-
tude, and skills/knowledge). In the end, they compared the two numeric sets of data.

Analysis of the Data

The specic form of data analysis used by Russek and Weinberg (1993) would, according to
Caracelli and Greene (1993), be called data transformation. For example, in a single table
Russek and Weinberg (1993) compared the degree of implementation of teacher verbal reports
(qualitative), classroom observations (qualitative), and lesson evaluation forms (quantitative)
(Table 2, p. 139). To do this, they transformed the verbal reports and class- room observations
into categories rated low, moderate, and high. For each teacher, then, the reader could visually
inspect ratings on the three methods of data collection. From this they concluded that ''there
appears to be good agreement among indicators; triangulation has resulted in convergence'' (p.
139).
As a second example, they created an index score for each teacher on calculator implementation.
This score represented the proportion of lessons completed and a quality weighting factor
describing the effectiveness of the teacher in fullling the objectives of the lessons completed.
Thus, a quanti- tative score was available. They then transformed qualitative expressions about
implementation on three factors, teacher commitment, preactivity at- titudes, calculator skill,
and knowledge into low, moderate and high ratings. In a Venn diagram, they plotted the
quantitative index score and the transformed qualitative views in a gure to show that those
individuals with a high quan- titative implementation score also had a high score when
measuring com- mitment, preactivity attitudes, and skills.

Quality Criteria
In terms of the criteria for assessing the quality of a mixed-method study mentioned earlier, the
Russek and Weinberg (1993) study adequately ad- dressed most of the criteria. Its strength was
in the use of multiple forms of quantitative and qualitative data; it presented a distinct model of
conver- gence and discussed the issue of triangulating data, and it was labeled a mixed-method
study. The authors might have enhanced their discussion by presenting a visual model and by
discussing the type of mixed-method ap- proach they employed and how it shifted during their
study.

CONCLUSION
This chapter suggests that mixed-method designs are beginning to emerge in policy
research. As policy researchers conduct or evaluate mixed-method studies, they need to
recognize that this form of design goes by different names and has a history of an increased
clarication of design topics in recent years. Accordingly, this chapter advances eight steps to
consider when planning or evaluating a mixed-method study.
Policy researchers should assess the need for such a study, followed by exploring its
feasibility in terms of time, cost, and expertise. Then the analyst needs to write research
questions that relate to qualitative and quantitative methods of data collection and identify the
specic forms of data to be collected. The analyst can then consider the relative weight, the
sequence, and a visual diagram for the study. Also, the specic procedures for data analysis
must relate closely to the model or purpose for the study identied at the outset of the project. In
the end, a policy study can combine the best of both quantitative and qualitative methods and
provide useful narrative as well as quantitative data for decision making.