University of Nebraska - Lincoln

DigitalCommons@University of Nebraska - Lincoln

Faculty Publications: Agricultural Leadership, Agricultural Leadership, Education &
Education & Communication Department Communication Department

April 1987

Effects of Classroom Testing by Microcomputer

Blannie E. Bowen
Ohio State University

David M. Agnew
University of Nebraska - Lincoln

Follow this and additional works at: http://digitalcommons.unl.edu/aglecfacpub

Part of the Other Public Affairs, Public Policy and Public Administration Commons

Bowen, Blannie E. and Agnew, David M., "Effects of Classroom Testing by Microcomputer" (1987). Faculty Publications: Agricultural
Leadership, Education & Communication Department. 2.
http://digitalcommons.unl.edu/aglecfacpub/2

This Article is brought to you for free and open access by the Agricultural Leadership, Education & Communication Department at
DigitalCommons@University of Nebraska - Lincoln. It has been accepted for inclusion in Faculty Publications: Agricultural Leadership, Education &
Communication Department by an authorized administrator of DigitalCommons@University of Nebraska - Lincoln.
 Effects of Classroom Testing by Microcomputer

Blannie E . Bowen, A s s o c i a t e P r o f e s s o r
Agricultural Education
The Ohio State University

David M. Agnew
Project Assistant
Agricultural Education
University of Nebraska-Lincoln

Accepted for Publication May 1986

Microcomputers are being u s e d f o r a v a r i e t y o f p u r p o s e s , but

research about their instructional effectiveness lags behind adoption
rates for the technology. However, severa I r e s e a r c h e r s h a v e e x a m i n e d
competencies vocational education teachers need to use effectively this
e m e r g i n g t e c h n o l o g y (Bowen, 1 9 8 4 ; C a n t r e l I , 1 9 8 2 ; C h a s e , G o r d o n , &
Makin, 1984; Hudson, 1 9 8 3 ; M i l l e r & F o s t e r , 1 9 8 5 ; R o t h , T e s o l o w s k i ,
Rankin, & B l a c k m a n , 1 9 8 4 ) . T h e s e s t u d i e s w e r e e x p l o r a t o r y a n d descr ip-
tive in nature.

Further, there is a limited research base about the effects of

microcomputers in vocational agriculture on learning in the affective,
cogn i t i v e a n d p s y c h o m o t o r doma ins. R o h r b a c h (1983) i n v e s t i g a t e d t h e
effects of microcomputer instruction versus lecture-discussion in teach-
ing college students a farm management lesson. The lecture-discussion
g r o u p i n t h e R o h r b a c h s t u d y p e r f o r m e d signi f i c a n t l y better than two
microcomputer groups did on a cognitive measure. However, Becker and
Shoup (1985) concluded that significant increases in student knowledge
of safe tractor operation concepts resulted when vocational agriculture
students as well as a class of University of Florida agriculture stu-
dents were taught using microcomputers,

The research base is even more shallow when effects of testing stu-
dents by computer technology are explored. B i s k i n a n d K o l o t k i n (1977)
found that computer-based, teletype and paper and pencil methods of test
administration produced similar results when students were given the
Minnesota Multiphasic Personality Inventory. C o r y (1977) f o u n d t h a t
paper and pencil and computerized administration methods predicted job
performance equally well on four of five attributes studied. These
findings about the effectiveness of computer technology are generally
c o n s i s t e n t w i t h t h e c o n c l u s i o n s C l a r k ( 1 9 8 3 ) m a d e a f t e r h i s meta-
analyses of research about the influence of media on learning. Cl a r k
contended that the teacher rather than instructional media brings about
student achievement.

Objectives and Hypotheses

This research tested hypotheses about how effectively microcompu-

ters could be used to administer an objective classroom test to students
who had studied and used computer technology. Specifically, the study
sought to determine: (a) t h e e f f e c t t a k i n g a n o b j e c t i v e f i n a l e x a m i n a -
tion by microcomputer would have upon student cognitive performance;
(b) t h e e f f e c t t h i s m e t h o d o f t e s t i n g w o u l d h a v e o n s t u d e n t a t t i t u d e
a b o u t c o m p u t e r s i m m e d i a t e l y a f t e r t h e e x a m i n a t i o n ; a n d (c) w h e t h e r t h i s
method of testing would require more time than conventionaI paper and
penc iI testing procedures.
 T h r e e h y p o t h e s e s w e r e f o r m u l a t e d f o r t e s t i n g (p<.05);

1. N o s i g n i f i c a n t d i f f e r e n c e i n s t u d e n t c o g n i t i v e k n o w l e d g e will
be observed for the two testing procedures when midterm examination
scores are used as a covariate.

2. There will be no significant difference in student attitude

about computers immediately after the final examination is given by the
two testing procedures.

3. No significant difference will be observed in the time needed

to complete the final examination using the two testing procedures.

Procedures

The study followed the posttest o n l y c o n t r o l g r o u p d e s i g n ( C a m p b e l l

& Stanley, 1963). Spector (1981) noted that experimental studies need
to be replicated to minimize external validity threats; consequently,
the study involved two replications. Replication 1 was conducted during
t h e 1 9 8 3 F a l I S e m e s t e r a t C a r t h a g e (MS) H i g h S c h o o l a n d R e p l i c a t i o n 2
d u r i n g t h e 1 9 8 4 S p r i n g S e m e s t e r a t P e t a l (MS) H i g h S c h o o l . Both repI i-
cations were conducted during a Mississippi State University course,
Application of Computer Technology to Agricultural and Extension Educa-
tion. T h e s a m e i n s t r u c t o r a n d t e a c h i n g a s s i s t a n t c o n d u c t e d b o t h r e p I i-
cations.

The three-hour u n d e r g r a d u a t e / g r a d u a t e l e v e l c o u r s e p r o v i d e d 3 0
hours of lecture and 30 hours of laboratory activities. The subjects in
each replication worked in pairs for 15 hours using a microcomputer and
a printer to complete a series of laboratory activities. They also com-
pleted 15 hours of laboratory activities away from a microcomputer.
Replication 1 met over a 15-week period. Each Replication 1 session
consisted of a two-hour lecture, one hour of microcomputer activities
per week and included 49 subjects. Replication 2 involved 28 subjects
and met for 10 weeks. E a c h R e p l i c a t i o n 2 s e s s i o n c o n s i s t e d o f 2-1/2
h o u r s o f l e c t u r e a n d 1-1/2 h o u r s o f m i c r o c o m p u t e r a c t i v i t i e s p e r w e e k .

The Treatment

During the session a week before the final examination was adminis-
tered, the instructor reviewed the areas the test would include. The
students were informed that half of them would take the cognitive test
on microcomputers and the other half with paper and pencil. They were
instructed to report to their regularly scheduled laboratory period to
take the examination. Two-stage random assignment was used to place the
students into either the microcomputer or conventional testing groups.
Participants in each replication were first randomly assigned to Group 1
o r 2 , and the treatment was then randomly assigned to the two groups.
The instructor administered the final examination in the laboratory with
one student per microcomputer while the teaching assistant administered
the same examination in a classroom using conventional test booklets and
optical scan sheets. The final test consisted of 35 multiple-choice
i tems.

The testing and grading software used by the experimental group was
available from a commercial vendor. This package presented one test
item on the screen at a time and did not advance to the next item until
the students entered the correct answer. However, only the first
response a student entered was used in tabulating that person’s score.
The testing package informed students of their scores on the test imme-
diately after the last item was answered. The microcomputer group was

3
given three sample items before starting the test. After the sample
items were completed, t h e i n s t r u c t o r f i e l d e d q u e s t i o n s a b o u t t h e p r o -
cess. Concurrently, the teaching assistant explained the directions and
fielded questions from students in the conventional group. Both teach-
ers then proceeded to record the number of minutes each student needed
to complete the test. After completing the cognitive test, the students
c o m p l e t e d a n instrument d e s i g n e d t o m e a s u r e t h e i r a t t i t u d e s t o w a r d com-
puters.

The three dependent variables measured in this study were:

(a) m i n u t e s t o c o m p l e t e t h e t e s t ; (b) s c o r e o n t h e t e s t ; a n d (c) s c o r e
on the attitudes about computers instrument. The test was developed by
the instructor , and r e l iabiIity coefficients from 10 prior administra-
t i o n s r a n g e d f r o m .78 t o .90. I n t h i s s t u d y , the conventional test
g r o u p d a t a w e r e u s e d t o c o m p u t e r e l i a b i l i t y c o e f f i c i e n t s w h i c h w e r e .80
f o r R e p l i c a t i o n 1 a n d .78 f o r R e p l i c a t i o n 2 . Data were not available to
compute reliability coefficients for the microcomputer testing group.
Although the software recorded each student’s score on the test, it did
not store data about individual test items. The l0-item attitudinal
i n s t r u m e n t w a s v a l i d a t e d b y a p a n e l o f e x p e r t s a n d y i e l d e d a .74 Cron-
bach's a l p h a r e l iabi I i t y c o e f f i c i e n t f o r R e p l i c a t i o n 1 a n d .76 f o r
Repl ication 2.

Findings

Personal data by treatment and control group are provided in Table

1 for Replications 1 and 2. The mean age for the 49 subjects in Repli-
cation 1 was 33.8 (treatment, 34.9; control, 32.5). The 28 subjects in
FG$ication 2 w e r e o l d e r , w i t h a m e a n a g e o f 3 8 . 4 ( t r e a t m e n t , 4 2 . 7 ; con-
34.1). T h e r e w e r e 3 8 f e m a l e s i n R e p l i c a t i o n 1 a n d 7 i n Replica-
t ion 2. In Replication 1, 33 subjects had no bachelor’s degree, and 16
had a bachelor’s degree or higher. Replication 2 consisted of 12 sub-
jects with a bachelor’s only and 16 with a master’s or educational spe-
cialist degree. The objective midterm examination (See Table 1) was
used as a covariate in the study. The midterm and final examinations
were very h i g h l y c o r r e l a t e d (r=.77) for Replication 1 and moderately
r e l a t e d i n R e p l i c a t i o n 2 (r=.43). Midterm scores for the treatment and
control groups were compared using a t-test, and no significant differ-
ence was found in either replication (Replication 1: t[47]=-.74, p>.05;
Repl ication 2: t[26]=1.04, p>.05).

Test of Hypothesis One: Final Evaluation Scores

A one-way analysis of covariance revealed that the two groups were

not significantly different in terms of their scores on the 35-item
final examination. Table 2 Iists the measures of central tendency and
the ANCOVA for the final examination scores. Hypothesis One was not
r e j e c t e d f o r e i t h e r r e p l ication s i n c e t h e e x p e r i m e n t a l and contro I
groups had similar scores on the final examination.

Test of Hypothesis Two: Student Attitudes Toward Microcomputers

The mean attitudinal scores of the two groups were positive in both
replications. The scale had a possible range of 10 (negative) to 50
(positive). The t-test performed for both replications indicated that
Hypothesis Two should not be rejected since the two groups had similar
p o s i t i v e a t t i t u d e s a b o u t c o m p u t e r s (Repl ication 1: microcomputer, 40.9,
a n d c o n t r o l , 4 1 . 0 ; R e p l ication 2: microcomputer, 39.3 and control,
41.4). Hypothesis Two was not rejected for either replication.

4
Table 1

A Comparison of Treatment and Control Groups on Selected Variables for

Replications 1 and 2

Treatment ControI Overa I I

Repl ication Replication RepI ication
Variable 1 2 1 2 1 2

Age
Mean 34.9 42.7 32.5 34.1 33.8 38.4
S.D. 12.9 8.8 9.4 7.9 11.0 9.3

Sex
Males 7 9 4 12 11 21
FemaIes 18 5 20 2 38 7

Degree
No B.S. 15 0 18 0 33 0
B.S. or Higher 10 0 6 0 16 0
Bachelor’s Only 0 6 0 6 0 12
Master’s or Higher 0 8 0 8 0 16

M a r ital S t a t u s
Single 4 2 6 1 10 3
Married 21 12 18 13 39 25

Midterm Score
(100-point s c a l e )
Mean 78.7 79.1 81.3 83.1 80.1 81.0
S.D. 12.2 11.0 12.2 9.4 12.1 10.2

Test of Hypothesis Three: Minutes to Take the Examination

A t-test used to compare the two groups indicated there was a sig-
nificant difference in minutes required to complete the examination in
Replication 1. The microcomputer group took 5.7 minutes longer to com-
p l e t e t h e e x a m i n a t i o n ( 3 2 . 7 v s . 2 7 . 0 ; t[47]=2.36, p<.05). There was no
significant difference in minutes needed to complete the examination in
Repl ication 2 ( m i c r o c o m p u t e r , 2 2 . 0 v s . 2 7 . 4 c o n t r o l ; t[ 16.15]=-1.40,
p>.05). H y p o t h e s i s T h r e e w a s r e j e c t e d f o r R e p l ication 1 but not for
Repl ication 2 . T h e c o n t r o l g r o u p n e e d e d 2 7 m i n u t e s i n b o t h replica-
tions, but the microcomputer group needed 10 minutes less in Replication
2 (32 v s . 22).

Conclusions and Implications

Conclusions and implications were formulated with the knowledge

that subjects for this study used microcomputer technology extensively
d u r i n g a c o m p u t e r a p p l ications c o u r s e . In this investigation, final
exami n a t i o n s c o r e s w e r e i n d e p e n d e n t o f t h e m e t h o d o f t e s t i n g . Further,
the objective midterm examination administered by paper and pencil pro-
cedures was moderately to highly correlated with the final examination.
This suggests that how students scored on the final examination was best
indicated by scores made on the midterm examination and not by the
method of testing.

5
Table 2

ANCOVA of Final Examination Scores for Replications 1 and 2

n Adjusted Means S.D.

Repl ication Replication Repl ication

Group 1 2 1 2 1 2

Treatment 24 14 28.4 26.5 4.27 4.29

Control 25 14 28.9 26.3 4.58 4.27

df ss MS F

Replication RepI ication Replication RepI icat ion

Source 1 2 1 2 1 2 1 2

Midterm
(Covariate) 1 1 565.8 90.3 565.3 90.3 69.5* 5.82*

Treatment 1 1 2.7 .3 2.7 .3 .3 .02

Residual 46 25 374.6 388.1 8.1 15.5 - -

TotaI 48 27 943.1 478.7

*p<.05.

T h e p o s i t i v e a t t i t u d e s o f t h e t r e a t m e n t a n d c o n t r o l g r o u p s i m m e d i-
ately after the final examination would suggest that the method of test-
ing was not a major determinant of how students felt about computers.
The fact that the subjects were completing a computer applications
course would suggest that the course and factors other than the method
of testing were probably responsible for the positive attitudes about
computers.

S i n c e t i m e r e q u i r e m e n t s f o r t h e m icrocomputer t e s t i n g p r o c e d u r e
w e r e n o t c o n s i s t e n t o v e r t w o r e p l i c a t i o n s , the method of testing as welI
a s o t h e r f a c t o r s a p p e a r t o i n f I uence t h e t i m e n e e d e d f o r t h i s t e s t i n g
procedure.

Recommendations

Additional research is needed in other classroom settings to see if

consistent findings about cognitive performance, time requirements and
attitude are achieved. Further, instructors who use this medium to
a d m i n i s t e r classroom e x a m i n a t i o n s s h o u l d b e c e r t a i n s t u d e n t s c a n u s e t h e
technology effectively since subjects for this study used microcomputers
throughout a semester-long course.

6
 References

B e c k e r , W . J . , & S h o u p , W . D. ( 1 9 8 5 , S u m m e r ) . The microcomputer as an

instructional tool. The Journal of the American Association of
Teacher Educators in Agriculture, - 26(2), 6 5 - 7 2 .

B i s k i n , B . H . , & Kolotkin, R. L. (1977). Effects of computerized admin-

istration on scores on the Minnesota Multiphasic Personality Inven-
tory. Applied Psychological Measurement, L 543-549.

Bowen, B . E . (1984). Microcomputer competencies needed by Cooperative

E x t e n s i o n S e r v i c e a g e n t s a n d v o c a t i o n a l a g r i c u l t u r e t e a c h e r s i n Mis-
sissippi. Unpublished report, Mississippi State University, Depart-
ment of Agricultural and Extension Education, Mississippi State.

C a m p b e l l , D. T . , & Stanley, J. C. (1963). Experimental and quasi-exper-

imental designs for research. Ch i cago: R a n d M c N a l l y Col l e g e Pub-
Iishing.

CantrelI, M. J. (1982). An assessment of attitudes, needs, and delivery

system for microcomputer applications by agricultural and extension
educators in Mississippi. Unpublished doctoral dissertation, Missis-
sippi State University, Mississippi State.

C h a s e , S. A . , G o r d o n , R . , & Makin, R . C . ( 1 9 8 4 ) . A s y s t e m f o r e v a l u a t -
ing microcomputer courseware for vocational and technical education,
final r e p o r t . C o l u m b u s : T h e National C e n t e r f o r R e s e a r c h in V o c a -
tional Education.

C l a r k , R . E . (1983). Reconsidering research on learning from media.

Review of Educational Research, 53, 445-459.

Cory, C. H. (1977). R e l a t i v e u t i l i t y o f c o m p u t e r i z e d v e r s u s paper-and-

pencil tests for predicting job performance. Applied Psychological
M e a s u r e m e n t , 1, 5 5 1 - 5 6 4 .

H u d s o n , C . J . (1983). Teacher competencies needed to utilize microcom-

puters in vocational agriculture. Unpublished doctoral dissertation,
Virginia Polytechnic Institute and State University, Blacksburg.

Mi ller, W. W., & F o s t e r , R . M . ( 1 9 8 5 , S p r i n g ) . An assessment of micro-

computer competencies needed by vocational agriculture instructors in
Nebraska and Iowa. The Journal of the American Association of
T e a c h e r E d u c a t o r s i n Agriculture,26(1), 30-38.

Rohrbach, N. F. (1983). Microcomputer use in teaching graduate students

i n a g r i c u l t u r a l education. Unpublished doctoral dissertation, Uni-
versity of Missouri-Columbia.

R o t h , G . L., T e s o l o wski, D . G . , Rankin, R . A . , & B l a c k m a n , H . S . (1984,

March). G e t t i n g r e a d y f o r m i c r o s . VocEd, 59(3), 3 0 - 3 1 .

S p e c t o r , P . E . ( 1 9 8 1). Research designs. Beverly Hills, CA: Sage Pub-

Iications.