CYBERPSYCHOLOGY & BEHAVIOR

Volume 11, Number 4, 2008

© Mary Ann Liebert, Inc.
DOI: 10.1089/cpb.2007.0080

Enhancing Computing Skills of Low-Achieving Students

via E-Learning: A Design Experiment of Web-Based,
Problem-Based Learning and Self-Regulated Learning

TSANG-HSIUNG LEE, Ph.D., 1 PEI-DI SHEN, Ph.D.,2 and CHIA-WEN TSAI, Ph.D.3

ABSTRACT

A design experiment was conducted applying Web-based problem-based learning (PBL), self-
regulated learning (SRL), or their combination to help low-achieving students improve their
skills of deploying application software in a compulsory course at a vocational school in Tai-
wan. The effects were encouraging and mostly positive. However, students’ inertia to change
routines of learning inherited from a culture of taking standard tests found in e-learning prac-
tice is quite common. A teacher’s awareness of these changes and associated resistance may
increase likelihood of e-learning success for future trials. Some modifications of the design
are thus suggested for the next round of the experiment.

INTRODUCTION order to increase students’ learning motivation and

develop practical skills, problem-based learning

P ROFESSIONALS WITH A VOCATIONAL DEGREE repre-

sent a major portion of the workforce in Taiwan.
Because of the system of university entrance exam-
inations and the high value Chinese culture places
on academic performance, most of the students who
choose to attend technological and vocational col-
leges tend to be those who do not perform well in
academic subjects. In general, students in the voca-
tional system show low confidence and motivation
in learning,1 have low interest and negative attitude
toward their learning,2 have inferior academic per-
formance at the high school level compared with
their counterparts in national and private universi-
ties,3 and so on. Teaching in such a context is a great
challenge to most educators.
The teaching of application software traditionally
emphasizes memorization by applying short, dis-
jointed, lack-of-context examples. The results of
such teaching can hardly be deemed as effective. In
(PBL) is considered to be a most appropriate peda-
gogy. For example, Yip4 points out that PBL can en-
hance competencies both in professional and infor-
mation systems education. PBL uses real-world,
simulated, contextualized problems in practice to
motivate, focus, and initiate content learning and
skill development.5–7 Students embedded in this
particular context may experience increased in-
volvement,8 attain significantly higher scores,9 and
so on. Therefore, we tend to believe that PBL could
help low-achieving students to develop practical
skills with application software.
Physical absence of an instructor and increased
responsibility of learners to effectively engage in
learning tasks may present difficulties, particularly
to those with low self-regulatory skills.10 It is sug-
gested that low-achieving students would benefit
from online learning.11,12 However, implementing
e-learning for low-achieving students inevitably

1Department of Information Management, National Chengchi University, Taipei, Taiwan, Republic of China.
2Department of Information Management, Ming Chuan University, Taipei, Taiwan, Republic of China.
3Graduate School of Management, Ming Chuan University, Taipei, Taiwan, Republic of China.

431
432 LEE ET AL.

runs high risks. For instance, Yang and Tung13 in- PBL non-PBL
dicated that vocational students were more Inter-
net-addicted than general students. When students
enter the traditional classroom, they might browse
shopping Web sites or play online games while a
SRL The most Medium effect
teacher is lecturing in the classroom. With these dis-
(C1, N = 30) (C3, N = 24)
tractions, it is quite difficult for students to concen-
trate on learning, online learning in particular. Stu-
dents’ motivation may benefit from Web-based
instruction with self-regulated learning (SRL)
strategies.14 To respond to this challenge, we pro-
pose an approach that may help students regulate non-SRL
their own learning, even in an online environment. Medium effect No difference
Few studies have discussed effective online (C2, N = 25) (C4, N = 27)
teaching methods for low academic achievers. To
bridge this gap of the literature, this study re-
designed two courses in application software to in-
tegrate innovative teaching methods and learning FIG. 1. Expected effects of variation in instructional
technologies to help students learn and apply what methods in the first phase.
they have learned. A two-phase design experiment
was conducted in this study, which was the first
trial of this kind for the instructor (third author),
who is experienced and successful in traditional first phase of the experiment, we developed con-
pedagogy and in helping low-achieving students structs and then explored their possible effects on
pass certification tests. Under the condition of lim- student learning of computing skills, largely based
ited financial and technical support, the instructor on related research on teaching methods: PBL17–19
20–22
sought the appropriate technologies and software and SRL. In the next phase, we revised the de-
for the development of the course Web site, adopted sign based on inadequacies found in the first phase.
and adapted innovative teaching methods and tech-
nologies suitable for this particular context, learned
how to install and deploy the technologies, and ex- THE LEARNING DESIGN IN
perimented on how to implement the teaching THE FIRST PHASE
methods in an online course. While climbing a steep
learning curve was inevitable, the design and im- Course setting
plementation in both pedagogies and course Web
The course under study in the first phase was a
site was far from elaborated and matured in the first
semester-long, 2-credit-hour class targeted at first-
attempt. After the first implementation of the Web-
year college students from different majors. There
based pedagogies, we evaluated the preliminary re-
were two classes in this study. One was a PBL class,
sults and advised refinements for the second phase
and the other a non-PBL class. PBL class students
of the experiment.
solved a series of authentic tasks by applying Mi-
crosoft Office. In addition, each class had two
groups divided according to whether or not the stu-
DESIGN EXPERIMENT
dents were involved in SRL. Therefore, there were
four groups (conditions) in this study.
The emerging design research paradigm regards
design as a strategy for developing and refining the-
Experimental design and procedure
ories. It explicitly takes advantage of the design pro-
cess to advance the researchers’ understanding of The experimental design was a 2  2 (PBL vs.
teaching, learning, and educational systems.15 The non-PBL  SRL vs. non-SRL) factorial pretest–post-
learning or instructional innovations may be incor- test design (see Fig. 1). Students from four groups
porated into a theory-based design and imple- solved the same task but in different learning con-
mented in a live setting over a series of continuing ditions. Participants were randomly assigned to one
research cycles.16 of the four experimental conditions. The course de-
Developing appropriate treatments for meeting sign in the present study consisted of three subse-
special needs of low achievers was the goal. In the quent modules: Word, Excel, and PowerPoint. A
A DESIGN EXPERIMENT OF WEB-BASED PBL AND SRL
TABLE 1. INDEPENDENT SAMPLES T TEST: DIFFERENCES IN IMPROVEMENT
PBL (SRL) AND NON-PBL (NON-SRL)
Groups n Mean SD
PBL 55 67.09 9.779
non-PBL 51 56.75 12.673
SRL 54 66.01 11.305
non-SRL 52 58.08 12.195
**p  0.05.
skill test was administered after the completion of
each module.
PBL treatment. The teacher demonstrated first
how he could approach the situation and solve the
problem with the help of a packaged software ap-
plication. Then, he proposed the problems and sit-
uation for students to solve. For example, students
played roles as if they were employed by an online
game company, and a marketing manager asked
them to compare expenses resulting from different
distribution channels. They had to survey and then
complete a worksheet with some graphs to contrast
differences between channels. In addition to teach-
ing students to use the application software, simi-
lar situations and related applications were also dis-
cussed in the class.
SRL treatment. Students in SRL groups were
given a 2-hour group lecture on how to manage
study time and regulate their learning. Content of
this SRL course was composed of four processes ad-
dressed by Zimmerman et al.:23 self-evaluation and
monitoring, goal-setting and strategy planning,
strategy implementation and monitoring, and mon-
itoring of the outcome of the strategy. Students in
SRL groups also were required to prepare and read
the textbook regularly before classes and to review
or practice the computing skills they had learned af-
ter school. They were also required to record their
learning behavior every week. Data was recorded
on the course Web site instead of in their notebooks
in order to prevent falsification of records.
Data collection and measurement approach
To examine levels of change manipulated by vari-
ations in experimental conditions, the authors mea-
sured students’ application software skills as a base-
line before the students entered the class. The
pretest grades show that their computer skills were
similarly low and differences were not statistically
significant. Skill tests were again given at the end
433
OF GRADES BETWEEN
F t value df p-value
5.430 4.722 104 0.000**
— — — —
0.769 3.473 104 0.001**
— — — —
of each module. The questions in the test related to
the content and examples in the course. A surrogate,
representing skills of application software, was av-
eraged from the scores of these three tests. Finally,
the measure of enhancement of skills in using the
application software in a module was the result of
the student’s grade minus his pretest on application
software. We tested the differences in the skills en-
hancement under different conditions.
In addition to the quantitative data, the third au-
thor in this study interviewed eight students from
the four groups. Two students were chosen ran-
domly from each of the four groups. The interviews
were tape-recorded, transcribed, and analyzed.
EVALUATION AND LESSON LEARNED
As revealed in the interview data, most students
in the PBL class had received a spoon-fed comput-
ing education in their senior high schools. Despite
having taken computing courses for three years,
they still could not apply skills and knowledge
learned to real-life problems. After the intervention
of PBL, they agreed that this series of authentic
problem solving had raised their interest and
helped them understand the situations and how to
apply their acquired skills. Results from Table 1
show that the enhancement of students’ computing
skills in terms of their averaged grades on three
modules in the PBL class (67.09) was significantly
higher than that in the non-PBL class (56.75), which
was consistent with the findings of the qualitative
data.
In contrast with students’ welcome of the imple-
mentation of PBL, students experienced difficulties
and persistently struggled to self-regulate their
learning. In the interview, most students in the SRL
groups admitted that they were still not used to the
practices of SRL, such as preview before class and
review after class. They were not willing to spend
additional time on their coursework after school
434 LEE ET AL.

TABLE 2. ONE-WAY ANOVA: DIFFERENCES IN IMPROVEMENT OF

GRADES AMONG FOUR GROUPS

Mean difference

(I) Group (J) Group (I–J) Significance

1 2 5.862 0.262
— 3 8.436* 0.047
— 4 17.064* 0.000
2 1 5.862 0.262
— 3 2.574 0.872
— 4 11.202* 0.004
3 1 8.436* 0.047
— 2 2.574 0.872
— 4 8.628* 0.048
4 1 17.064* 0.000
— 2 11.202* 0.004
— 3 8.628* 0.048

*The mean difference is significant at the 0.05 level.

and resisted taking responsibility for their own
learning. Worse, three students in the C1 group
even requested to drop out of this group because
they did not want to spend more time on this course
after school. However, the teacher still encouraged
the students in SRL groups and insisted that stu-
dents should persist in regulating their learning. At
the end of the course, the exhilarating achievement
suggested that this effort was valuable. The quanti-
tative results from Table 1 supported that the en-
hancement of computing skills in the SRL group
(66.01) was significantly higher than in the non-SRL
group (58.08).
Finally, the data in Table 2 shows that the com-
bination of PBL and SRL resulted in that group hav-
ing the highest grades among the four groups. The
enhancement in computing skills of students in C1
was higher than those of C3 and C4 in a statistically
significant manner, and also higher than C2, though
insignificantly. Thus, we may conclude that the
skills of application software were improved
through Web-based problem-based learning and
self-regulated learning interventions.
pleted by the end of the semester. The course fo-
cuses on the development of students’ skills in pro-
gramming a Web page and planning a Web site. Stu-
dents must solve a series of tasks using several
software applications, including Web site building,
Web page programming, graphic design, and file
transfer software. Therefore, a course such as Web
Page Programming and Website Planning may pro-
vide a broader stage for students to learn and ap-
ply what they have learned.
Refinement for the treatments of PBL and SRL
PBL treatment. We plan to scaffold student learn-
ing in a more systematic manner in the second
phase. In this regard, Barrows’s whiteboards
PBL non-PBL
A significant effect
SRL
(C1 Group)

PROPOSED LEARNING DESIGNS FOR

THE SECOND PHASE
No difference
Better select of courses for the experiment non-SRL
(C2 Group)
The second phase of the experiment will be con-
ducted with students taking the course “Web Page
Programming and Website Planning.” This course
is appropriate for our experiment because it offers FIG. 2. The expected effects of variation in instructional
real products—Web sites—to be designed and com- methods in the second phase.
A DESIGN EXPERIMENT OF WEB-BASED PBL AND SRL
model24 will be applied to help students solve prob-
lems and learn from the problem-solving processes.
The whiteboard supports students’ planning and
monitoring as they identify what should be
recorded on or later removed from the board.25 The
students of the PBL group will receive a 1-hour lec-
ture about PBL. The PBL lecture will consist of the
four-column chart put forth by Barrows: Facts,
Ideas, Learning Issues, and Action Plan.
Students in the PBL group will be required to
build a prototype of their Web site. In each class,
the teacher will provide guidelines and review mis-
takes and bugs in the students’ Web pages. The stu-
dents will then reconsider and revise their Web
pages according to their new knowledge. The
knowledge and skills they learn will immediately
be applied in their Web pages. Applying this model
makes the problem-solving processes more feasible
for low achievers.
SRL treatment. The instructor faced many chal-
lenges when implementing SRL among the low
achievers in the first phase. The rigid and extra re-
quirements of SRL may have caused the low achiev-
ers’ resistance and antipathy to the treatment. We
therefore redesigned the treatment of SRL, particu-
larly for low-achieving students.
In Zimmerman and Martinez-Pons’s study,26
high-achieving students utilized the strategies of re-
viewing notes, keeping records, and monitoring
their work often. Moreover, they used these strate-
gies significantly more frequently and consistently
than did the low achievers. Ross27 also indicated
that keeping records and self-evaluation were the
two most-often-mentioned learning strategies in his
analysis. We plan to adopt Zimmerman and Mar-
tinez-Pons’s26 and Ross’s27 strategies and to em-
phasize keeping records, reviewing notes, and self-
evaluation when implementing SRL in the second
phase.
Refinement of the experimental design and procedure
In the second phase, we plan to divide students
only into two groups: one a PBL and SRL group and
the other a non-PBL and non-SRL group. There are
two main reasons why we adopt this simpler group-
ing method. First, preliminary results of the exper-
iment clearly suggest that a combination of Web-
based PBL and Web-based SRL is the most effective
of the approaches we used. Second, PBL facilitates
SRL because it places the responsibility on the stu-
dents to discover information, to coordinate actions
and people, to monitor understanding, and to reach
goals.28 Therefore, the experimental design in the
second phase is a PBL and SRL pedagogy versus the
435
traditional teaching method (non-PBL and non-
SRL) pretest–posttest design (see Fig. 2). Students
in the two groups solve the same tasks but under
different learning conditions.
CONCLUSION AND IMPLICATIONS
In our study, effects of Web-based PBL, Web-
based SRL, and their combination on enhancing stu-
dents’ skills in deploying application software dif-
fered significantly. It is suggested that teachers
should adopt these innovative instructional meth-
ods and technologies and then redesign their
courses to fully exploit the benefits of Web-based
learning environments.
To develop students’ skills and regulate their
learning in online courses, the two innovative teach-
ing methods, Web-based PBL and Web-based SRL,
could be applied simultaneously rather than sepa-
rately. Researchers and teachers may imitate or
modify this design to fit their needs. Students’ learn-
ing could be improved through online courses that
are thoughtfully considered and designed.
The implementation of innovative teaching meth-
ods via e-learning and the redesign of online courses
are imperative. We hope the lessons we learned and
the problems we experienced in the first phase, and
the proposal for refinements in the second phase,
may provide valuable insights and shed light on
new and effective practices for schools (particularly
vocational schools), scholars, and teachers planning
to be or presently engaged in implementing e-learn-
ing.
REFERENCES
1. Su MHM. A study of EFL technological and voca-
tional college students’ language learning strategies
and their self-perceived English proficiency. Elec-
tronic Journal of Foreign Language Teaching 2005;
2:44–56.
2. Chen CH, Tien CJ. (2005) Market segmentation anal-
ysis for taking skill test by students in an institute
of technology. www.projects.education.uts.edu.au/
RWL4/RWL4Papers/8167.doc (accessed Aug. 15,
2007).
3. Lee YM. An investigation of Taiwanese graduate stu-
dents’ level of civic scientific literacy. Ph. D. disserta-
tion, University of Texas, 2003.
4. Yip W. (2001) Utilisation of the problem-based learn-
ing approach facilitated by information technology
to teach information on systems development.
http://citeseer.ist.psu.edu/update/535807 (accessed
Aug. 15, 2007).
436
5. Boud D, Feletti G. (1991) The challenge of problem-based
learning. London: Kogan Page.
6. Bruer JT. (1993) Schools for thought: a science of learning
in the classroom. Cambridge, MA: MIT Press.
7. Williams SM. Putting case-based learning into con-
text: examples from legal, business, and medical ed-
ucation. Journal of Learning Sciences 1993; 2:367–427.
8. Chanlin LJ, Chan KC. Assessment of PBL design ap-
proach in a dietetic Web-based instruction. Journal of
Educational Computing Research 2004; 31:437–52.
9. Polanco R, Calderón P, Delgado F. Effects of a prob-
lem-based learning program on engineering students’
academic achievements in a Mexican university. In-
novations in Education & Teaching International
2004; 41:145–55.
10. Dabbagh N, Kitsantas K. Using Web-based pedagog-
ical tools as scaffolds for self-regulated learning. In-
structional Science 2005; 33:513–40.
11. Beal CR, Walles R, Arroyo I, et al. Online tutoring for
math achievement testing: a controlled evaluation.
Journal of Interactive Online Learning 2007; 6:43–55.
12. Graff MG, Lebens M. (2007) Web-based direct in-
struction in mathematics for low achievers. In Pro-
ceedings of the 6th IASTED Conference. Chamonix,
France, pp. 98–103.
13. Yang SC Tung CJ. Comparison of Internet addicts and
non-addicts in Taiwanese high school. Computers in
Human Behavior 2007; 23:79–96.
14. Chang MM. Applying self-regulated learning strate-
gies in a Web-based instruction—an investigation of
motivation perception. Computer Assisted Language
Learning 2005; 18:217–30.
15. Edelson DC. Design research: what we learn when we
engage in design. Journal of the Learning Science
2002; 11:105–21.
16. Bunderson CV. (2000) Design experiments, design sci-
ence, and the philosophy of measured realism: philo-
sophical foundations of design experiments. Sympo-
sium session at the annual meeting of the American
Educational Research Association, New Orleans, LA.
17. Greening T, Kay J, Kingston J. (1996) Problem-based
learning of first year computer science. In Proceedings
of the First Australasian Conference on Computer Science
Education, Sydney: ACM Press, pp. 13–8.
18. Perels F, Gürtler T, Schmitz B. Training of self-regu-
latory and problem-solving competence. Learning &
Instruction 2005; 15:123–39.
LEE ET AL.
19. Polanco R, Calderón P, Delgado F. Effects of a prob-
lem-based learning program on engineering students’
academic achievements in a Mexican university. In-
novations in Education & Teaching International
2004; 41:145–55.
20. Kauffman DF. Self-regulated learning in Web-based
environments: instructional tools designed to facili-
tate cognitive strategy use, metacognitive processing,
and motivational beliefs. Journal of Educational Com-
puting Research 2004; 30:139–61.
21. Reed JH, Schallert DL, Deithloff LF. Investigating the
interface between self-regulation and involvement
processes. Educational Psychologist 2002; 37:53–7.
22. Zimmerman BJ. (1998) Developing self-regulation cy-
cles of academic regulation: an analysis of exemplary
instructional model. In Schunk DH, Zimmerman BJ,
eds. Self-regulated learning: from teaching to self-reflec-
tive practice. New York: Guilford, pp. 1–19.
23. Zimmerman BJ, Bonner S, Kovach R. (1996) Developing
self-regulated learners: beyond achievement to self-efficacy.
Washington, DC: American Psychological Association.
24. Barrows HS. (1992) The tutorial process. Springfield:
Southern Illinois University School of Medicine.
25. Hmelo-Silver CE. Problem-based learning: what and
how do students learn? Educational Psychology Re-
view 2004; 16:235–66.
26. Zimmerman BJ, Martinez-Pons M. Development of a
structured interview for assessing student use of self-
regulated learning strategies. American Educational
Research Journal 1986; 23:614–28.
27. Ross JD. (1999) Regulating hypermedia: self-regulation
learning strategies in a hypermedia environment. [Ph. D.
dissertation]. Virginia Polytechnic Institute and State
University.
28. Paris SG, Paris AH. Classroom applications of re-
search in self-regulated learning. Educational Psy-
chologist 2001; 36:89–101.
Address reprint requests to:
Dr. Tsang-Hsiung Lee
Department of Information Management
National Chengchi University
64, Sec.2, ZhiNan Rd., Wenshan District,
Taipei City 11605, Taiwan
Republic of China
E-mail: thlee@mis.nccu.edu.tw