Professional Documents
Culture Documents
How Pre Service Teachers Understandand Perform Science Process Skills 4238
How Pre Service Teachers Understandand Perform Science Process Skills 4238
How Pre Service Teachers Understandand Perform Science Process Skills 4238
www.ejmste.com
APA style referencing for this article: Chabalengula, V.M., Mumba, F. & Mbewe, S. (2012) How
Pre-service Teachers’ Understand and Perform Science Process Skills. Eurasia Journal of
Mathematics, Science & Technology Education, 8(3), 167-176.
Linking to this article: DOI: 10.12973/eurasia.2012.832a
URL: http://dx.doi.org/10.12973/eurasia.2012.832a
Terms and conditions for use: By downloading this article from the EURASIA Journal website you agree that
it can be used for the following purposes only: educational, instructional, scholarly research, personal use. You
also agree that it cannot be redistributed (including emailing to a list-serve or such large groups), reproduced in
any form, or published on a website for free or for a fee.
Disclaimer: Publication of any material submitted by authors to the EURASIA Journal does not necessarily
mean that the journal, publisher, editors, any of the editorial board members, or those who serve as reviewers
approve, endorse or suggest the content. Publishing decisions are based and given only on scholarly
evaluations. Apart from that, decisions and responsibility for adopting or using partly or in whole any of the
methods, ideas or the like presented in EURASIA Journal pages solely depend on the readers’ own judgment.
© 2013 by ESER, Eurasian Society of Educational Research. All Rights Reserved. No part of this publication may be
reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any
information storage and retrieval system, without permission from ESER.
State of the literature elementary and middle school teachers. As a result, they
proposed that science process skills serve as a very
• Though science process skills form an integral part important way for beginning teachers to learn about
of inquiry teaching (Anderson, 2002; Minstrell & science teaching. Therefore, teachers should possess a
van Zee, 2000), and emphasized in science strong conceptual understanding and be able to perform
education reforms (AAAS, 1993; NRC, 1996), well on the science process skills if they have to
some studies have demonstrated that pre-service effectively teach them in their classrooms. Scharmann
teachers have poor understanding of the process (1989) points out that science process skills foster
skills (e.g. Emereole, 2009; Mbewe, Chabalengula significant increases in subject matter understanding and
& Mumba, 2010). science content knowledge, arguing that science content
• A few studies in the domain of science process and science process skills should be taught together as
skills rarely discuss elementary education pre- they complement each other. Similarly, Rillero (1998)
service teachers’ conceptual understanding of and points out that both science content and science process
performance on the science process skills. skills are mutually valuable and complementary. Settlage
and Southerland (2007) also emphasize how the science
Contribution of this paper to the literature process skills provide a foundation for inquiry.
Though science process skills form an integral part
• Our pre-service teachers’ limited conceptual
of inquiry teaching (Anderson, 2002; Glynn & Duit,
understanding of the science process skills is
1995; Minstrell & van Zee, 2000), and emphasized in
consistent with the findings in previous studies
science education reforms (AAAS, 1993; NRC, 1996),
(e.g. Emereole, 2009; Farsakoglu, Sahin, Karsli,
some studies have demonstrated that pre-service
Akpinar & Ultay, 2008; Mbewe, Chabalengula &
teachers have poor understanding of the process skills
Mumba, 2010).
(e.g. Emereole, 2009; Mbewe, et al., 2010). For instance,
• However, a unique contribution to the literature is Emereole (2009) investigated conceptual understanding
that whilst pre-service teachers were unable to of science process skills among high school pre-service
provide correct definitions of the science process science teachers in Botswana. Emereole’s study found
skills, they performed well on the test that that pre-service high school science teachers did not
involved novel situations of the process skills. A have sufficient conceptual understanding of science
possible explanation for this good performance process skills. Similarly, many studies have examined
could be that the performance test items were teachers’ understanding of inquiry and have concluded
presented in a real-world type situation, which that they lack a sufficient understanding of such a
could have assisted the pre-service teachers in process (Lotter, Harwood, & Bonner, 2007). Mbewe,
solving them because they were familiar with the Chabalengula, and Mumba (2010) found that nearly all
contexts. pre-service elementary teachers were unable to provide
• The poor conceptual understanding held by the correct definitions and explanations of the basic and
pre-service teachers in our study is of great integrated science process skills, but the majority of
concern and a call to action on the part of science them provided partially correct and incorrect answers.
teacher education and professional development Yet, science process skills are essential for teaching
programs. As such, we recommend an explicit science content knowledge and scientific inquiry (Cain,
intervention on science process skills in teacher 2002).
education programs for pre-service teachers to Although previous research has examined teachers’
develop conceptual understanding of the science conceptual understanding of the science process skills,
processes. few studies, if any, have examined the extent to which
elementary education pre-service teachers are able to
The premium placed on conceptual understanding is translate their conceptual understanding to novel and
illustrated by its prominence as an objective in the everyday life situations involving science process skills.
National Assessment of Educational Progress (NAEP) Yet, it is important for the pre-service teachers to
science assessment (O’Sullivan, Reese, & Mazzeo, demonstrate a sound conceptual understanding and be
1997). Among the definitions of characteristic elements able to perform well on test items involving novel
of knowing and doing science, conceptual situations of the science process skills in order to
understanding is included. Similarly, Settlage and effectively create conditions for their development
Southerland (2007) viewed the science process skills as among their students. Therefore, this study attempted to
an integral feature of the actions of the scientific culture, examine pre-service teachers’ conceptual understanding
although not as all there is to science. Settlage et al of and performance on science process skills.
further state that teaching with an eye toward science
process skills is an appropriate entry point for beginning
168 © 2012 ESER, Eurasia J. Math. Sci. & Tech. Ed., 8(3), 167-176
Teachers’ Understanding of and Performance on Process Skills
© 2012 ESER, Eurasia J. Math. Sci. & Tech. Ed., 8(3), 167-176 169
V. M. Chabalengula, F. Mumba & S. Mbewe
Table 2. Percentages for Conceptual Understanding of Process Skills by Course Level
Incorrect (%) Partially Correct (%) Correct (%)
SPS Name SPS Type
IC AC IC AC IC AC
Classifying Basic 8.3 16.1 90.0 77.4 1.7 6.5
Observing Basic 15.0 16.1 85.0 83.9 0.0 0.0
Measuring Basic 23.3 29.0 76.7 67.7 0.0 3.2
Predicting Basic 40.0 64.5 58.3 35.5 1.7 0.0
Communication Basic 61.7 22.6 38.3 74.2 0.0 3.2
Inferring Basic 75.0 67.7 25.0 32.3 0.0 0.0
Hypothesizing Integrated 41.7 67.7 58.3 25.8 0.0 6.5
Interpreting data Integrated 55.7 51.6 43.3 35.5 0.0 12.9
Experimenting Integrated 61.7 35.5 36.7 61.3 1.7 3.2
Formulating models Integrated 88.3 93.5 11.7 6.5 0.0 0.0
Identifying variables Integrated 88.3 96.8 11.7 3.2 0.0 0.0
Graphing Integrated 95.5 100.0 3.3 0.0 1.7 0.0
IC: Introductory course
AC: Advanced Course
This shows that a large number of pre-service teachers
Data analysis did not have a complete conceptual understanding of
the science process skills. Of particular interest, the
Data analyses involved computing reliability values results show that nearly all the pre-service teachers
for conceptual understanding and performance tests. failed to provide correct definitions/explanations for
The SPCUT instrument had a very high Cronbach alpha observation, measurement, inferring, formulating
reliability value of 0.935. To ensure validity of the models and identifying variables. The analysis of the
compiled SPPT, three science education experts were open-ended responses revealed the following trends in
asked to identify each question with its associated the definitions or explanations provided by the
process skill being tested. A Cohen’s kappa score of participants: many participants used tautology in
0.764 was found, and indicates a strong inter-rater defining the terms in almost all categories. Many
reliability for the performance test. participants failed to distinguish between predicting and
Participants’ responses to the SPCUT were scored inferring. A number of them used everyday language in
and categorized as correct, partially correct, and defining communication. There was frequent use of the
incorrect. The responses were compared to the standard phrase “educated guess” on defining hypothesis, and
answers used by Emereole (2009). The correct response prediction; many defined observation in terms of senses
was assigned a value of 3, partially correct response was with the sense of sight being most prevalent; many
assigned a value of 2, and an incorrect response was defined classification on basis of similarities ignoring the
assigned a value of 1. A response was considered correct differences; and many did not mention measuring tools
if it contained all the aspects in the standard answers when defining measurement.
completely. The response was considered partially
correct if it contained some of the aspects in the Conceptual understanding by demographics
standard answer. The response was considered incorrect
if it was either completely wrong when compared to the The results show that there were no significant
standard answer or if the question was not answered or differences between pre-service teachers’ conceptual
left blank. Then, the responses were analyzed and coded understanding of science process skills and their
to identify recurring themes. Participants’ responses to demographic variables. That is, this group of pre-service
the SPPT were either scored as correct or incorrect as teachers’ conceptual understanding was similar
there was only one correct answer for each test item. regardless of whether: they were enrolled in an
Then, statistical tests were done, which included t-tests, introductory or advance science education course, their
ANOVA and person correlation coefficient. teaching subject major was science or not, they had
taken the integrated science core course or not, they had
RESULTS taken 1-3 or 4-6 university science courses, and whether
they were male or female.
Conceptual understanding of science process
skills
Overall conceptual understanding level
Table 2 below shows that very few pre-service
teachers in both groups had “correct” answers. A Table 4 shows that the overall conceptual
majority had “partially correct” and “incorrect” answers. understanding of the process skills was not good for
170 © 2012 ESER, Eurasia J. Math. Sci. & Tech. Ed., 8(3), 167-176
Teachers’ Understanding of and Performance on Process Skills
Performance on individual science process skills To determine the relationship between conceptual
understanding and performance, a Pearson correlation
In order to determine how well participants coefficient was computed. The results showed a low,
performed on each process skill, test items for each skill positive, but non-significant correlation between
were analyzed separately. Table 6 below shows that the conceptual understanding and performance, r = 0.050, n
participants performed better on the skills of modeling, = 91, p = 0.637. A positive relationship implies that
predicting, inferring, classification, and interpreting data. To the conceptual understanding is required in order for one to
contrary, they performed most poorly on experimenting, perform on science process skills tasks. However, this
identifying variables, graphing, communicating, hypothesizing and relationship is not significant enough to be certain that
observing. conceptual understanding and performance are
correlated.
Comparing performance on process skills by
demographics DISCUSSION
The results in Table 7 show that there were no The purpose of this study was to examine pre-
significant differences between performance and service teachers’ conceptual understanding and
demographic variables. That is, pre-service teachers’ performance on the science process skills. The results
performance is similar regardless of whether: they were show that pre-service teachers had a poorer conceptual
enrolled in an introductory or advance science education understanding of the science process skills than their
course, their teaching subject major was science or not, performance in the science process skills. The
they had taken the integrated science core course, they definitions and explanations provided by the
had taken 1-3 or 4-6 university science courses, and participants revealed that they did not have complete
whether they were male or female. conceptual knowledge of the science processes, and had
difficulties in defining and explaining processes such as
Ratings on science process skills for conceptual inferring, communicating, formulating hypothesis, experimenting,
understanding and performance formulating models, interpreting data, predicting, identifying
variables and graphing. The poor conceptual
Table 8 shows the ratings to highlight the process understanding revealed in this study is consistent with
skills pre-service teachers understood and performed the findings in previous studies (e.g. Emereole, 2009;
well and poorly. The pre-service teachers exhibited Farsakoglu, Sahin, Karsli, Akpinar &Ultay, 2008). For
better conceptual understanding of the science process example, Emereole (2009) found that pre-service high
skills in this descending order: classifying, observing, school science teachers did not have sufficient
measuring, predicting, hypothesizing, interpreting data, conceptual understanding of science process skills.
experimenting, communicating, inferring, identifying variables, Similarly, studies on teachers’ understanding of inquiry
modeling and graphing. With respect to performance, they have reported that teachers teaching different grade
performed better on the science process skills in this levels lack sufficient understanding of science process
descending order: modeling, predicting, inferring, classifying, skills (Lotter, Harwood, & Bonner, 2007; Luft, 2001).
interpreting data, measuring, observing, hypothesizing,
communicating, graphing, identifying variables and experimenting.
© 2012 ESER, Eurasia J. Math. Sci. & Tech. Ed., 8(3), 167-176 171
V. M. Chabalengula, F. Mumba & S. Mbewe
Table 3. Comparison of conceptual understanding between demographics
Demographic N Mean (SD) t df p-value Sig
Introductory 60 17.5 (1.6)
Science education course level -0.525 89 0.60 NS
Advanced 31 17.7 (2.5)
Science 27 17.4 (1.90)
Teaching subject major -0.594 89 0.55 NS
Non-science 64 17.7 (1.95)
Yes 60 17.9 (1.96)
Taken integrated science core course 1.904 89 0.06 NS
No 31 17.1 (1.79)
1-3 courses 37 17.6 (1.52)
Number of university science courses taken -0.105 89 0.917 NS
4-6 courses 54 17.6 (2.18)
Female 75 17.7 (1.95)
Gender 1.360 89 0.177 NS
Male 16 17.0 (1.75)
Sig at p<.05; NS means Non Significant
172 © 2012 ESER, Eurasia J. Math. Sci. & Tech. Ed., 8(3), 167-176
Teachers’ Understanding of and Performance on Process Skills
Table 7. Comparison of performance between demographics
Demographic N Mean (SD) t df p-value Sig
Introductory 60 32.3 (13.8)
Science methods course 0.652 89 0.516 NS
Advanced 31 30.4 (12.1)
Science 27 29.3 (13.5)
Teaching subject major -1.101 89 0.274 NS
Non-science 64 32.7 (13.1)
Yes 60 32.0 (13.4)
Taken integrated science core course 0.351 89 0.726 NS
No 31 31.0 (13.0)
1-3 courses 37 31.9 (13.5)
Number of college science courses taken 0.141 89 0.888 NS
4-6 courses 54 31.5 (13.2)
Female 75 31.9 (13.2)
Gender 0.371 89 0.712 NS
Male 16 30.5 (13.9)
Sig at p<.05
Table 8. Ratings on Science Process Skills for Conceptual Understanding & Performance
Rating Conceptual Understanding Performance
High Understanding Classifying Modeling
& Performance Observing Predicting
Measuring Inferring
Predicting Classifying
Hypothesizing Interpreting data
Interpreting data Measuring
Experimenting Observing
Communicating Hypothesizing
Inferring Communicating
Identifying variables Graphing
Low Understanding Modeling Identifying variables
& Performance Graphing Experimenting
In this study, the majority of the pre-service teachers concept. However, our results did not support this logic
who attempted to provide traditional and correct – as illustrated by non significant correlation between
definitions of classification and experimenting, also pre-service teachers’ conceptual understanding of and
included additional incorrect statements - suggesting performance on science process skills. Therefore, these
that some participants just rote-learned the definitions. pre-service teachers do not possess adequate conceptual
In this case, it was difficult to determine for sure understanding of the science process skills despite
whether the participants really understood the process performing well on the skills test. However, the finding
skills or not. Similarly, Duit (1984) argues that it is offers some hope in that some pre-service teachers may
difficult to distinguish whether partially correct teach the science process skills because they performed
definitions and explanations of a concept provided by well on the test.
participants are based on their understanding or merely Particularly, the pre-service teachers performed well
rote-learned. on the following skills in descending order: classification,
With respect to performance, the results showed that predicting, experimenting. The skills on which they
pre-service teachers performed better on science performed most poorly were graphing, observation,
process skills compared to their conceptual identifying variables, and measuring. With respect to graphing
understanding. A possible explanation for this good skill, previous research (e.g. Roth, McGinn, & Bowen,
performance could be based on the contexts in which 1998) report that pre-service teachers have graphing
performance test items were presented (Song & Black, difficulties and such difficulties were attributed to a lack
1992). The performance test in this study presented of appropriate training in the graphing skills. Other
these skills in a real-world type situation, which could researchers have also found that teachers have
have assisted the pre-service teachers in solving them difficulties with measurements, especially in science
because they were familiar with the contexts. Multiple experiments (Rollnick, Lubben, Lotz & Dlamini, 2002).
choice questions also allow respondents a greater An interesting finding in this study is on the skills of
chance at guessing the right answer. Ideally, we argue classification and experimenting. In both conceptual
that in order for one to be able to apply their knowledge understanding and performance tests, pre-service
to given situations or scenarios, that individual must at teachers exhibited highest understanding and
least have some conceptual understanding of a given performance on these two skills. One possible
© 2012 ESER, Eurasia J. Math. Sci. & Tech. Ed., 8(3), 167-176 173
V. M. Chabalengula, F. Mumba & S. Mbewe
postulation for this trend is that people in general use REFERENCES
classification to sort out things in everyday life chores –
thereby enabling these pre-service teachers to do well American Association for the Advancement of Science
on this skill. With regard to experimenting, one would (AAAS) (1989). Science for all Americans: Project 2061.
argue that pre-service teachers would often hear or talk Washington, DC.
about experimenting simply because science is American Association for the Advancement of Science.
(1993). Benchmarks for science literacy. New York:
technically an “experiment” subject. Therefore, at one
Oxford University Press.
point or the other these pre-service teachers have always Anderson, R. D. (2002). Reforming Science Teaching: What
heard of the word “experiment” and have been required research says about inquiry. Journal of Science Teacher
to conduct experiments in their teacher education Education, 13(1), 1-12.
science courses. All these experiences would have made Barbosa, P. & Alexander, L. (2004). Science inquiry in the
them more adept at conceptually articulating and CORI framework. In J. T. Guthrie, A. Wigfield, & K. C.
performing well on classification and experimenting Perencevich (Eds.), Motivating Reading Comprehension:
process skills. Concept-Oriented Reading Instruction, pp.113-141. Mahwah,
NJ: Erlbaum.
Implications and recommendations Boardman, L. & Zembal-Saul, C. (2000). Exploring
prospective teachers’ conception of scientific inquiry.
Paper presented at the annual meeting of the National association
The poor conceptual understanding held by the for Research in Science Teaching (NARST), New Orleans,
elementary education pre-service teachers in this study is LA.
of great concern and a call to action on the part of Burns, J. C., Okey, J. R. & Wise, K. C. (1985). Development
science teacher education and professional development of an integrated process skill test: TIPS II. Journal of
programs. As such, we recommend an explicit Research in Science Teaching, 22(2),169-177.
intervention on science process skills in teacher Cain, S. (2002). Sciencing. (4th ed). Upper Saddle River, NJ:
education programs for pre-service teachers to develop Pearson Education.
conceptual understanding of the science processes Dana, T., Boardman, L., Friedrichsen, P., Taylor, J., &
outlined in the national science education reforms and Zembal-Saul, C. (2000). A framework for preparing
primary science teachers to support children’s scientific
standards. It is also important to inform teacher
inquiry. Paper presented at the annual meeting of the Association
educators of the fact that even those pre-service for the Education of Teachers of Science (AETS). Akron, OH.
teachers who appear to perform well on a performance Duit, R. (1984). Learning the energy concept in school -
test may not have adequate understanding of the science empirical results from the Philippines and West
processes. Teacher educators should identify pre-service Germany. Physics Education, 19(2), 59-66.
teachers’ prior ideas about science processes in science Emereole, H. U. (2009). Learners’ and Teachers’ Conceptual
methods courses. understanding of Science Processes: The Case of
Future research should investigate how pre-service Botswana. International Journal of Science and Mathematics
teachers with poor conceptual understanding of science Education, 7, 1033-1056.
process skills tend to perform better on science process Farsakoglu, O.F., Sahin, C., Karsli, F., Akpinar, M. & Ultay,
skills performance tests. Studies should also investigate N. (2008). A study on awareness levels of prospective
science teachers on science process skills in science
how teachers’ good performance on science process education. World Applied Sciences Journal, 4, 174-182.16
skills translates into classroom instruction. Gao, L. (2002). Conceptions of teaching held by school
science teachers in P.R. China: Identification and cross-
CONCLUSIONS cultural comparisons. International Journal of Science
Education, 24(1), 61-79.
The elementary education pre-service teachers had Glynn, S. M. & Duit, R. (1995). Learning science
very limited conceptual understanding of the science meaningfully: Constructing conceptual models. In S. M.
process skills but performed better on science process Glynn and R. Duit (Eds.), Learning science in the schools:
skills test. Their incorrect definitions of science Research reforming practice, pp. 3-33. Mahwah, New Jersey:
processes ranged from not having any idea to tautology. Lawrence Erlbaum Associates.
Lotter, C., Harwood, W. S. & Bonner, J. J. (2007). The
Moreover, most participants interchanged definitions of
influence of core teaching conceptions on teachers' use
some science processes notably predicting and inferring. of inquiry teaching practices. Journal of Research in Science
As such, there was a gap between their performance on Teaching, 44, 1318-1347
the process skills and their ability to provide reasonable Luft, J. A. (2001). Changing inquiry practices and beliefs: The
conceptual definitions and explanations of the science impact of an inquiry-based professional development
process skills. The results suggest that this group of pre- programme on beginning teachers and experienced
service teachers did not have sufficient conceptual secondary science teachers. International Journal of Science
understanding of science process skills to help their Education , 23(5), 517-534.
future students to understand them in a meaningful way. Marion, R., Hewson, P. W., Tabachnick, B. R., & Blomker, K.
(1999). Teaching for conceptual change in primary and
174 © 2012 ESER, Eurasia J. Math. Sci. & Tech. Ed., 8(3), 167-176
Teachers’ Understanding of and Performance on Process Skills
secondary science methods courses. Science Education, 83, examining preservice primary teachers’ emerging ideas
323–346. about teaching science. Paper presented at the Annual
Mbewe, S., Chabalengula, V. M. & Mumba, F. (2010). Pre- Meeting of the National Association for Research on Science
service teachers’ familiarity, interest and conceptual Teaching (NARST), Boston, MA.
understanding of science process skills. Problems of Zembal-Saul, C. & Oliver, M. (1998, January). Meeting the
Education in the 21st Century, 22(22): 76-86. science content needs of prospective primary teachers:
Ministrell, J. & van Zee, E. H. (2000). Inquiring into Inquiry An innovative biology laboratory/recitation course.
Learning and Teaching in Science. Washington, DC: Paper presented at the Annual Meeting of the Association for the
American Association for the Advancement of Science Education of Teachers of Science (AETS), Minneapolis, MN.
National Assessment Governing Board (NAGB). (2005).
Science framework for the 2009 National Assessment
of Educational Progress. Washington, DC: American
Institutes for Research.
National Research Council (NRC). (1996). National Science
Education Standards. Washington, DC: National Academy
Press.
National Science Teachers Association (NSTA). (2002).
NSTA Position Statement: Elementary school science.
Retrieved from National Science Teachers Association
website: http://www.nsta.org/about/ positions/
elementary.aspx.
O’Sullivan, C. Y., Reese, C. M. & Mazzeo, J. (1997). NAEP
1996 science report card for the nation and the states.
Washington, DC: National Center for Education
Statistics.
Padilla, M., Cronin, L., & Twiest, M. (1985). The
development and validation of the test of basic process
skills. Paper presented at the annual meeting of the
National Association for Research in Science Teaching,
French Lick, IN.
Padilla, M.J. (1990). The science process skills (Research matters –
to the science teacher No. 9004). Retrieved from
National Association of Research in Science Teaching
website: http://www.narst.org/ publications/
research/skill.cfm.
Rillero, P. (1998). Process skills and content knowledge
[Editorial]. Science Activities, 35(3), 3.
Rollnick, M., Lubben, F., Lotz, S. & Dlamini, B. (2002). What
do underprepared students learn about measurement
from introductory laboratory work? Research in Science
Education, 32(1), 1-18.
Roth, W. M., McGinn, M. K. & Bowen, G. M. (1998). How
prepared are pre-service teachers to teach scientific
inquiry? Levels of performance in scientific
representation practices. Journal of Science Teacher
Education, 9(1), 25-48.
Scharmann, L. C., (1989). Developmental influences of
science process skill instruction. Journal of Research in
Science Teaching, 26(8), 715-726.
Settlage, J., & Southerland, S. A. (2007). Teaching science to every
child: Using culture as a starting point. New York: Taylor &
Francis.
Song, J. & Black, P. J. (1992). The effects of concept
requirements and task contexts on pupils' performance
in control of variables. International Journal of Science
Education, 14(1) , 83-93.
Virginia Department of Education. (Spring, 2007). Virginia
Standards of Learning Test Form S0117, Core 1.
Released Test, Grades 3 & 5, Science.
Zembal-Saul, C., Boardman, L.A., Severs, M. & Dana, T.
(1999, March). Web-based portfolios: A vehicle for
© 2012 ESER, Eurasia J. Math. Sci. & Tech. Ed., 8(3), 167-176 175