Discussion: Validity and Reliability of Assessment Tools.

I recently had the opportunity to work on a research proposal that studied the effects of life coaching on community college students. Part of my motivation to develop this proposal derived from my experience in teaching Academic Strategies. As a classroom facilitator, I find fascinating to see how students have a set of perceptions and beliefs at the beginning of the semester and later on, as the class progresses, these perceptions are shaped by the dynamics of the environment in which these individuals develop. Frequently in my Academic Strategies class, I work with students that have low self-esteem, are anxious about school, do not test well, and do not have a clear direction of what they want to achieve while they attend school. But, how can these variables be measured? How can it be determined if, after weeks of instruction, these variables have changed for the better or the worse? At times, students and classrooms facilitators have the idea that an assessment is a tool that measures how well you apply theory into practice. Some even think that an assessment measures how smart one can be when it comes to mastering a new concept. From a personal standpoint, an assessment is a tool that allows me, as a facilitator of change, to determine areas of weaknesses that need to be addressed during regular hours of classroom instruction or by providing students with an opportunity to take advantage of all the resources available to them in college, i.e., tutoring labs. In designing my classroom activities and performance objectives, I administer the Learning and Study Strategies Inventory (LASSI) to all the students enrolled in the Academic Strategies class at the beginning of the semester. The LASSI is a standardized, 10scale instrument that includes 80 questions to assess students learning strategies. Three major psychometric constructs are present in this inventory: affective strategies, goal strategies, and comprehension monitoring strategies. The affective strategies involve the students time

management skills, motivation, concentration and attitude. On the other hand, the goal strategies construct looks at the anxiety levels that students exhibit towards school, test strategies, and selection of main ideas. In terms of comprehension monitoring strategies, particular attention is paid to information processing, self-testing, and study aids. So, can one assume that this measuring tool is valid and/or reliable? Given that I am measuring the students perceptions about school, their concerns and attitudes towards the college system, I find some validity in this test as it is a tool intended to measure students learning strategies. In terms of reliability, I consider that this test can posit incongruences when it comes to producing the same results every time the test is administered. These incongruences relate to the unchanging nature of human beings. Additionally, it is important to remember that students perceptions about schools are affected by their peers, family members, and even the professors with whom they interact on a weekly basis. Students that do not like their instructors, or those that cannot deal with all the diversity that abounds at the college level, will most likely have a different opinion from students that are motivated and eager to work diligently to earn a degree. According to Cano (2006), the internal consistency and reliability of the LASSI was measured using Cronbachs Alpha. The validity of the LASSI was measured by administering the test to 527 college students representing different faculties (divisions) from the University of Granada in Spain. The test was administered twice throughout the semester: at the beginning and at the end of the academic year. For each question, students were prompted to give responses using a Likert-type scale, from 1 for not at all like me to 5 very much like me (Cano, 2006). After scores were obtained, two-standard multiple regression analysis were conducted to establish a relationship among affective, goal, comprehension strategies and academic

performance. The analysis revealed that the first two latent constructs assessed by LASSI, affective strategies and goal strategies, were statistically significant predictors of students academic performance in both samples (Cano, pp. 1035 & 1036, 2006). Although the LASSI was useful at measuring different learning strategies, Cano (2006) asserts that the LASSI is a relatively new tool that has not been widely used. Therefore, its reliability may be threatened as it may not yield the expected outcome hypothesized by researchers. When this test is administered in my Academic Strategies class at the beginning of the semester, students scores are lower in the areas of affective and goal strategies, particularly time management skills, motivation, attitudes, and test strategies, than in the comprehension monitoring strategies. Most of my hours of classroom instruction are devoted to creating practical applications that allow students to improve their time management skills and test strategies. I believe that if students learn how to set goals and manage their time effectively they will understand how to better prepare for tests. If students are well-prepared for an exam, assuming that the instructor has provided students with clear learning objectives and useful information, they will perform better. Higher performance, in turn, will serve as a motivational factor for students to rise to their fullest potential. During week 16 of the semester, I administer the test again. Based on previous experiences, students show an improvement in their affective and goal strategies after 72 hours of classroom instruction. At least 75% of the students in the class know how to set up goals, create a sound time management plan that includes study time, and how to prepare, well in advance, for different types of examinations. This improvement is measured by the LASSI; however, I have to remind myself that every group of students is different and so are their strengths and weaknesses. Therefore, using the scores from one group to predict the scores of a

different group may not accurately describe the true characteristic of the population being studied.

Reference: Cano, F. (2006). An in-depth analysis of the learning and study strategies inventory (LASSI). Educational and Psychological Measurement, 66(6), 1023-1038.

Miguel Llovera Da Corte