Evaluation of the LOGO Programs Impact on Teachers

Rebeca Trudith Pachn

Delivered to Courtney Browne-LOGO Project Coordinator San Diego Unified School District

Pachon-LOGO Program Evaluation

Table of Contents

Summary .................................................................................................. 3 Introduction .............................................................................................. 3 Program Description ................................................................................. 4

Program Objectives ............................................................................................................ 4 Program Components ........................................................................................................ 4

Evaluation Method ................................................................................... 5

Participants......................................................................................................................... 5 Procedures.......................................................................................................................... 6 Data Sources ....................................................................................................................... 6

Results ...................................................................................................... 7
Teacher Practice ................................................................................................................. 7 Teacher Training Hours ...................................................................................................... 9

Discussion ............................................................................................... 10 Project Cost............................................................................................. 11 Citations ................................................................................................. 11 Appendices ............................................................................................. 12

Data Sources:.................................................................................................................... 12 Survey Questions and Raw Data from Survey Monkey .................................................... 12

Pachon-LOGO Program Evaluation

Summary
Learning on the Go (LOGO) provides broadband access to all 6th grade students in 10 middle schools in the San Diego Unified School District. This evaluation looked at the impact of LOGO on teacher practice, perception and training hours. Our preliminary data provide evidence that the program is significantly impacting the way teachers deliver instruction. Just five months into the program, almost half of the LOGO teachers are using their own websites daily. The teachers are also participating in many hours of training after-hours. Teachers participating in LOGO also perceive that the program has had an impact on the learning activities they provide their students, the use digital resources and the way they communicate with parents. The majority of LOGO teachers had positive perceptions about LOGO and how it affected their teaching.

Introduction
LOGO is a pilot program funded through the (FCC) to grant increased wireless access to districts around the nation. The FCC funding for LOGO is a part of the E-Rate program (Schools and Libraries program) whose goal is to make telecommunication and information services more affordable for these institutions (Federal Communications Commission, 2011). The broad goal of the program is to increase broadband access in schools and at home. The more targeted goal of SDUSD is to integrate technology into all teaching and learning (Chief Wireline Competition Bureau Federal, 2011). The LOGO funds were granted to 20 districts around the nation, each district having unique goals and objectives for implementation of the broader FCC vision. The specific project goals for the San Diego Unified School District were: San Diego Unified School District (SDUSD) has established a Mobile Learning Program to seamlessly integrate ubiquitous, one-to- one computing and other 21st century technology into all teaching and learning throughout the curriculum. Its program will serve 6th grade in eight middle schools and schoolwide in two middle schools. (Fiske, 2011) In order to evaluate integration of 1 to 1 computing into all teaching and learning, this evaluation looks at the impact of LOGO on teachers. The number of hours of voluntary technology training, and the impact of LOGO on teacher attitudes and practices were the metrics used in the evaluation. Some of the research from this evaluation was a part of the reporting requirements to the FCC on the effect of the LOGO program on teachers.

Pachon-LOGO Program Evaluation

Program Description
The LOGO program provides 3G and 4G access to students in the San Diego Unified School District. This is a pilot program introduced in 10 middle schools around the district. The schools were chosen from different areas in the district and provided a good representation of our student population and demographics. The program was rolled out in early October and will run through the end of May. The program has been extended for next year in all of the schools involved in the original pilot.

Program Objectives
The goals for the LOGO program as defined by SDUSD included true 1:1 computing for LOGO students, and to integrate technology into all teaching and learning throughout the curriculum. In order to measure the impact and effectiveness of these goals, teacher practice, attitudes and voluntary training were evaluated. The objectives for the program through the teacher metrics were: Teachers will use technology to deliver instruction to LOGO students, both at home and at school. Teachers will perceive LOGO as having a significant impact on their teaching practice. Teachers will participate voluntarily in technology training to increase their effectiveness in teaching with the tools provided to the LOGO students.

Program Components
Computers Students in the LOGO program were provided district computers to use both at home and at school. The computers were checked out to the students and they were trained on how to connect at home to the 3G or 4G connection provided by the LOGO program. 3G or 4G connectivity With the LOGO funding, 3G access was purchased though AT & T and 4G access was purchased through Verizon. The choice of AT & T or Verizon was based on the area in the district the school was located. Teacher Training The SDUSD Educational Technology department provided teacher training and support. There were LOGO specific trainings as well as technology workshops offered to all teachers after school and on Saturdays. LOGO workdays were also offered for teachers to come in and plan lesson collaboratively for use with the LOGO computers.

Pachon-LOGO Program Evaluation Technical Support An IT tech was assigned to work with the LOGO schools to service technical issues that came up with the LOGO computers. Each school also had students involved in supporting the LOGO computers on Tech Squads. These courses were created to support a technical services curriculum and are a part of our College, Career and Technical Education department. Resource Teacher Support The LOGO schools had a resource teacher specifically assigned to help with coordinating LOGO related activities and logistical support. Each school also had a resource teacher who conducted technology integration workshops, and provided one on one coaching to the LOGO teachers.

Evaluation Method
Participants
Students All sixth grade students from ten schools were selected to participate in the LOGO program. The schools and students were selected based on the following criteria: demographics, administrative support, need and willingness to pilot the program. The students represented virtually every demographic and need level in the district. Teachers Teachers dictated how the LOGO computers were used instructionally. They were in control of directing and creating learning opportunities and activities on their computers. There were about 160 teachers in total who participated in LOGO. Administrators Administrators in the schools were heavily involved in the LOGO program. They were responsible for organizing the computer distribution, providing the leadership needed to make the program a success and in involving all community stakeholders, especially parents. The administrators of the LOGO schools had to express a willingness to attend additional trainings and meetings to support the program as well as administer it at their sites. Parents Parents of LOGO students had to agree to have their students participate as well as assume financial responsibility for the LOGO computers should their children damage or lose a computer. Parents also had to attend a LOGO training to understand how the

Pachon-LOGO Program Evaluation program works and how to support their children. Parent training and permission were mandatory.

Procedures
Since the LOGO program is a pilot, monitoring activities were in place and data were being collected for evaluative purposes. The program coordinator established activities and procedures to monitor the various metrics associated with the LOGO program. This evaluation only focused on a small part of the overall program, the impact of LOGO on teachers. Impact of LOGO on Teacher Perception and Practice Teacher perceptions and practices were measured using a survey sent out after five months of participation in the program. Taking the survey was strongly encouraged by the administration, and about 90 teachers participated in the survey. Questions were created to determine teacher perceptions about how LOGO affected their teaching practice. Teacher Training In order to take voluntary technology trainings and workshops, teachers have to sign up in our Electronic Registrar Online, ERO. In this way, we can monitor the number of hours of training each teacher in the district has taken, and which trainings they have taken. Attendance is taken at each training and entered into the system. Data were later pulled from the system and aggregated for the LOGO teachers.

Data Sources
Data about teacher practices and perceptions were collected from a survey given to them through Survey Monkey. The survey link was sent out through an email to all LOGO teachers and administrators. The data for teacher training were collected through the ERO. The number of trainings, the number of hours spent in training and the types of training were all pulled from the system and aggregated for the evaluation.

Results
Teacher Practice
Figure 1 summarizes the changes in teacher practice creating learning opportunities outside of the classroom. Before teachers had equity of access outside of the class, most were not using digital tools or websites outside of the classroom. Most are now using instructional videos, games and interactive websites and class websites. Almost 90% of LOGO teachers are now using class websites to provide learning opportunities outside of the classroom. Figure 1: So far this year, how often do you require LOGO students to complete the following types of assignments outside of school (ie. for homework and NOT in the classroom)?

Pachon-LOGO Program Evaluation Figure 2 shows that over 75% of LOGO teachers are using class websites like Moodle to post links and resources for students. This means most of these teachers created websites for their students and are using them often to communicate with students and provide learning opportunities for them. Only 13% of teachers never use websites to post links. Figure 2.

Teacher Perceptions
Figure 3 shows that most LOGO teachers perceive that the program has changed their teaching practices. Especially significant is the perception of their use of digital resources as well as student use of digital resources. Fewer than 5 % of teachers saw no change in their use of digital resources both for themselves as well as their students. Communication with parents was the least changed, but even there, almost 2/3 saw a change in the way communication was approached with parents.

Pachon-LOGO Program Evaluation Figure 3.

Teacher Training Hours

Figure 4 shows that on average, teachers attended over 20 hours of additional training in the five months of the program to support their effective implementation of the LOGO program. These trainings were usually held on Saturdays and afterschool. Figure 4.

Discussion
The LOGO program has had an impact on the teachers in the San Diego Unified School District in the five months since its inception in SDUSD. The data from our surveys indicates that LOGO increased technology integration in curriculum both at school and for students at home. Almost 80 % of the LOGO teachers are now using digital resources outside of the classroom with their students. They are also creating and using websites often. Almost 50% of LOGO teachers are using their own websites daily. Teacher perceptions of how the LOGO program is affecting their practice were also significant. Almost all teachers thought LOGO has changed their practice, their communication with parents and the types of assignments they are giving students. LOGO has also affected the number of hours of training the teachers in the program seek out. Within the first five months of the program, teachers, on average, had already attended more than 20 additional hours of training to support technology integration into their teaching and curriculum. These metrics are an indicator that the program is having an impact on teachers and their practices. LOGO appears provide an impetus for teachers to change their practices by integrating technology and to provide more opportunities for learning outside of the classroom. Having equity to computer broadband access is an issue teachers must consider when they are developing projects and activities for students outside of the class. With LOGO, the access is the same for all students, so teachers are branching out to learn how to provide digital opportunities for learning they couldnt consider before. The data collected in this evaluation provide support for the idea that LOGO type programs can significantly impact teaching and learning, especially where there are inequities to access and a large digital divide. The objectives of the program, through the metrics of teacher perceptions, practice and training attendance were achieved.

Pachon-LOGO Program Evaluation

Project Cost
Budget Item Personnel Travel Expenses Total Cost Cost per Day 500 100 100 Number of Days 10 7 10 Total Cost 5000 700 1000 6700

Citations
1. Fiske, D. Federal Communications Commission, (2011). Fcc authorizes release of funding to qualified chools and libraries for off-premises wireless internet projects (DOC-30833A1). Retrieved from FCC News website: http://hraunfoss.fcc.gov/edocs_public/attachmatch/DOC-308331A1.pdf 2. Chief Wireline Competition Bureau Federal. (2011). Order: In the matter of e-rate deployed ubiquitously 2011 pilot program Communications Commission (FCC) (DA 11-1181). Retrieved from FCC website: http://hraunfoss.fcc.gov/edocs_public/attachmatch/DA-11-1181A1.pdf 3. Fedral Communications Commission (FCC). (2011, April 25). Universal service program for schools and libraries. Retrieved from http://www.fcc.gov/guides/universal-service-program-schools-and-libraries

11

Pachon-LOGO Program Evaluation

Appendices
Data Sources:
Survey Monkey: https://www.surveymonkey.com/ ERO: https://ero1.eschoolsolutions.com/user/login.taf?orgid=92103

Survey Questions and Raw Data from Survey Monkey

LOGO Teacher Survey Jan. 2012

So far this year, how often do you require LOGO students to complete the following types of assignments outside of school (ie. for homework and NOT in the classroom)? Answer Options Research on the internet Typed reports Presentations (ex. powerpoint, flipchart,) Group assignments (not necessarily on a computer) Access a class website to complete assignments Wiki or blog Online games or interactive websites The use of Web 2.0 tools to create projects Show instructional videos Create videos Graphing or charting using a computer Online discussion (ex. Moodle forum or Edmodo) Online textbook Every day 9 4 7 1 30 8 16 3 8 3 1 7 6 More than once a week 24 10 8 13 17 3 20 10 17 4 6 13 14 Once a week 15 18 16 23 13 4 21 7 20 7 5 15 11 Once a month 19 32 31 22 7 7 8 15 19 15 18 13 14 Never 10 13 15 18 10 55 12 42 13 48 47 29 32 Response Count 77 77 77 77 77 77 77 77 77 77 77 77 77 77 13

answered question skipped question

12

Pachon-LOGO Program Evaluation

So far this year, how often do you use a class website like Moodle to post links, assignments, and other resources for students? Answer Options Everyday More than once a week Once a week Once a month Never Response Percent 48.7% 28.9% 5.3% 3.9% 13.2% Response Count 37 22 4 3 10 76 14

answered question skipped question

So far this year, how often do you use instructional videos to increase student understanding? Answer Options More than once a week Once a week Twice a month Once a month Never Response Percent 30.3% 22.4% 26.3% 15.8% 5.3% Response Count 23 17 20 12 4 76 14

answered question skipped question

13

Pachon-LOGO Program Evaluation

Are you currently using an online resource (ex. ParentConnect) where parents could view student weekly grades, attendance, and behavior? Answer Options Yes No Response Percent 89.3% 10.7% Response Count 67 8 75 15

answered question skipped question

So far this year, how do you perceive that LOGO has changed your: Answer Options lesson design lesson delivery use of digital resources students' assignments students' use of digital resources students' collaboration students' outputs (in regard to showing evidence of learning) Communication with parents Extremely significant 35 35 41 39 42 17 28 17 Somewhat significant 32 31 27 30 28 47 40 30 No change 7 8 6 5 4 10 6 27 Response Count 74 74 74 74 74 74 74 74 74 16

answered question skipped question

14

Pachon-LOGO Program Evaluation

What support do you need to reach these goals? Answer Options Moodle training Wiki training Edmodo training ActivExpression/Engage training DataDirector training Safari Montage training Someone to help me teach technology infused lessons in my classroom Help in designing technology infused lessons i21 workshops Planning Time Visits to classrooms where teachers in my content area are using the technology effectively Interactions (online or in person) with other LOGO teachers to get and share ideas Other (please specify) Response Percent 26.0% 16.4% 23.3% 15.1% 17.8% 15.1% 15.1% 35.6% 27.4% 76.7% 39.7% 28.8% 13.7% Response Count 19 12 17 11 13 11 11 26 20 56 29 21 10 73 17

answered question skipped question

15