Endline Report-Visakhapatnam Smart City

Third Party Impact Study
Google Future Classroom Initiative
Impact Assessment Report – Visakhapatnam Smart City
Greater Visakhapatnam Smart City Corporation

Submitted to
Limited (GVSCCL)
Implemented by IL&FS Education and Technology Services Ltd
Prepared by Vidura Education Solutions
Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 1

Table of Contents
About Vidura Edu ......................................................................................................................................... 3

Executive Summary ...................................................................................................................................... 4
1. Introduction ........................................................................................................................................... 5
1.1. Background ................................................................................................................................... 5
1.2. Project Outcomes .......................................................................................................................... 5
2. Snapshot of the Evaluation ................................................................................................................... 6
2.1. Methodology ................................................................................................................................. 8
2.1.1. Population ............................................................................................................................. 9
2.1.2. Instruments .......................................................................................................................... 10
3. Analysis Approach .............................................................................................................................. 11
4. Results of the Baseline Evaluation...................................................................................................... 12
4.1. Overall program performance on each subject – Treatment vs. Control .................................... 12
4.2. Overall subject performance on each academic skill - Treatment vs. Control............................ 13
4.3. Overall school-wise performance on each subject - Treatment vs. Control ............................... 17
5. Results of the Endline Evaluation ....................................................................................................... 21
5.1. Overall program performance on each subject – Treatment vs. Control .................................... 21
5.2. Overall subject performance on each academic skill - Treatment vs. Control............................ 22
5.3. Overall school-wise performance on each subject - Treatment vs. Control ............................... 26
6. Improvement from Baseline to Endline .............................................................................................. 28
6.1. Program Performance – Grade-wise ........................................................................................... 29
6.2. Program Performance – Subject-wise ......................................................................................... 30
6.3. Program Performance – Academic Skill-wise for each Subject ................................................. 31
6.4. Program Performance – School-wise .......................................................................................... 33
7. Conclusion .......................................................................................................................................... 34

About Vidura Edu
Vidura grew out of the founder's experience with diverse organisations in the social sector in India:
governments, for-profits, non-profits, social enterprises and multilateral organisations. Vidura has
completed curriculum and learning assessments projects to more than 500,000 students across India-
cities including Delhi, Pune, Jaipur, Chandigarh, Bangalore, Bhubaneshwar, Ahmedabad, Mumbai,
Hyderabad, and Guwahati. Our clients are from non-profit, the private sector, government, and social
enterprise backgrounds.

Executive Summary
This report presents the results of a longitudinal study that tests the knowledge of students involved in a
pilot project being conducted by IL&FS Education in Andhra Pradesh. The baseline study was conducted
in the month of October and the endline study was conducted in the month of March. The objective of the
study was to capture the knowledge improvement of the students who received the pilot intervention
(Google Future Classroom Technology to improve learning outcomes) in comparison to the control
group students who were studying in the regular classrooms. The study compares scores across different
subjects and skills in both the baseline and endline.
The objective of the pilot project is to improve learning outcomes of students by introducing Google
Future Classroom Technology and Professional Development Program (PDP) in the government-run
schools. Using these objectives, Vidura Education has conducted baseline and endline assessments to
collect data on the improvement in the knowledge level. Quantitative data was collected for 1987
treatment students across 10 schools and 298 control students across 5 schools. Findings of the baseline
test, endline test and improvement over control group are described in Section 4, 5 and 6 respectively.
Some highlights from the analysis are indicated below:
 A relative improvement of 15% in seen in the treatment group students as compared to the
control group students
 This improvement is seen across grades with highest improvement in grade 9 and least
improvement in grade 6.
 Improvement is seen across subjects with highest improvement in Biology and the least
improvement in Science.
 In English, academic skill focussed on reading comprehension showed the highest improvement.
 In Mathematics, academic skill focussed on Reasoning and Proof showed highest improvement.
 In Science subjects, academic skill focussed on Appreciation and Aesthetic Sense, Values showed
highest improvement.
 In Social, academic skill focussed on Appreciation and Sensitivity showed highest improvement.
 Both the schools that participated in the both baseline and endline (K.D.P.M.High school, MVD
High School) showed positive improvement as compared to their counterparts in the control
group.
The inputs from this pilot study will be useful to improve the program further. These reports will be
analysed by the implementation team to make necessary improvement in program design, training or
implementation.

1. Introduction
1.1. Background
The advent of Information and Communication Technologies (ICT) in education has highly
reconfigured the teaching-learning-assessing experiences in education landscape across the globe.
Our country has witnessed a proliferating growth in employing Education Technology (ET) to
redefine the learning spaces to improve the quality of education. The Ministry of Human Resource
Development (MHRD), Government of India recognizes the importance and need of ICTs in the
education sector. This need for reimagining the educational experiences through ICT has been
highlighted in the National Policy on Education (1992) and the same was echoed in the National
Curriculum Framework (NCF) 2005. With the introduction of a policy on ICT called National Policy
on ICT in School Education (revised, 2012), the Ministry has proposed desirable changes to the
school education.
In order to equip students to compete in the knowledge economy; develop much-needed skills such as
inquiry, critical thinking, problem-solving, schools must integrate Information and Communication
Technology (ICT) in the Teaching and Learning process effectively. New technology tools like
digital support, Google Chrome books and G suite for education have been active enablers in
changing the teaching learning process in classroom and enhancing the learner experiences.
Considering the proven operational abilities in public schools in India Google has appointed IL&FS
Edeucation as implementation partner for Google Future Classroom Pilot in Vishakhapatnam District
in the state of Andhra Pradesh. Vidura Edu proposes a framework to evaluate the efficacy of this
technology intervention in these schools. The evaluation framework design incorporates a system to
understand the increase in the learning outcomes (Detailed description in Pg 10 in this report) of the
children fostered by Google Future Classrooms through Lab model implemented by IL&FS.
Project Outcomes
This project will be implemented with the aim to positively impact learning outcomes for students
who attend Government run schools. Considering the objectives of IL&FS program implementation,
as given below, Vidura Edu focuses on the evaluation of the objective pertaining to accelerating the
learning facilitated by state-of-the-art technology resources.
a) Transform the teaching - learning process by improving communication and collaboration among
the faculty, students and administrators of the school
b) Improve integration of technology into teaching-learning spectrum
c) Encourage students to become creators of their own learning by incorporating 21st-century skills
such as creativity and critical thinking
d) Accelerate learning through differentiated instruction and a broader set of learning resources

2. Snapshot of the Evaluation
The objective of this evaluation is to assess the impact of the pilot project on the students exposed to
the intervention. To achieve and validate the same in the proposed program, these are the specific
steps followed for both baseline and endline.
 Mapping of the examination papers to desired skills to be attained in subjects and create an
OMR sheet aligned to the same
 Equip teachers with skills to use Vidura’s OMR sheets
 Collecting and analysing the exam data to identify the Google Future Classroom’s impact on
student’s learning by deploying state-of-the-art technology driven systems in the back-end.
 Demonstrating the efficacy of this approach by evaluating the impact of this intervention using
statistically sound methodologies.
Image: Students Writing the Endline Test
The evaluation program intends to understand and evaluate the efficacy of the Lab Model
Solution proposed by IL&FS Education. The solution implements a framework that
reconfigures teaching-learning experience by employing multimedia content, chromebooks,
G Suite for education, K-Yan, based on an integrated and innovative technology platform,
called Integrated Community Computer. The teacher is the core of K-Yan and innovations
around it. It enables a teacher to focus on teaching rather than technology. While Google
Future Classrooms offer a holistic solution to change the way learning happens in the new
technology-enabled classrooms, we, at Vidura focus on the ultimate outcome of this
intervention, which is the ability of this system to increase the learning outcomes in students’
learning.

Image: Vidura Executive overseeing the Endline Test
 Vidura Edu’s association with the 10 identified schools for this program is as depicted in
the below figure.
Vidura Edu
•Google Future Classroom
intervention in the • Measure impact using
schools of Vizag, AP scientific methods
• Mapping skills and • Evaluate impact of the
• Students Test measure learning
performance data intervention
• Data analysis by state- • Present empirical
of-the-art technology evidence
•Evaluate the efficacy of
the intervention
Public Schools Proof of concept

The purpose of this evaluation is to identify the current levels of the students’ learning outcome and
compare the improvements recorded vis a vis with performances shown in baseline. This analysis
will enable our understanding of the impact of the intervention.
Image: Students attempting the baseline test paper
2.1. Methodology
The quantitative evaluation can be mapped onto the PICOT framework for posing the relevant
impact evaluation questions:
 Population (P) – About 1987 children enrolled in grades 6-9 spread across 10 schools. This is
the Treatment group where the impact of the intervention is to be evaluated.
 Intervention (I) – Google Classroom + K-Yan Lab model: Integrated technology-enabled

classroom experience.
 Comparison (C) – Identify an appropriately similar comparison group of 298 Students across 5
schools for indicative purposes. This is the Control group used for estimating the differential
impact of the intervention.
 Outcome (O) –Analyse subject specific learning competency improvement among students
 Time frame (T) – from June 2017 to April 2018

2.1.1. Population
Program Group Students who participated in the study –
No of Students No of Schools
Class Baseline Endline Baseline Endline
6 155 535 2 10
7 149 528 2 10
8 166 476 2 9
9 146 447 2 9
Total 616 1987
Non Program Group (Control) students who participated in the study -
No of Students No of Schools
Class Baseline Endline Baseline Endline
6 58 66
7 72 82 3 5
8 65 54
9 73 96
Total 268 298

2.1.2. Instruments
For the quantitative data collection, the objectives and outcomes are listed here. The instrument is
designed based on the academic learning outcomes prescribed by the APSCERT syllabus.
Science % weightage
Application to Daily Life, Concern to Biodiversity 10%
Appreciation and Aesthetic Sense, Values 10%
Asking Questions and Making Hypothesis 10%
Communication through Drawing and Model-Making 15%
Conceptual Understanding 25%
Experimentation and Field Investigation 15%
Information Skills and Projects 15%
English % weightage
Grammar 37.5%
Reading Comprehension 25%
Vocabulary 37.5%
Maths % weightage
Communication 15%
Connections 15%
Problem Solving 30%
Reasoning Proof 20%
Visualisation & Representation 20%
Social % weightage
Appreciation and Sensitivity 10%
Conceptual Understanding 40%
Information Skills 15%
Mapping Skills 15%
Reading the Text, Understanding and Interpretation 10%
Reflection on Contemporary Issues and Questioning 10%
Image: Teacher Distributing Baseline Test Papers

3. Analysis Approach
As a part of the quantitative study measuring the baseline and endline knowledge of children, data was
collected and analysed on academic skills relating to the specific questions in the instruments developed.
These academic skills were further categorized into subjects for both control and treatment groups.
To calculate average percentage score each question was assigned a weight (score) of one. Summary
statistics for the outcome variables are provided in the graphs and/or in percentages below. The analysis is
done at three levels for both baseline and endline - (reported in sections 4 and 5)
1) Overall program performance on each subject – Treatment vs. Control

2) Overall subject performance on each academic skill - Treatment vs. Control
3) Overall school-wise performance on each subject - Treatment vs. Control
All the questions were of MCQ type. All questions not attempted were scored zero to calculate the
average score.
To understand the impact of the program, an analysis is done at three levels to calculate the difference of
difference –(reported in section 6)
1) Overall program performance on each of the subjects – Improvement over control schools
2) Overall program performance on each of the classes – Improvement over control schools
3) Overall subject performance on each of the academic skills – Improvement over control schools
4) School wise improvement for Common schools which appeared for both baseline and endline
The report contains the improvement measured from all baseline schools (2) to all endline schools (10)
and also cover the improvements for the common schools (2).

4. Results of the Baseline Evaluation
The results of each variable studied are discussed below. Comparisons between the control and treatment
groups are made to evaluate the initial (baseline) knowledge base of children as well as to inform the team
on evaluation design for the end-line study. Each graph shows a visual display of the comparison
between control and treatment groups. The graphs show the scores as percent of the total maximum
scores for the variable.
4.1. Overall program performance on each subject – Treatment vs. Control
Class 6 Class 7 Class 8 Class 9

English IL&FS-Vizag 25% 28% 39% 30%
Control 39% 43% 48% 51%
Maths IL&FS-Vizag 34% 26% 35% 27%
Control 40% 44% 34% 58%
Science IL&FS-Vizag 27% 27%
Control 34% 46%
Physics IL&FS-Vizag 30% 28%
Control 37% 47%
Biology IL&FS-Vizag 27% 25%
Control 44% 41%
Social IL&FS-Vizag 27% 26% 29% 25%
Control 31% 41% 49% 42%
Children in the control groups showed an average understanding on the learning outcomes. Treatment
school students achieved scores visibly lower than the control group students. No specific trend in the
score was observed in relation to the subject or class at the cohort level.

4.2. Overall subject performance on each academic skill - Treatment vs. Control
4.2.1 English
Control group students outperformed treatment group students in all skills. Baseline scores of reading
comprehension were also found to be higher than other two skills.
English Skillwise Performance

70%
60%
50%
40%
30%
20%
10%
0%
Grammar Reading Comprehension Vocabulary
Test Control
The table below captures similar information at a grade level.
Academic Skill School Type Class 6 Class 7 Class 8 Class 9

Grammar IL&FS-Vizag 22% 24% 35% 26%
Control 27% 40% 38% 43%
Reading Comprehension IL&FS-Vizag 35% 38% 54% 42%
Control 64% 60% 71% 64%
Vocabulary IL&FS-Vizag 20% 26% 34% 26%
Control 33% 35% 43% 53%

4.2.2 Maths
Control group students performed marginally better than test group students in all skills.
Maths Skillwise Performance

60%
50%
40%
30%
20%
10%
0%
Communication Connections Problem Solving Reasoning Proof Visualisation &
Representation
Test Control

Communication IL&FS - Vizag 28% 20% 22% 27%
Control 29% 54% 24% 48%
Connections IL&FS - Vizag 25% 28% 24% 20%
Control 34% 32% 35% 72%
Problem Solving IL&FS - Vizag 39% 30% 45% 33%
Control 47% 44% 36% 72%
Reasoning Proof IL&FS - Vizag 42% 23% 37% 24%
Control 40% 58% 29% 55%
Visualisation & IL&FS - Vizag 29% 26% 36% 27%
Representation Control 40% 34% 43% 43%

4.2.3 Science
Control group students outperformed the treatment group students in most of the skills/academic
standards except in “Experimentation and Field investigation” and “Appreciation and Aesthetic Sense”.
Science + Physics + Biology

Skillwise Performance
Information Skills and Projects

Experimentation and Field Investigation
Conceptual Understanding
Communication through Drawing and Model-Making
Asking Questions and Making Hypothesis
Appreciation and Aesthetic Sense, Values
Application to Daily Life, Concern to Biodiversity
0% 10% 20% 30% 40% 50% 60%
Control Test

Science Physical Sciences Biological Sciences
School Class Class 7 Class 8 Class 9 Class 8 Class 9
Academic Skill 6
Type
Application to IL&FS- 26% 28% 31% 19% 31% 24%
Daily Life, Vizag
Concern to Control 26% 47% 28% 14% 52% 64%
Biodiversity
Appreciation and IL&FS- 26% 20% 23% 28% 26% 20%
Aesthetic Sense, Vizag
Values Control 28% 31% 35% 64% 25% 32%
Asking Questions IL&FS- 31% 19% 24% 26% 18% 18%
and Making Vizag
Hypothesis Control 33% 45% 32% 29% 45% 18%
Communication IL&FS- 29% 32% 31% 29% 30% 23%
through Drawing Vizag
and Model Control 38% 48% 42% 28% 52% 71%
Conceptual IL&FS- 26% 27% 34% 31% 32% 26%
Understanding Vizag
Control 37% 50% 47% 66% 56% 39%
Experimentation IL&FS- 23% 29% 33% 29% 20% 33%
and Field Vizag
Investigation Control 39% 44% 23% 47% 25% 15%
Information Skills IL&FS- 28% 26% 24% 26% 24% 23%
and Projects Vizag
Control 31% 48% 43% 60% 42% 47%

4.2.4 Social Studies
Control group students outperformed the treatment group students in most of the skills/academic
standards.
Social Skillwise Performance
Reflection on Contemporary Issues and Questioning
Reading the Text, Understanding and Interpretation
Mapping Skills
Information Skills
Appreciation and Sensitivity
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
Control Test

Appreciation and Sensitivity IL&FS - Vizag 23% 23% 32% 25%
Control 18% 42% 55% 46%
Conceptual Understanding IL&FS - Vizag 26% 25% 29% 20%
Control 26% 47% 45% 45%
Information Skills IL&FS - Vizag 25% 33% 26% 24%
Control 34% 53% 46% 35%
Mapping Skills IL&FS - Vizag 40% 28% 34% 40%
Control 49% 30% 56% 48%
Reading the Text, IL&FS - Vizag 22% 25% 26% 22%
Understanding and Control 34% 35% 33% 36%
Interpretation
Reflection on Contemporary IL&FS - Vizag 21% 23% 24% 24%
Issues and Questioning Control 30% 23% 53% 33%

4.3. Overall school-wise performance on each subject - Treatment vs. Control
4.3.1 English
Both the Schools – KDPM High School and MVD High School achieved lower scores compared to
the control group scores.
School Name Class 6 Class 7 Class 8 Class 9

K.D.P.M.High school 26% 28% 41% 32%
MVD High School 20% 28% 37% 27%
Control 39% 43% 48% 51%
The overall score distribution of English remained similar around their respective means in both test
and control groups. The horizontal axis here represents score buckets in multiple of five and the
vertical axis represents percentage of students in that score bucket.
English - Score Distribution

40%
35%
% Students in the
30%
Score Bucket
25%
20%
15%
10%
5%
0%
c) 10 to d) 15 to e) 20 to f) 25 to g) 30 to h) 35 to
a) 0 to 5 b) 5 to 10
15 20 25 30 35 40
Control 0% 4% 21% 32% 34% 7% 1% 0%
Test 4% 29% 38% 19% 9% 2% 0% 0%
Score Buckets

4.3.2 Maths

Control 40% 44% 34% 58%
The overall score distribution of Maths in test and control groups is as shown below.
Maths - Score Distribution

45%
40%
% Students in the
35%
Score Bucket
30%
25%
20%
15%
10%
5%
0%
c) 10 to d) 15 to e) 20 to g) 30 to h) 35 to
a) 0 to 5 b) 5 to 10 f) 25 to 30
15 20 25 35 40
Control 2% 11% 26% 18% 17% 21% 4% 0%
Test 4% 26% 39% 23% 7% 1% 0% 0%
Score Buckets
4.3.3 Science
Science Physics Biology

School Name Class 6 Class 7 Class 8 Class 9 Class 8 Class 9
K.D.P.M.High school 30% 27% 31% 28% 27% 22%
MVD High School 21% 25% 28% 26% 26% 28%
Control 34% 46% 37% 47% 44% 41%

The overall score distribution of Science, Physics and Biology remained similar in both test and
control groups.
Science - Score Distribution

50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
a) 0 to 5 b) 5 to 10 c) 10 to 15 d) 15 to 20 e) 20 to 25 f) 25 to 30 g) 30 to 35
Control 2% 5% 35% 30% 21% 6% 2%
Test 9% 29% 45% 12% 5% 0% 0%
Physics - Score Distribution

70%
60%
50%
40%
30%
20%
10%
0%
a) 0 to 5 b) 5 to 10 c) 10 to 15 d) 15 to 20
Control 11% 43% 46% 0%
Test 29% 66% 5% 0%
Biology - Score Distribution

70%
60%
50%
Axis Title
40%
30%
20%
10%
0%
a) 0 to 5 b) 5 to 10 c) 10 to 15 d) 15 to 20
Control 12% 49% 34% 5%
Test 40% 58% 3% 0%


Control 31% 41% 49% 42%
The overall score distribution of Social Studies remained similar in both test and control groups.
Social Studies - Score Distribution

60%
50%
40%
30%
20%
10%
0%
b) 5 to c) 10 to d) 15 to e) 20 to f) 25 to g) 30 to h) 35 to
a) 0 to 5
10 15 20 25 30 35 40
Control 4% 12% 26% 22% 21% 10% 3% 0%
Test 8% 26% 51% 13% 2% 0% 0% 0%

5. Results of the Endline Evaluation
The results of endline evaluation over different parameters are discussed below. Comparisons between
the control and treatment groups are made to evaluate the Endline knowledge of children. Each graph
shows a visual display of the comparison between control and treatment groups. The graphs show the
scores as percent of the total maximum scores for the variable.
5.1. Overall program performance on each subject – Treatment vs. Control
Subject Control/Test 6 7 8 9
ILFS-Vizag 36% 36% 47% 42%
English
Control 34% 42% 48% 41%
ILFS-Vizag 33% 37% 39% 37%
Maths
Control 37% 32% 35% 51%
ILFS-Vizag 22% 27%
Science
Control 27% 30%
ILFS-Vizag 38% 37%
Physics
Control 36% 34%
ILFS-Vizag 39% 34%
Biology
Control 33% 31%
ILFS-Vizag 36% 37% 41% 45%
Social
Control 30% 34% 32% 38%
In the endline, children in the treatment groups showed slightly better scores in most cases over
control group. This is a major improvement from baseline where the control groups students average
scores were clearly higher. No specific trend in the score was observed in relation to the subject or
class at the cohort level. Subject wise and class wise improvement from baseline to endline are
documented in Section 6.

5.2. Overall subject performance on each academic skill - Treatment vs. Control
5.2.1 English
The endline scores are mixed with higher scores for treatment group in reading comprehension,
higher scores for control group in vocabulary and similar scores on grammar. This contrasts with the
baseline where control group students outperformed treatment group students in all skills. Baseline
and endline scores of reading comprehension were also found to be higher than other two skills.
English Skillwise Performance

60%
50%
40%
30%
20%
10%
0%
Grammar Reading Comprehension Vocabulary
Test Control

IL&FS-Vizag 35% 37% 38% 35%
Grammar
Control 31% 39% 37% 36%
Reading IL&FS-Vizag 41% 38% 77% 45%
Comprehension Control 38% 48% 51% 47%
IL&FS-Vizag 35% 34% 37% 48%
Vocabulary
Control 34% 40% 58% 43%

5.2.2 Maths
Control group students still performed marginally better than treatment group students in all skills
(except reasoning and proof), however the difference has reduced a lot as compared to the
baseline, hence showing in better improvement in treatment group students. The same will be
evident while looking at the skill wise improvement differences between control and test groups
in section 6.
Maths Skillwise Performance

45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
Communication Connections Problem Reasoning Visualisation &
Solving Proof Representation
Test Control

IL&FS - Vizag 29% 34% 35% 36%
Communication
Control 31% 31% 33% 54%
IL&FS - Vizag 31% 26% 26% 34%
Connections
Control 40% 27% 33% 43%
Problem IL&FS - Vizag 36% 40% 47% 38%
Solving Control 42% 32% 34% 56%
Reasoning IL&FS - Vizag 33% 38% 46% 34%
Proof Control 31% 26% 39% 44%
Visualisation & IL&FS - Vizag 36% 40% 35% 42%
Representation Control 37% 41% 36% 53%

5.2.3 Science + Physics + Biology
Control group students still performed marginally better than treatment group students in all skills
(except Appreciation and Aesthetic Sense, Values), however the difference has reduced by a good
margin as compared to the baseline, hence showing in better improvement in treatment group
students. The same will be evident while looking at the skill wise improvement differences
between control and test groups in section 6.
Science + Physics + Biology

Skillwise Performance
Information Skills and Projects
Asking Questions and Making Hypothesis
Application to Daily Life, Concern to…

0% 5% 10% 15% 20% 25% 30% 35% 40%
Control Test
Science Physical Sciences Biological Sciences

School Class Class 7 Class 8 Class 9 Class 8 Class 9
Academic Skill 6
Type
Application to IL&FS- 23% 23% 29% 32% 54% 44%
Daily Life, Concern Vizag
to Biodiversity Control 26% 26% 30% 33% 51% 31%
Appreciation and IL&FS- 21% 22% 64% 39% 30% 30%
Aesthetic Sense, Vizag
Values Control 29% 22% 29% 26% 34% 33%
Asking Questions IL&FS- 27% 27% 32% 33% 31% 34%
and Making Vizag
Hypothesis Control 36% 25% 25% 25% 33% 38%
Communication IL&FS- 22% 34% 41% 31% 44% 40%
through Drawing Vizag
and Model Control 31% 35% 43% 39% 31% 41%
Conceptual IL&FS- 21% 30% 38% 46% 40% 32%
Understanding Vizag
Control 22% 37% 47% 38% 29% 30%
Experimentation IL&FS- 22% 22% 35% 31% 42% 38%
and Field Vizag
Investigation Control 29% 29% 36% 31% 30% 27%
Information Skills IL&FS- 21% 24% 31% 37% 29% 21%
and Projects Vizag
Control 23% 25% 29% 40% 35% 19%

In the endline, treatment group students outperformed control group students in all the academic skills
while it was a complete flip in the baseline with Control group students outperforming the treatment
group students in most of the skills/academic standards – evidencing a clear positive impact of the
intervention in the treatment group schools.
Social Skillwise Performance
Reflection on Contemporary Issues and

Questioning
Reading the Text, Understanding and
Interpretation
Mapping Skills
Information Skills
Appreciation and Sensitivity
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
Control Test

Appreciation and Sensitivity IL&FS-Vizag 35% 37% 39% 49%
Control 20% 43% 33% 41%
Conceptual Understanding IL&FS-Vizag 35% 39% 41% 44%
Control 28% 37% 28% 34%
Information Skills IL&FS-Vizag 41% 35% 39% 47%
Control 39% 27% 28% 53%
Mapping Skills IL&FS-Vizag 36% 33% 42% 42%
Control 31% 31% 40% 39%
Reading the Text, IL&FS-Vizag 38% 41% 43% 45%
Understanding and Control 29% 27% 34% 35%
Interpretation
Reflection on Contemporary IL&FS-Vizag 36% 34% 40% 44%
Issues and Questioning Control 28% 32% 40% 35%

5.3. Overall school-wise performance on each subject - Treatment vs. Control
5.3.1 English
Both the Schools – KDPM High School and MVD High School (2 schools that are common in both
the baseline and the endline) achieved lower scores compared to the control group scores in baseline
and both improved in their scores for the endline to have scores almost similar to that of the control
group (except in Class 6 at MVD).
School_Name Class 6 Class 7 Class 8 Class 9

MVDM High School 30% 32% 51% 58%
GVMC NGGO 31% 34% - -
GVMC MALKAPURAM 30% 33% 48% 39%
GVMC GANDHIGRAM 44% 40% 37% 27%
GVMC 66% 49% 55% 31%
MADHURANAGAR
GVMC 77% 34% 32% 35%
PRAKASHRAOPETA
GVMC RAILWAY 26% 26% 22% 27%
COLONY
GVMC RP PETA 36% 47% 62% 86%
Control Group Total 36% 37% 47% 42%
GVMC Madhuranagar achieved above average scores in English consistently in all grades (except
grade 9). There is no visible pattern or trend in schools or grades as the performance is mixed across
grades and schools.
The score distributions curves for the endline are very similar to the baseline and hence we are not
taking up the exercise of showing all the curves for the endline as well.
5.3.2 Maths
the control group scores in baseline and both improved in their scores for the endline to have scores
almost like that of the control group (except in Class 9). GVMC Madhuranagar achieved consistently
high scores in Mathematics across grades.
School_Name 6 7 8 9
GVMC NGGO 23% 22% - -
GVMC MADHURANAGAR 72% 60% 59% 71%
GVMC PRAKASHRAOPETA 32% 77% 29% 39%

GVMC RAILWAY COLONY 20% 21% 23% 25%
GVMC RP PETA 26% 24% 34% 45%
Control Group 37% 32% 35% 51%
5.3.3 Science
There is no visible pattern or trend in schools or grades as the performance is mixed across grades and
schools. GVMC Railway Colony School achieved below average scores in all the grades for science
subjects.
Science Physics Biology

School Name Class 6 Class 7 Class 8 Class 9 Class 8 Class 9
K.D.P.M.High school 23% 22% 33% 37% 39% 36%
MVD High School 19% 22% 42% 38% 38% 40%
MVDM High School 24% 32% 38% 43% 43% 44%
GVMC NGGO 20% 24% - - - -
GVMC MALKAPURAM 25% 26% 42% 33% 35% 23%
GVMC GANDHIGRAM 24% 30% 36% 32% 27% 22%
GVMC 19% 58% 60% 59% 82% 61%
MADHURANAGAR
GVMC 28% 24% 26% 35% 48% 30%
PRAKASHRAOPETA
GVMC RAILWAY 17% 24% 32% 19% 19% 20%
COLONY
GVMC RP PETA 14% 24% 33% 26% 28% 25%
Control 27% 30% 36% 34% 38% 33%

Most scores are above the control group scores. The lowest and the highest scores for each graded are
highlighted in red and green respectively.
School_Name 6 7 8 9
GVMC NGGO 31% 34% - -
GVMC 42% 44% 45% 50%
MADHURANAGAR
GVMC 35% 39% 40% 62%
PRAKASHRAOPETA
GVMC RAILWAY 33% 40% 38% 37%
COLONY
GVMC RP PETA 31% 36% 41% 38%
Control 30% 34% 32% 38%

6. Improvement from Baseline to Endline
Improvement in the scores is calculated using “difference in difference” approach.
Difference in difference is the Incremental improvement in treatment(test) group’s scores as compared to

the incremental improvement in the control group’s scores.
DID Score = (Test EL – Test BL) – (Control EL – Control BL)
And this is done for 2 sub-selection of schools:
1) For Common Schools: For like to like schools from baseline to endline in test group (2 Schools)
and control group (3 Schools).
2) For All Schools: For all schools participating in the baseline and endline –test group (10
Schools) and control group (5 Schools).
Overall Improvement (Common Schools)

50%
40%
30%
20%
10%
0%
BL EL
Control 43% 36%
Test 29% 36%
Overall a DID of 15% is achieved which is significant improvement, implying that the scores of treatment
group improved 15% more than the scores of the control group, hence indicating the efficacy of the
intervention.
Overall Improvement (All Schools)

50%
40%
Axis Title
30%
20%
10%
0%
BL EL
Control 43% 36%
Test 29% 37%

6.1. Program Performance – Grade-wise
Program Improvement (Common Schools)

Control Test Improvement
Class BL* EL* BL EL (DID Score)
6 36% 32% 28% 32% 9%
7 44% 34% 27% 33% 16%
8 42% 37% 33% 41% 14%
9 49% 41% 27% 39% 20%
Total 43% 36% 29% 36% 15%
*BL – Baseline; EL – Endline
The improvement is consistent across all grades with class 9 showing highest improvement and
class 6 the lowest.
Gradewise Score Improvements

60%
50%
40%
30%
BL
20% EL
10%
0%
Control Test Control Test Control Test Control Test
6 7 8 9
The above chart makes it evident that the endline scores dropped in the control groups of all the
grades and rose in treatment groups of all the grades. The overall drop in the average scores can
be due to the change in the syllabus from the baseline to the endline with endline containing more
syllabus. Despite this, the scores in the treatment groups improved pointing to the impact of the
intervention.
Class-wise Program Improvement (All Schools)

Class BL EL BL EL DID Score
6 36% 32% 28% 32% 8%
7 44% 34% 27% 34% 17%
8 42% 38% 33% 42% 13%
9 49% 41% 27% 40% 21%
Total 43% 36% 29% 37% 14%

6.2. Program Performance – Subject-wise
Subject-wise Program Improvement (Common Schools)

Subject BL EL BL EL DIDScore
Biology 43% 30% 26% 38% 25%
English 45% 40% 31% 40% 14%
Maths 45% 40% 31% 38% 12%
Physics 43% 35% 29% 36% 15%
Science 41% 28% 27% 22% 7%
Social 41% 34% 27% 38% 19%
Total 43% 36% 29% 36% 15%
DID scores are high across subjects showing that the intervention is helping in all the subjects –
directly or indirectly. The highest positive impact is seen in Biology and the least in Science.
English and Maths showed healthy improvements of 14% and 12% as compared to the control
groups students.
Subjectwise Score Improvements

50%
45%
40%
35%
30%
25%
20%
15% BL
10%
5% EL
0%
Control
Control
Control
Control
Control
Control
Test
Test
Test
Test
Test
Test
Biology English Maths Physics Science Social
Across all the subjects, the control group scores have dropped while the test groups scores
increased.
Subject-wise Program Improvement (All Schools)

Subject BL EL BL EL DID Score
Biology 43% 32% 26% 36% 21%
English 45% 42% 31% 40% 13%
Maths 45% 39% 31% 37% 12%
Physics 43% 35% 29% 37% 16%
Science 41% 29% 27% 24% 10%
Social 41% 34% 27% 40% 20%
Total 43% 36% 29% 37% 14%

6.3. Program Performance – Academic Skill-wise for each Subject
Below tables capture skill-wise score improvements in the treatment group as compared to the control
group. This data can be used to deep dive and understand which skills are being impacted the least
and the most from intervention – as highlighted in red and green respectively.
6.3.1. English:
Skill Wise Performance Improvement in English (Common Schools)

Skills BL EL BL EL DID
Grammar 37% 37% 27% 32% 6%
Reading Comprehension 65% 45% 43% 57% 34%
Vocabulary 40% 41% 27% 37% 10%
Grand Total 45% 40% 31% 40% 14%
Skill Wise Performance Improvement in English (All Schools)

Grammar 37% 36% 27% 36% 10%
Reading Comprehension 65% 47% 43% 49% 25%
Vocabulary 40% 44% 27% 38% 8%
Grand Total 45% 42% 31% 40% 13%
6.3.2. Maths:
Skill Wise Performance Improvement in Maths (Common Schools)

Communication 40% 41% 24% 29% 3%
Connections 44% 38% 24% 24% 5%
Problem Solving 50% 43% 37% 44% 14%
Reasoning Proof 46% 35% 32% 44% 22%
Visualisation & 40% 43% 30% 41% 7%
Representation
Total 45% 40% 31% 38% 12%
Skill Wise Performance Improvement in Maths (All Schools)

Communication 40% 38% 24% 33% 11%
Connections 44% 36% 24% 29% 13%
Problem Solving 50% 41% 37% 40% 12%
Reasoning Proof 46% 35% 32% 38% 16%
Visualisation & 40% 42% 30% 38% 6%
Representation
Total 45% 39% 31% 37% 12%

6.3.3. Science+Physics+Biology
Skill Wise Performance Improvement in Science+Physics+Biology (Common Schools)

Application to Daily Life, Concern 39% 27% 27% 31% 16%
to Biodiversity
Appreciation and Aesthetic Sense, 35% 26% 24% 37% 23%
Values
Asking Questions and Making 35% 31% 23% 25% 6%
Hypothesis
Communication through Drawing 46% 37% 30% 30% 10%
and Model-Making
Conceptual Understanding 48% 32% 29% 31% 18%
Experimentation and Field 34% 29% 27% 31% 9%
Investigation
Information Skills and Projects 45% 27% 26% 21% 13%
Grand Total 42% 30% 27% 30% 14%
Skill Wise Performance Improvement in Science+Physics+Biology (All Schools)

Application to Daily Life, Concern 39% 27% 27% 31% 16%
to Biodiversity
Appreciation and Aesthetic Sense, 35% 26% 24% 37% 23%
Values
Asking Questions and Making 35% 31% 23% 25% 6%
Hypothesis
Communication through Drawing 46% 37% 30% 30% 10%
and Model-Making
Experimentation and Field 34% 29% 27% 31% 9%
Investigation
Information Skills and Projects 45% 27% 26% 21% 13%
Grand Total 42% 30% 27% 30% 14%

6.3.4. Social
Skill Wise Performance Improvement in Social (Common Schools)

Appreciation and Sensitivity 41% 34% 26% 44% 25%
Information Skills 42% 38% 27% 38% 15%
Mapping Skills 45% 35% 36% 36% 11%
Reading the Text, Understanding 35% 33% 24% 39% 17%
and Interpretation
Reflection on Contemporary Issues 34% 29% 23% 43% 26%
and Questioning
Grand Total 41% 34% 27% 38% 19%
Skill Wise Performance Improvement in Social (ALL Schools)

Appreciation and Sensitivity 41% 35% 26% 40% 20%
Information Skills 42% 37% 27% 40% 18%
Mapping Skills 45% 36% 36% 38% 13%
Reading the Text, Understanding 35% 31% 24% 41% 21%
and Interpretation
Reflection on Contemporary Issues 34% 34% 23% 38% 16%
and Questioning
Grand Total 41% 34% 27% 40% 20%
6.4. Program Performance – School-wise
The below table shows improvement in both the schools in the treatment group that gave both
baseline and endline exams as compared to the control group students. Both the schools showed
clear improvement as compared to their control group counterparts.
School Name Improvement (DID Score)

K.D.P.M.High school 14%
MVD High School 15%

7. Conclusion
Overall a significant improvement of 15% (Difference in Difference scores between control and treatment
group score changes from baseline to endline) is seen, indicating a significant impact of the intervention
on the student performance.
Overall Improvement Summary

50%
40%
30%
20%
10%
0%
BL EL
Control 43% 36%
Test 29% 36%
The scores of the treatment group students increased from baseline to the endline while the same have
dropped for the control group students. The drop in the scores could be attributed to the increased syllabus
for the endline. However, the relatively better performance of the treatment group could be attributed to
the positive impact of the intervention, hence indicating the efficacy of the intervention.
The score improvements are shown across different learning dimensions like grade, subject, academic
skill so that it will provide detailed insights on the efficacy of the intervention across each of this
dimension.
This data can be used to understand areas where the improvements are higher and continue efforts there
while looking at areas with low score improvements and devise possible measures to work upon the same.

Endline Report-Visakhapatnam Smart City

Uploaded by

Copyright:

Available Formats

You might also like

Endline Report-Visakhapatnam Smart City

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Endline Report-Visakhapatnam Smart City

Uploaded by

Copyright:

Available Formats

Third Party Impact Study

Google Future Classroom Initiative

Impact Assessment Report – Visakhapatnam Smart City

Greater Visakhapatnam Smart City Corporation

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 1

About Vidura Edu ......................................................................................................................................... 3

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 2

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 3

Some highlights from the analysis are indicated below:

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 4

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 5

Image: Students Writing the Endline Test

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 6

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 7

Image: Students attempting the baseline test paper

 Intervention (I) – Google Classroom + K-Yan Lab model: Integrated technology-enabled

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 8

Program Group Students who participated in the study –

Non Program Group (Control) students who participated in the study -

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 9

Image: Teacher Distributing Baseline Test Papers

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 10

1) Overall program performance on each subject – Treatment vs. Control

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 11

4.1. Overall program performance on each subject – Treatment vs. Control

Class 6 Class 7 Class 8 Class 9

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 12

English Skillwise Performance

The table below captures similar information at a grade level.

Academic Skill School Type Class 6 Class 7 Class 8 Class 9

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 13

Maths Skillwise Performance

The table below captures similar information at a grade level.

Academic Skill School Type Class 6 Class 7 Class 8 Class 9

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 14

Science + Physics + Biology

Information Skills and Projects

0% 10% 20% 30% 40% 50% 60%

The table below captures similar information at a grade level.

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 15

Social Skillwise Performance

Reflection on Contemporary Issues and Questioning

Reading the Text, Understanding and Interpretation

Appreciation and Sensitivity

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

The table below captures similar information at a grade level.

Academic Skill School Type Class 6 Class 7 Class 8 Class 9

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 16

School Name Class 6 Class 7 Class 8 Class 9

English - Score Distribution

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 17

School Name Class 6 Class 7 Class 8 Class 9

Maths - Score Distribution

Science Physics Biology

Endline Report - Google Future Classroom Program in Visakhapatnam Smart City 18

Science - Score Distribution

Physics - Score Distribution