Synopsis of Data Collection and Analysis

Katharine S. Jacobs
Report Card Data for School Year 2010-2011
Program Quality Data
Youth Outcomes Survey



















Report Card Data for School Year 2010-2011
Big Brothers Big Sisters of the Coastal Empire (BBBSCE) collected report cards for 160 youth
who had been matched with a mentor for at least six months (Littles). Report cards were
obtained from Chatham and Effingham counties school systems. BBBSCE also collected report
cards for 50 youth who were either waiting for a mentor or had been matched with a mentor
for less than six months (control group). Six months was chosen as the cut-off point because
mentoring research agrees on six months being the minimum amount of time a mentoring
relationship needs to produce positive outcomes for youth.
By comparing the first quarter grades and the last quarter grades (beginning of the school year
and end of the school year) we attempted to quantify the positive effects of mentoring
relationships.
For the mentored youth, grades improved from:
77.38 83.53 in Social Studies (p < .01)
76.03 82.19 in Math (p < .01)
75.53 81.57 in Reading (p < .01)
76.41 83.93 in English Language Arts (p < .01)
79.11 84.01 in Science (p < .01)

Additionally, conduct grades, which are issued in the form of U (Unacceptable), N (Needs
Improvement), S (Satisfactory) and E (Excellent), were coded on a four-point numerical scale
and examined. Students in mentoring relationships began the year with an average conduct
70
75
80
85
90
Social Studies Math Reading Language Arts Science
Mentored Youth's Grades 2010-2011
First Quarter
Fourth Quarter
rating of 2.77 (below Satisfactory) and ended the school year with an average conduct score of
3.00 (Satisfactory). This improvement was statistically significant (p < .05).
In looking at the control group of 50 students, their grades improved from:
81.25 83.89 in Social Studies (p < .03)
78.32 78.98 in Math (p = .73)
84.90 86.09 in Reading (p = .26)
79.67 80.59 in English Language Arts (p = .53)
79.91 83.00 in Science (p = .05)










70
75
80
85
90
Social Studies Math Reading Language Arts Science
Control Group's Grades 2010-2011
First Quarter
Fourth Quarter
The rate at which students were retained (held back a grade) was also examined. Regarding
mentored students, of the 67 report cards which contained this information, only five students
were retained. This works out to a retention rate of .07, less than the Savannah-Chatham
County Public School Systems overall retention rate of .09 (or 2,970 / 33,230, reference:
http://reportcard2010.gaosa.org/%28S%283nwpqx55pnlxo3ettyll3v55%29%29/k12/Indicators.aspX?ID=625:ALL&TestKey=Retained&TestType=indicators)
Students in the control group were retained at a rate of .24 (11/46). This discrepancy, with
mentored students less likely to be retained a grade, is statistically significant (p < .001).
Furthermore, this data reinforces our belief that the agency is reaching their target population,
i.e. children at risk of academic failure, and is affecting change in their clients.









0
5
10
15
20
25
Littles District Overall Unmatched Littles
(Control)
Percent of Students Retained
Attendance was also examined by comparing the percentage of students who missed 0-5 days
of school, 6-15 days and more than 15 days. (These categories are used by the state of Georgia
in their indicators.)
0-5 days: 57% of control group
6-15 days: 30% of the control group
16+ days: 13% of control group

0-5 days: 52% of students in district overall
6-15 days: 37% of students in district overall
16+ days: 11% of students in district overall
(reference: http://reportcard2010.gaosa.org/%28S%28gttslt1sgzjwd34v0bw0hfkn%29%29/k12/Indicators.aspX?ID=625:ALL&TestKey=Attend&TestType=indicators)

0-5 days: 64% of mentored youth
6-15 days: 31% of mentored youth
16+: 5% of mentored youth

Mentored students miss fewer days of school as compared to both the district overall as well as
to their un-mentored peers. This again lends strength to the argument that the agency is
serving youth most at risk of academic failure and helping them achieve positive outcomes.




0
20
40
60
80
100
Mentored Youth Control Group District Overall
Days Absent: Percent of Students in Three
Categories
more than fifteen
six - fifteen
five or fewer
Program Quality Data
Match Retention Rates:
A measure of match quality, match retention rates reflect the percentage of matches still open
and active at specific time intervals. Because mentoring research demonstrates that lasting
relationships lead to better outcomes for youth, match retention rates are a critical indicator of
successful mentoring programs.
BBBSCE Six-month Retention Rate:
School-Based Mentoring: 85%; national BBBS average is 81%
Community-Based Mentoring: 86%; national BBBS average is 84%
BBBSCE Twelve-month Retention Rate:
School-Based Mentoring: 63%; national BBBS average is 42%
Community-Based Mentoring: 43.2%; national BBBS average is 68%

Strength of Relationship Survey:
Mentoring research clearly shows that the stronger the mentoring relationship, the better
youth fare. Seeing the strength of the relationship in measurable terms will guide program
improvements that lead to fewer premature match closures. The Strength of Relationship
survey, SOR, is administered to both mentors and mentees at three months and one year.
Averages are below with higher scores indicating more positive responses.
School-Based Mentoring Average Score (of 5)
Child, 3 months 4.82
Child, End of School Year 4.85
Volunteer, 3 months 4.05
Volunteer, End of School Year 4.19

Community-Based Mentoring Average Score (of 5)
Child, 3 months 4.59
Child, Annual 4.75
Volunteer, 3 months 3.99
Volunteer, Annual 4.08

Additionally, Strength of Relationship Surveys reveal:
Of 287 children and youth surveyed, 203 (71%) report always feeling close to their
mentor and 255 (89%) report feeling mostly or always feeling close to their mentor.
Of 287 children and youth, 227 (79%) stated their relationship with their mentor is
always important to them and 264 (92%) stated their relationship with their mentor
is mostly or always important to them.

Youth Outcome Survey
The Youth Outcomes Survey, YOS, is a self-report survey administered to youth ages nine years
and older. Nationally developed and validated, the YOS examines eight critical areas; social
acceptance, scholastic competency, educational expectations, grades, risk attitudes, parental
trust, truancy and the presence of a special adult in youths lives.








After one calendar year (Community-Based Mentoring) or school year (School-Based
Mentoring) of mentoring, the following percent of students improved by area:
Area Community-Based School-Based
Social Acceptance 69.2% 60.8%*
Scholastic
Competency
30.8% 62.5%**
Educational
Expectations
23.1% 28.8%*
Grades 46.2% 55.6%*
Risk Attitudes 61.5% 30.6%
Parental Trust 46.2% 40.4%
Truancy 61.5% 51%*
Special Adult 46.2% 21.6%
* p-value < .05, ** p-value < .01
Community-Based YOS were not an adequate sample size (n=17) to detect statistically-significant
improvements.

Additionally, the numbers below represent the percent of youth who reported improvements
in the following number of areas:
% Improved Community-Based (%) School-Based (%)
In 1 major category 100.0 98.1
In 2 major categories 94.1 90.4
In 3 major categories 88.2 71.2
In 4 major categories 47.1 40.4
In 5 major categories 11.8 21.2
In 6 major categories 5.9 7.7