RP Group Strengthening Student Success Conference * Session #35 * October 3, 2012

Method-ists: Building a DataInformed Culture

Los Angeles Harbor College
Kristi Vollmer Blackburn, Ph.D. Rhea Estoya Elena Reigadas, Ph.D.
Dean of Institutional Effectiveness Research Analyst Faculty Member, SLO Coordinator

Objectives
Participants will be able to identify members of their campus community who could serve as a data team Participants will be able to identify tasks and analyses that their data teams can perform

Achieving the Dream

LAHC Joined AtD Summer 2011 (9 college in District) Lumina Foundation, AtD is a social justice movement to increase equity in success by minority students 5 Goals of AtD:
Students Successfully complete the courses they take Students Advance from remedial to credit-bearing courses Students Enroll in and successfully complete gatekeeper courses Students Enroll from one semester to the next Students Earn degrees and/or certificates

Commit to find out why there are gaps in attainment Examine how to eliminate barriers in order to achieve equity

Approach of AtD
AtD Data Team (data/evaluation/evidence)
Focus on colleges performance with respect to student outcomes, with a special focus on low-income students, students of color and others who face barriers to success Examine data (qual/quant), present findings in a compelling way Involve students/faculty to identify strengths/weaknesses of college policies, structures, services Aid the core team in engaging campus community about data analysis and proposed strategies/goals

AtD Core Team (implementation)

Who should be on the Data Team?

AtD Suggests:
IR/IE Director Senior Planning Admin IT representative Faculty (developmental, gatekeeper, high enrmt/high failure courses) Diversity staff member Student Services Students

What we actually did

All on the left plus: SLO/Assessment Coordinator Director of Financial Aid (although could not participate due to time) Learning Assistance Center Director Admissions and Records Supervisor CTE/Econ. Workforce Devmt Dean Other campus constituencies who wanted to participate Invitations to those who are change agents

WHO SHOULD BE ON YOUR DATA TEAM?

ACTIVITY:
Identify by position at your college who has data/evidence that can be brought to the Team Identify by personality who can be a change agent on your campus Identify those who have authority to enact change and how you will engage them with the Data Team (member?)

AtD DATA TEAM SHARED UNDERSTANDING

Data discussions need to be held within a context. Resist the desire to draw conclusions from incomplete data. Avoid jumping to conclusions. Data discussions are that: discussions. Resist desire to draw conclusions about courses/departments/personnel from what could be incomplete data. Be mindful of how we discuss the data and be cautious of how people hear what we report to the AtD Core Team and CPC. Data in itself is free from judgmenthow it is interpreted and presented makes all the difference as to what people do with it. Keep an open mind and a critical eye. Watch for what makes sense and what doesnt. Keep an open heart: the work we are doing is to support Student Success! Our students will ultimately benefit from this work!

Component One
Whats Wrong?
(Outcome Measures)

Component Two
Why?
(Underlying Factors)

Component Three
Intervention(s) Use data from Component Two to revise or design new interventions to effectively address the underlying factors impeding student success. Review and consider changes to existing college policies that impact the underlying factors impeding student success.

Component Four
Evaluation &
Modification

Use Longitudinal, Disaggregated, Cohort data to assess Student Success Outcomes (e.g., Persistence,
Course Completion rates, Degree comp. rates) to determine:

Collect, analyze, and use second set of LOCAL data to identify the underlying factors (barriers or challenges) impeding student success: Focus Groups Surveys Literature Reviews Learning Outcome Assessment
Many Colleges: (a) Skip (b) Loosely rely on national literature (Engagement) (c) Lack a local understanding based on qualitative data

Collect, analyze, and use evaluation data to answer: 1) To what extent did the interventions (or policy changes) effectively address the underlying factors impeding student success? 2) To what extent did the interventions increase student success? Make modifications based on evaluation results.

1) Which student groups are less successful than others (Equity Gaps in Student Success). 2) Which high enrollment courses have the lowest success rates.

Reference:

Gonzalez, K. P. (2009). Using data to increase student success: A focus on diagnosis. Achieving the Dream Inc. www.achievingthedream.org

DATA TOOLBOX:
2011 Fact Book (hot off the presses!) Leakage Point Analysis Hand out (from Lumina) IPEDS Factsheets for 2005, 2006, 2007, 2008, 2009, 2010, 2011 LAHC readiness submission to Lumina for the grant Alignment of College/District Strategic Plan (presented to the Board June, 2011) Powerpoint of Aligning AtD with Accreditation activities Drop Survey results from Spring 2011 Financial Aid data Learning Assistance Center datawork in progress Powerpoint from Dr. RichardsWho are our Students? Article7 Myths of Student Retention

In the Beginning
LAHC Highest Enrollment Courses X demographic LAHC Highest failure courses X demographic Financial Aid guidelines provided and discussed Matriculation Committee Report/ Assessment Data Summary of Orientation data from E. Colocho Exit Point Analysis (aka Leakage Points or momentum points) Course availability based on placement data (Report) Articles: A Period of Adjustment? Raceadjusted Rates for a State Accountability; TrickleAcross Theory: Student Flow Into and Away from the California Community Colleges Multiple files on Qualitative data collection technique (Focus Groups) ARCC data report from LATTC which has all colleges in District comparison

OUR DATA TEAMS PROCESS OF INQUIRY AND DIALOGUE:

What are our pain points? Retention, Completion, Success What additional data would be useful to know? What research questions should drive our data campaign? What data do we have versus what do we need? What approach do we want to take?

GOAL OF AtD

DATA SHARED

Factbook; exit point analysis; highest enrmt/lowest success data; lowest retention courses data; low success courses data; drop 1. Students Successfully survey results Spring2011; in progress: complete the courses they take dismissal/probation student analysis (which may have some spillover into the other 5 goals) 2. Students Advance from remedial to credit-bearing courses 3. Students Enroll in and successfully complete gatekeeper courses 4. Students Enroll from one semester to the next 5. Students Earn degrees and/or certificates Factbook; exit point analysis; learning center data (in progress); Course availability report (Matriculation) exit point analysis; Factbook; IPEDS; ARCC exit point analysis; Factbook; IPEDS; ARCC Kick Off presentation data slides; exit point analysis; IPEDS data; Factbook; ARCC

PERSISTENCE: The Pipeline

of First Time College Students by Ethnicity & Gender
100% Lowest Persistence In The First Year: African American Male African American Female Hispanic Male White Male

80%

60%
47.0% 44.1%

53.6% 53.0% 41.5% 42.6%

40%
30.3% 25.5%

20%

0%
Fa04 Sp05 Fa05

Asian F

Asian M

AfrAmer F

AfrAmer M

Hisp F

Hisp M

White F

White M

PERSISTENCE
of First Time College Students by Ethnicity & Gender
100% Lowest Persistence In The Second Year: African American Male African American Female Hispanic Male White Male

80%

60%

40%

34.3%

36.5% 27.5% 26.6% 27.8%

20%

22.5% 19.7% 12.1%

0%
Fa04 Sp06 Fa06

Asian F

Asian M

AfrAmer F

AfrAmer M

Hisp F

Hisp M

White F

White M

PERSISTENCE
of First Time College Students by Ethnicity & Gender
100% Lowest Persistence In The Third Year: African American Male African American Female Hispanic Male White Male

80%

60%

40%

21.7%

20%

18.8% 13.7% 12.1% 15.9% 11.8% 7.6% 11.3%

0%
Fa04 Sp07 Fa07

Asian F

Asian M

AfrAmer F

AfrAmer M

Hisp F

Hisp M

White F

White M

PERSISTENCE
of First Time College Students by Ethnicity & Gender
100%

80%

60%

Lowest Persistence In The Fourth Year: African American Male African American Female Hispanic Male White Male Asian Male

40%

20%
13.0% 9.8% 1.5% 11.3% 8.8% 7.6% 6.9% 7.8%

0%
Fa04 Sp08 Fa08

Asian F

Asian M

AfrAmer F

AfrAmer M

Hisp F

Hisp M

White F

White M

PERSISTENCE
of First Time College Students by Ethnicity & Gender
100% Lowest Persistence In The Fifth Year: African American Male African American Female Hispanic Male White Male Asian Male White Female

80%

60%

40%

20%
10.0% 6.1% 5.9% 7.0% 6.3% 0.0% 8.8% 7.0%

0%
Fa04 Sp09

Fa09

Asian F

Asian M

AfrAmer F

AfrAmer M

Hisp F

Hisp M

White F

White M

PERSISTENCE
of First Time College Students -- Summary

Ethnicity and Gender With Lowest Persistence In 5-Year Trend:

African American Males African American Females Hispanic Males White Males

DEGREE & CERTIFICATE

Attainment of First Time College Students Over Time
65

39
Count

36
20 19 7

7
Within 1 year 2 years 3 years 4 years 5 years

6 years More than 6 years

DEGREE & CERTIFICATE

Attainment of First Time College Students - Summary

Awards Received:
Within 2 years -- 4% received a degree or certificate. Within 4 years 12% received a degree or certificate.

Overall, 16% (or 193) of 1,209 First Time College Students in Fall 2004 received a degree or certificate.

Fall 2004 and Fall 2005 Comparison

FALL 2005: Age, ethnicity, and gender fairly consistent between the two semesters. Percentage of awards received also consistent at 16% (6 or more years). Slight shift on lowest persistence between disaggregated groups-Fall 2004: African American F African American M Hispanic M Fall 2005: African American F African American M White M

English and Math Within the First Year of College

ENGLISH Out of 1,354, 806 (60%) took an English course within the first year.
-16% (126) Asian -15% (119) White -11% (90) African American -52% (418) Hispanic

MATH Out of 1,354, 749(55 %) took a Math course within the first year.
-15% (112) Asian -15% (109) White -10% (77) African American -54% (408) Hispanic

Lowest groups to successfully complete:

52% - White M (25 out of 48) 45% - Afr Amer M (18 out of 40) 52% - Afr Amer F (26 out of 50) 62% - Hispanic F (160 out of 259)

Lowest groups to successfully complete Math:

33% - White M (15 out of 45) 20% - Afr Amer M (7 out of 35) 38% - Afr Amer F (16 out of 42) 39% - Hispanic M (61 out of 158)

Fall 2004 and Fall 2005 Comparison

FALL 2005: Age, ethnicity, and gender fairly consistent between the two semesters. Percentage of awards received also consistent at 16% (6 or more years). Slight shift on lowest persistence between disaggregated groups-Fall 2004: African American F African American M Hispanic M Fall 2005: African American F African American M White M

Unexpected Outcomes of Sharing Data

Discovering our SIS could not hold major code semester to semester unless the student declared upon entry
Assessment Committee CTE Deans Admissions & Records Deans Recommended change to the D.O. for the SIS
New SIS coming on board and resources scarce!

Discussion of Majors fair/Declare Day

Students major choice and relationship to math (Focus Groups and Survey)
Students choose their majors based on least amount of math possible Sequence too long; want acceleration Students very savvy about other colleges in/out of District Language issues by Instructor awareness, not sure how to address

English 28 is harder than English 101 (Focus Groups) Overwhelming theme that students know it is their responsibility to learn
BUT, do they have the skills to meet the responsibility?

Outcomes leading to changes

Student Services Interventions
Created an In-person Orientation Creation of FYE cohorts with Learning Coaches Puente

Math/English Interventions
Need more Math/English sections Hiring of Specialists, not just content experts On Course Training Fast Track/Math acceleration

Equity Intervention
Culture of poverty (first year) Creation of an Urban Center (second year, funding dependent)

Evaluation (all of the above)

Data shared Data discussed

How will you guide the process on your campus?

ACTIVITY: What data will you find useful? Where is the data located? How will you share the data? Can you use existing structure on your campus ie. Assessment Committee

Wrap Up