Professional Documents
Culture Documents
Public Policy Report: UC Student Default Python Model: The Council For Education (CED)
Public Policy Report: UC Student Default Python Model: The Council For Education (CED)
UC Student Default
Python Model
The Council for Education (CED)
Publication Date: October 3, 2017
The CED is registered with the Nevada Secretary of State as a Non-Profit Corporation.
EIN: 205036299
13337 South St., 416 t. 800-307-1076 x 101 director@CforED.com
Cerritos, CA 90703 f. 877-459-7907 http://CforED.com
Out[1]: 'This project analyzes the changes in non-resident student enrollment in \nUniversity of California campuses and its effect on outstanding student loans, \ndefault rates and oth
er metrics'
In [23]: fl = 'data/Input_For_Model_Data_filled_v10.csv'
------------------------------------------------------
Inferred types from first 100 line(s) of file as
column_type_hints=[int,int,int,int,int,str,float,float,str,float]
If parsing fails due to incorrect types, you can correct
the inferred type list above and pass it to read_csv in
the column_type_hints argument
------------------------------------------------------
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 1/19
10/3/2017 Student_Default_Model_janu_v2.0
In [25]: print(df1)
+----+------+-------------------------------+-------------------------------+-------------------------------+
| X1 | Year | Number of Residential UC ... | Number of Non Residential ... | Percentage of UC Students... |
+----+------+-------------------------------+-------------------------------+-------------------------------+
| 0 | 1999 | 128779 | 7879 | 6 |
| 1 | 2000 | 132458 | 8479 | 6 |
| 2 | 2001 | 138735 | 8952 | 6 |
| 3 | 2002 | 145702 | 8845 | 6 |
| 4 | 2003 | 150177 | 8867 | 6 |
| 5 | 2004 | 149630 | 8422 | 5 |
| 6 | 2005 | 150777 | 7953 | 5 |
| 7 | 2006 | 154950 | 8024 | 5 |
| 8 | 2007 | 159055 | 8272 | 5 |
| 9 | 2008 | 163773 | 9000 | 5 |
+----+------+-------------------------------+-------------------------------+-------------------------------+
+-------------------------------+-------------------------------+-------------------------------+
| National Cohort Default R... | Total Volume of Student Lo... | Total Volume of Default St... |
+-------------------------------+-------------------------------+-------------------------------+
| 5.60% | 31582873484.0 | 1768640915.1 |
| 5.90% | 33176168436.0 | 1957393937.72 |
| 5.40% | 36387561831.0 | 1964928338.87 |
| 5.20% | 41872392524.0 | 2177364411.25 |
| 4.50% | 47989697452.0 | 2159536385.34 |
| 5.10% | 53115729625.0 | 2708902210.88 |
| 4.60% | 56288062618.0 | 2589250880.43 |
| 5.20% | 60335198802.0 | 3137430337.7 |
| 6.70% | 68891841340.0 | 4615753369.78 |
| 7.00% | 85279308917.0 | 5969551624.19 |
+-------------------------------+-------------------------------+-------------------------------+
+-------------------------------+-------+
| UC Overcharges / NonReside... | DRate |
+-------------------------------+-------+
| Data Not Available | 0.056 |
| Data Not Available | 0.059 |
| Data Not Available | 0.054 |
| Data Not Available | 0.052 |
| Data Not Available | 0.045 |
| Data Not Available | 0.051 |
| Data Not Available | 0.046 |
| $227,000,000.00 | 0.052 |
| $225,000,000.00 | 0.067 |
| $250,000,000.00 | 0.07 |
+-------------------------------+-------+
[18 rows x 10 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 2/19
10/3/2017 Student_Default_Model_janu_v2.0
In [27]: pd1.columns
In [159]: pd1
0 1999 128779 7879 6 5.60% 3.158287e+10 1.768641e+09 Data Not Available 0.056
1 2000 132458 8479 6 5.90% 3.317617e+10 1.957394e+09 Data Not Available 0.059
2 2001 138735 8952 6 5.40% 3.638756e+10 1.964928e+09 Data Not Available 0.054
3 2002 145702 8845 6 5.20% 4.187239e+10 2.177364e+09 Data Not Available 0.052
4 2003 150177 8867 6 4.50% 4.798970e+10 2.159536e+09 Data Not Available 0.045
5 2004 149630 8422 5 5.10% 5.311573e+10 2.708902e+09 Data Not Available 0.051
6 2005 150777 7953 5 4.60% 5.628806e+10 2.589251e+09 Data Not Available 0.046
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 3/19
10/3/2017 Student_Default_Model_janu_v2.0
In [31]: pd1=pd1.dropna();
pd1
Year
1999 128779 7879 6 5.60% 3.158287e+10 1.768641e+09 Data Not Available 0.056
2000 132458 8479 6 5.90% 3.317617e+10 1.957394e+09 Data Not Available 0.059
2001 138735 8952 6 5.40% 3.638756e+10 1.964928e+09 Data Not Available 0.054
2002 145702 8845 6 5.20% 4.187239e+10 2.177364e+09 Data Not Available 0.052
2003 150177 8867 6 4.50% 4.798970e+10 2.159536e+09 Data Not Available 0.045
2004 149630 8422 5 5.10% 5.311573e+10 2.708902e+09 Data Not Available 0.051
2005 150777 7953 5 4.60% 5.628806e+10 2.589251e+09 Data Not Available 0.046
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 4/19
10/3/2017 Student_Default_Model_janu_v2.0
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 5/19
10/3/2017 Student_Default_Model_janu_v2.0
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 6/19
10/3/2017 Student_Default_Model_janu_v2.0
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 7/19
10/3/2017 Student_Default_Model_janu_v2.0
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 8/19
10/3/2017 Student_Default_Model_janu_v2.0
In [169]: pd2
Out[169]:
Year Non_Residential_Count Non_Residential_Percent Default_Rate_Percent Student_Loans_Disbursements Default_Student_Loans_in_1000000_$
In [170]: pd2.iloc[0]['Default_Rate_Percent']
Out[170]: 5.5300000000000002
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 9/19
10/3/2017 Student_Default_Model_janu_v2.0
In [171]: Class = []
for index,row in pd2.iterrows():
if (row['Default_Rate_Percent'] < 5.0):
Class.append('Low')
elif(row['Default_Rate_Percent'] > 10.0):
Class.append('High')
else:
Class.append ('Moderate')
print(Class)
Out[173]: Student_Loans_Disbursemen
Year Non_Residential_Count Non_Residential_Percent Default_Rate_Percent
ts ...
Default_Student_Loans_in_
Default_Class
1000000_$ ...
$7,908,950,494.00 Moderate
$10,559,841,392.00 Low
$32,268,679,001.00 Moderate
$52,089,862,755.00 High
[4 rows x 7 columns]
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 10/19
10/3/2017 Student_Default_Model_janu_v2.0
Logistic regression:
--------------------------------------------------------
Number of examples : 4
Number of classes : 3
Number of coefficients : 6
--------------------------------------------------------
+-----------+----------+--------------+-------------------+
+-----------+----------+--------------+-------------------+
| 1 | 2 | 0.050926 | 0.750000 |
| 2 | 3 | 0.051541 | 0.750000 |
| 3 | 4 | 0.052005 | 0.750000 |
+-----------+----------+--------------+-------------------+
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 11/19
10/3/2017 Student_Default_Model_janu_v2.0
In [175]: model.evaluate(df2)
Rows: 3
Data:
+--------------+-----------------+-------+
| target_label | predicted_label | count |
+--------------+-----------------+-------+
| Moderate | Moderate | 2 |
| High | High | 1 |
| Low | Moderate | 1 |
+--------------+-----------------+-------+
[3 rows x 3 columns], 'f1_score': 0.6, 'log_loss': 0.5560577881480753, 'precision': 0.8333333333333333, 'recall': 0.6666666666666666, 'roc_curve': Columns:
threshold float
fpr float
tpr float
p int
n int
class int
Rows: 300003
Data:
+-----------+-----+-----+---+---+-------+
| threshold | fpr | tpr | p | n | class |
+-----------+-----+-----+---+---+-------+
| 0.0 | 1.0 | 1.0 | 1 | 3 | 0 |
| 1e-05 | 1.0 | 1.0 | 1 | 3 | 0 |
| 2e-05 | 1.0 | 1.0 | 1 | 3 | 0 |
| 3e-05 | 1.0 | 1.0 | 1 | 3 | 0 |
| 4e-05 | 1.0 | 1.0 | 1 | 3 | 0 |
| 5e-05 | 1.0 | 1.0 | 1 | 3 | 0 |
| 6e-05 | 1.0 | 1.0 | 1 | 3 | 0 |
| 7e-05 | 1.0 | 1.0 | 1 | 3 | 0 |
| 8e-05 | 1.0 | 1.0 | 1 | 3 | 0 |
| 9e-05 | 1.0 | 1.0 | 1 | 3 | 0 |
+-----------+-----+-----+---+---+-------+
[300003 rows x 6 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.}
In [178]: '''The above results show that the model can predict the level of default rate accuracy': 0.75
and 'precision': 0.83 using the features = ['Non_Residential_Percent','Non_Residential_Count']
For this sample data set this proves the predictive power of these features '''
Out[178]: "The above results show that the model can predict the level of default rate accuracy': 0.75 \nand 'precision': 0.83 using the features = ['Non_Residential_Percent','Non_Reside
ntial_Count']\nFor this sample data set this proves the predictive "
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 12/19
10/3/2017 Student_Default_Model_janu_v2.0
In [49]: ''' The table and the graphs below show the growth of revenue from Non-resident students in UC-campuses and
Campuswide from 2005-2016 '''
Out[49]: ' The table and the graphs below show the growth of revenue from Non-resident students in UC-campuses and \nCampuswide from 2005-2016 '
Out[50]: Year Session Non-Resident Total Tuition UC Berkeley UC Davis UCLA UCIrvine UCSanDiego UCSantaBarbara UCRiverside UCMerced UCSantaCruz UCSFO
0 2005 2004-05 236131 62528 24976 50293 24976 27803 22273 10255 0 10646 2381
1 2006 2005-06 236533 62909 22076 48870 23981 27745 21709 9990 4927 12007 2319
2 2007 2006-07 230830 61208 22165 52692 21776 29654 20561 9701 422 10407 2244
3 2008 2007-08 254241 65977 24272 58424 24440 33225 23071 10588 773 10600 2871
4 2009 2008-09 276891 72438 26350 68053 25681 35473 23338 11530 1400 10061 2567
5 2010 2009-10 305236 80961 28247 77067 25407 42088 24170 12093 1871 10500 2832
6 2011 2010-11 332975 96678 30459 84897 26612 46406 23849 11070 1882 8404 2718
7 2012 2011-12 433041 131451 36557 105286 38344 62125 28637 14266 2927 10395 3053
8 2013 2012-13 534540 145806 46518 132155 51274 81743 39967 18603 4476 11087 2911
9 2014 2013-14 656418 174014 60924 157077 69280 104448 45419 20454 6307 15200 3295
10 2015 2014-15 769998 185960 78779 176958 87971 126933 54559 23338 8013 24388 3099
11 2016 2015-16 938080 202918 106679 201946 120037 164644 68822 24266 11776 34004 2988
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 13/19
10/3/2017 Student_Default_Model_janu_v2.0
In [61]: pd3
Out[61]: Session Non-Resident Total Tuition UC Berkeley UC Davis UCLA UCIrvine UCSanDiego UCSantaBarbara UCRiverside UCMerced UCSantaCruz UCSFO
Year
2005 2004-05 236131 62528 24976 50293 24976 27803 22273 10255 0 10646 2381
2006 2005-06 236533 62909 22076 48870 23981 27745 21709 9990 4927 12007 2319
2007 2006-07 230830 61208 22165 52692 21776 29654 20561 9701 422 10407 2244
2008 2007-08 254241 65977 24272 58424 24440 33225 23071 10588 773 10600 2871
2009 2008-09 276891 72438 26350 68053 25681 35473 23338 11530 1400 10061 2567
2010 2009-10 305236 80961 28247 77067 25407 42088 24170 12093 1871 10500 2832
2011 2010-11 332975 96678 30459 84897 26612 46406 23849 11070 1882 8404 2718
2012 2011-12 433041 131451 36557 105286 38344 62125 28637 14266 2927 10395 3053
2013 2012-13 534540 145806 46518 132155 51274 81743 39967 18603 4476 11087 2911
2014 2013-14 656418 174014 60924 157077 69280 104448 45419 20454 6307 15200 3295
2015 2014-15 769998 185960 78779 176958 87971 126933 54559 23338 8013 24388 3099
2016 2015-16 938080 202918 106679 201946 120037 164644 68822 24266 11776 34004 2988
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 14/19
10/3/2017 Student_Default_Model_janu_v2.0
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 15/19
10/3/2017 Student_Default_Model_janu_v2.0
In [63]: pd3.columns
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 16/19
10/3/2017 Student_Default_Model_janu_v2.0
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 17/19
10/3/2017 Student_Default_Model_janu_v2.0
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 18/19
10/3/2017 Student_Default_Model_janu_v2.0
In [ ]:
file:///C:/Users/admin/Google%20Drive/financials/______deliverables/batch%2016%20+/Student_Default_Model_janu_v2.0.html 19/19