A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction

Purpose Terminology Glicko Rating System

A Bayesian Skill Rating System for NCAA Teams

Alvin Chen

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Senior Seminar, 2014

Outline
Introduction Purpose Terminology Glicko Rating System

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Outline
Introduction Purpose Terminology Glicko Rating System Predictive Models Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Outline
Introduction Purpose Terminology Glicko Rating System Predictive Models Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation Outlook

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Purpose

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Research goal: Improve the speed and accuracy of the Glicko rating system for NCAA Mens Basketball

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Purpose

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Research goal: Improve the speed and accuracy of the Glicko rating system for NCAA Mens Basketball Developed predictive ratings system for NCAA Mens Basketball

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Purpose

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Research goal: Improve the speed and accuracy of the Glicko rating system for NCAA Mens Basketball Developed predictive ratings system for NCAA Mens Basketball Focused on quantifying predictions

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Terminology

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Ranking Ordering of teams

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Terminology

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Ranking Ordering of teams Rating (points) An estimation of a teams relative skill level

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Terminology

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Ranking Ordering of teams Rating (points) An estimation of a teams relative skill level Rating deviation Uncertainty of rating

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Terminology

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Ranking Ordering of teams Rating (points) An estimation of a teams relative skill level Rating deviation Uncertainty of rating Rating adjustment Change in rating after additional evidence

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Glicko Rating System

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Designed for head to head competition

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Glicko Rating System

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Designed for head to head competition Considers strength of schedule

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Glicko Rating System

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Designed for head to head competition Considers strength of schedule Includes uncertainty for ratings

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Glicko Rating System

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Designed for head to head competition Considers strength of schedule Includes uncertainty for ratings Ratings deviations begin large and shrink over time

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Glicko Rating System

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Uses logistic function for expected outcomes 1 E= r1 r2 1 + 10 400

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Glicko Rating System

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Uses logistic function for expected outcomes 1 E= r1 r2 1 + 10 400 Ratings describe skill in relation to other teams

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Glicko Rating System

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Uses logistic function for expected outcomes 1 E= r1 r2 1 + 10 400 Ratings describe skill in relation to other teams Updates rating by comparing outcomes to predictions R = R + (RD )(s E )

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Model

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

No prior information on teams

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Model

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

No prior information on teams All teams begin with rating of 1500 and ratings deviation of 200

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Model

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

No prior information on teams All teams begin with rating of 1500 and ratings deviation of 200 Only considers wins and losses

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Model

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

No prior information on teams All teams begin with rating of 1500 and ratings deviation of 200 Only considers wins and losses Wins are worth 1, losses are worth 0

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Model

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

No prior information on teams All teams begin with rating of 1500 and ratings deviation of 200 Only considers wins and losses Wins are worth 1, losses are worth 0 Rating adjusted by performance vs expectations

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Sample Calculations
Team A Rating 1500.00 RD 200 R = 1500 + 200(1 0.5) Rating 1600.00 RD 185 Team B Rating 1500.00 RD 200 R = 1500 + 200(0 0.5) Rating 1400.00 RD 185

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Sample Calculations
Team A Rating 1600.00 RD 185 R = 1600 + 185(1 0.5) Rating 1692.50 RD 170 Team C Rating 1600.00 RD 130 R = 1600 + 130(0 0.5) Rating 1535.00 RD 120

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Sample Calculations
Team A Rating 1692.50 RD 170 R = 1692.50 + 170(1 0.89) Rating 1710.79 RD 160 Team D Rating 1325.00 RD 160 R = 1325 + 160(0 0.11) Rating 1307.78 RD 151

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Sample Calculations
Team A Rating 1710.79 RD 160 R = 1710.79 + 160(1 0.95) Rating 1558.82 RD 157 Team E Rating 1200.00 RD 150 R = 1200 + 150(0 0.05) Rating 1342.47 RD 148

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Sample Calculations
Team A Rating 1710.79 RD 160 R = 1710.79 + 160(1 0.95) Rating 1558.82 RD 157 Team E Rating 1200.00 RD 150 R = 1200 + 150(0 0.05) Rating 1342.47 RD 148

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Performance

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Discarded games before December 15th, 2012

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Performance

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Discarded games before December 15th, 2012 First 10 games considered provisional rating period Model Correct Incorrect % Correct Naive 3455 2168 61.4 RPI 3790 1833 67.4

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Performance
Statistically signicant deviation from 0.5 (P < 1066 )

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Performance
Statistically signicant deviation from 0.5 (P < 1066 ) Ratings spread insuciently

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Performance
Statistically signicant deviation from 0.5 (P < 1066 ) Ratings spread insuciently Range of 341 for Division I ratings

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Naive Performance
Statistically signicant deviation from 0.5 (P < 1066 ) Ratings spread insuciently Range of 341 for Division I ratings Implies 11% chance for the worst team to beat the best team

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Prior Seasons

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Assumes a teams performance predicts its performance next season

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Prior Seasons

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Assumes a teams performance predicts its performance next season Assumes a 30 game season with over 500 teams is insucient

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Prior Seasons Performance

0 61.444% 1 61.672% 2 61.702% 3 61.704% 4 61.704%

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Prior Seasons Performance

0 61.444% 1 61.672% 2 61.702% 3 61.704% 4 61.704%

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Noticeable eect on accuracy from prior seasons No signicant dierence between 3 seasons and 4 seasons

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Prior Seasons Performance

0 61.444% 1 61.672% 2 61.702% 3 61.704% 4 61.704%

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Noticeable eect on accuracy from prior seasons No signicant dierence between 3 seasons and 4 seasons

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Spreading Ratings

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Based on historic results for post season tournament

Outlook

Spreading Ratings

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Based on historic results for post season tournament Considered only games with at least 40 occurences

Outlook

Spreading Ratings

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Based on historic results for post season tournament Considered only games with at least 40 occurences Assumed no favorable matchups

Outlook

Spreading Ratings

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Spreading Ratings Results

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Home Court Advantage

Kvam and Sokol determined a home court advantage of 6.7 points

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Home Court Advantage

Kvam and Sokol determined a home court advantage of 6.7 points

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Margin of Victory

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Performed poorly compared to naive approach

Outlook

Margin of Victory

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Performed poorly compared to naive approach Punished winning teams and awarded losing teams

Outlook

Margin of Victory

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Performed poorly compared to naive approach Punished winning teams and awarded losing teams Inferred information that was unavailable

Outlook

Home Court Rating

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction

Home team wins 68% of the time

Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Home Court Rating

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction

Home team wins 68% of the time Translates into 130 eective rating increase for home team

Purpose Terminology Glicko Rating System

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Home Court Results

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Model Naive Priors Home Court

Correct 3455 3469 3593

Incorrect 2168 2154 2030

% Correct 61.4 61.7 63.9

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Ratings Deviation

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction

1 1 1 RD = ( RD 2 + d2 ) m 10 (g (RDj ))2 E (s |r , rj , RDj )(1 d 2 = ( ln 400 j =1

Purpose Terminology Glicko Rating System

E (s |r , rj , RDj )))

Predictive Models
Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Outlook

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Investigate ratings deviation

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Outlook

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Investigate ratings deviation Use recently updated volatility rating to change ratings deviation based on time between games

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook

Outlook

A Bayesian Skill Rating System for NCAA Teams Alvin Chen Introduction
Purpose Terminology Glicko Rating System

Predictive Models

Investigate ratings deviation Use recently updated volatility rating to change ratings deviation based on time between games Use updated iterative algorithm to determine optimize initial RD

Naive Model Sample Calculations Prior Seasons Spreading Ratings Home Court Advantage Ratings Deviation

Outlook