Boosting

Honghao Zhao

PhD, Associate Professor

Department of Decision Sciences, School of Business
Macau University of Science and Technology, Macau, P. R. China

March 28, 2023

1 / 10
Boosting

Adaboost
1 Boosting
Example

2 Adaboost

3 Example

2 / 10
Boosting

Adaboost
1 Boosting
Example

2 Adaboost

3 Example

3 / 10
 Boosting

Boosting
Motivation
Adaboost

Example
Combines the outputs of many weak learners to produce a
powerful prediction rule.

Adaboost
1 Most popular boosting algorithm.

2 Trains weak learners on weighted versions of the training

sample, giving higher weights to the cases which are
currently misclassified.

4 / 10
Boosting

Adaboost
1 Boosting
Example

2 Adaboost

3 Example

5 / 10
 Schematic of AdaBoost

Boosting

Adaboost

Example

6 / 10
 Adaboost

Algorithm
Boosting

Adaboost
1 Given (x1 , y1 ), ..., (xn , yn ) with xi a vector and
Example yi ∈ {−1, +1}
2 Start with weights wi = 1/n, i = 1, ..., n.
3 For m = 1 to M:
1 Fit fm (x) ∈ {−1, +1} using weights wi on the training
data. Pn Pn
2 Compute em = i =1 wi I (yi 6= fm (xi ))/ i =1 wi
3 Compute αm = 21 log 1−e em .
m

(−αm yi fm (xi ))
4 Update wi ← wi e
Output F (x) = sign( M
P
m=1 αm fm (x))
4

7 / 10
Boosting

Adaboost
1 Boosting
Example

2 Adaboost

3 Example

8 / 10
 Example

xi 0 1 2 3 4 5 6 7 8 9
yi 1 1 1 -1 -1 -1 1 1 1 -1
Boosting
wi 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
Adaboost

Example


1, x < 2.5
g1 =
−1, x > 2.5


−1, x < 5.5
g2 =
1, x > 5.5


1, x < 8.5
g3 =
−1, x > 8.5
9 / 10
 Questions and Answers

Boosting

Adaboost

Example

Thank you for your attention!

10 / 10