n

input

Space
o
X set of valid inputs e.g IR

values Y
Target

set of labels e
g oil

Concept

8 c X Y a function which assigns

label to input

want to learn
Concept class c set of C's that we

may
e triangles etc

g Half spaces

Distribution 0 which governs the samples drawn from

X D is unknown underlying distribution

Sample S i.yD.Cnc.yD Cam ymD

where c ni c C
for

Yi
some

Samples drawn iid from X D

c is the unknown function that we

want to learn

Hypothesis set H set h X y

of which are

considered as candidates the

by

learner

 Risky 3 Rcn Expected mismatch of h from

the true unknown c over
Generalization
error X D
True Risk
had CUD then cars
hypothesish Par nE
of
e is unknown Dis unknown

so Rch is not accessible

directly
It is estimated by empirical methods

Empirical Risk Rasch Average rate of mismatch

b w hypothesis
h d trueunknown
h
tf hypothesis her the sample S
n and

on a sample s niacin
is
Ras h E
ie I
thenistani

estimate the
Sample error Rsch is an of
aka empirical unaccessible true risk
generalization erorror
Rcn
with some theoretical
guarantees
Observation
h the expected empirical risk
For a
fixed hypothesis
h
Over X D is the same as true risk
the of
ie E Rasch Rch
S Dm

lhcnist
Rasch
ccniss
ccnis
ChD upmfmtEmlhcnist
E.fr gE

D
If
then can
mL

RCD
R h
Lm
m
RID a
PAI
A concept class he is PAC leernable if
Some algorithm Age can produce

for every sample SD c of

reasonable size
C BE

a
hypothesis hs.ge
It Risk of w r t D c

generalization ghs e

error

is smell CE E

with l s
high probability

E 3 i s
s
E.cl fhs e

Reesoneble Sire Polynomial in Le Lg

Risk is writ the same 0

Lte that
from which S was drawn

doesn't D
PAC assume
anything about

To Do Think about the limitation of PAC

e.g finding Algorithm which could be hard
PAC A concept class C is PAC learnable if
Whoo with a reasonable Isambles some algorithm
can
get a

hypothesis which has low risk with

good

high probability

or some Algorithm with a reasonable samples

can produce a low risk hypothesis

C
ie of the actual
c
an approximation
with low error

with high probability

ie most
of the time the hypothesis is
low risk