Scribd Logo
Upload

Welcome to Scribd!

  • Supervised Learning Live Lecture Notes CS229

    Uploaded by

    Shivam Patel
    6 views8 pages
    1. The document discusses various machine learning models including perception, exponential families, generalized linear models, softmax regression, logistic regression, and softmax regression for multiclass classification. 2. Softmax regression is a generalization of logistic regression that can be used for multiclass classification problems. It works by taking the softmax of the logits to output a probability distribution over classes. 3. The goal of softmax regression is to minimize the cross entropy between the predicted probabilities and true labels using gradient descent. This has the effect of maximizing the log likelihood of the correct class.

    Original Description:

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    1. The document discusses various machine learning models including perception, exponential families, generalized linear models, softmax regression, logistic regression, and softmax regression for multiclass classification. 2. Softmax regression is a generalization of logistic regression that can be used for multiclass classification problems. It works by taking the softmax of the logits to output a probability distribution over classes. 3. The goal of softmax regression is to minimize the cross entropy between the predicted probabilities and true labels using gradient descent. This has the effect of maximizing the log likelihood of the correct class.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    6 views8 pages

    Supervised Learning Live Lecture Notes CS229

    Uploaded by

    Shivam Patel
    1. The document discusses various machine learning models including perception, exponential families, generalized linear models, softmax regression, logistic regression, and softmax regression for multiclass classification. 2. Softmax regression is a generalization of logistic regression that can be used for multiclass classification problems. It works by taking the softmax of the logits to output a probability distribution over classes. 3. The goal of softmax regression is to minimize the cross entropy between the predicted probabilities and true labels using gradient descent. This has the effect of maximizing the log likelihood of the correct class.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 8

    Perception

    Exponential Family
    Generalized Linear Models
    Soft max Regression
    Multiclass Classification

    Logistic Regression

    96 I 270
    z
    g z O Z CO

    ho n hole Ota
    pox g

    0 it 0 a ye hold nj
    T
    different
    ye hold
    O algorithm got it right
    11 1 I if wrong yo I
    I
    if wrong y O
    Eno
    o
    o or an
    n Ot autre
    On t

    t.IT
    Exponential Families
    PD F

    p ly i n b y exp
    qt Tcg acn
    data
    y
    1 natural parameter
    T ly sufficient statistic
    y today
    b Base measure
    y
    acn log partition function

    ply n bly
    exp Etsy

    scalar
    y
    vector scalar
    n
    T k
    y
    b y scalar

    Bernoulli Binary Data


    Q probability of event
    ty
    ply o q's i o
    y
    exp log O 1 0
    log
    exp 4 y log 1 0

    T Ey Ts
    bly 1

    y Tcg
    2 1091,4 4 n
    sigmoid

    acn logo 4 log l


    ten
    log Iten
    Gaussian w
    fixed variance Assume 0 1

    ply m exp 19 5
    e 9
    H exp my m

    it
    bly exp 42

    Ty y
    ne m
    acn ME Ez
    Properties natural parameter

    MLE w.it is concave


    I
    negative Log Likelihood NLL is convex

    Ely 2
    Zach
    Var lyin acn
    Ig
    GLM
    Assumptions Design Choices
    1 y n O Exponential family
    Real Gaussian
    Binary Bernoulli
    Count Poisson
    Rt Gamma Exponential
    Dist Beta Dirichlet

    4 U OTR n Ird
    Rd Oe
    Test time Output Ely n o
    hole Ely ago
    LinearModel

    Elgin
    offs thing
    x
    Ely Ota
    ho a Cest

    ota
    my log ply Gram

    Learning Update Rule


    O t O 2
    g hold Nj
    T in appropriate ho u
    plug

    Terminology n natural parameter

    µ Ely in 9 q canonical response for

    q g pi
    n link th
    gin Inam
    3 parametrizations
    Model parameters Natural param
    O Ote y 4 Bernoulli

    T g 9 pyo Gaussian
    x Poisson
    Design 1
    learning g
    g

    Logistic Regression
    holn E Lyla 03 9 Hn For
    Assumptions
    Regression Assumption
    distributed as
    y forany n
    gaussian

    y R
    w mean
    pOTR

    Ote n p

    Data

    KEY
    i
    Edo
    Classification

    X X X X X X X X

    Ltd

    tiger
    Soft Max Regression
    Member
    of GLM family
    Cross Entropy Minimization

    Multiclass Classification

    a E Ird
    Label YE O I CO O 1,0 one hot vector
    eg
    d
    class E IR
    class E D O D
    if't
    I
    Softmax regression generalization of logistic regression

    07 0

    In 0

    Given K

    XM

    logtspace

    I If

    gag PG ply

    I th
    th
    8

    min distance between dumb


    goal
    Mun cross entropy
    Cross Entropy
    p p E p y log ply
    ye o o O

    log p yo

    e.gg E
    ie 0,0 o

    p
    Gradient descent

    You might also like