Scribd Logo
Upload

Welcome to Scribd!

  • 1.5 Linear Regression

    Uploaded by

    Yau Stein
    17 views28 pages
    The document discusses linear regression and the multivariate normal distribution. It describes: 1) Linear regression aims to find the weights w that minimize the least squares problem ||wTX - y||^2. 2) A Bayesian perspective treats w and y as random variables. w is assumed to follow a multivariate normal prior N(w|0, I) and y follows N(y|w^TX, I) given w and X. 3) Maximum likelihood estimation aims to find the w that maximizes the joint probability P(w,y|X). This is equivalent to minimizing the objective ||w^TX - y||^2 + C||w||^2.

    Original Description:

    linear regression

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document discusses linear regression and the multivariate normal distribution. It describes: 1) Linear regression aims to find the weights w that minimize the least squares problem ||wTX - y||^2. 2) A Bayesian perspective treats w and y as random variables. w is assumed to follow a multivariate normal prior N(w|0, I) and y follows N(y|w^TX, I) given w and X. 3) Maximum likelihood estimation aims to find the w that maximizes the joint probability P(w,y|X). This is equivalent to minimizing the objective ||w^TX - y||^2 + C||w||^2.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    17 views28 pages

    1.5 Linear Regression

    Uploaded by

    Yau Stein
    The document discusses linear regression and the multivariate normal distribution. It describes: 1) Linear regression aims to find the weights w that minimize the least squares problem ||wTX - y||^2. 2) A Bayesian perspective treats w and y as random variables. w is assumed to follow a multivariate normal prior N(w|0, I) and y follows N(y|w^TX, I) given w and X. 3) Maximum likelihood estimation aims to find the w that maximizes the joint probability P(w,y|X). This is equivalent to minimizing the objective ||w^TX - y||^2 + C||w||^2.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 28

    Example: linear regression

    Univariate normal
    Univariate normal
    Univariate normal
    Univariate normal
    Univariate normal: mean
    Univariate normal: variance
    Multivariate normal
    Multivariate normal
    Multivariate normal
    Multivariate normal
    Multivariate normal

    1 1 3 0 2 0
    Σ= Σ= Σ=
    1 2 0 1 0 2

    Full Diagonal Spherical


    !(!#$)
    Parameters: Parameters: 𝐷 Parameters: 1
    &
    Linear regression
    Linear regression
    Linear regression
    Least squares problem
    N
    X
    T 2 T 2
    L(w) = (w xi yi ) = w X y ! min
    w
    i=1
    Least squares problem
    N
    X
    T 2 T 2
    L(w) = (w xi yi ) = w X y ! min
    w
    i=1

    b = arg min L(w)


    w
    w
    Model

    Weights Data

    W X
    Target

    y
    Model

    Weights Data

    W X
    Target

    P (w, y|X) = P (y|X, w)P (w)


    Model

    Weights Data

    W X
    Target

    P (w, y|X) = P (y|X, w)P (w)


    P (y|w, X) = N (y|wT X, 2
    I)
    Model

    Weights Data

    W X
    Target

    P (w, y|X) = P (y|X, w)P (w)


    P (y|w, X) = N (y|wT X, 2
    I)
    2
    P (w) = N (w|0, I)
    Training ТЕХНИЧЕСКИЙ СЛАЙД (НА ДОСКЕ), 6
    минут
    P (w, y|X) = P (y|X, w)P (w)
    P (y|w, X) = N (y|wT X, 2
    I)
    2
    P (w) = N (w|0, I)

    P (w|y, X) =?
    Training ТЕХНИЧЕСКИЙ СЛАЙД (НА ДОСКЕ)

    P (w, y|X) = P (y|X, w)P (w)


    P (y|w, X) = N (y|wT X, 2
    I)
    2
    P (w) = N (w|0, I)

    P (w, y|X)
    P (w|y, X) = / P (w, y|X)
    P (y|X)
    Training ТЕХНИЧЕСКИЙ СЛАЙД (НА ДОСКЕ)

    P (w, y|X) = P (y|X, w)P (w)


    P (y|w, X) = N (y|wT X, 2
    I)
    2
    P (w) = N (w|0, I)

    P (w, y|X)
    P (w|y, X) = / P (w, y|X)
    P (y|X)

    P (w, y|X) ! max


    w
    Training ТЕХНИЧЕСКИЙ СЛАЙД (НА ДОСКЕ)

    P (w, y|X) = P (y|X, w)P (w)


    P (y|w, X) = N (y|wT X, 2
    I)
    2
    P (w) = N (w|0, I)

    P (w, y|X)
    P (w|y, X) = / P (w, y|X)
    P (y|X)

    P (w, y|X) ! max , log P (w, y|X) ! max


    w w
    Training ТЕХНИЧЕСКИЙ СЛАЙД (НА ДОСКЕ)

    P (w, y|X) = P (y|X, w)P (w)


    P (y|w, X) = N (y|wT X, 2
    I)
    2
    P (w) = N (w|0, I)

    log P (w, y|X) ! max


    w
    Training ТЕХНИЧЕСКИЙ СЛАЙД (НА ДОСКЕ)

    P (w, y|X) = P (y|X, w)P (w)


    P (y|w, X) = N (y|wT X, 2
    I)
    2
    P (w) = N (w|0, I)

    log P (w, y|X) ! max


    w
    1 T 2 1 2
    2
    ||w X y|| 2
    ||w|| ! max
    2 2 w
    Training ТЕХНИЧЕСКИЙ СЛАЙД (НА ДОСКЕ)

    P (w, y|X) = P (y|X, w)P (w)


    P (y|w, X) = N (y|wT X, 2
    I)
    2
    P (w) = N (w|0, I)

    log P (w, y|X) ! max


    w
    1 T 2 1 2
    2
    ||w X y|| 2
    ||w|| ! max
    2 2 w

    ||wT X y||2 + C||w||2 ! min


    w

    You might also like