Scribd Logo
Upload

Welcome to Scribd!

  • 1 views

    Interpretatiof Logistic Regression

    Uploaded by

    Milton Espinoza
    The document describes using a logistic regression model to classify images from the MNIST dataset. The MNIST dataset contains images of handwritten digits that are encoded as numbers between 0-255. Logistic regression is used to build a predictive model that takes these encoded images as input and predicts the digit in the image.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content

    You might also like

    Interpretatiof Logistic Regression

    Uploaded by

    Milton Espinoza
    1 views24 pages
    The document describes using a logistic regression model to classify images from the MNIST dataset. The MNIST dataset contains images of handwritten digits that are encoded as numbers between 0-255. Logistic regression is used to build a predictive model that takes these encoded images as input and predicts the digit in the image.

    Original Description:

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document describes using a logistic regression model to classify images from the MNIST dataset. The MNIST dataset contains images of handwritten digits that are encoded as numbers between 0-255. Logistic regression is used to build a predictive model that takes these encoded images as input and predicts the digit in the image.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    1 views24 pages

    Interpretatiof Logistic Regression

    Uploaded by

    Milton Espinoza
    The document describes using a logistic regression model to classify images from the MNIST dataset. The MNIST dataset contains images of handwritten digits that are encoded as numbers between 0-255. Logistic regression is used to build a predictive model that takes these encoded images as input and predicts the digit in the image.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 24

    Interpretation of

    Logistic Regression
    MNIST Data Set

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Number
    0-9
    DATA MACHINE MODEL PREDICTION
    Images are Encoded as Numbers
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Images are Encoded as Numbers
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Images are Encoded as Numbers
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Images are Encoded as Numbers
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 118 200 223 155 155 23 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 6 1248214 253 253 253 253 254 229 213 67 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 43 198 253 254 253 247 175 175 235 253 253 108 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 9 212 253 253 224 58 19 0 0 16 139 247 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 254 254 186 14 0 0 0 0 0 0 189 254 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 58 253 183 8 0 0 0 0 0 0 36 222 253 118 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 54 250 128 2 0 0 0 6 71 192 237 253 247 71 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 213 253 50 0 18 123 198 253 254 253 247 85 0 0 0 0 0 0 0

    X=
    0 0 0 0 0 0 0 0 0 69 241 227 136 200 253 253 253 254 192 34 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 175 254 256 254 2545254 149 59 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 101 253 253 254 253 253 253 42 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 8 138 247 253 243 159 196 243 253 19 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 12 183 253 253 253 50 0 0 49 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 180 254 253 213 50 2 0 0 711 253 214 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 234 217 97 10 0 0 0 23 207 254 215 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 174 253 156 0 0 0 0 45 215 253 253 95 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 9 210 253 163 5 19 49 130 244 253 251 137 4 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 13 229 253 254 192 253 253 253 254 253 137 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 160 253 254 253 253 253 253 230 132 4 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 18 112 194 254 254 254 163 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    Output is Binary

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Learning on MNIST
    Training Set Logistic Regression Model
    σ(zi) (or “Network”)
    zi = (b1 X xi1) + (b2 X xi2 (b2 X xi2) + ꔇ + (bM X xiM) + b0
    zi

    bM
    b1

    xi1 xiM

    (b0, b1,… bM)


    Learned Parameters
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Zooming in on 0/1

    subject learned parameters


    – 0.2

    – 0.1

    σ( x – 0.0

    – –0.1
    )
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Zooming in on 0/1

    – 0.2

    – 0.1

    σ( – 0.0

    – –0.1
    ) = 0.006
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Zooming in on 0/1

    negative sections – 0.2

    – 0.1

    σ( – 0.0

    – –0.1
    ) = 0.006
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Zooming in on 0/1

    – 0.2

    – 0.1

    σ( – 0.0

    – –0.1
    ) = 0.991
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Zooming in on 0/1

    positive sections – 0.2

    – 0.1

    σ( – 0.0

    – –0.1
    ) = 0.991
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Learned Weights for 0/1
    – 0.2

    – 0.1

    σ( x – 0.0

    – –0.1
    ) = 0.006
    (6% chance it is a 1)
    – –0.2

    – 0.2

    – 0.1

    σ( x – 0.0

    – –0.1
    ) = 0.991
    (99.1% chance it is a 1)
    – –0.2

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Interpretation of
    Logistic Regression
    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
    xi1 xi2 xiM
    add up the M products
    xi
    M

    b
    x x x
    Σ
    m=1
    xim X bm

    b1 b2 bM
    Interpretation of
    Logistic Regression
    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
    xi1 xi2 xiM
    add up the M products
    xi
    M

    b
    x x x
    Σ
    m=1
    xim X bm

    b1 b2 bM

    Compact Notation: xi ⊙b
    Interpretation of
    Logistic Regression
    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
    xi1 xi2 xiM
    add up the M products
    xi
    M

    b
    x x x
    Σ
    m=1
    xim X bm

    b1 b2 bM

    Compact Notation: xi ⊙b

    inner product
    Interpretation of Model

    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0


    = b0 + xi ⊙ b
    – 0.2

    – 0.1

    – 0.0
    examples of class y = 0
    – –0.1

    – –0.2

    filter
    examples of class y = 1
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Interpretation of Model

    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0


    = b0 + xi ⊙ b
    – 0.2

    – 0.1

    – 0.0
    examples of class y = 0
    – –0.1

    – –0.2

    filter
    examples of class y = 1
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Interpretation of Model

    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0


    = b0 + xi ⊙ b
    – 0.2

    – 0.1

    – 0.0
    examples of class y = 0
    – –0.1

    – –0.2

    filter b
    examples of class y = 1
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Interpretation of Model

    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0


    Outcome of y
    = b0 + xi ⊙ b
    Strong match between filter
    – 0.2
    and data indicates p(yi = 1|xi)
    is high
    – 0.1
    Low match between filter
    – 0.0 and data indicates p(yi = 1|xi)
    examples of class y = 0 is low
    – –0.1

    – –0.2

    filter b
    examples of class y = 1
    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0
    Interpretation of
    Logistic Regression
    zi = (b1 x xi1) + (b2 x xi2) + + (bM x xiM) + b0
    Sigmoid Function p(yi = 1|xi) = σ(zi)
    1
    – 0.2

    – 0.1

    – 0.0
    0.5
    – –0.1

    – –0.2

    –6
    0 zi filter b
    –4 –2 0 2 4 6

    Data Source: MNIST Dataset by LeCun et al. (1999) / CC-by-SA 3.0


    Credits
    MNIST Dataset of Handwritten Digits (Images)
    Yann LeCun (Courant Institue, NYU) and Corinna Cortes (Google Labs, New York) CC-by-SA 3.0
    http://yann.lecun.com/exdb/mnist/

    You might also like