Scribd Logo
Upload

Welcome to Scribd!

  • 26 views

    Machine Learning Tutorial

    Uploaded by

    Daniel T. Dinka
    This document provides an overview of machine learning techniques including regression, classification, and ensembling methods. It discusses linear regression, logistic regression, nearest neighbor classification, support vector machines, bagging, boosting, and random forests. Evaluation metrics like confusion matrices, ROC curves, PR curves, and F1 scores are also covered. Key aspects of machine learning like model training, feature selection, and example importance are outlined.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content

    You might also like

    Machine Learning Tutorial

    Uploaded by

    Daniel T. Dinka
    26 views20 pages
    This document provides an overview of machine learning techniques including regression, classification, and ensembling methods. It discusses linear regression, logistic regression, nearest neighbor classification, support vector machines, bagging, boosting, and random forests. Evaluation metrics like confusion matrices, ROC curves, PR curves, and F1 scores are also covered. Key aspects of machine learning like model training, feature selection, and example importance are outlined.

    Original Description:

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document provides an overview of machine learning techniques including regression, classification, and ensembling methods. It discusses linear regression, logistic regression, nearest neighbor classification, support vector machines, bagging, boosting, and random forests. Evaluation metrics like confusion matrices, ROC curves, PR curves, and F1 scores are also covered. Key aspects of machine learning like model training, feature selection, and example importance are outlined.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    26 views20 pages

    Machine Learning Tutorial

    Uploaded by

    Daniel T. Dinka
    This document provides an overview of machine learning techniques including regression, classification, and ensembling methods. It discusses linear regression, logistic regression, nearest neighbor classification, support vector machines, bagging, boosting, and random forests. Evaluation metrics like confusion matrices, ROC curves, PR curves, and F1 scores are also covered. Key aspects of machine learning like model training, feature selection, and example importance are outlined.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 20

    MACHINE LEARNING

    An Overview

    deeplearning4j.org
    MACHINE
    deeplearning4j.org

    input weights output

    X f(X,β) f(X)≡Y
    LEARNING
    deeplearning4j.org

    N features or dimensions

    M examples or observations

    X= Y=

    Xi= Ŷ=?
    OUTLINE
    deeplearning4j.org

    • Prediction/Regression (temperature|day)
    • Linear Regression
    • Classification (cold/hot|day)
    • Logistic Regression
    • Nearest Neighbor Classifier • Evaluation Metrics
    • Support Vector Machine • Confusion Matrix
    • Ensembling • Cheatsheet
    • Bagging • ROC Curve
    • Boosting • PR Curve
    • Random Forest • F1 Score
    deeplearning4j.org

    REGRESSION

    XMN= YM=
    Linear Regression deeplearning4j.org

    • A solution if found in one step • Non-zero mean distributions • Linear problem


    • Multiple performance metrics
    • Mean Quadratic error • Extra columns/dimensions
    • R squared • Categorical inputs
    • …
    deeplearning4j.org

    CLASSIFICATION

    XMN= YM=
    Logistic Regression (classifier) deeplearning4j.org

    Logit - odds

    • Probabilistic model • Un-biased distributions • Observations are supposed to fit


    • Solution in a few steps to the non-linearity curve
    • Extra columns/dimensions • Prone to underfitting
    • Categorical inputs
    Nearest Neighbor Classifier deeplearning4j.org

    • Simple model • Confidence value • M too big • Generalization


    • No training • Outlayers • sub-sampling • Exhaustive search
    • k-NN • prototypes • O(M)
    • Skewness • N too big • Ball tree
    • w-k-NN • dimensionality • O(log(M))
    reduction
    Support Vector Machine (SVM) deeplearning4j.org

    • New example O(k) • Outlayers • Training O(M) for linear


    • # of Support Vectors • Training O(M2) for non-linear
    • Non-separable • Separable?
    • Kernel trick
    deeplearning4j.org

    ENSEMBLING
    Q classifiers
    hq(X)
    Bagging deeplearning4j.org

    (Bootstrapping)

    • Improves overfitting
    • Simple to implement
    • Alternative to cross-validation

    • Error correlation X
    • picking examples
    • weighted average Sample with repacement
    • … 63% unique examples

    • Complexity O(Q)
    • Brute force
    Boosting (AdaBoost) deeplearning4j.org

    • Fast with dummy classifiers • Choice of β • Generalization capability?


    • Ensemble vastly improves
    dummies
    • Example importance update
    Random Forests deeplearning4j.org

    Combining trees into a forest


    • Random X
    • Random feature selection

    • New example O(Q·log(M)) • Well suited for the cloud • Training O(Q·M·log(M))
    • Feature selection • Access to a portion of • Generalization?
    • the database
    • each example
    • Output weighted average
    deeplearning4j.org

    EVALUATION METRICS
    (classifiers)
    Confusion Matrix deeplearning4j.org
    Cheatsheet deeplearning4j.org
    ROC Curve deeplearning4j.org
    PR Curve deeplearning4j.org

    ‘usefulness’ ‘completeness’
    F1 Score deeplearning4j.org

    (harmonic mean) β=2 recall precision

    β=0.5 recall precision

    You might also like