MACHINE LEARNING MODELS

LECTURE 8
Machine learning models are algorithms
or mathematical frameworks that enable
computers to learn patterns and make
predictions or decisions without being
explicitly programmed.
These models learn from example data,
called training data, and generalize that
knowledge to make predictions on new,
unseen data.
 THERE ARE VARIOUS TYPES OF MACHINE LEARNING MODELS, EACH
WITH ITS OWN CHARACTERISTICS AND APPLICATIONS:
• Linear Regression
Linear regression is a simple and widely used supervised learning model for regression tasks. It models the relationship
between independent variables and a continuous dependent variable, fitting a linear equation to the data.
• Logistic Regression
Logistic regression is a supervised learning model used for binary classification tasks. It models the probability of a binary
outcome based on input variables using a logistic function.
• Decision Trees
Decision trees are versatile supervised learning models that learn decision rules from the data. They divide the data into
branches based on feature values, creating a tree-like structure for making predictions or decisions.
• Random Forest
Random forest is an ensemble learning method that combines multiple decision trees to make more accurate predictions. It
uses bootstrapping and feature randomization to create a diverse set of trees.
• Support Vector Machines (SVM)
SVM is a powerful supervised learning model that can be used for both classification and regression tasks. It finds an
optimal Neural networks hyperplane that separates or predicts data points with maximum margin.
• Neural Networks
are complex models inspired by the structure and functioning of the human brain. They consist of interconnected nodes, or
artificial neurons, organized in layers. Deep neural networks, known as deep learning models, have multiple hidden layers
and can learn intricate patterns from large-scale data.
• Naive Bayes
Naive Bayes is a probabilistic classification model based on Bayes' theorem. It assumes that the features are conditionally
independent, making it efficient and effective for text classification and other applications.
• K-Nearest Neighbors (KNN)
KNN is a simple and intuitive model for both classification and regression tasks. It classifies or predicts new data points
based on the similarity to their k-nearest neighbors in the training data.
• Gradient Boosting Models
Gradient boosting models, such as XGBoost and LightGBM, are ensemble learning methods that combine weak prediction
models in a sequential manner. They iteratively train new models that focus on the errors of previous models, leading to
highly accurate predictions.
 MACHINE LEARNING
ALGORITHMS

Machine learning algorithms are computational methods or

mathematical models that enable machines to automatically learn
from data and improve their performance on a given task. These
algorithms are the core components of machine learning, a subset of
artificial intelligence (AI) that focuses on training systems to learn
patterns and make predictions or decisions without explicit
programming.
SUPERVISED LEARNING ALGORITHMS

SUPERVISED LEARNING ALGORITHMS LEARN FROM LABELED TRAINING DATA,

WHERE EACH EXAMPLE IS ASSOCIATED WITH A CORRESPONDING TARGET OR
OUTPUT VALUE. THE ALGORITHM LEARNS TO MAP INPUT FEATURES TO THE
CORRECT OUTPUT BASED ON THE PROVIDED LABELS.

Support Vector Machines

Linear Regression: Fits a
Logistic Regression: Models (SVM): Finds an optimal
linear relationship between
the probability of a binary hyperplane that separates
input features and
or categorical outcome. data points of different
continuous output values.
classes.

Neural Networks: Deep

Decision Trees: Learn
decision rules to predict the
target variable based on
feature values.
Random Forest: Ensemble
learning models composed
model that combines
of interconnected nodes
multiple decision trees to
(neurons) organized in
improve accuracy.
layers.
UNSUPERVISED LEARNING ALGORITHMS: UNSUPERVISED LEARNING
ALGORITHMS DEAL WITH UNLABELED DATA, WHERE THE GOAL IS TO DISCOVER
PATTERNS OR STRUCTURES WITHIN THE DATA WITHOUT ANY PREDEFINED
TARGET VARIABLES. THESE ALGORITHMS FIND INHERENT RELATIONSHIPS OR
CLUSTERS IN THE DATA.

K-Means Clustering: Divides
data points into k clusters
based on similarity.
Hierarchical Clustering: Creates
a hierarchy of clusters by
iteratively merging or splitting
them.
Principal Component Analysis
(PCA): Reduces the
dimensionality of data by
extracting important features.
Gaussian Mixture Models
(GMM): Represents the
probability distribution of a
dataset as a mixture of
Gaussian distributions.
Association Rule Learning:
Identifies interesting
relationships or associations
between variables in a dataset.
REINFORCEMENT
LEARNING
ALGORITHMS:
REINFORCEMENT Deep Q-Networks (DQN):
Q-Learning: Learns an
Combines deep neural
LEARNING ALGORITHMS optimal policy by estimating
networks with Q-Learning to
LEARN THROUGH the values of state-action
handle high-dimensional
INTERACTIONS WITH AN pairs.
state spaces.
ENVIRONMENT,
RECEIVING FEEDBACK
IN THE FORM OF
Actor-Critic Methods: Utilize
REWARDS OR Policy Gradient Methods:
both an actor network to
PENALTIES. THE Optimize the policy directly
select actions and a critic
ALGORITHMS AIM TO by estimating gradients and
network to estimate the
MAXIMIZE CUMULATIVE updating policy parameters.
value function.
REWARDS BY LEARNING
OPTIMAL ACTIONS.
• It's important to note that these categories are not mutually
exclusive, and some algorithms may exhibit characteristics of
multiple types. Additionally, within each category, there are
numerous specific algorithms and variations, each suited to
different types of problems and data.
• Selecting an appropriate machine learning algorithm depends
on factors such as the nature of the data, the problem type
(classification, regression, clustering, etc.), the available
resources, and the desired outcome. Experimentation,
evaluation, and validation of the algorithms are key to finding
the best fit for a given task.

Preprocess and
MODEL TRAINING,
EVALUATION, AND
INTERPRETATION ARE
CRUCIAL STEPS
IN THE MACHINE LEARNING
WORKFLOW
MODEL TRAINING INVOLVES
THE PROCESS OF FITTING A
MACHINE LEARNING MODEL
TO THE TRAINING DATA. THE
GOAL IS TO LET THE MODEL
LEARN THE UNDERLYING
PATTERNS AND
RELATIONSHIPS BETWEEN
THE INPUT FEATURES AND
THE TARGET VARIABLE.
Iterate and refine
Data Preparation: Preprocess and prepare the data by handling missing
values, encoding categorical variables, and scaling features, among other
prepare tasks.
Splitting Data: Divide the dataset into training and validation sets. The
Splitting training set is used to train the model, while the validation set helps
assess its performance during training.
Model
Model Selection: Choose an appropriate model based on the problem
type, data characteristics, and requirements.
Training: Fit the model to the training data by optimizing its parameters or
Training weights using a specific algorithm or optimization technique.
Hyperparameter Tuning: Adjust the hyperparameters of the model, such
Adjust as learning rate, regularization strength, or number of hidden layers, to
improve performance.
Iterative Refinement: Iterate and refine the training process, adjusting
parameters or trying different models until satisfactory performance is
achieved.
MODEL EVALUATION IS
THE PROCESS OF Metrics: Calculate evaluation metrics such as accuracy, precision,
recall, F1 score, or mean squared error, depending on the
ASSESSING THE problem type.
PERFORMANCE OF THE
TRAINED MODEL ON
Cross-Validation: Perform cross-validation to estimate the model's
UNSEEN DATA. THE GOAL performance by splitting the data into multiple folds and
IS TO UNDERSTAND HOW evaluating it on different subsets.
WELL THE MODEL
GENERALIZES AND Confusion Matrix: Construct a confusion matrix to analyze the
MAKES PREDICTIONS ON model's predictions and identify the number of true positives,
NEW INSTANCES. true negatives, false positives, and false negatives.
EVALUATION HELPS
DETERMINE IF THE ROC Curve and AUC: Plot the receiver operating characteristic
MODEL MEETS THE (ROC) curve and calculate the area under the curve (AUC) to
assess the model's performance for binary classification tasks.
DESIRED PERFORMANCE
CRITERIA AND PROVIDES
INSIGHTS INTO ITS Regression Metrics: For regression problems, evaluate metrics like
STRENGTHS AND mean absolute error (MAE), mean squared error (MSE), or R-
squared to measure the model's predictive accuracy.
WEAKNESSES.
MODEL Feature Importance: Determine the importance of input
INTERPRETATION features in the model's predictions. Techniques like feature
importance scores, permutation importance, or SHAP values
INVOLVES can provide insights into the impact of each feature on the
model's output.
UNDERSTANDING
AND EXTRACTING
Partial Dependence Plots: Visualize how the predicted outcome
INSIGHTS FROM THE changes with variations in a specific feature while holding other
TRAINED MODEL. IT features constant.

AIMS TO EXPLAIN
THE REASONING
Local Interpretability: Analyze and interpret individual
BEHIND THE MODEL'S predictions to understand the model's decision-making process
PREDICTIONS AND for specific instances.

PROVIDE INSIGHTS
INTO THE
Model Visualization: Use visualization techniques to visualize
IMPORTANCE OF complex models, such as decision trees or neural networks, to
FEATURES. gain insights into their structure and behavior.