Institute of Business Administration, Karachi

ECO-580: Machine Learning Methods

(with Application Economics and Finance)
Spring 2023
Instructor Dr. Wali Ullah
Office (Room No.) 205, Fauji Foundation Building, Main Campus
Office Hours Monday and Thursday: 3:30pm-5:30pm (or by appointment)
Email waliullah@iba.edu.pk
Telephone Ext. 2642

COURSE BASICS
Credit Hours 3
Lecture(s) No. of Lec(s) per Week 1 Duration 120 minutes
Recitation/Lab (per week) No. of Lec(s) per Week 1 Duration 60 minutes

COURSE DESCRIPTION
Machine learning (ML) studies the design and development of algorithms that learn from the data
and improve their performance through experience. ML refers to a set of methods and that help
computers to learn, optimize and adapt on their own. ML has been employed to devise algorithms for
diverse applications including object detection or identification in computer vision, sentiment analysis
of speaker or writer, detection of disease and planning of therapy in healthcare, product
recommendation in e-commerce, learning strategies for playing games, recommending movies to
customers, speech recognition systems, fraudulent transaction detection or loan application approval
in banking sector, to name a few.
This course provides a thorough introduction to the theoretical foundations and practical applications
of ML. We will learn fundamental algorithms in supervised learning and unsupervised learning. We
will not only learn how to use ML methods and algorithms but will also try to explain the underlying
theory building on mathematical foundations. While reviewing the several problems and algorithms
to carry out classification, regression, clustering, dimensionality reduction, we will focus on the core
fundamentals which unify all the algorithms. The theory discussed in class will be tested in
assignments, quizzes and exams.
Prerequisites
Machine Learning is a mathematical discipline, and students will benefit from a good background in
probability, linear algebra, and calculus.
PROGRAMMING LANGUAGE
We will use Python language in this course. Python is the ideal coding language used for machine
learning. Python can be used even if you are new to ML since it is flexible and comes with pre-existing
libraries like Pandas, SciPy, and nltk. Python language is lauded for its simple syntax and minimal
codes.

Learning outcomes
On completion of the course students will be expected to:
 Have a good understanding of the fundamental issues and challenges of machine learning:

-1-
 data, model selection, model complexity, etc.
 Have an understanding of the strengths and weaknesses of many popular machine learning
approaches.
 Appreciate the underlying mathematical relationships within and across Machine Learning
algorithms and the paradigms of supervised learning.
 Be able to design and implement various machine learning algorithms in a range of real-world
applications.

GRADING BREAKUP AND POLICY

 Programming Assignments and HomeWorks, 25 %
 Project, 15 %
 Quizzes, 10 %
 Two Hourly Exams, 50%

CORE TEXTBOOKS
 (CB) Pattern Recognition and Machine Learning, Christopher M. Bishop
 (KM) Machine Learning: a Probabilistic Perspective, Kevin Murphy
 (TM) Machine Learning, Tom Mitchell
 Class Lecture Notes

Lecture Topics Required Readings

Lecture-1  Course Introduction and Machine Learning Lecture Slides
Overview (Class Lecture Notes)
 Supervised Learning: Formulation, Setup,
Train-test split, Generalization
Lecture-2  k-Nearest Neighbor (kNN) Algorithm, Lecture Slides
Algorithm Formulation (Class Lecture Notes)
 Distance Metrics, Choice of k, Algorithm
Convergence, Storage, Time Complexity
Analysis, Fast kNN
Lecture-3  The Curse of Dimensionality and Connection Lecture Slides
with kNN (Class Lecture Notes)
 Dimensionality Reduction: Feature Selection
and Extraction, Principal Component
Analysis
Lecture-4  Classifier Performance Evaluation Lecture Slides
 Confusion Matrix Sensitivity, Specificity, (Class Lecture Notes)
Precision Trade-offs, ROC, AUC, F1-Score and
Matthew’s Correlation Coefficient
Lecture-5  Multi-class Classification, Evaluation, Micro, Bishop, Chapter 1.0-1.1; 3.1
Macro Averaging lecture2 (handout)
 Regression: Linear Regression, Polynomial Lecture Slides
Regression, Overfitting (Class Lecture Notes)
Lecture-6  Gradient Descent Algorithm Lecture Slides

-2-
  Regularization (Class Lecture Notes)
Lecture-7  Probability Review and Bayesian Learning Lecture Slides
Framework, MAP and ML Hypothesis (Class Lecture Notes)
 Linear Regression as ML estimation
 Naive Bayes Classifier
Lecture-8  Naïve Bayes Classifier Lecture Slides
 Bayesian Networks Introduction (Class Lecture Notes)
Lecture-9  Logistic Regression: Mathematical Model, Lecture Slides
Decision Boundaries, Loss/Cost Function, (Class Lecture Notes)
Gradient Descent
 Multi-class Logistic Regression
Lecture-10  Decision Tree and Random Forest Lecture Slides
 Perceptron and Perceptron Classifier, (Class Lecture Notes)
Perceptron Learning Algorithm and its
Geometric Intuition
 Perceptron Learning Algorithm Convergence
Lecture-11  SVM Overview Lecture Slides
 Hard SVM, (Class Lecture Notes)
 Soft SVM,
 Kernel Trick
Lecture-12  Neural Networks Introduction, Model, Lecture Slides
Forward Pass (Class Lecture Notes)
Lecture-13  Neural Networks: Back Propagation Lecture Slides
(Class Lecture Notes)
Lecture-14  Unsupervised Learning, Clustering Overview Lecture Slides
 K-means Clustering (Class Lecture Notes)
 Agglomerative Clustering

-3-