CS-871

Machine Learning

Lecture 1
Instructor: Dr Hashir Kiani
 2

General Conduct

• Be respectful of others
• Only speak at your turn and preferably raise your hand if you want to say
something
• Do not cut off others when they are talking
• Join the class on time
 3

About me

• Bachelors in Electrical Engineering from MCS, NUST (2011)

• Masters in Communications Engineering and Networks from University of
Birmingham, UK (2013)
• PhD in Computer Science from University of Manchester, UK (2020)
• Research interests are applications of Machine Learning and Deep Learning in
different domains including energy, health and climate change.
• Contact Information:
• Email : hashir.moheed@seecs.edu.pk
• Office: C-302, Second Floor, IAEC
 Assessment
Weightage (%)
Assignments 5
Quizzes 10
Project 15
Mid Semester Exam 25
End Semester Exam 45

• Approximately three hours of lectures every week.

• Lecture slides will be shared on LMS. (Enrolment Code: 658124370)
• Assignments given and submitted on LMS. Late submission penalty of 10% per late day.
• Your writings must be your own thoughts.
• Cheating and plagiarism will not be tolerated and will lead to strict penalties.
 5

Assignments

• Assignment 1: Week 3
• Given on: 25th September 2023
• Due Date: 8th October 2023

• Assignment 2: Week 5
• Given on: 9th October 2023
• Due Date: 22nd October 2023

• Assignment 3: Week 7
• Given on: 23rd October 2023
• Due Date: 5th November 2023
 6

Quizzes

• Quiz 1: 25th September 2023 (Week 3)

• Quiz 2: 11th October 2023 (Week 5)
• Quiz 3: 25th October 2023 (Week 7)
• Quiz 4: 29th November 2023 (Week 12)
• Quiz 5: 20th December 2023 (Week 15)
 7

Project Timeline

• Project will be given on 23rd October 2023.

• You must get the Project proposal approved by 5th November 2023.

• Project Report must be submitted by 31st December 2023.

• Project Presentations will be taken during the last week of course starting
from 1st January 2024.
 8

Mid and Final Exam

• Mid Semester Exam: Week 9 starting from 6th November 2023.

• End Semester Exam: Week 18 starting from 8th January 2024.

• Class Attendance: Minimum 75%

 Course Objectives

• Implement and analyze existing learning algorithms for classification,

regression, clustering, and representation learning.
• Integrate multiple facets of practical machine learning in a single system:
data preprocessing, learning, regularization, and model selection.
• Describe the formal properties of models and algorithms for learning and
explain the practical implications of those results.
• Design experiments to evaluate and compare different machine learning
techniques on real-world problems.
 Course Prerequisites

• Linear Algebra
• Probability
• Calculus
• Programming
 Course Contents

• Linear Regression • Model Selection and Experimental

• Gradient Descent Design

• Logistic Regression • Feature Engineering

• Decision Trees
• K Nearest Neighbours
• Support Vector Machines
• Naïve Bayes
• Bayesian Networks
• Regularization
• PCA and LDA
• Perceptron and Backpropagation
• Convolutional Neural Networks
• Advanced ML Topics
 Recommended Books

• Textbook:
• RS Michalski, JG Carbonell, TM Mitchell, “Machine Learning: An Artificial Intelligence
Approach”, Springer, 2014
• Christopher M. Bishop, “Pattern Recognition and Machine Learning”, 2007
• Reference Books/Materials:
• Richard Duda, Peter Hart and David Stork, Pattern Classification, 2nd ed. John
Wiley & Sons, 2001.
• Richard Sutton and Andrew Barto, Reinforcement Learning: An introduction. MIT
Press, 1998
• http://cs229.stanford.edu
 13

History of ML

• Alan Turing’s 1950 paper “Computing Machines and Intelligence”

• “Can machines think?”
• Turing Test
• Interrogator passes written questions to a human and machine
• Based on the answers, interrogator must differentiate between the two
• Turing discusses two possible approaches to machine intelligence
• Symbolic AI: Solve problems like chess
• Neural AI: Teach like a child (Machine Learning)
 14

Ideas from outside ML

• 1801: Linear Regression

• 1936: Linear Classification

• 1957: Markov decision processes

 15

Statistical Machine Learning

• 1985: Bayesian Networks

• 1995: Support Vector Machines

 16

History of Neural ML

• 1943: Artificial Neural Networks: Mathematical representation

• 1949: Learning rule: cells that fire together wire together
• 1958: Perceptron algorithm for linear classifiers
• 1959: ADALINE device for linear regression
• 1960s: Single layered neural nets not that powerful, could not
do XOR etc
 17

Revival of Neural ML

• 1980: Convolutional neural networks for images

• 1986: Backpropagation for training multi-layer networks

• 1989: Convolutional neural networks were used to recognize handwritten

digits
 18

Era of Deep Learning

• 2006: Training of deep networks was developed (deep learning coined)

• 2012: AlexNet obtained huge gains in object recognition

• 2016: AlphaGo used deep reinforcement learning to defeat world

champion in Go
 19

ML Applications

• ML has moved from the lab to the real world

 20

ML Applications

• Machine translation has improved a lot and has started helping people in
terms of connecting on a global level
• Autonomous driving has the potential to reduce road accident and
congestion.
 21

ML Applications

• ML in healthcare:
• Interpreting chest x rays

• Diagnosing diabetic retinopathy

• Response of covid19 infected cells to certain drugs

 22

ML Applications

• Large Language Models

 23

ML Risks

• Privacy Concerns
• Might lead to violation of privacy
• Security risks
• Bias problem
• Fairness problem
• Creation of fake content
• Energy Consumption
 Definition of Machine Learning

• Machine Learning is the field of study that

gives the computer the ability to learn
without being explicitly programmed - Arthur
Samuel (1959)

• A computer program is said to learn from

experience E with respect to some class of
tasks T and performance measure P, if its
performance at tasks in T, as measured by P,
improves with experience E - Tom Mitchell
(1998)
 25

Machine Learning

• Uses historical data to find parameters of a model.

• Model can then be used to make future predictions.
• Can be used to estimate parameters of any type of model.
Taxonomy of Machine Learning
Supervised Learning
 Housing Price Prediction

• Given: A dataset of n samples

• Task: If a house has x square feet, predict its price.
 Housing Price Prediction

• Given: A dataset of n samples

• Task: If a house has x square feet, predict its price.
• Can have more features
 Regression vs Classification

• Regression: Prediction of a continuous variable.

• Classification: Prediction of a discrete variable.

Image Classification
Object Localization and Detection
Machine Translation
Unsupervised Learning
 Unsupervised Learning

• Dataset contains no labels

• Task: To find structure in the data
Clustering
Clustering Genes
Reinforcement Learning
Reinforcement Learning
 AlphaGo

• AlphaGo is a computer program that plays the board game Go.

• Succeeded by an even more powerful version known as AlphaGo Zero,
which was completely self-taught without learning from human games.
• Succeeded by a program known as MuZero which learns without being
taught the rules.
• Use reinforcement learning
Questions?