Department of Computer Science

School of Science and Engineering

Lahore University of Management Sciences

CS436 / CS5310: Computer Vision Fundamentals

Summer Semester 2016-17

Course Team
Mon Tue Wed Thu
Instructor Sohaib Ahmad Khan, Ph.D.
Room No. TBA 10:00 am
Office Hours MoTuWe 12:00 pm – 1:00 pm 11:00 am
Email sohaib@lums.edu.pk
Telephone TBA 12:00 pm Office Office Office
Hours Hours Hours
Secretary Zulfiqar Nasim Malik x8313 1:00 pm
Course URL On LMS
2:00 pm Lecture Lecture Lecture Lecture

Course Basics 3:00 pm Quiz

Tutorial Tutorial
Credit Hours 3
4:00 pm
Course Status Elective for CS Undergrad
and Graduate Program
Lectures MoTuWeTh 2:00pm – 3:15pm
Tutorial TuWe 3:30 – 4:30 pm

Course Description
Computer Vision is one of the fastest growing fields of computer science. Computer Vision applications have gained
significant maturity in the past five years and are rapidly transitioning from research labs to real-world applications. This
course aims to give a flavor of this transition. What is behind the revolution of self-driving cars or the recent Google Lens
announcement? This course will explore the technology driving these exciting applications.
The course has been re-designed from scratch in this offering, primarily with an industry-readiness perspective. We aim
to equip students with hot practical skills as well as a sound theoretical grounding to design and implement computer
vision solutions at the cutting edge. Hence, the course is designed with an application perspective – students will be
expected to implement several techniques discussed in the lectures.
The course is broadly divided into two broad themes – that of Geometric Computer Vision, consisting of an
understanding of camera geometry, and Deep Learning, which attempts to give a flavor of the revolution in object
recognition that is driving the most exciting applications in the industry.

Course Prerequisites
Enrollment open for all SBASSE students who have fulfilled both of the following pre-requisites:
CS200 Introduction to Programming, Math120 Linear Algebra
Students should be familiar with:
• Basic Calculus: Being able to take derivative of a function, maximize a function by taking derivative, and
application of chain-rule.
• Linear Algebra: Be comfortable with matrix operations such as multiplication, transpose, inverse, and basic
algebra with matrix expressions. Vector spaces, basis vectors, rank and null space of a matrix are important
concepts for this course. Transformations will be used extensively to, for example, apply rotations or scaling to
points. We will deal with matrix decompositions such as Singular Value Decomposition, and Eigen
Decomposition. However, there will be tutorials on these concepts during the course, so if you can brush up
your concepts quickly, you will be okay.
• Good programming skills. We will be using MATLAB and Caffe for the assignments.
 Department of Computer Science
School of Science and Engineering
Lahore University of Management Sciences

Course Objectives
1. Acquire a broad overview of the theory and applications of Computer Vision.
2. Understand and appreciate the difficulty of developing computer vision software and its reasons.
3. Acquire an understanding of basic computer vision techniques and mathematical models in the area of camera
geometry, feature detection and robust estimation.
4. Acquire an understanding of state-of-the-art image recognition techniques, in particular, the application of
convolutional neural networks for supervised learning.
5. Develop skills to write (and debug) programs to solve computer vision problems, through solving several
programming assignments.
6. Acquire preparation to take on industry jobs in the field

Learning Outcomes
1. Explain some successful applications of computer vision algorithms and how they work.
2. Understand the mathematical basis of camera and scene geometry and apply your understanding to write
programs.
3. Write programs to solve basic computer vision problems using MATLAB.
4. Understand how Convolutional Neural Networks work.
5. Be familiar with the revolutionary impact of Deep Learning on image recognition problems.
6. Setup and train a transfer-learning problem using Caffe.
7. Be poised to undertake further study and research in this area.
8. Be poised to contribute to industrial work in this field.

Textbooks / Supplementary Material

[RS] Richard Szeliski, “Computer Vision: Algorithms and Applications”, Springer 2010
Available for free at http://szeliski.org/Book/
This is an excellent text of classical computer vision, but does not include coverage of Deep
Learning

[GYC] Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”, MIT Press 2016
This is the most comprehensive and latest text on Deep Learning, and particularly discusses
applications in computer vision.

[HZ] Richard Hartley and Andrew Zisserman, “Multiple View Geometry in Computer Vision”,
2nd Ed, Cambridge University Press, 2004
This is a authoritative, albeit, more mathematically advanced, text on geometric aspects of
computer vision.
 Department of Computer Science
School of Science and Engineering
Lahore University of Management Sciences

Grading Breakup
Assignments 50%
Quiz 10%
Class Participation & Attendance 10%
Final Exam 30%

Policies
Late Days • There are a fixed number of 5 late days that can be used throughout the semester for
homework submission.
• A single homework cannot utilize more than 3 late days.
• Homework will not be accepted after 3 days of the deadline.
• Clearly mark the number of late days that you have utilized for your assignment.
• If you have utilized all your late days, each day that the homework is late will carry a
penalty of 25%.
• Late days cannot be utilized for the last homework.
Quiz Drop Policy Your worst quiz will be dropped from the evaluation of the final grade.
Policies for Co- The course outline for the graduate and undergraduate course is the same. The lectures will
Located Graduate take place jointly. However, the undergraduate students will have less homework deliverables
Class The “extra credit” deliverables for the undergraduate students will be required for the graduate
students.
Honor Code We expect all LUMS students to act honestly at all times. Cheating and plagiarism will not be
tolerated. Resorting to unfair means may result in an F in the course and can result in
expulsion from LUMS, even for first-time offenders.
Some examples of what constitutes cheating and plagiarism are listed below, though this is not
an exhaustive list:
• Copying code from someone else’s screen or notes.
• Providing your own written code to another student.
• Letting another student type solutions for you on the machine you are working on, for
assignment submission.
• Looking at someone else’s answer sheet during a quiz or an exam.
• Downloading code from the internet or any other source and submitting portions of it
as your own.
• Writing code, whether on a computer or a sheet of paper, for someone else’s
assignment and providing it to him or her.
• Emailing your own code to someone else.
• Debugging someone else’s code.
Some examples of what does not constitute cheating and plagiarism in this course.
• Answering questions on the web forum.
• Discussing project ideas with another student.
• Helping other students understand concepts.
If in doubt, discuss with your instructor.
 Department of Computer Science
School of Science and Engineering
Lahore University of Management Sciences

Supplementary
Wk Lec Date Module Material Activity
Readings
1 1 Mon, 05-Jun-17 Geometric Primitives Introduction to the course, Digital Images, 2D [RS] 2.1.1 HW1 Assigned
and 2D points, homogeneous representation, ideal [HZ] 2.1
Transformations points, 2D lines, line at infinity,
2 Tue, 06-Jun-17 basic 2D transformations, Projective group [RS] 2.1.2 Linear Algebra
[HZ] 2.2, 2.3, 2.4 Primer
3 Wed, 07-Jun-17 Concatenation of transformations, [RS] A.1, A.2 HW Review
Decomposition of transformations, Estimation [HZ] 4.1
of transformation from point correspondences
4 Thu, 08-Jun-17 Points and planes in 3D, Transformations in [RS] 2.1.3 Quiz 1,
3D, Rotations in 3D [HZ] 3.2.1
2 5 Mon, 12-Jun-17 Camera Model Camera model, Setting up a camera model [RS] 2.1.5 HW1 Due
from parameters [HZ] 6.1 HW2 Assigned
6 Tue, 13-Jun-17 Camera looking at a plane, Relationship of [HZ] 8.4 Tutorial
plane and horizon line, Rotation about
camera center
7 Wed, 14-Jun-17 Camera Calibration [RS] 6.2, 6.3 HW Review

8 Thu, 15-Jun-17 Camera properties from Camera Matrix [HZ] 6.2 Quiz 2

3 9 Mon, 19-Jun-17 Convolution, Feature Convolution, Edge Detection, [RS] 4.2 HW2 Due
Detection, Matching, HW3 Assigned
10 Tue, 20-Jun-17 Model Fitting, Least Square fitting, RANSAC for line fitting [RS] 4.3 Tutorial
RANSAC
11 Wed, 21-Jun-17 Corner Features, KLT [RS] 4.1 HW Review

12 Thu, 22-Jun-17 SIFT [RS] 4.1 Quiz 3

Mon, 26-Jun-17
Eid Break
to Fri, 30-Jun-
17
4 13 Mon, 03-Jul-17 Object Recognition, Supervised Learning: Data Driven Approach [GYC] 5.1 HW3 Due
Supervised Learning, to Learning, K Nearest Neighbor HW4 Assigned
14 Tue, 04-Jul-17 Linear Classifiers Linear Classification Tutorial

15 Wed, 05-Jul-17 Loss Functions, Gradient Descent HW Review

16 Thu, 06-Jul-17 HoG features + SVM example Quiz 4

5 17 Mon, 10-Jul-17 Neural Networks, Perceptron [GYC] 6.1 HW4 Due

Back Propagation HW5 Assigned
18 Tue, 11-Jul-17 Multi-layer Perceptron Tutorial

19 Wed, 12-Jul-17 Back propagation [GYC] 6.5 HW Review

20 Thu, 13-Jul-17 Training Considerations, Results Quiz 5

6 21 Mon, 17-Jul-17 Convolutional Neural Convolution and Pooling [GYC] 9.1-9.4 HW5 Due
Networks HW6 Assigned
22 Tue, 18-Jul-17 CNN Architectures Tutorial

23 Wed, 19-Jul-17 Transfer Learning HW Review

24 Thu, 20-Jul-17 Object Detection and Localization Quiz 6

7 25 Mon, 24-Jul-17 Stereo*, Optical Stereo, Epipolar Constraint [RS] 11.1 HW6 Due
Flow*, Tracking*
26 Tue, 25-Jul-17 Fundamental Matrix Estimation [RS] 11.1

27 Wed, 26-Jul-17 Brightness constancy equation. LK Least [RS] 8.4

Squares solution
28 Thu, 27-Jul-17 TLD Tracker

* Tentative topics, based on time availability