AIN2001

Principles of Artificial Intelligence

Week-1
04.10.2022

Mustafa Ümit Öner

mustafaumit.oner@eng.bau.edu.tr

 x2

x1
0

2
 x2 x2

x1 x1
0 0

3
 x2 x2

x1 x1
0 0

4
 x2 x2

x1 x1
0 0

{1 if x1 < 0
0 if x1 ≥ 0
square (y = 0) y=
circle (y = 1)
Linear Classifier

5
6
7
8
About me
Bahcesehir University (Jan 2022)
Artificial Intelligence Engineering
National University of Singapore (Aug 2017 - Dec 2021)
Ph.D. in Computer Science
Thesis: Novel Multiple Instance Learning Models for Digital
Histopathology
Thesis Advisors: Hwee Kuan Lee and Wing-Kin Sung
Middle East Technical University (Sep 2013 - Sep 2016)
M.S. in Electrical and Electronics Engineering
Thesis: Metastasis Detection and Localization in Lymph Nodes by
Using Convolutional Neural Networks
Thesis Advisor: Uğur Halıcı
Middle East Technical University (Sep 2008 - Jun 2013)
B.S. in Electrical and Electronics Engineering
B.S. in Industrial Engineering - Double Major

My research
Genomics Transcriptomics Proteomics Pathology
Omics (DNA) (RNA) (Protein) (Cell)
analysis

• Digital Pathology • Omics

• Cellular level information • Cancer biology
• Tumor microenvironment • Personalized oncology

Genomics Transcriptomics Proteomics Pathology

(DNA) (RNA) (Protein) (Cell)

Machine learning-based information systems

Survival Recurrence Drug response

To develop machine learning-based systems for digital

histopathology and integrative multi-omics to support
diagnostic and therapeutic decision-making in cancer.

10

My research
Genomics Transcriptomics Proteomics Pathology
Omics (DNA) (RNA) (Protein) (Cell)
analysis

• Digital Pathology • Omics

• Cellular level information • Cancer biology
• Tumor microenvironment • Personalized oncology

Genomics Transcriptomics Proteomics Pathology

(DNA) (RNA) (Protein) (Cell)

Machine learning-based information systems

Survival Recurrence Drug response

Looking for self-motivated, and hard-working students

enthusiastic about pursuing a career in medical AI research!

11

About you
Tell a little about yourself

What do you expect to learn in this class?

Teach us one thing for today

12

Why is AI so popular?
Why is AI so popular?
• It has become a primary tool for many tasks in different domains
• computer vision • machine translation
• pattern recognition • bioinformatics
• natural language processing • game playing

• It has penetrated different industries

• healthcare • entertainment
• finance • social media
• self-driving cars • games

As a result, it has become a highly demanded skill

in both research and industry.

14

Deep Learning Examples - Cancer Diagnosis

Paige Prostate - FDA approval (Sep 2021)

https://www.fda.gov/news-events/press-announcements/ https://www.paige.ai/news/news-press-release1/ 15
fda-authorizes-software-can-help-identify-prostate-cancer https://www.nature.com/articles/s41591-019-0508-1

Deep Learning Examples - Object Detection

Facebook Detectron
https://research.facebook.com/downloads/detectron/ 16
https://github.com/facebookresearch/Detectron

Deep Learning Examples - NLP

• Neural machine translation - Google Translate
• https://arxiv.org/pdf/1609.08144.pdf

• Question answering - BERT

• https://arxiv.org/pdf/1810.04805.pdf

• Speech recognition - USR

• https://arxiv.org/pdf/2105.11084.pdf

17

Deep Learning Examples - Protein Folding

• Proteins are building blocks of life

• ~100M proteins, yet structure of only a tiny fraction known
• Expensive (~$100K) and time-consuming (years)
• “What took us months and years to do, AlphaFold was able to do in
a weekend.” - Prof. John McGeehan, Portsmouth University
https://www.nature.com/articles/s41586-021-03819-2 https://youtu.be/KpedmJdrTpY 18
https://deepmind.com/research/case-studies/alphafold https://deepmind.com/blog/article/AlphaFold-Using-AI-for-

Deep Learning Examples - Protein Folding

DeepMind AlphaFold

https://www.nature.com/articles/s41586-021-03819-2 https://youtu.be/KpedmJdrTpY 19
https://deepmind.com/research/case-studies/alphafold https://deepmind.com/blog/article/AlphaFold-Using-AI-for-

Deep Learning Examples - GANs

Refik Anadol - Machine Hallucinations

https://www.nvidia.com/en-us/research/ai-art-gallery/artists/ 20
refik-anadol/
https://youtu.be/6U-bI3On1Ww 21
What is (artificial) intelligence?
23
Closest Words in GloVe Embedding

https://www.cs.toronto.edu/~lczhang/360/lec/w05/w2v.html 24

 “Algorithms enabled by constraints exposed by

representations that support models targeted at thinking,
perception, and action [tied with loops and acting together]”

Patrick Henry Winston, MIT

25

Can machines think?

Turing’s Imitation Game

Aim:
A - player X
to fool C
C - interrogator

Aim: Aim: to determine which

B - player Y of the other two is the man
to help C and which is the woman.

Turing, A. M. (2009). Computing machinery and intelligence. Mind, 49, 433-460. 27

Turing’s Imitation Game

Aim:
A - player X
to fool C
C - interrogator

Aim: Aim: to determine which

B - player Y of the other two is the man
to help C and which is the woman.

C: Will X please tell me the length of his or her hair?

X: My hair is shingled, and the longest strands are about nine inches long.
Y: I am the woman, don't listen to him!

Turing, A. M. (2009). Computing machinery and intelligence. Mind, 49, 433-460. 28

Turing’s Imitation Game

Aim:
A - player X
to fool C
C - interrogator

Aim: Aim: to determine which

B - player Y of the other two is the man
to help C and which is the woman.

C: Will X please tell me the length of his or her hair?

X: My hair is shingled, and the longest strands are about nine inches long.
Y: I am the woman, don't listen to him!

"What will happen when a machine takes

the part of A in this game?"

Turing, A. M. (2009). Computing machinery and intelligence. Mind, 49, 433-460. 29

 https://images.newscientist.com/wp-content/uploads/2017/01/04171935/gettyimages-5153584581.jpg

Think about AlphaGo with this perspective!

Silver, David, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser et al. 30
"Mastering the game of Go with deep neural networks and tree search." nature 529, no. 7587 (2016): 484-489.
http://aima.cs.berkeley.edu/cover.jpg 31

 Alan Turing
Computing
machinery
Ada Lovelace and
The world’s first intelligence
computer
programmer

A statue of
Aristotle

http://aima.cs.berkeley.edu/cover.jpg 32

Deep Blue Garry Kasparov

defeated Garry
Kasparov, 1997
(Final position of
the match)

Alan Turing
Computing
machinery
Ada Lovelace and
The world’s first intelligence
computer
programmer

A statue of
Aristotle

http://aima.cs.berkeley.edu/cover.jpg 33

Deep Blue Garry Kasparov AlphaGo

defeated Garry defeated Lee
Kasparov, 1997 Sedol, 2016 (A
(Final position of pivotal
the match) position in the
Go match)

Alan Turing
Computing
machinery
Ada Lovelace and
The world’s first intelligence
computer
programmer

A statue of
Aristotle

http://aima.cs.berkeley.edu/cover.jpg 34

Deep Blue Garry Kasparov AlphaGo

defeated Garry defeated Lee
Kasparov, 1997 Sedol, 2016 (A
(Final position of pivotal
the match) position in the
Go match)
Atlas
humanoid
robot by
Boston
Dynamics
Alan Turing
Computing
machinery
Ada Lovelace and
The world’s first intelligence
computer
programmer

A statue of
Aristotle

http://aima.cs.berkeley.edu/cover.jpg 35

Deep Blue Garry Kasparov AlphaGo

defeated Garry defeated Lee
Kasparov, 1997 Sedol, 2016 (A
(Final position of pivotal
the match) position in the
Go match)
Atlas
humanoid
robot by
Boston
Dynamics
Alan Turing
Computing
machinery
Ada Lovelace and
The world’s first intelligence
computer
programmer
Self-driving
Mars cars
Exploration
Rovers
Spirit and A statue of
Opportunity Aristotle

A probabilistic model to detect nuclear

explosions from seismic signals

http://aima.cs.berkeley.edu/cover.jpg 36

NNs and Deep Learning History by Yann LeCun

Y. LeCun

Inspiration for Deep Learning: The Brain!

1943: McCulloch & Pitts, networks of binary neurons can do logic
1947: Donald Hebb, Hebbian synaptic plasticity
1948: Norbert Wiener, cybernetics, optimal filter,
feedback, autopoïesis, auto-organization.
1957: Frank Rosenblatt, Perceptron
1961: Bernie Widrow, Adaline
1962: Hubel & Wiesel, visual cortex architecture
1969: Minsky & Papert, limits of the Perceptron

https://drive.google.com/file/d/1vVNUye-1JNJnqP4A0704sjtF7gs_MpCI/edit 37
https://youtu.be/mTtDfKgLm54

NNs and Deep Learning History by Yann LeCun

Y. LeCun

More History
1970s: statistical patter recognition (Duda & Hart 1973)
1979: Kunihiko Fukushima, Neocognitron
1982: Hopfield Networks
1983: Hinton & Sejnowski, Boltzmann Machines
1985/1986: Practical Backpropagation for neural net training
1989: Convolutional Networks
1991: Bottou & Gallinari, module-based automatic differentiation
1995: Hochreiter & Schmidhuber, LSTM recurrent net.
1996: structured prediction with neural nets, graph transformer nets
…..
2003: Yoshua Bengio, neural language model
2006: Layer-wise unsupervised pre-training of deep networks
2010: Collobert & Weston, self-supervised neural nets in NLP

https://drive.google.com/file/d/1vVNUye-1JNJnqP4A0704sjtF7gs_MpCI/edit 38
https://youtu.be/mTtDfKgLm54

NNs and Deep Learning History by Yann LeCun

Y. LeCun

More History
2012: AlexNet / convnet on GPU / object classification
2015: I. Sutskever, neural machine translation with multilayer LSTM
2015: Weston, Chopra, Bordes: Memory Networks
2016: Bahdanau, Cho, Bengio: GRU, attention mechanism
2016: Kaiming He, ResNet

https://drive.google.com/file/d/1vVNUye-1JNJnqP4A0704sjtF7gs_MpCI/edit 39
https://youtu.be/mTtDfKgLm54

Artificial Intelligence, Machine Learning and Deep Learning

Artificial
Intelligence

Machine
Learning

Deep
Learning

40
Artificial Intelligence, Machine Learning and Deep Learning
Artificial
Intelligence

Machine
Learning

Deep
Learning

Machine Learning

Unsupervised Weakly Supervised Fully Supervised

(no supervision) (weak supervision) (strong supervision)

incomplete supervision inexact supervision inaccurate supervision

(i) semi-supervised (i) multiple instance
learning learning

41

 AIN2001
Principles of Artificial Intelligence

AIN2001 - Principles of Artificial Intelligence

This course will teach you nothing!

43

AIN2001 - Principles of Artificial Intelligence

This course will teach you nothing!
(unless you work for it)

44

AIN2001 - Principles of Artificial Intelligence

This course will teach you nothing!
(unless you work for it)

The course is not teaching-centric, rather

learning-centric.

45

AIN2001 - Principles of Artificial Intelligence

This course will teach you nothing!
(unless you work for it)

The course is not teaching-centric, rather

learning-centric.

As you put effort, you can learn anything in

this course!

46

Course philosophy: learn from each other!

• This is an interactive class
• Interaction helps us learn better
• We like questions
• Asking questions is more important than giving answers
• No question is silly/wrong/embarrassing
• If you have a question, most probably others also have it
• Please pay attention and participate
• We may ask questions to random people, you must answer
• “I don’t know” is an acceptable answer
• Don’t be embarrassed for wrong answers
• Some questions are to make you think and start a discussion

47

Course and Learning Objectives

• We will learn the basics of artificial intelligence and applications of
AI models in various domains.

• By the end of the course, we are expected to:

• know the fundamentals of Artificial Intelligence
• science
• be able to apply AI models to different tasks
• engineering
• have a strong basis to understand the related work in the
literature and extend it through further studies
• research

48

Course Structure
• Learning foundations
• covering pointed out resources before the class
• discussions during the class
• quizzes after the class

• Gaining hands-on experience (using Python)

• tutorials in the class
• programming assignments

49

No Electronics During Lectures!

https://static01.nyt.com/images/2017/11/26/business/26ViewArt/26ViewArt-jumbo.jpg?quality=75&auto=webp

“Laptops Are Great. But Not During a Lecture or a Meeting”

Susan Dynarski, The New York Times, Nov. 22, 2017
https://www.nytimes.com/2017/11/22/business/laptops-not-during-lecture-or-meeting.html

50

Course Content

Week Topics
A review of AI concepts
1
Rational agents
Solving problems by searching
2
• Search algorithms (uniformed and informed)
Solving problems by searching
3
• Constraint satisfaction problems
Games
4 • Adversarial search
• Game theory
Logical agents
5 • Propositional logic
• First order logic and inference
6 Planning
Probabilistic reasoning
7 • Basic probability concepts
• Inference using joint distributions
Probabilistic reasoning
8 • Naïve Bayes’ model
• Bayesian networks

51

Course Content

Week Topics
9 Machine learning
10 Neural networks
Deep learning
11
• Convolutional neural networks
Deep learning
12
• Recurrent neural networks
13 Reinforcement learning
14 AI, ethics, and society

52

Resources

Type Name Description

Textbook Artificial Intelligence: A The Deep Learning textbook is a resource intended to help
Modern Approach (Stuart students and practitioners enter the field of machine learning
Russell and Peter in general and deep learning in particular.
Norwig, 2021)
Lecture MIT 6.034 Artifficial Lecture of Patrick Henry Winston from MIT
Intelligence

Lecture Berkeley CS188 Intro to Lecture of Igor Mordatch and Peyrin Kao from Berkeley
Artificial Intelligence

53
Grading
Item Weight (%)
Weekly Quizzes 24
Programming Assignments 16
Midterm Exam 20
Final Exam 40
Contribution Bonus 5 [bonus]

General grading and expectations

F/E Those who do not know what they are doing. Their
results are unreasonable

D/C Those who know how to get some good results but
cannot explain them.

Those who understand what is going on with their

B experiments. Able to explain their results.

Those who know enough to combine different

A methods and be creative in using Deep Learning

Those who break the frontiers of Deep Learning

research.
4 NUS Deep Learning SP22, Lee Hwee Kuan

54
Academic Integrity
• Academic integrity is a serious matter in this course. Any
violation will be reported to the university’s highest levels
and maximum punishment will be argued for.
• You are encouraged to form study groups to discuss quizzes
and programming assignments. However, you
• must write down your own solutions without referring to the
notes taken during discussions
• DO NOT copy, refer to or look at written solutions and code
from another student or any other sources.
• DO NOT post your solutions and code online.
• must write down the names of your collaborators in your
submissions.
• You are expected to comply with the University Policy on
Academic Integrity and Plagiarism. If you have any questions
about this policy and any work you are doing in the course,
please feel free to contact your instructor for help.
55

Attendance
• You must attend at least 10 out of 14 classes. Otherwise, you
will get an “NA” and will NOT be allowed to take the final exam.

56
Weekly Quizzes
• There will be 14 weekly quizzes, weighted equally.
• Your best 12 out of 14 quizzes will be retained.
• Quizzes will generally (but not always) be released on Friday and
due 72 hours later.
• Each quiz may cover the topics of previous weeks and the next
week.
• Quizzes may contain multiple-choice questions, True/False
questions, written questions, or programming questions. If there
are any written questions, solutions must be in PDF format and
typeset in LaTeX.
• Online editors such as Overleaf may be helpful in writing
solutions: https://www.overleaf.com
• A quick tutorial: Learn LaTeX in 30 minutes

57

Programming Assignments
• There will be four programming assignments, weighted equally.
• Programming parts will be auto graded using scripts. You must
follow the submission guidelines; otherwise, you may get zero.
• Programming assignments may also include written parts.
Solutions to these parts must be in PDF format and typeset in
LaTeX.
• Selected solutions will be invited to present in the class.

58

Exams
• Midterm Exam
• There will be one midterm exam covering all subjects in the first
8 weeks. It will be given face to face in the classroom during
Week-9.
• Final Exam
• There will be one final exam covering all subjects. It will be
given face to face in the classroom during final exams. A
student with “NA” will NOT be allowed to take the final
exam.

59

Contribution Bonus
• Attending the lectures and contributing to the discussions will be
awarded a 5% bonus.
• The bonus will be prorated based on the number of weeks a
student contributed to the lecture.
• The followings during a lecture are accepted as a contribution:
• Asking at least one question
• Commenting on a subject at least once
• Please indicate your contribution in ItsLearning

60

Late Submission
• You are given a total of 7 free late (calendar) days. You can use
your late days in quizzes and programming assignments.
• Each late day is bound to only one assignment.
• If you submit one quiz and one programming assignment 3
hours after the deadline, you will be charged for 2 late days.

• Once you run out of late days, your late submissions will be
penalized 20% per late day.
• However, submissions will NOT be accepted more than 3
days after the deadline.

61

To Do
Coming Deadlines
• First quiz will be released on Friday (07.10.2022) and due on
10.10.2022 @23.59

62