AI LEARNING

PARADIGMS

Nguyễn Ngọc Thảo

nnthao@fit.hcmus.edu.vn
 • Supervised learning
Learning
• Unsupervised learning
problems • Reinforcement learning
Supervised learning (SL)
• Learn a function that maps an input to an output based on
examples, which are pairs of input-output values.

Supervised
𝑋 𝑦 learning Model
algorithm

Image credit:
𝑦ො EnjoyAlgorithms

3
 Spam detection

Recommender systems in BI
Object recognition
on MNIST dataset 4
 Classify protein structures at different levels

Tuan-Luc Huynh, Thanh-Danh Le, Van-Tam Nguyen, Trung-Nghia Le,

and Minh-Triet Tran. “Efficient 3D Brain Tumor Segmentation with
Axial-Coronal-Sagittal Embedding“. PSIVT

Bui, Nhat-Tan, Dinh-Hieu Hoang, Quang-Thuc Nguyen, Minh-Triet Tran,

and Ngan Le. "MEGANet: Multi-Scale Edge-Guided Attention Network
for Weak Boundary Polyp Segmentation." WACV 2024. (Arkansas Uni. -
HCMUS)
5
Classification vs. Regression

Regression

• Train a model to predict a continuous dependent variable

Classification
• Train a model to predict a categorical dependent variable
6
Classification vs. Regression

Set up a decision boundary that Find a curve that best fits

separates different classes the available data points
7
Classification vs. Regression
• There are also some overlaps between the two types.
• A regression algorithm can predict a discrete value which is
in the form of an integer quantity
• A classification algorithm can predict a continuous value if it
is in the form of a class label probability

8
Classification: Problem types
• Binary vs. Multiclass
classification

• Single-label vs. Multi-label classification

9
Unsupervised learning (USL)
• Infer a function to describe hidden structure from "unlabeled"
data, i.e., labels are not included in the observations.

Unsupervised
𝑋 𝑦 learning Model
algorithm

Image credit: EnjoyAlgorithms

𝑦ො
10
Common unsupervised learning tasks
High dimensional data Raw data points Transactional data

Dimensionality Reduction Clustering Association Analysis

Low dimensional data Clustered data points Causal relationships

11
 Social
network
Biology: a taxonomy of living things analysis:
community
detection

Outlier detection

Information retrieval: document clustering

12
Ref: Shevtsov, Alexander, et al. "Analysis of Twitter and
YouTube during US elections 2020."
arXiv e-prints (2020): arXiv-2010.

13
Supervised vs. Unsupervised learning

Supervised Unsupervised
learning learning

Predefined classes No predefined classes

Learning by examples Learning by observations

14
 Human
action
recognition
problem

Semi-Supervised Learning
and Capsule Network for
Video Action Detection.
RIVF 2023. (HCMUS)
15
Reinforcement learning (RL)
• An agent learns to interact with an environment based on
feedback signals it receives from the environment.

An agent spilled water on the

floor, changing the room’s state
→ he learns from feedback not
to do it again.
16
Concept vocabulary in RL

Image credit: Anyscale

17
Reinforcement learning: Demo

Source: YouTube video 18

Monte Carlo tree search (MCTS)
• A heuristic search method that combines the precision of
tree search with the generality of random sampling.

Node: a state of the domain

Each node contains statistics describing at Link: an action leading to a
least a reward value and number of visits subsequent state

19
From AlphaGO to AlphaZero

https://deepmind.com/blog/article/
alphazero-shedding-new-light-
grand-games-chess-shogi-and-go

• AlphaZero gains human behavior in board games without

relying on external knowledge.
• Trained solely via self-play: 5000 first-generation TPUs to generate
the games and 64 second-generation TPUs to train the networks
20
A taxonomy of RL algorithms

A non-exhaustive, but useful taxonomy of algorithms in modern RL.

Image credit: OpenAI Spinning Up 21

22
Automated game testing and tuning
23
RL toolkits/libraries

Gym is a standard API for RL, and a diverse collection

of reference environments
Gym Retro is a platform for RL research on games.
There are around 70 Atari games and 30 Sega games
to over 1,000 games across a variety of backing
emulators.

RLlib is an open-source library that offers support for

production-level, highly distributed RL workloads

24
Hybrid • Semi-supervised learning

learning • Self-supervised learning

problems • Multi-instance learning

Semi-supervised learning
• That is supervised learning where the training data includes
very few labeled examples and lots of unlabeled examples.
• Unsupervised methods (e.g., clustering or density estimation) help
utilize unlabeled data effectively.

Image credit: HAL 26

Semi-supervised learning

self-training co-training

Image credit: ResearchGate 27

Self-supervised learning
• The model generates its own supervision signals from the
input data to guide the learning process for pretext tasks.
• The acquired knowledge can then be passed to downstream
tasks of interest.

Generative adversarial Contrastive

Autoencoder
network (GAN) learning

28
Autoencoder
• An autoencoder captures the most important patterns in the
input data to produce efficient representations.

Image credit: OpenGenus IQ

29
Autoencoder: Applications

Image colorization

Image denoising

Image credit:
1,2,3
Image
super-resolution 30
Generative Adversarial Networks (GANs)
• The key idea behind GANs is to train two neural networks, a
generator, and a discriminator, in a competitive framework.

31
GAN Applications: Image inpainting

Zhou, Yuqian, et al. "TransFill: Reference-guided Image Inpainting by Merging Multiple

Color and Spatial Transformations." Proceedings of the IEEE/CVF CVPR. 2021. 32
GAN Applications: Image synthesis

Nvidia StyleGAN 33
GAN Applications: Image synthesis

Nvidia StyleGAN 3 34
Multi-instance learning
• An entire collection of examples is labeled as containing or
not containing an example of a class.
• However, the individual members are not labeled.

Supervised learning Multi-instance learning

Image credit: 1, 2 35
Multi-instance learning: Pathology

Patches sampled from clusters of different whole slide images in

decreasing order of attention importance for detecting Celiac disease.
Sharma,
Image Yash,Medium
credit: Aman Shrivastava, Lubaina Ehsan, Christopher A. Moskaluk, Sana Syed, and
Donald Brown. "Cluster-to-conquer: A framework for end-to-end multi-instance learning
for whole slide image classification." In Medical Imaging with Deep Learning, pp. 682-698.
PMLR, 2021. 36
 • Inductive learning
Statistical
• Deductive learning
inference • Transductive learning
Statistical inference

• Induction: Learn a general model from specific examples

• Deduction: Use a model to make predictions
• Transduction: Use specific examples to make predictions

38
 • Multi-task learning

• Active learning
Learning
• Online learning
techniques • Transfer learning

• Ensemble learning
Multi-task learning
• A type of supervised learning that involves fitting a model on
one dataset that addresses multiple related problems.
• E.g., the same word embedding is used to learn a distributed
representation of words and then shared across multiple NLP tasks.

Soft parameter sharing

Hard parameter sharing

Image credit: Ruder.io 40

Multi-task learning: An example

The architecture takes a single monocular RGB image as input and produces a pixel-wise
classification, an instance semantic segmentation and an estimate of per pixel depth.
Multi-task learning can improve accuracy over separately trained models because cues
from one task, such as depth, are used to regularize and improve the generalization of
another domain, such as segmentation.
Kendall, Alex, Yarin Gal, and Roberto Cipolla. "Multi-task learning using uncertainty to
weigh losses for scene geometry and semantics." Proceedings of the IEEE conference on
computer vision and pattern recognition. 2018. 41
Active learning
• The learner resolves ambiguity during the learning process by
querying an oracle to request labels for new points.

The number of tuples used

to learn a concept this way
is often much smaller than
those in typical SL.

• Useful in domains where labels are expensive to obtain.

• E.g., computational biology applications

42
Active learning

• How to pick the data points to be queried?

• Uncertainty sampling
• Reduce expected entropy over 𝑈
• Reduce the subset of hypotheses consistent with the training data
43
Active learning
• We can view active learning as an approach to solve semi-
supervised learning problems.

Semi-supervised methods Active methods

exploit what the learner thinks it explore the unknown aspects
knows about the unlabeled data by querying an oracle

44
Online learning
• The model is updated as each new data point arrives rather
than waiting until “the end” (which may never occur).
• The probability distribution of observations changes over time.
• There are too many observations to reasonably fit into memory.
• Stochastic gradient descent (SGD)

45
46
Transfer learning
• Models learning from one type of problem and the learning
is then applied to solve a different but related problem.

47
Transfer learning
Classifier
The pre-trained model is used directly to classify samples.

Feature extractor
The pre-trained model is used to extract relevant features.

Weight initialization
The pre-trained model is integrated into a new model, and
its layers are trained in concert with the new model.

48
Ensemble learning
• Two or more models are fit on the same data and the
predictions from each model are combined.

49
Ensemble learning
• An ensemble goes more accurate than its base learners,
especially when the models are significant diverse.
• Parallelizability: each base learner to a different CPU
• Types of ensemble learning

Predict via simple majority voting over a collection

Bagging of base learners

Boosting Weighting votes over a collection of base learners

Combine a set of heterogeneous weak learners

Stacking
using one ore more meta-models
50
Ensemble learning: Demonstration

Image credit: mksaad.wordpress 51

Ensemble learning: Demonstration

Image credit: mksaad.wordpress 52

Ensemble learning: Demonstration

Image credit: mksaad.wordpress 53

Ensemble learning: An example

Decision boundary by (a) a single decision tree and (b) an ensemble of decision
trees for a linearly separable problem (i.e., where the actual decision boundary is
a straight line). The decision tree struggles with approximating a linear boundary.
The decision boundary of the ensemble is closer to the true boundary. 54
Random Forest (Breiman 2001)
• Each base learner is a decision tree, generated using a random
selection of attributes at each node to determine the split.
• Classification: each tree votes and the most popular class is returned.

55
AdaBoost (Freund and Schapire, 1997)

Referece: Explaining AdaBoost

56
Deepfake detection

Reference: GitHub 57
Video generation: Deepfake

Source: YouTube video 58

Anomaly detection in videos
Zahid, Yumna, et al. "Ibaggedfcnet: An
ensemble framework for anomaly
detection in surveillance videos." IEEE
Access 8 (2020): 220620-220630.

59
 • Federated learning
Advanced
• Explainable AI
learning • Generative AI
Federated learning (FL)
• Privacy-preserving models can be trained in heterogeneous
and distributed networks.

61
Federated learning (FL)
Generate a global model shared by all nodes

Exchange
parameters
between
these local
nodes

Train local models on local data samples

62
A typical FL workflow
• First, the initial model is distributed to the edge devices and
trained based on data generated by the user.

63
A typical FL workflow
• Only the updated model will be sent to the server side.
• Any actual data based on user behavior is not necessarily included.

64
A typical FL workflow
• The new model is aggregated on the server side and will be
distributed to all client devices.

65
Federated learning: Pros and Cons

Hyper-personalized Low Cloud Minimum Privacy preserving

Infra Overheads latencies

Expensive System Statistical Privacy concerns

communication heterogeneity heterogeneity

66
Federated learning platforms

TensorFlow Federated

IBM Federated Learning

67
Explainable AI (XAI)
• XAI enables a ML model to provide an explanation for its
decisions and action, which can be understood by humans.
• It contrasts with the "black box" concept in ML where we cannot
explain why an AI arrived at a specific decision.
• XAI is desired in use cases involving accountability, fairness
and transparency, with the aims of enhancing the trust
between humans and machines
• Higher visibility into model decision-making process (which
helps with transparency)

68
Explainable AI (XAI)

Image credit: Lawtomated 69

Explainable AI: Challenges
• How to generate explanations that are both accurate and
understandable.
• XAI models can be more difficult to be trained and tuned,
compared to non-explainable machine learning models.
• XAI systems are more difficult to deploy since explainability
features need some level of human-in-the-loop intervention.
• Explainability is how to take an ML model and explain the behavior in
human terms.

70
 Marked areas in the image were relevant for
Context image the answer and hidden areas were irrelevant

The AI
answer

XAILab Demo: Explainable VQA

71
Context image Marked areas in the image were relevant for
the answer and hidden areas were irrelevant

The AI
answer

XAILab Demo: Explainable VQA

72
Context image Marked areas in the image were relevant for
the answer and hidden areas were irrelevant

The AI
answer

XAILab Demo: Explainable VQA

73
 Generative Artificial Intelligence
• Generative artificial intelligence (Generative AI) refers to AI
models that can generate new, original content.
• These systems can create images, text, or other types of content
that were not explicitly provided to them beforehand.

74
Image credit: Turing
ChatGPT: An introduction
• ChatGPT is built upon either GPT-3.5 or GPT-4, that were
fine-tuned to target conversational usage.
• It leverages both supervised learning and reinforcement learning in a
process called reinforcement learning from human feedback (RLHF).

ChatGPT ChatGPT Plus

75
ChatGPT: An introduction
• ChatGPT is built upon either GPT-3.5 or GPT-4, that were
fine-tuned to target conversational usage.
• It leverages both supervised learning and reinforcement learning in a
process called reinforcement learning from human feedback (RLHF).

Image credit: SearchUnify unofficial though76

Transformer:
The key technique
• Transformer is a versatile
neural network architecture
for both unsupervised and
supervised learning tasks.

Vaswani, Ashish, Noam Shazeer, Niki

Parmar, Jakob Uszkoreit, Llion Jones,
Aidan N. Gomez, Łukasz Kaiser, and Illia
Polosukhin. "Attention is all you
need." Advances in neural information
processing systems 30 (2017). 77
Image generation and manipulation
• Image generation include art creation, data augmentation for
ML models, creating realistic scenes for movies, and more.
• It is an evolving field with continuous advancements in both
traditional and machine learning-based approaches.

StyleGAN

78
DALL·E: An introduction
• DALL·E is a text-to-image model using deep learning to
generate digital images from natural language prompts.
3.5 billion parameters,
a diffusion model
conditioned on CLIP
image embeddings

A multimodal
implementation of
GPT-3 with 12 Released natively into
billion parameters ChatGPT Plus
Implemented in Bing's
Image Creator

79
DALL·E 2: Demonstration

Source: YouTube video 80

Imagen: Demonstration

Source: YouTube video 81

Text-to-image translation
• The task aims to generate an image that corresponds to a
given textual description.

A modern architectural
building with large glass Meaningful
windows, situated on a representation
cliff overlooking a serene
ocean at sunset

Created by DALLE 3

• It is a reverse task to image captioning.

82
Virtual Interior
Design “Japanese style,
wooden wall,
cotton puff”

“A Van Gogh
style painting
Original image on the wall”

Le, Minh-Hien, Chi-Bien Chu, Khanh-Duy Le, Tam V. Nguyen, Minh-Triet Tran, and Trung-
Nghia Le. "VIDES: Virtual Interior Design via Natural Language and Visual Guidance." ISMAR
2023. HCMUS research work.
83
Audio generation: AI music
• Generative AI empowers creators to generate new warbles,
chimes, measures, and even entire songs.

84
85