THE APPLICATION OF DEEP

LEARNING
FOR ANOMALY DETECTION

Kun-chan Lan
CSIE NCKU
http://www.csie.ncku.edu.tw/~klan
klan@csie.ncku.edu.tw

1
 Important Information
• Course :深度學習在異常偵測的應
用
• Taught by : 藍崑展
• Class No. : NM61100
• Credit : 3 units

2
 Today

All about the class admin

3
 課程概述 Course Description
• 本課程主要透過專案學習（PjBL）的方式作為學
習媒介，藉由在執行專案時遇到 問題驅使學生
自發性尋求協助並從中學習相關知識；在發展的
階段，以Deep Learning技術構思解決方法，並
實作設計一個跌倒偵測系統
• This course will implement the project-based
learning (PjBL) method as a learning mechanism,
with a hope that, by solving problems during the
implementation of a project, students are motivated
to seek help spontaneously and learn relevant
knowledge by themselves. The context of the
projects will focus on the Fall Detection system

64
 Objectives
• Overview of Deep Learning (DL)
• Introduction to methods for one-class
classification
• Implementation of DL for an anomaly
detection

75
 PBL teaching
•Project-based learning
•Problem-based learning
•Product-based learning
–Novelty (demand-based)
–Business model (how to sell)
–Market value (whom to sell)
–Resource-limited thinking
(cost-effective thinking)

6
 PBL
• Motivation/Problem
• Idea/hypothesis
• Problem formulation
–Optimization
–Classification
–Integration
–…..
• Experiment/prototype
• Performance Evaluation
• Sell it
–Business model
–User interface
–……

7
Maybe something new for you
•We will adopt ‘Flipped Classroom’ teaching
•Preview class materials before coming to class

Intro to Computer 8
 Flipped classroom
•Students learn new content online by
watching video lectures
•Homework/assigned problems is done in
class with teachers offering more
personalized guidance and interaction with
students,

Intro to Computer 9
PLEASE DO NOT eat in my class!
Finish your meal before stepping into my
classroom, please!

10
 Today

All about the class admin

11
 Roadmap
• The essentials
• Administrative Information
• Content
– Course objective and scope
– Syllabus
• Your responsibility
– Homework & quiz
– Projects
• Grading policy
• Class material
12
19
 The Essentials
• Course page
– https://lens.csie.ncku.edu.tw/course?id=dlad-2023-
fall
– It is YOUR responsibility to frequently check the
‘Announcement’ link on the course page

• From my web page

– http://www.csie.ncku.edu.tw/~klan
– Click the ‘LENS’ link
– Then, click the ‘Courses’ link

13
20
 Roadmap
• The essentials
• Administrative Information
• Content
– Course objective and scope
– Syllabus
• Your responsibility
– Homework & Quiz
– Projects
• Grading policy
• Class material
14
21
 Lecture Info
• Location
– EE Room 92185
• Time
– Wed 2:10-5:00pm
– Normally two breaks

15
22
 The Instructor
• Kun-chan Lan (藍崑展)
– Office: new CSIE building 12F 65C05
– Phone: 2757575 x62550
– Email: klan@csie.ncku.edu.tw
– Homepage: http://www.csie.ncku.edu.tw/~klan
– Office hours: by appointment via email

16
23
 The TA
• 張文彥
• Office: new CSIE building 5F, Laboratory
for experimental network and system (LENS)
–Office Hour:5-5:30 pm Wednesday

• Or, by appointment via email

–Email:p78111098@gs.ncku.edu.tw

17
25
 Roadmap
• The essentials
• Administrative Information
• Content
– Course objective and scope
– Syllabus
• Your responsibility
– Homework
– Project
• Grading policy
• Class material
18
27
 Prerequisite
• Programming knowledge in Java/C++
• python programming experiences

19
 Syllabus: Part I (theory)
• week 1 (9/6) Administration issue
• week 2 (9/13) Tutorial for homework I and II (by TA)
• week 3 (9/20) Introduction to Anomaly detection
• week 4 (9/27) Introduction to Deep Learning (I)
• week 5 (10/4) Introduction to Deep Learning (II) ), Homework I
due
• week 6 (10/11) Introduction to Deep Learning (III)
• Week 7 (10/18) Unsupervised Learning
• week 8 (10/25) midterm exam

20
29
Syllabus: Part II (implementation)
• week 9 (11/1 ) Graphic Neural Network
• week 10 (11/8) Generative AI (I), Homework II due
• week 11 (11/15) Generative AI (II)
• week 12 (11/22) Speaker Talk (from聯詠科技), Project
proposal due
• Week 13 (11/29) Anomaly detection for Image data
• week 14 (12/6) Anomaly detection for Image data
• week 15 (12/13) Anomaly detection for Time series data
• week 16 (12/20) Anomaly detection for Time series data
• week 17 (12/27) in-class project demo
• Week 18 (1/3) final exam

21
30
One class classification

22
Generative AI

23
 Waiver for class attendance
• From week 4 to week 7, a waiver of
attendance is allowed if you can demonstrate
your experience in basic Deep Learning
materials
• Make an appointment with me to discuss
the possibility for such a waiver

24
 Be aware
• This is an Introductory course (which
meaning we only talk about things on the
surface)
• Self-studying/practicing after classes is
strongly required for you to complete the
term project

25
 Roadmap
• The essentials
• Administrative Information
• Content
– Course objective and scope
– Syllabus
• Your responsibility
– Homework and Quiz
– Project
• Grading policy
• Class material
26
33
 How are you evaluated
• Homework and Project (80%) – codes will be
provided for your reference
– Homework I (15%)
– Homework II (15%)
– Term Project (50%)
• Written exams (10 questions)
– Midterm (10%)
– Final (10%)
• Class participation (up to +/-10%)
• In-class discussion

27
34
Homework I: handwriting recognition

28
 MNIST dataset
•How to download
–https://ithelp.ithome.com.tw/articles/101
86473
•Three parts
–Training data - 55000
–Testing data – 10000
–Validation data - 5000
29
 Criteria for marking
•Many example codes online
–We’ll also show an example Keras
program in class
•Accuracy (both training error and testing
error)
•You can use
–Fully connected network (FCN)
–Convolutional Neural Network (CNN)
30
 Homework II: Semi-supervised
Anomaly Detection using AutoEncoders
 Homework II: Semi-supervised
Anomaly Detection using AutoEncoders
model

33
dataset

34
 Criteria for marking
Data-set F1 Score
• DAGMC8 0.96
• RSDDsI 0.81

•Codes and data

–https://github.com/msminhas93/anomaly-
detection-using-autoencoders
–https://zenodo.org/record/8086136
35
 Term project
Anomaly detection for on Fall
Detection

36
Fall detection as anomaly detection

•Falling data is not easy to collect

–Unbalanced data set
•No need to label
–Unsupervised learning

37
 Possible approaches for video-
based anomaly detection

• One-class classification
–High-dimensional image data suggests high
computation overhead
• Autoencoder
–With the high-dimensionality of images, pure
autoencoders suffer from learning robust data
representations
–use a feature extractor in the pre-processing phase o
obtain a robust and efficient representation

38
Example

39
Use ST-GCN for feature extraction

40
CNN vs. ST-GCN

41
ST-GCN

• Blue dots denote the body joints.

The intra-body edges between
body joints are defined based on
the natural connections in human
bodies. The inter-frame edges
connect the same joints between
consecutive frames. Joint
coordinates (obtained through a
pose estimator, e.g. OpenPose) are
used as inputs to the ST-GCN.

42
Convolution as an aggregation
operation

43
GCN vs. CNN
• CNN
– Images, pixels arranged in a grid-like structure.
– An image can be thought of as a graph where each pixel is
a node and is connected to all its neighboring pixel.
• all nodes have same amount of neighbors except
those which are at boundary
• There is order in the arrangement of these nodes.
– convolution operation is done on the spatial representation
of the image
– learn a node-agnostic filter where the same filter matrix is
applied to all nodes regardless of its position in the graph
• GCN
– similar to a CNN. In effect, every nodes information is
learnt by aggregating weighted information from its
neighbors.
– get node-specific filters governed by the Laplacian

44
 Resources

• A Framework for Anomaly Identification Applied on Fall Detection

– Paper: https://ieeexplore.ieee.org/document/9439497
• Spatial Temporal Graph Convolutional Networks for Skeleton-
Based Action Recognition (ST-GCN) Sijie Yan, Yuanjun Xiong and
Dahua Lin, AAAI 2018.
– Paper: https://arxiv.org/abs/1801.07455
– Codes: https://github.com/KingMV/st-gcn_ad
• Using Tiny-YOLO oneclass to detect each person in the frame and
use AlphaPose to get skeleton-pose and then use ST-GCN model to
predict action from every 30 frames of each person tracks.
– https://github.com/GajuuzZ/Human-Falling-Detect-Tracks

45
 Dataset (I)
1. UR Fall Detection Dataset http://fenix.ur.edu.pl/~mkepski/ds/uf.html

46
 Dataset (II)
1. UP-FALL Dataset https://sites.google.com/up.edu.mx/har-up/

47
Benchmark

48
 Your grade
• choose one of the datasets
• ( yours – benchmark) * 10 + 90
• e.g.
–Your got an accuracy for 99% for UP-FALL dataset
• (99%-98%) x 100 + 90 = 91
– Your got an accuracy for 70% for UP-FALL dataset
• (70%-98%) x 100 + 90 = 61
–We only consider ‘accuracy’ and ‘F1’
•Your grade will be the average score for these 2 metrics

49
 Demo your project
• In the end of semester,
you will demo your
project in-class
• Each team has to make
5-min video to demo
your project
• The video should be
uploaded to youtube in
advance

50
 Project document
• proposal (11/22)
– 5 page, 11-pt-font, double-spaced proposal
describes/explain the following
• How you plan to implement the project
• Responsibility for each of your team members
• Final
– Project implementation (including source codes and
demo video)

51
 Class participation (10%)
•Up to 1% or -0.5%of total credits for each
class

52
Select your group members now!
• Find your group members NOW!
– 3 members for each group (data preparation, model and
n evaluation, demonstration)
– You should sit together with your group members every
class for in-class discussion

• Email your group members (name, ID) to TA

(NOT me) before the end of this Friday

53
50
Questions?

54