C1 Introduction

Faculty of Information Technology
Chapter 1. Introduction
(Digital Image Processing)
PGS.TS. Hoàng Văn Dũng
Email: dunghv@hcmute.edu.vn
Outline
1 Introduction to image processing
2 Concepts
3 Image processing stages
4 Components of image processing
5 Introduction to OpenCV
6 Exercises
ISLab- Intelligent Systems Laboratory 2

Introduction
• Digital image processing (DIP) is a method to perform some operations on
an image, in order to get an enhanced image or to extract some useful
information from image.
• It is a type of signal processing in which input is an image and output may
be image or characteristics/ features associated with that image.
• Image processing is among rapidly growing technologies.
• It forms core research area within engineering and computer science
disciplines too.

Introduction
• Image processing basically includes the following three steps:
– Importing the image via image acquisition tools;
– Analyzing and manipulating the image;
– Output in which result can be altered image or report that is based on image
analysis.

Introduction
• There are two types of methods:
– Analogue image processing
– Digital image processing
• Digital image processing techniques help in manipulation of the digital
images by using computers.
• The three general phases that all types of data have to undergo while using
digital technique are
• Preprocessing
• Enhancement image
• Display, information extraction.

Application of DIP
• Image sharpening and restoration
• Automatic inspection, e.g. manufacturing applications
• Assisting humans, e.g., species identification system
• Controlling processes, e.g., industrial robot
• Detecting events, e.g., surveillance or people counting
• Interaction, e.g., as input to a device for CHI
• Modeling objects, e.g., medical image analysis
• Navigation, e.g., autonomous vehicle, mobile robot
• Organizing information, e.g., indexing databases of images.
• ………….

History of DIP
Early 1920s: One of the first applications of digital imaging was in the news-
paper industry
– The Bartlane cable picture transmission service
– Images were transferred by submarine cable between London and New York
– Pictures were coded for cable transfer and reconstructed at the receiving end on
a telegraph printer
Early digital image

History of DIP
Mid to late 1920s: Improvements to the
Bartlane system resulted in higher quality
images
– New reproduction processes based on
photographic techniques
Improved digital image
– Increased number of tones in reproduced
images
Early 15 tone digital

image
History of DIP
1960s: Improvements in computing technology and the onset of the space
race led to a surge of work in digital image processing
– 1964: Computers used to
improve the quality of
images of the moon taken
by the Ranger 7 probe
– Such techniques were used
in other space missions
including the Apollo landings
A picture of the moon taken

by the Ranger 7 probe
minutes before landing

History of DIP
1970s: Digital image processing
begins to be used in medical
applications
– 1979: Sir Godfrey N. Hounsfield &
Prof. Allan M. Cormack share the
Nobel Prize in medicine for the
invention of tomography, the
technology behind Computerised
Axial Tomography (CAT) scans
Typical head slice CAT image

History of DIP
1980s - Today: The use of digital image processing techniques has exploded
and they are now used for all kinds of tasks in all kinds of areas
– Image enhancement/restoration
– Artistic effects
– Medical visualisation
– Industrial inspection
– Law enforcement
– Human computer interfaces

Outline
2 Concepts
6 Exercises

Digital Images
• An image is a two-dimensional function f(x,y), where x and y are the spatial
(plane) coordinates, and the amplitude of f at any pair of coordinates (x,y) is
called the intensity of the image at that level.
• If x,y and the amplitude values of f are finite and discrete quantities, we call the
image a digital image. A digital image is composed of a finite number of
elements called pixels, each of which has a particular location and value.
1 pixel

Digital Images
f(1,1) = 103 Pixel intensity value
Pixel location Consider the following

image (2724x2336
pixels) to be 2D function
rows or a matrix with rows
columns
and columns
f(645:650,1323:1328) =
83 82 82 82 82 82 In 8-bit representation
82 82 82 81 81 81 Pixel intensity values
82 82 81 81 80 80 change between 0 (Black)
82 82 81 80 80 79
80 79 78 77 77 77 and 255 (White)
80 79 78 78 77 77
f(2724,2336) = 88

Digital Images
Remember digitization implies that a digital image is an approximation
of a real scene
One pixel

Digital Images
• Sources of Digital Images
– The principal source for the images is the electromagnetic
(EM) energy spectrum.

Digital Images
• Gamma rays
Gamma-Ray imaging
of
A starburst galaxy
Gamma-Ray about 12 million
Imaging light-years away
Cherenkov
Telescope Gamma-Ray Imaging
in nuclear medicine
Digital Images
• X- ray images Figure ad
Examples of b
X-ray imaging ce
(a) Chest X-ray.

(b) Aortic
angiogram.
(c) Head CT.
(d) Circuit boards.
(e) Cygnus Loop
X-ray images from the space

The Chandra X-Ray Observatory

Digital Images
• Ultraviolet images

Digital Images
• Visible light images R
G B

Digital Images
• Infrared images
infrared ("thermal") image Snake around the arm
Messier 51 in ultraviolet (GALEX), visible (DSS), and near

infrared (2MASS). Courtesy of James Fanson.

Digital Images
Microwave images
Synthetic Aperture Radar

System

Digital Images
• Radio wave images
MRI image slices from the brain

Digital Images
Digital Images based on the EM Spectrum
An example showing Imaging in all of the bands
Visible light

Digital Images
• Ultrasound imaging
Ultrasonic spectrum
Ultrasonic baby image Ultrasound image

during pragnancy acquisition device
Digital Images
The continuum from image processing to computer vision
can be broken up into low- mid- and high-level processes
Low Level Process Mid Level Process High Level Process

Input: Image Input: Image Input: Attributes
Output: Image Output: Attributes Output: Understanding
Examples: Noise Examples: Object Examples: Scene

removal, image recognition, understanding,
sharpening segmentation autonomous navigation

Summary: Digital Images
▪ We have looked at:
▪ What is a digital image?
▪ What is digital image processing?
▪ State of the art examples of digital image processing
▪ Image acquisition
▪ Next time we will start to see the key stages in digital image processing and talk
about interpolation

Outline
2 Concepts
6 Exercises

Key Stages in Digital Image Processing
Image Morphological
Restoration Processing
Image
Segmentation
Enhancement
Image Object
Acquisition Recognition
Representation &
Problem Domain
Description
Colour Image Image
Processing Compression

• Image Aquisition
Image Morphological
Image
Segmentation
Enhancement
Image
Object Recognition
Acquisition
Single Sensor
Representation &
Problem Domain Line
Sensor
Description
Array
Sensor

• Image Enhancement
Image Morphological
Image
Segmentation
Enhancement
Image
Object Recognition
Acquisition
Representation &
Problem Domain
Description

• Image Restoration
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation &
Problem Domain
Description

• Morphological Processing
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation &
Problem Domain
Description

• Segmentation
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation &
Problem Domain
Description

• Object Recognition
Image Morphological
Image
Enhancement
Segmentation
Image
Acquisition Object
Recognition
Problem Domain
Representation &
Description

• Representation & Description
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation &
Problem Domain
Description

• Image Compression
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation &
Problem Domain
Description
Colour Image Image

• Colour Image Processing
Image Morphological
Image
Segmentation
Enhancement
Image
Object Recognition
Acquisition
Representation &
Problem Domain
Description
Colour Image Image

Outline
2 Concepts
6 Exercises

Components of IP systems
• Components of a
general-purpose
image processing
system

• Acquire digital images:
– (1) Physical sensor responds to the energy radiated by the object we wish to image;
– (2) Digitizer is a device for converting the output of the sensing device into digital form.
• Specialized image processing hardware:
– Arithmetic logic unit (ALU), that performs arithmetic and logical operations in
parallel on entire images
– GPU
• The computer: is a general computer and can range from a PC to a
supercomputer.
–

• Software: consists of specialized modules that perform specific tasks.
• Mass storage is a must in image processing applications.
• Image displays in use today are mainly color, flat screen monitors.
• Hardcopy devices for recording images include laser printers, film
cameras, digital units …..
• Networking, cloud communication are almost default functions in computer
system in use today.

Outline
2 Concepts
6 Exercises

Introduction to OpenCV
• What Open Source Computer Vision
Library: http://opencv.org) is an open-source library that
includes several hundreds of computer vision algorithm
• Documents:
– https://docs.opencv.org
• https://docs.opencv.org/4.x/d9/df8/tutorial_root.html
• https://docs.opencv.org/4.x/d6/d00/tutorial_py_root.html
– https://www.geeksforgeeks.org/opencv-python-tutorial/

• OpenCV has a modular structure.
• The package includes several shared or static libraries.
• The following modules:
– Core functionality (core)
– Image Processing (imgproc)
– Video Analysis (video)
– Camera Calibration and 3D Reconstruction (calib3d).
– 2D Features Framework (features2d)
– Object Detection (objdetect)
– High-level GUI (highgui)
– Video I/O (videoio).

• Core functionality: Defines basic data structures, including the dense multi-
dimensional array Mat and basic functions used by all other modules.
• Image Processing: includes linear and non-linear image filtering, geometrical
image transformations (resize, affine and perspective warping, generic table-
based remapping), color space conversion, histograms, and so on.
• 2D Features Framework: Salient feature detectors, descriptors, and descriptor
matchers.
• Object Detection: Detection of objects and instances of the predefined classes.
E.g. faces, eyes, mugs, people, cars,…..

• Video Analysis: Includes motion estimation, background subtraction, and object
tracking algorithms.
• Camera Calibration and 3D Reconstruction: Includes multiple-view geometry
algorithms, single- stereo camera calibration, object pose estimation, stereo
correspondence algorithms, and elements of 3D reconstruction.
• High-level GUI: Supports an easy-to-use interface to simple UI capabilities.
• Video I/O: Supports an easy-to-use interface to video capturing and video
codecs.
• Some other modules FLANN and Google test wrappers, Python bindings, and
others.

Implement OpenCV
➢ OpenCV in Python
➢ OpenCV in C++

OpenCV in Python
• Install Python
– Using Anaconda https://www.anaconda.com
• Install openCV
pip install opencv-python
• Install other (option)

pip install numpy
pip install matplotlib

OpenCV in Python
• Using Spyder IDE

Example
• Implement GUI to show image folders

Homeworks
• Take some landscape images and portrait images; store to *.jpg, .png files;
compare sizes of files.
• Install and configure to use OpenCV Python/C++
• Self learning OpenCV API https://docs.opencv.org
• Coding Python/C++ program read two image files and show on two
windows and same window.

Homeworks
1. Collecting at least 10 images, then save to 3 following format PNG, BMP,
JPG.
2. Programming using OpenCV in Python/C++
• Read image set and show each image on each window.
• Separate RGB image to 3 layers and show each layer on each window.
• Convert RGB image to grey and show it.
• Rotate an image in 50 times: In each step, rotated 15 degrees and zoom out
to 90% of its size and then show it on the same window.
• Read an image, then crop image with ¼ its size from image center and then
show it.

Bài tập
For Vi only:
Câu 1. Thu thập tối thiểu 10 ảnh màu ngoại cảnh và ảnh chụp đồ vật. Mỗi ảnh lưu
theo 3 loại sau PNG, BMP, JPG.
Câu 2. Viết chương trình Python/C++:
• Đọc các file ảnh ở câu 1.
• Hiển thị ảnh màu;
• Hiển thị ảnh theo từng lớp R, G, B riêng lẻ.
• Chuyển ảnh màu thành ảnh grey và hiển thị chúng.
• Thực hiện lặp 50 lần xoay- mỗi lần 15 độ. Mỗi lần xoay giảm kích thước xuống
90% rồi hiển thị và dừng chờ 1 giây (hiện thị trên cùng cửa sổ, xóa trạng thái cũ, hiển
thị trạng thái mới).
• Trích ra ¼ ảnh với tâm vùng lấy là tâm của ảnh, sau đó hiển thị.

Thanks for your attention!
Q&A
Appendix
Lập trình Xử lý ảnh với OpenCV & C++

OpenCV in C++
➢ Install VS Code (Visual Studio Code)
1. Install VS Code C/C++ extensions (ms-vscode.cpptools)
1. Open VS Code.
2. Click the Extensions view icon on the Sidebar (Ctrl+Shift+X).
3. Search of C++. After searching, you should find the extensions shown below:

VS Code with OpenCV C++
2. Install C/C++ Compiler and Debugger
– In this part I choose Mingw-w64. Notice you should choose the following
setting when installing:

3. Download pre-built OpenCV package
• OpenCV-MinGW-Build: https://github.com/huihut/OpenCV-MinGW-
Build/tree/OpenCV-4.1.1-x64
MinGW-x86_64-8.1.0-posix-seh-rt_v6-rev0

4. Add the Compiler Path and Pre-built OpenCV Packages to Path Variable
• For editing the Path variable, you can go Control
Panel > System > Advanced system settings. Then switch to "Advanced"
tab in System Properties window and click "Environment Variables"
button. At last, edit the Path variable
• E.g. C:\opencv\OpenCV-MinGW-Build-OpenCV-4.1.1-x64

5. Configure the .vscode files of your project.
• First, create a new project called opencvexample. Then copy and paste
the task.json, launch.json, and c_cpp_properties.json I created from this repo.
• task.json
This file tells VS Code to invoke the compiler to create an executable file based on the source
code. I have made some alterations of command and args:
• The command will invode the compiler. Here, I change the path to the MinGW compiler I
desired.
• The args will be passed to the compile command as compiler flags. You can see the program
will link with OpenCV libraries.
• As a thumb up rules, only include the libraries you are going to use, though I include all the
libraries of OpenCV.

• Example:
"command": "C:\\mingw-w64\\x86_64-8.1.0-posix-seh-rt_v6-rev0\\mingw64\\bin\\g++.exe",...
"args": [
"-g",
"${file}",
"-o",
"${fileDirname}\\${fileBasenameNoExtension}.exe",
"-I","C:\\opencv\\OpenCV-MinGW-Build-OpenCV-4.1.1-x64\\include",
"-L","C:\\opencv\\OpenCV-MinGW-Build-OpenCV-4.1.1-x64\\x64\\mingw\\lib",
"-llibopencv_calib3d411",
"-llibopencv_core411",
"-llibopencv_dnn411",
"-llibopencv_features2d411",
"-llibopencv_flann411",
"-llibopencv_highgui411",
"-llibopencv_imgcodecs411",
"-llibopencv_imgproc411",
"-llibopencv_ml411",
"-llibopencv_objdetect411",
"-llibopencv_photo411",
"-llibopencv_stitching411",
"-llibopencv_video411",
"-llibopencv_videoio411"
],
...

• launch.json
• The launch.json file will launch the GDB debugger
when you start to debug the program, and I change
some lines shown below:
"program": "${fileDirname}\\${fileBasenameNoExtension}.exe",
"miDebuggerPath": "C:\\mingw-w64\\x86_64-8.1.0-posix-seh-rt_v6-rev0\\mingw64\\bin\\gdb.exe",
• The program is the executable of this program.

• The miDebuggerPath find the executable of debugger,
and I alter to the GDB debugger I would like to use.

• c_cpp_properties.json
• The c_cpp_properties.json lets you have more control over the C/C++
extensions:
• The includePath search the path that will include the libraries you want, and I
add the path of OpenCV.
• The compilerPath finds the path of compiler, so I set to the compiler I am going
to use.

6. Create Code, Compile It, and Run It
Create a file named main.cpp:
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace cv;
int main( int argc, char** argv )
{
std::cout << "aa" << std::endl;
std::string img = "lenna.jpg";
Mat srcImage = imread(img);
if (!srcImage.data) {
return 1;
}
imshow("srcImage", srcImage);
waitKey(0);
return 0;
}

API Concepts
• cv Namespace
#include "opencv2/core.hpp"
using namespace cv;
...
Mat H = findHomography(points1, points2, RANSAC, 5 );
...

• Automatic Memory Management
– OpenCV handles all the memory automatically.
– std::vector, cv::Mat, and other data structures used by the functions and
methods have destructors that deallocate the underlying memory buffers when
needed.
• Means that the destructors do not always deallocate the buffers as in case of Mat.
• They take into account possible data sharing.
• A destructor decrements the reference counter associated with the matrix data buffer.

• Automatic memory management
Example:
// create a big 8Mb matrix

Mat A(1000, 1000, CV_64F);
// create another header for the same matrix; this is an instant operation, regardless of the matrix size.
Mat B = A;
// create another header for the 3-rd row of A; no data is copied either
Mat C = B.row(3);
// now create a separate copy of the matrix
Mat D = B.clone();
// copy the 5-th row of B to C, that is, copy the 5-th row of A to the 3-rd row of A.
B.row(5).copyTo(C);
// now let A and D share the data; after that the modified version of A is still referenced by B and C.
A = D;
// now make B an empty matrix (which references no memory buffers), but the modified version of A will still be
referenced by C, despite that C is just a single row of the original A
B.release();
// finally, make a full copy of C. As a result, the big modified matrix will be deallocated, since it is not referenced by
anyone
C = C.clone();

• Automatic allocation of the output data
– if a function has one or more input arrays (cv::Mat) and some output arrays,
the output arrays are automatically allocated or reallocated.
– The size and type of the output arrays are determined from the size and type of
input arrays.

• Automatic allocation of the output data
Example:
#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"
using namespace cv;
int main(int, char**)
{
VideoCapture cap(0);
if(!cap.isOpened()) return -1;
Mat frame, edges;
namedWindow("edges", WINDOW_AUTOSIZE);
for(;;)
{
cap >> frame;
cvtColor(frame, edges, COLOR_BGR2GRAY);
GaussianBlur(edges, edges, Size(7,7), 1.5, 1.5);
Canny(edges, edges, 0, 30, 3);
imshow("edges", edges);
if(waitKey(30) >= 0) break;
}
return 0;
}
• InputArray and OutputArray
– Many OpenCV functions process dense 2-dimensional or multi-dimensional
numerical arrays. Usually, such functions take cppMat as parameters, but in
some cases it's more convenient to use std::vector<> (for a point set, for
example) or cv::Matx<> (for 3x3 homography matrix and such).

• Fixed Pixel Types, Limited use of templates
– Templates is a great feature of C++ that enables implementation of very
powerful, efficient and yet safe data structures and algorithms.
– However, the extensive use of templates may dramatically increase compilation
time and code size; difficult to separate an interface and implementation when
templates are used exclusively.
– OpenCV implementation use of templates is limited

• There is a limited fixed set of primitive data types the library can operate on. Array elements
should have one of the following types:
– 8-bit unsigned integer (uchar)
– 8-bit signed integer (schar)
– 16-bit unsigned integer (ushort)
– 16-bit signed integer (short)
– 32-bit signed integer (int)
– 32-bit floating-point number (float)
– 64-bit floating-point number (double)
– a tuple of several elements where all elements have the same type as one of the above.
– Multi-channel arrays, whose elements are such tuples
– The single-channel arrays, whose elements are scalar values.
Max possible number of channels is defined CV_CN_MAX (=512).





C1 Introduction

Uploaded by

Copyright:

Available Formats

You might also like

C1 Introduction

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

C1 Introduction

Uploaded by

Copyright:

Available Formats

Faculty of Information Technology

3 Image processing stages

4 Components of image processing

ISLab- Intelligent Systems Laboratory 2

ISLab- Intelligent Systems Laboratory 3

ISLab- Intelligent Systems Laboratory 4

ISLab- Intelligent Systems Laboratory 5

ISLab- Intelligent Systems Laboratory 6

Early digital image

Early 15 tone digital

A picture of the moon taken

ISLab- Intelligent Systems Laboratory 9

Typical head slice CAT image

ISLab- Intelligent Systems Laboratory 10

ISLab- Intelligent Systems Laboratory 11

3 Image processing stages

4 Components of image processing

ISLab- Intelligent Systems Laboratory 12

ISLab- Intelligent Systems Laboratory 13

Pixel location Consider the following

ISLab- Intelligent Systems Laboratory 14

ISLab- Intelligent Systems Laboratory 15

ISLab- Intelligent Systems Laboratory 16

(a) Chest X-ray.

X-ray images from the space

ISLab- Intelligent Systems Laboratory 18

ISLab- Intelligent Systems Laboratory 19

ISLab- Intelligent Systems Laboratory 20

infrared ("thermal") image Snake around the arm

Messier 51 in ultraviolet (GALEX), visible (DSS), and near

ISLab- Intelligent Systems Laboratory 21

Synthetic Aperture Radar

ISLab- Intelligent Systems Laboratory 22

MRI image slices from the brain

An example showing Imaging in all of the bands

ISLab- Intelligent Systems Laboratory 24

Ultrasonic baby image Ultrasound image

Low Level Process Mid Level Process High Level Process

Examples: Noise Examples: Object Examples: Scene

ISLab- Intelligent Systems Laboratory 26

ISLab- Intelligent Systems Laboratory 27

3 Image processing stages

4 Components of image processing

ISLab- Intelligent Systems Laboratory 28

ISLab- Intelligent Systems Laboratory 29

ISLab- Intelligent Systems Laboratory 30

ISLab- Intelligent Systems Laboratory 31

ISLab- Intelligent Systems Laboratory 32

ISLab- Intelligent Systems Laboratory 33

ISLab- Intelligent Systems Laboratory 34

ISLab- Intelligent Systems Laboratory 35

ISLab- Intelligent Systems Laboratory 36

ISLab- Intelligent Systems Laboratory 37

ISLab- Intelligent Systems Laboratory 38

3 Image processing stages

4 Components of image processing

ISLab- Intelligent Systems Laboratory 39

ISLab- Intelligent Systems Laboratory 40