EEE6218

Visual Information Processing (VIP)

Dr Charith Abhayaratne
Tel: 25893
Email: c.abhayaratne@sheffield.ac.uk
Office: Portobello C11

Course Delivery:
Lectures: 2 hours/week From Week 3 Semester 1 to Week 8 Semester 2
In Semester 1 Tuesday @ 1 pm & Wednesday @ 3 pm
In Semester 2 Thusrday @ 2pm & Friday @ 4pm

Visual Information Engineering

Production

Visual Content
Capturing

Content

Storage

Image

Compression:
to reduce file
sizes
Metadata (data
about content)
Creation:
e.g.,
Annotation,
Descriptors for
indexing

Usage

Displaying
Analysis

Video
Graphics

Studio Editing

Transmission

Networks

Browsing,
Search
and
retrieval
Personalization

Multimedia adaptations, e.g.,

subtitles, sign-language icons,
transmoding

 Visual Information Processing

Visual content
Images/video/graphics etc.
Visual content engineering
Capturing
Pre-processing
Compression
Content adaptation (change of
format/resolution/quality/mode according to usage
requirements and available resources)
Image/ video quality - enhancement /restoration/fusion

Mainly focus on 3 concepts:

Signal
Data
Information
Signal
Detectable transmitted energy that can be used to carry information.
A time/spatially-varying characteristic of a physical phenomenon, used to convey
information.
Amplitude

Visual content can be regarded as signal

with higher dimensions.
e.g., images -2D, video - 3D
1D Signal Processing concepts can be easily extended to higher dimensions

Time

Data
Representation of
facts,
concepts, or instructions
in a formalized manner suitable for
communication,
interpretation, or processing by
humans or by automatic means.
E.g., any representations such as characters or analogue quantities to which
meaning could be assigned.

Information
The meaning that a human assigns to data
by means of the known conventions used in their representation.
Entropy is a measure of information content.

Information
The meaning that a human assigns to data
by means of the known conventions used in their representation.
Entropy is a measure of information content.

Higher the entropy of a message, the more information it contains.

The entropy is high if the uncertainty is high.

How much information?

Message

Tomorrow is Wednesday.
(Saying this on a Tuesday)

How much
Information?
No information at all

P=1

Very little information

Very high

Fairly high

Slightly low

Very high

Very low

It will rain tomorrow.

Lewis Hamilton wins the F1
title in 2014.
Next Saturdays lottery winning
combination is
4, 8, 15, 16, 23, 42.

What is the
probability?

Information content
From the previous slide we can conclude that the information content is inversely
proportional to the probability of occurrence.
We measure the information content of an event (symbol) using entropy (H).

1
P

1
H = log2
P
= log2 (P)

Bits

Information content in the winning number combination

= -log2(1/13,983,816)
= 23.7 bits

Information content
The total entropy for a collection of symbols:
The sum of the entropy of all symbols
T

t =1

i =1

HTotal = log2 (1/ pt ) = si log2 (1/ pi )

The average entropy For C groups of different symbols in the message

C
C
1 C
si
H AVG = si log2 (1/ pi ) = log2 (1/ pi ) = pi log2 (1/ pi ) Bits per sample
T i =1
i =1 T
i =1

This is called Shannons Entropy formula

Homework: For a binary source find the probabilities of symbols when HAVG is maximum.

Information content
of an image

A picture is worth ten thousand words

Literally means
A picture's meaning can express ten thousand words ?

Image databases requires huge amount of metadata

(keywords) image indexing.

Information content
of an image
Only one colour.
What is the entropy?

16x16 pixels
In total 256 pixels.
Each pixel represent
Each of the 256 colours.
What is the entropy?

Information content
of an image
Gray scale images usually use 256 gray
levels. That means 8 bits per pixel (bpp).

P robability

The Shannons entropy 7.46 bpp

Probability distribution of Lena image.

We have to modify this distribution to one with a
narrow peak and long tails to reduce entropy.
0

50

100
150
Grayscale values

200

250

Information content
of an image

compressing

decoding
5

WT and Q=8 Lena Histogram

x 10

1.8
1.6
1.4

P robability

1.2
1
0.8
0.6
0.4
0.2
0
-30

50

100
150
Grayscale values

200

-20

-10

10

20

30

250

In this course we will discuss on signal processing tools for content engineering and efficient
representation.

EEE 6218 Visual Information Processing

Aims:
Introduce concepts of filtering and signal transform techniques
in image and video processing.
Emphasise the use of these techniques for image and video
processing in modern video applications.
Introduce the concepts of image and video compression.
To provide an understanding of visual quality, techniques for
quality enhancement and evaluation
to provide an understanding of using software tools in designing
and implementing technologies for visual information
engineering.

EEE 6218 Visual Information Processing

Learning Outcomes:
By the end of the unit, students will be able to

demonstrate the understanding of the basic theory on signal

transforms and its application in video/image processing.
demonstrate the understanding of Multi resolution analysis and
its application in video/image processing.
use the common transform techniques for image/video
compression.
Demonstrate the understanding of concepts of visual quality
and video/image processing methodology for enhancing and
evaluation of quality.
design and software implementation of simple algorithms for
visual information processing and performance evaluation

EEE 6218 Visual Information Processing

Course Syllabus: 3 Parts
Filtering and transform concepts:
Filters, Filter banks and wavelets
2D/3D filtering and transforms
Multi-resolution (MR) representation
MR Application on Image and video
Image and video processing:
Digital imaging concepts
Multi-resolution (MR) analysis and MR domain processing
Image and video compression
Motion estimation
Applications: e.g. restoration, enhancement, fusion
Algorithm Evaluation:
Complexity
Picture Quality
Software implementation (MATLAB)

EEE 6218 Visual Information Processing

Reference Books (for background reading only)
Digital image processing
R. Gonzalez & R. Wood (Prentice Hall).
Digital video processing
A. M. Tekalp (Prentice Hall).
Multimedia and Communications Technology
S. Heath (Focal Press).
Wavelets & Subband coding
M. Vetterli & J. Kovacavic.
(available online at http://www.waveletsandsubbandcoding.org/)

EEE 6218 Visual Information Processing

Course delivery
2 hours lecture / week
30 lectures on VIP topics
Printed handouts and/or online lecture notes
Class activities / Demos
Research paper reading

3 lectures on the coursework + Matlab

Tutorials
Problem sheets
3 problem solving classes
Other problem solving during lectures.

EEE 6218 Visual Information Processing

Relevant course materials available in MOLE.
Login via MUSE
Module description / Handouts / lecture notes /
problem sheets + solutions / (Mock exam +
solutions) / revision questions / online tests/
Coursework information/ relevant research papers/
Matlab exercises/ assignments
Discussion forum for the coursework

EEE 6218 Visual Information Processing

Assessment
Continuous assessment (20%)
Course work
An assignment based on problem-based learning on a VIP
related application.

More details later

Submission in Semester 2

EEE 6218 Visual Information Processing

Assessment
End of Semester 2 exam (80%)
In May/June
2 hour exam
Answer 3 questions out of 4.

EEE 6218 Visual Information Processing

Revision: Imaging, Signal processing and Matlab
preliminaries
Homework:
Revise
Discrete time signals
Convolution
Impulse response
Correlation
FFT
Frequency response
Filters (low pass and high pass)