Hyperspectral Imaging:

Techniques, Issues, and

Applications

ANTONY LAM
HUMAN-MACHINE INTERACTION SYSTEMS
ENGINEERING
 Biography

 Name: Antony Lam

 Home Town: Los Angeles
 Education (Graduated Sept. 2010)
 PhD in Computer Science (Computer Vision, Machine
Learning); University of California, Riverside
 Postdoc (Oct. 2010 – June 2014)
 National Institute of Informatics, Tokyo

 Assistant Professor (July 2014-Present)

 Saitama University
 Quiz

 There will be a short online quiz.

 You can do the quiz after the talk.

 Please finish it by Monday.

Link: http://goo.gl/forms/5ONgty6d64
You can also find the link on my
homepage:
http://yankee.cv.ics.saitama-
u.ac.jp/~kunolab/antonylam/
Click!
Click!
Enter
Name,
ID,
E-mail
Remember
to submit!
Light Waves and Color

Light
Prism Splitting White Light into Different Colors
Color Images

Need spectrum over

wavelengths to
described color.
Typical RGB Color Image
(Only 3 Bands)

Why do we only have

three colors in images?
Human Vision is Trichromatic
 Spectral
Distribution
of a Point
Specific Wavelengths can Reveal Characteristics of an Object

Sugar Distribution of Melon Imaged at 676 nm (top) and

RGB Image (bottom)

Taken from Sugiyama (1999)

 Different wavelengths
reveal characteristics of an
object. (E.g. strawberry.)

Bruising
easily
identified
at 1000
nm.

Bruising
not easily
identified
Taken from “Computer Vision Technology for Food Quality Evaluation” Edited by
at 640 nm. Da-Wen Sun
Hyperspectral Imaging Applications

Satellite Imaging
Medical Diagnostics Food Quality Inspection …and more!
How to Capture?

y
How to Capture? Standard Methods (Slow and noisy.)

Prototype introduced
Fastest but in 2014.
expensive (Needs moving of
(May have lower imaging equipment to
resolution.) cover the entire cube.)
 Narrowband
Filter to Only
Let Certain
Wavelengths In
 HySpex (Norsk Elektro Optikk AS)

Collimation
makes light rays
parallel.

Diffraction
Grating Splits
Light Like a
Prism
 Optics Split Light
Make Light Rays into Different Result is
Parallel Spectral Multiple
Channels Band
Images

Limits Field of View

 Combines Ideas of
Camera Obscura
with a Grating

Connects
Wavelength
and Viewing
Directions

Grusche (Applied Optics 2014)

 Combines Ideas of
Camera Obscura
with a Grating

Connects
Wavelength
and Viewing
Directions

Full Spectral Cube

Scanned by Moving the
Camera
Grusche (Applied Optics 2014)
 Problems with Hyperspectral Imaging

 Noise
 Low Resolution
 Slow
 We address these problems.
Denoising: Sample Result
Past Work
Spatial Structure

Improvement of target-detection
algorithms based on adaptive three-
dimensional filtering. Bourennane,
et al., 2011 y
Adaptive linear minimum mean
square error restoration: influence
on hyperspectral detection strategy.
Gaucel, et al. 2008
x
Noise removal from hyperspectral
images by multidimensional
filtering. Lextexierand and
Bourennane. 2008 Denoised
HSI
Past Work
Separable Spatial and Spectral
Domains

Denoising of hyperspectral imagery

using principal component analysis
and wavelet shrinkage. Chen and y
Qian, 2011

Noise reduction of hyperspectral

imagery using hybrid spatial-
spectral derivative-domain wavelet
shrinkage. Othman and Qian, 2006 λ x

Denoised
HSI
Our Approach

Leverage
Statistics λ x

Denoised
HSI
 Why Use the Spectral Domain?

 Real life spectra have less varied patterns than

spatial patches.

6-8 Principal Dozens of

Components Principal
Components
Many Spectra in a Single HSI

λ
Say band image is of There are 90,000
size 300 X 300. spectra.
Overview of Our Method
Spectral Domain Processing

Spatial Domain Processing

Denoising Results
Noisy
Ours
520 nm

Noisy Original Ours

High Resolution Hyperspectral Imaging
Problem: Hyperspectral Camera Spatial Resolution is Low

Combine

High-Res Low-Res
RGB Hyperspectral
High-Res
Hyperspectral Image

Kawakami et al. (CVPR 2011)

High Resolution Hyperspectral Imaging

Input RGB Input (550nm) Estimated (550nm) Input (620nm) Estimated (620nm)

Estimated High Estimated High

Low Resolution Low Resolution
Resolution Band Resolution Band
Band Image Band Image
Image Image

Kawakami et al. (CVPR 2011)

 Different
Lighting
Changes
Color
Appearance
Reflectance L(400 nm)R(400 nm) = 1*0.8 = 0.8

80% Reflectance at
400 nm

1 0.8
Finding R is
important for
analyzing material.
Object Surface
400 nm light

Reflected Light at Wavelength  = L()R()

 Fundus (眼底) Hyperspectral Imaging

 Fundus Hyperspectral Imaging

 For diagnostic purposes.

 Take images of eye over different wavelengths.

Camera

Diagnostic
Hyperspectral
Image
 Drawback of Standard Fundus Photography

Need to change color Using incident lights from fundus

filter. Slow statistics
Recovered Spectral
Reflectance

E.g. only four images required instead of 31

Patient remains still for
in standard methods.
about 31 band images.
Fast Fundus Photography

Advantages

ELS Fast, Less Noise

But…

Capture Images under 4

Incident Lights
Recovered Images
for Medical Diagnosis
Drawbacks

ELS is expensive.
Need to synchronize with camera. +
Informative Sparse Wavelengths

Six Most
Informative
Wavelengths
Statistics
Database of
Fundus
Spectral
Reflectance

Capture Only Under

Six Wavelengths
Estimate
Full
Spectra

Full Hyperspectral
Image (E.g. 31
bands)
Sparse Wavelength Results

560 nm

630 nm

Ground Truth Estimated

Fast Spectral Reflectance Recovery Using DLP Projector

Fast color switch with

fast camera can capture
scene under multiple
lights quickly.

The spectral reflectance

can be recovered for
moving targets.

Taken from Han et al. (ACCV 2012)

Application: Face Recognition
Pose

Expression Lighting
Application: Face Recognition

NIR Spectral Reflectance of Skin:

1. Sense Subsurface Tissue
2. Stable Over Time
3. Nearly Invariant to Face
Orientations and Expressions

Pan et al. (PAMI 2003)

Application: Face Recognition
Application: Remote Sensing

Shaw and Burke (Lincoln Lab Journal 2003)

Application: Remote Sensing

Shaw and Burke (Lincoln Lab Journal 2003)

Application: Remote Sensing

Shaw and Burke (Lincoln Lab Journal 2003)

Application: Security and Defense
Camouflage and Target Identification

http://www.resonon.com/
 Conclusion

 Hyperspectral Imaging Sees More than the Human

Eye
 Various Imaging Techniques Developed and Being
Developed
 Many Applications