TERM PAPER ON

The Compressive Sensing Based on

Biorthogonal Wavelet Basis

Submitted By:
Amrita Mishra 11104163
Manoj C 11104059

Under the Guidance of

Dr. Sumana Gupta
Professor
Department of Electrical Engineering
Indian Institute of Technology Kanpur
ABSTRACT: Compressive Sensing is one of the latest tools for simultaneous sensing and
compression of data. It enables a significant reduction in the sampling and computation costs
for signals having sparse representation in some basis. In this term paper, we use wavelet
transformations such as Haar, db4, db6 and db8 wavelets for implementation of CS. We also
test several wavelet bases from Biorthogonal family. The Error Ratio between the original
coefficient and the reconstructed coefficient, the PSNR of the original image and
reconstructed image, and the Elapsed Time were used as the measurement indexes. Lena
along with CVX and l1-regularised least squares optimizer (in MATLAB) is used in
experimental for obtaining the results. The section Discussion and Results talks about the
observations and inferences in detail.

INTRODUCTION:
COMPRESSIVE SENSING

Conventional approaches to sampling signals or images follow Shannon‘s celebrated

theorem: the sampling rate must be at least twice the maximum frequency present in the
signal (the so-called Nyquist rate). In fact, this principle underlies nearly all signal acquisition
protocols used in consumer audio and visual electronics, medical imaging devices, radio
receivers, and so on. (For some signals, such as images that are not naturally bandlimited, the
sampling rate is dictated not by the Shannon theorem but by the desired temporal or spatial
resolution. However, it is common in such systems to use an antialiasing low-pass filter to
bandlimit the signal before sampling, and so the Shannon theorem plays an implicit role.) In
the field of data conversion, for example, standard analog-to-digital converter (ADC)
technology implements the usual quantized Shannon representation: the signal is uniformly
sampled at or above the Nyquist rate. The theory of compressive sampling, also known as
compressed sensing or CS, a novel sensing/sampling paradigm goes against the common
wisdom in data acquisition. CS theory asserts that one can recover certain signals and images
from far fewer samples or measurements than traditional methods use. To make this possible,
CS relies on two principles:
sparsity, which pertains to the signals of interest, and incoherence, which pertains to the
sensing modality.
 Sparsity expresses the idea that the ―information rate‖ of a continuous time signal
may be much smaller than suggested by its bandwidth, or that a discrete-time signal
depends on a number of degrees of freedom which is comparably much smaller than
its (finite) length. More precisely, CS exploits the fact that many natural signals are
sparse or compressible in the sense that they have concise representations when
expressed in the proper basis Ψ.
 Incoherence extends the duality between time and frequency and expresses the idea
that objects having a sparse representation in Ψ must be spread out in the domain in
which they are acquired, just as a Dirac or a spike in the time domain is spread out in
the frequency domain. Put differently, incoherence says that unlike the signal of
interest, the sampling/sensing waveforms have an extremely dense representation in
Ψ.
 The crucial observation is that one can design efficient sensing or sampling protocols
that capture the useful information content embedded in a sparse signal and condense it into a
small amount of data. These protocols are non-adaptive and simply require correlating the
signal with a small number of fixed waveforms that are incoherent with the sparsifying basis.
What is most remarkable about these sampling protocols is that they allow a sensor to very
efficiently capture the information in a sparse signal without trying to comprehend that
signal. Further, there is a way to use numerical optimization to reconstruct the full-length
signal from the small amount of collected data. In other words, CS is a very simple and
efficient signal acquisition protocol which samples—in a signal independent fashion—at a
low rate and later uses computational power for reconstruction from what appears to be an
incomplete set of measurements.

Mathematical Framework of CS technique

1. Acquire n << N measurements, using a ‗special‘ sampling matrix Φ , by computing
for a signal x

2. Since the dimension of vector of the acquired samples y is ‗substantially‘ smaller than
the dimension of the signal, we obviously obtain some initial compression, which can
be further augmented by applying lossy or lossless compression to the vector y.
3. Similarly to standard transform-based compression techniques, the paradigm of CS is
based on the assumption that the signal x has a sparse representation in some basis
such as wavelets. This means that we assume that there exists a known fixed
transform T, such that from the N (or more) transform coefficients c = Tx , only k < n
coefficients are significant. Working under this ‗sparsity‘ assumption an
approximation to x can be reconstructed from y by ‗sparsity‘ minimization, such as 1 l
minimization

A key assumption in the theory of CS is that the sampling process determined by the matrix
Φand the sparsity transform T are ‗incoherent‘. Roughly speaking, this means that if a signal
has a sparse representation in one, then it must have a dense representation in the other and
visa versa, but a signal cannot have a sparse representation in both.
Figure 1: (a) Sparse wavelet representation of an image. Black- significant coefficient, white
– insignificant coefficient (b) JPEG2000 compressed image based on the sparse
representation of (a)

BIORTHOGONAL WAVELET FAMILIES

The discrete wavelet transform can be represented in matrix form as equation, where Ψ is a
matrix with columns corresponding to othonormal scaling and wavelet basis vectors. The
functions that qualify as orthonormal wavelets, such as Daubechies wavelets, lack desirable
symmetry properties. The Biorthogonal wavelets use two different wavelet bases, ψ(x) and
. One is used for decomposition (analysis) and the other one for reconstruction
(synthesis) i.e

We then have,

for decomposition
and

for reconstruction.
The two scaling functions given in the frequency domain are

and the wavelets are

and

where

The biorthogonal wavelets for the forward two-dimentional transform are given as

For the inverse transform

WAVELET TREE STRUCTURE COMPRESSIVE SENSING

The wavelet coefficients may be represented in terms of a tree structure. To a real

image, most of the significant wavelet coefficients are located in the vicinity of edges.
Wavelets can be regarded as multi-scale local edge detectors. The absolute value of a wavelet
coefficient corresponds to the local strength of the edge. The 3 level wavelet tree structure is
depicted in Figure 2 and the wavelet tree structure of Lena is shown in Figure 3. The
coefficients at the highest of the left in Figure 1 correspond to ―root nodes‖. And the
coefficients at the bottom or right in the figure correspond to ―leaf nodes‖. The top-left block
corresponds to the scaling coefficients which capture the coarse-scale representation of the
image. Each wavelet coefficient has four ―children‖ coefficients at the next level, and it has
the statistical relationship between the ―parent‖ and ―children‖ coefficients. This could be
exploited in the CS inversion model.
 Figure 2: The 3 level wavelet tree structure

Figure 3: The 3 level wavelet tree structure of Lena.

(Two wavelet trees are shown in the figure)

A wavelet coefficient has a small value, then its children coefficients are likely to also
negligible. The statistics of the wavelet coefficients may be represented by the hidden
Markov tree. The structure of the wavelet tree is exploited explicitly. Each wavelet
coefficient is assumed to be drawn from one of two zero-mean Gaussian distributions in
hidden Markov tree. These distributions define the observation statistics for two hidden
states. One of the states is a ―low‖ state, defined by a small Gaussian variance. And the
―high‖ state is defined by a large variance. If a wavelet coefficient is relatively small, it is
more likely to reside in the ―low‖ state. A large wavelet coefficient has a high probability of
coming from the ―high‖state.

We do not get the wavelet coefficients directly in compressive sensing. We only get
the projections of these coefficients. The form of the hidden Markov tree will be used in the
compressive sensing inversion. If a given coefficient is negligible, we can scale its children
coefficients as ―zero‖. The subtrees of these wavelet coefficients may all be set to zero with
a little effect on the reconstruction accuracy.
Experiment:
We performed compressive sampling on the standard Lena image. Due to
computational limitations, we resized the Lena image to 64×64 pixels and performed
compressive sampling on the resized image. The original Lena image used is shown in Figure
4. The experiments were carried out in MATLAB 7.11.0 running on PC with Intel® Core™ i7
2.93 GHz CPU with 4 GB RAM.

Figure 4: Original Lena image

CS was performed using various wavelets – Haar, Daubechies-4 (db4), Daubechies-6

(db6), Daubechies-8 (db8), biorthogonal 2.8 and biorthogonal 3.5 wavelets. We also used
CVX and l1-regularized lease squares solver (l1_ls) for reconstructing the image from the
compressed samples. The performance indicators Peak Signal to Noise Ratio (PSNR), time
taken and the visual appearance of the reconstructed image were used for comparing the
performance of various wavelets and the solvers. We reconstructed the 64×64 image (4096
coefficients) using 750, 1000, 1250, 1500, 1750, 200, 2500, 3000, 3500, 4000 measurements.

Results:
The experimental results obtained by compressing the Lena image using CVX are
shown in Table 1. The performance indicators time taken in seconds, PSNR in dB and the
reconstructed image are tabulated. As expected, the PSNR and the image quality increase as
the number of measurements increases. Among the several wavelets considered, we can
observe that the biorthogonal wavelets give a good reconstruction. The time taken for
reconstruction of biorthogonal 3.5 wavelets is the least compared to other wavelets.

Table 1: Experimental results obtained using CVX.

No. of
haar db4 db6 db8 bior2.8 bior3.5
measurements
Time 97 99 193 189 118 99
PSNR 12.88 13.17 13.11 12.88 13.03 12.88
750
Image

Time 162 178 305 303 200 182

PSNR 13.00 13.18 12.79 12.91 12.65 12.74
1000
Image
 Time 270 279 389 352 300 273
PSNR 13.26 13.18 13.06 13.47 13.12 12.60
1250
Image

Time 383 397 52 529 467 386

PSNR 12.98 13.38 13.42 13.29 13.47 12.81
1500
Image

Time 513 551 714 648 605 529

PSNR 13.46 13.70 13.5 13.52 13.20 13.05
1750
Image

Time 723 722 847 871 733 712

PSNR 14.16 14.22 14.11 14.17 13.34 12.65
2000
Image

Time 1096 1167 1268 1397 1157 1185

PSNR 15.44 16.66 16.10 16.29 13.96 13.28
2500
Image

Time 1583 1695 1972 1981 1806 1844

PSNR 22.85 21.49 20.68 21.39 19.90 16.31
3000
Image

Time 2342 2236 2387 2423 2327 2219

PSNR 30.26 28.09 28.08 28.25 28.92 28.25
3500
Image

Time 2641 2597 2769 2728 2879 2687

PSNR 43.74 43.12 41.99 41.68 44.20 43.50
4000
Image

We also performed the same experiment using l1-regularised least squares solver
(l1_ls) to reconstruct the image from compressed samples. The results corresponding to l1_ls
are tabulated in Table 2.
 Table 2: Experimental results obtained using l1-regularised least squares solver.

No. of
haar db4 db6 db8 bior2.8 bior3.5
measurements
Time 142 178 231 178 168 375
PSNR 14.30 14.27 14.58 14.24 13.69 14.61
750
Image

Time 277 241 351 233 476 635

PSNR 15.23 15.00 15.36 15.29 14.36 15.23
1000
Image

Time 300 326 425 395 483 721

PSNR 15.86 15.78 15.75 15.51 15.69 15.73
1250
Image

Time 388 368 357 462 582 1505

PSNR 16.73 16.98 15.43 17.03 16.96 16.20
1500
Image

Time 469 564 431 447 596.85 1237

PSNR 18.31 18.04 17.70 17.81 17.03 17.24
1750
Image

Time 590 480 641 529 836 981

PSNR 19.78 18.29 18.04 17.46 18.61 16.00
2000
Image

Time 813 631 870 592 1155 24110

PSNR 21.38 20.09 21.33 20.00 20.60 20.18
2500
Image

Time 1103 1097 713 1128 1065 2784

PSNR 26.72 23.13 24.78 24.63 24.29 22.92
3000
Image
 Time 1070 1211 1214 1278 2293 3192
PSNR 32.67 29.84 29.24 27.73 29.68 28.04
3500
Image

Time 962 1024 1023 1035 1293 3523

PSNR 44.87 42.05 41.77 40.57 41.58 40.68
4000
Image

We can observe that l1_s gives better performance in terms of both PSNR and time
taken. We also conclude that the compression performance depends on the solver used to
reconstruct the image from the compressed samples. This is the reason why the performance
observed is different from that given in the paper.

Contribution:
The reconstructed image is not of very good quality when number of measurements is
less. So, we tried denoising the image with by applying Total Variation denoising. Total
variation denoising was applied on images reconstructed using l1_ls only and the results are
shown in Table 3.

Table 3: Experimental results obtained using l1-regularised least squares solver.

No. of
haar db4 db6 db8 bior2.8 bior3.5
measurements
Time 142 178 231 178 168 375
PSNR 15.49 15.33 15.56 15.21 14.55 15.06
750
Image

Time 277 241 351 233 476 635

PSNR 15.72 15.74 16.62 15.82 15.97 15.90
1000
Image

Time 300 326 425 395 483 721

PSNR 17.68 17.04 17.05 17.19 16.72 16.69
1250
Image

Time 388 368 357 462 582 1505

PSNR 17.57 17.62 17.42 17.64 18.00 17.29
1500
Image
 Time 469 564 431 447 596.85 1237
PSNR 19.22 19.16 18.89 18.61 18.01 17.89
1750
Image

Time 590 480 641 529 836 981

PSNR 20.26 19.79 19.62 19.99 19.79 18.79
2000
Image

Time 813 631 870 592 1155 24110

PSNR 21.27 19.82 20.47 20.43 19.79 20.61
2500
Image

Time 1103 1097 713 1128 1065 2784

PSNR 22.44 22.80 22.67 22.72 22.90 22.41
3000
Image

Time 1070 1211 1214 1278 2293 3192

PSNR 23.95 23.58 23.36 23.19 23.69 23.57
3500
Image

Time 962 1024 1023 1035 1293 3523

PSNR 24.13 24.14 24.12 24.11 24.11 24.11
4000
Image

From the results, we can infer that the total variation filtering improves the image
quality and PSNR when number of measurements taken is less. The improvement in PSNR is
around 1-2 dB. When the number of measurements increases, the image reconstructed is
almost similar to the original image. So, the total variation filtering has resulted in blurring of
the image, resulting in a decreased PSNR. Since compression and compressive sampling aim
at reducing the number of measurements taken, number of measurements taken in practice is
very less. So, the PSNR can be increased by using denoising after reconstructing the image.
Conclusion:
In this paper, we test the quality of Haar, db4, db6, db8,bior2.8 and bior3.5 wavelet
basis for the implementation of CS and it is observed that Biorthogonal wavelets given a
better reconstruction. We used both CVX and l1-regularised least squares optimizer (in
MATLAB) to perform the optimisation step in CS and observe that l1-regularised least
squares optimizer gave better performance with respect to both PSNR and time taken. We
observed that the reconstructed image is not of very good quality when number of
measurements is less. So, we tried denoising the image with by applying Total Variation
denoising (results provided only for l1-regularised least squares optimized images) and
observed the PSNR can be increased by using denoising after reconstructing the image. We
also concluded that the type of wavelet basis used in CS depends upon the image to be
compressed. For an image with more details like Lena, the best performance was achieved for
Biorthogonal Wavelets. It can be seen that for a image with fewer details other wavelets like
Daubechies also do a fair job. Even the number of pixels required for proper reconstruction
would be less for an image with fewer details.