MEAN-GAIN-SHAPE VECTOR QUANTIZATION
Karen L . Oehler Robert M. Gray
Information Systems Laboratory, Department of Electrical Engineering
Stanford University, Stanford, CA 94305-4055 USA
ABSTRACT
Mean-gain-shape (MGS) is a product code formulation
of vector quantization (VQ) which allows flexible bit al-
location between the mean, gain, and shape features of a
vector. Product codes provide memory and complexity ad-
vantages over non-product VQ. This makes the use of larger
vectors more feasible, and increasing vector dimension im-
proves compression efficiency. T h e product code presented
here obtains the minimum distortion reproduction vector
by successive encoding in each of the three codebooks. We
use pruned tree-structured vector quantizers (PTSVQ) to
provide variable rate codes a t low encoding complexity. Si-
multaneous pruning of the three codebooks provides opti-
mal bit allocation. Prediction and concatenation is used to
take advantage of interblock correlation. The results com-
pare favorably with other tree-structured VQ methods.
1. I N T R O D U C T I O N
A variety of VQ algorithms have been proposed and stud-
ied for image compression [I]. A properly designed product
code structure can offer several advantages over unstruc-
tured VQ including: lower memory requirements, better
representation of important features of the input vectors
and more robust performance. Typical product codebooks
require exhaustive search t o determine the best combina-
tion of component codewords. T h e MGS algorithm, how-
ever, can be implemented so that the minimum distortion
representation is found by successive encoding in each of
the three codebooks. This algorithm works in the spatial
domain requiring no computationally intensive transforms
or filtering.
Given a k-dimensional vector X = (ZI,2 2 , . . . ~ k we)
compute the vector mean m x , gain u x , and shape 8x as
k
1 1
mx = - Z, = -XT1 where 1 = (1, 1 . . .
k k
1=1
k
These features can be combined to the original vet-
tor: X = dxsx + m x l ,where we require that the gain u x
is nonnegative whenever expressing in this form, Note
that the shape vector has mean and unit gain. we
‘This work was SUDDOrtedbv the National ScienceFoundation
under Grant MIP-90i6974. -
V-241
develop a codebook for each feature producing a product
codebook, & ,, x &, x € 8 .
MGS was first proposed for image compression applica-
tions by Murakami et al. [2], but they did not describe the
detailed structure of the code design and the optimality
properties of the sequential encoding techniques given here.
Previous work in MGS encoding examined the same fea-
tures as defined above, but coded the mean and gain fea-
tures noiselessly and coded the shape feature with a full-
search vector quantizer [2, 31. Sitaram and Huang demon-
strate how product codes, including MGS, save memory
using a balanced 16-ary tree-structured vector quantizer
(TSVQ) to encode shape features, but again do not inves-
tigate the bit allocation between the three features [4].
Here, variable rate PTSVQ are used for all three fea-
ture codebooks, which provides low complexity encoding,
further compression of the mean and gain features, and a
natural method of bit allocation. T h e optimum encoding
algorithm is shown to decompose naturally into a sequence
of separate quantization steps as in the gainlshape VQ case
[SI. T h e algorithms for codebook generation are extended
to the tree-structured case.
In encoding, we wish to minimize the quantizer distortion
d ( X , X ) which is defined as simple mean squared error,
IIX - Xl12:
d ( X ,X) = IIX - [ui+ T h l j 1 1 2
= (IXII2 - 2 u Y i - 2&(XT1)
+ b’llill2 + 2 8 i n i T 1 + Th2k
If the shape codewords in €8 are designed t o be consistent
~ with the shape training vectors so that 11&112 = 1 and
s T l = 0 we have:
-
4x9 X)
= ~ I X I I ~ - 2UxTi- 2mmxk + u2 + h’k
---
Shape
- ux’ + 2 - 2&xTir + l i ( ~ h - m x (1)
)~
No Effect Gain Mean
Examination of this distortion equation shows that given
the mean, gain, and shape codebooks, the Optimal repro-
duction choice can be obtained by encoding each feature
successively. T h e variance U; of the input vector X is
obviously independent of the codewords chosen. T h e best
mean codeword is independent of the choices of gain and
shape. Additionally, the best gain and shape cidewords
0-7803-0946-4/93$3.00 0 1993 IEEE
can be found by first determining the best shape codeword
and then using the result t o determine the best gain code-
word. Not all product codes have this characteristic; other
structures may require an exhaustive search of the various
codeword combinations t o determine the optimal choice of
codewords. Note that the shape codeword is selected by
examining the inner product of the input vector and the
codeword candidates. There is no need to compute the in-
put vector’s gain or normalized shape. For details on the
gain/shape interaction see [5].
2. CODEBOOK GENERATION
An unbalanced tree-structured vector quantizer is greedily
grown for each feature [6]. This provides a variable rate
code which can be optimally pruned to provide a set of
nested codebooks over a range of bit rates [7].
T h e distortion term in (1) can be rewritten t o indicate
the effect of each feature:
d ( X ,X) +
= d m ( m x ,i“) d s ( X ,8 ) + d,(XTB,
with
dm(mx,7jL) = k(riz-mx)’
d e ( X ,8) = - ([XT8]+)’
c7:
d , ( X T i , b) = b2 - 26XTB + ([X%]+)’
where [.It represents the larger o f . and 0. T h e mean dis-
tortion, d m ( m x , h ) , is simply mean squared error. T h e
optimal shape codeword, 8 , minimizes the shape distortion,
d e ( X , 8 ) ,regardless of the gain codeword. Once ii is c h e
sen, the ideal gain is c7* = [XTii]+and the optimal gain
codeword minimizes the gain distortion, d , ( X T 8 , &). The
centroids used in codebook generation are derived t o mini-
mize distortion for each feature.
Although the optimal encoding process is sequential, the
optimal codebook generation process is not sequential. Be-
cause the distortion term for the mean is completely in-
dependent of the shape and gain, the mean codebook can
be grown independently without loss of optimality. In con-
trast, the gain and shape codebooks d o depend upon each
other and joint optimization of the codebooks is required
for optimal codebook design. However, Sabin and Gray
demonstrate that independently optimized algorithms give
results very close to jointly optimized algorithms a t reduced
complexity and design time [5]. Hence, we adopt their in-
dependently optimized algorithm to the tree growing case.
T h e mean codebook is constructed using a training set of
vector means. T h e shape codebook is constructed using a
training set of mean-removed vectors. Once the shape code-
book is constructed, it is used to encode the training vectors
providing the d a t a for the gain codebook to be trained.
We grow a large TSVQ for the shape features, and then
optimally prune to achieve the desired bit rate [7]. T h e
tree-growing algorithm is a greedy algorithm, which only
optimizes for the current split. Once grown the tree is opti-
mally pruned by repeatedly removing branches (either ter-
minal nodes or internal nodes with descendants) of the tree.
T h e pruning algorithm removes the branch which minimizes
the increase in average distortion per decrease in bit rate.
T h e resulting nested subtrees provide optimal TSVQs a t
various bit rates (given the initial tree.)
Because the shape and gain codebooks are interdepen-
dent, we need t o grow a unique gain TSVQ for each shape
subtree we wish to use. However, we found experimen-
tally that developing and pruning a single gain TSVQ from
the large shape TSVQ, provides gain coding equivalent t o
constructing unique gain codebooks for each shape bit rate.
Hence, training a single gain codebook using the large shape
TSVQ is satisfactory.
3. BIT ALLOCATION AND JOINT PRUNING
Since encoding requires three codebooks, we have to decide
how many bits we should use (on average) to describe the
mean, the gain, and the shape. One alternative is t o o p
timally prune each of the three codebooks and try various
combinations of the resulting subtrees. Better bit alloca-
tion can be obtained by jointly pruning the three codebook
trees simultaneously. Figure 1 demonstrates how the si-
multaneously pruned codebooks outperform various combi-
nations of individually pruned codebooks on the training
set. Given sufficient experimentation, combined individ-
ual pruning can eventually improve to the simultaneously
pruned performance, but simultaneous pruning provides a
b) quick
.~ path to the best bit allocation.
Joint pruning is a straightforward extension of previous
TSVQ pruning [7]where all three feature trees are consid-
ered and branches are removed from each. This is possible
because the overall distortion is simply the sum of each fea-
ture distortion and the quantizer functions of the the three
TSVQs are convex. For more details see Riskin [a] where
a similar problem involving optimal bit allocation pruning
for Classified VQ is considered. While the pruning here
minimizes the overall distortion, other criteria such as psy-
chovisual knowledge could easily be incorporated during the
pruning step. For instance, if the shape distortion was con-
sidered to be more noticeable than mean or gain distortion
because of the sensitivity of the human eye t o edges, the
pruning algorithm could weight the shape distortion more
heavily resulting in codes with more bits allocated to shape
and fewer bits allocated to mean and gain.
4. PREDICTION
T h e means of adjacent blocks are highly correlated. Hence,
one can use a simple linear predictor on the mean, predict-
ing a block’s mean by averaging the quantized means of the
nearest upper and leftmost blocks. T h e resulting predic-
tion residual is encoded. T h e decoder adds the quantized
residual t o the predicted value. T h e quantized means are
available to both the encoder and decoder so that no side
information is required. Such prediction improves the en-
coded image quality with only a small increase in complex-
ity. While adjacent gains are correlated to a lesser degree,
their nonnegative nature and more complicated distortion
term make prediction more complex and it is not considered
here.
5. CODING MEAN AND GAIN AS VECTORS
The scalar mean and gain often require a disproportion-
ate number of bits for quality representation. Both the
mean and gain show significant degree of correlation be-
tween neighboring blocks. In Section 4, we used prediction
to take advantage of the mean correlation. We can also use
VQ to code both the mean and the gain values more effec-
tively. T h e mean and gain values for each block are con-
catenated together int,o vectors before being encoded with
a PTSVQ. Prediction of the means based on preceding vec-
tors can still be used to reduce redundancy.
V-242
 36
35
34
33
*
6
a
32
31
2
-3 30
G
29
28
27
26
6. OTHER IMPROVEMENTS
We have also implemented two heuristic extensions which
improve quantizer performance. These extensions use the
same codebooks as described before; they alter the encod-
ing process, but not the codebook generation process. The
first extension takes advantage of the fact that quality shape
representation is much less important when the vector gain
is small. We propose a simple alteration in the original al-
gorithm. We encode the shape, then the gain as before.
However, if the quantized gain value (known t o both en-
coder and decoder) falls below a given threshold, then fewer
shape bits are sent. Because a TSVQ is used to encode the
shape, this corresponds t o sending only the first bits of the
path through the tree. This can substantially reduce the
bit rate without a significant increase in distortion.
T h e second extension improves the prediction of means.
Mean distortion has a strong effect on both PSNR and per-
ceived image quality. However, for those vectors which the
prediction is poor (e.g. around edges) the mean distortion
can be quite high. We can use two methods of encoding the
mean, using the usual codebook if the resulting mean dis-
tortion is below a certain threshold and using a non-residual
codebook otherwise (here, we simply sent the actual mean
values as integers). We send one extra bit per concatenated
vector t o tell the decoder which codebook to use. When us-
ing mean vectors from 4 x 4 blocks concatenated into 2 x 2
vectors, this adds one bit per 64 pixels, a small increase in
rate compared t o the improvement in encoded mean qual-
ity. This allows us to use a low rate residual coder most
of the time, but prevents large distortion in those vectors
which are badly predicted.
7. RESULTS
The MGS algorithm was used to encode images from the
USC d a t a base using 4 x 4 pixel blocks. Results were com-
pared t o regular (non-product) predictive PTSVQ also us-
V-243
I
ing 4 x 4 blocks. A training set of 10 512x512 grayscale
images was used. T h e PSNR (dB) obtained by encoding
a test image from outside the training sequence using vari-
ous algorithms are shown in Figure 2. (PSNR is defined
as 10 log,, [(255)2/mean squared error] .) T h e predictive
MGS is comparable to the predictive PTSVQ. At higher
rates (above 0.9 B P P ) the MGS algorithm is clearly su-
perior, due to the robustness of the product code. This
makes MGS particularly useful when high quality encod-
ing is necessary. Encoding the mean and gain as concate-
nated 2 x 2 vectors as described in Section 5 significantly
improves the MGS performance a t lower rates. T h e ex-
tensions mentioned in Section 6, further improve the quan-
tizer performance with little additional complexity. Here,
we show results where only the first 2 bits of the chosen
shape codeword index are transmitted if the gain codeword
is less than 15, and the actual mean values are transmit-
ted instead of the quantized ones if the mean distortion is
above 1000. The test image encoded using MGS with con-
catenation and these extensions is shown in Figure 3 with
overall bit rate of 0.58 bpp and PSNR of 33.18 dB. T h e
mean rate is 0.26 bpp (16.96 bits per concatenated vector),
the gain rate is 0.07 bpp (4.57 bits per concatenated vec-
tor), and the shape rate is 0.24 bpp (3.88 bits per vector).
In this example, 67% of the vectors have sufficiently small
gains so that their truncated indexes are sent, and 11%
of the mean concatenated vectors have sufficient distortion
to warrant alternate rendering. At the same overall rate of
0.58 bpp, the ordinary predictive PTSVQ produces a PSNR
of 32.68 d B and its codebook requires 5 times the memory
needed for all three MGS codebooks combined. T h e MGS
rates could be further reduced using entropy coding t o en-
code the codeword indices.
 361
35 _/*
>-
-

341
_-*
*.-*

_.-.-
_,_.-.-.- -

6
a
& 33 -
a,
2 32- -
.!i
31 - -
c)

Gcm
30 -
.. = Pred PTSVQ
-
29 - - = Pred MGS with Concat. and Exten. -
.- = Pred MGS with Concat.
-- = Red MGS
I
0.2 0.4 0.6 0.8 1 1.2 1.4
O v e r a l l bit rate (BPP)
Figure 2. Comparison of various MGS algorithms with ordinary PTSVQ. Shown are predictive PTSVQ, predictive MGS, predictive
MGS with concatenated mean and gain
- vectors, and predictive MGS with concatenated vectors and the extensions (gain threshold
15 and mean threshold 1000).
8. CONCLUSIONS REFERENCES
We have demonstrated a MGS product code using PTSVQ [l] A. Gersho and R.M. Gray, Vector Quantization and
for the compression of digitized images. Simultaneous prun- Signal Compression, Kluwer Academic Publishers,
ing of the tree-structured codebooks provides optimal bit Boston, 1992.
allocation between the three features. Prediction and other [2] T. Murakami, K. Asai and E. Yamazaki, Vector Quan-
enhancements are implemented to further improve the en- tiser of Video Signals, Electronic Letters, pp. 1005-
coder efficiency. The algorithm produces quantized images 1006, Vol. 18, No. 23, November 1982.
of good quality with low encoding complexity and reduced
memory requirements. [3] H.J. Lee and D.T.L. Lee, A Gain-Shape Vector Quan-
tizer for Image Coding, Proc. lEEE lnt. Conf. Acoust.,
Speech and Signal Proc., Vol. 1, pp. 141-144, Tokyo,
1986.
[4] V.S. Sitaram and C.M. Huang, Efficient Codebooks
for Vector Quantization, Proceedings of the 1992
IEEE Data Compression Conference, p. 396, Snowbird,
Utah, March 1992.
[5] M.J. Sabin and R.M. Gray, Product code vector quan-
tizers for waveform and voice coding, IEEE Trans.
Acoust., Speech and Signal Proc., ASSP-32:474-488,
June 1984.
[6] E.A. Riskin and R.M. Gray, A greedy tree growing al-
gorithm for the design of variable rate vector quan-
tizers, IEEE Trans. Signal Process., pp. 2500-2507,
November 1991.
[7] P.A. Chou, T. Lookabaugh, and R.M Gray, Optimal
pruning with application to tree-structured source cod-
ing and modeling, IEEE Trans. lnform. Theory, pp.
299-315, March 1989.
[8] E.A. Riskin, IEEE Trans. Info Theory, Vol. 37, No. 2,
pp. 400-402, March 1991.

Figure 3. Test image encoded a t 0.58 bpp. PSNR = 33.18 dB.

V-244