Distributed Source Coding in Wireless Sensor Networks

Guogang Hua and Chang Wen Chen

Wireless Center of Excellence
Florida Institute of Technology, Melbourne, FL32901
ghua@fit.edu, cchen@fit.edu

Abstract
This paper offers a practical distributed data
compression in wireless sensor networks based on
convolutional code and turbo code. An improved
Viterbi algorithm for distributed source coding (VADSC) is proposed to take the advantage of the known
parity bits at the decoder. When the algorithm is
applied to recursive systematic convolutional (RSC)
and Turbo codes, it can decrease both the decoding
error probability and the computation complexity. Also
a scheme of applying distributed source coding to
wireless sensor networks is proposed to ensure
receiving the data correctly as well as reducing the
energy consumption in the networks.

1. Introduction
In recent years, sensor networks attract more and
more research attention. Networked microsensors
technology is a key technology for the future. In
September 1999 [1], Business Week heralded it as one
of the 21 most important technologies for the 21st
century. The MIT Technology Review ranked wireless
sensor networks that consist of many tiny, low-power
and cheap wireless sensors [2] as the number one
emerging technology.
Wireless sensor networks consist of batteryoperated sensing devices with computing, data
processing and communicating components. Since in
many cases, battery recharging is very difficult or
impossible and the life of the networks is expected as
long as to several years, the primary performance
objective of wireless sensor networks is energy
conservation. Most of research so far has focused on
issues such as energy efficient MAC and routing
protocols, a large number of protocols [3-7] have been
proposed in this category.
However, in wireless sensor networks, multiple
nodes collectively perform the sensing task and

communicate the sensed results. In many cases, sensor

nodes are densely deployed and sensing the same kind
of events, so the information from different sensor
nodes is greatly correlated, which implies that if the
data can be compressed in networks by using the
correlation, few data will be sent and received in
networks, which can reduce the energy consumption
greatly. Actually, most wireless sensor networks are
application-orientated, in which multiple nodes
collectively perform the sensing task and communicate
the sensed results to the sink. It does not have to send
every bit in network (sensed or received for
forwarding) with fidelity. Instead, it only needs to
gather its interested information. This leads to another
kind of research in sensor networks, information
aggregation. This kind of research has just started
recently [2, 8, 9].
Considering a wireless sensor network in which
many sensor nodes sense a common event
independently, and send their sensed data (readings) to
a base station (BS) to do information processing. Since
their readings usually are highly correlated, that is to
say much redundancy exists in their readings. It is
straightforward that they can avoid send many
redundant information to BS if the sensor nodes can
communicate each other. However, in some cases, it is
difficult for all sensors to communicate each other; in
the cases that sensors can communicate each other, the
communication among sensor nodes consumes energy,
which usually is the most significant issue in designing
wireless sensor networks. So here comes the idea: can
the redundancy be reduced without the direct
communication among sensor nodes in the networks?
The answer is yes after Slepian and Wolf proposed the
famous Slepian-Wolf coding theorem [10] in 1973.
This paper is the theoretical base of distributed
compression.
Lets look at the communication system shown as
Fig. 1. X and Y are correlated information sources and
Y is usually called side information. In the encoder
end, a switch K is set. When K switches to on, it means

Proceedings of the 2nd Int'l Conf. on Quality of Service in Heterogeneous Wired/Wireless Networks (QShine05)
0-7695-2423-0/05 $20.00 2005

IEEE

the encoder can access to Y , otherwise,

when K switches to off, the encoder can not access to
Y . Decoder always can access to Y . It is well known
that if encoder can access to Y , X can be encoded at a
rate of H ( X | Y ) , where
(1)
H ( X | Y )  P ( y ) P ( x | y ) log P ( x | y )

H ( X | Y ) is often interpreted as the uncertainty

remaining in the random variable X given the
observation of Y [8].

X Decoder

X Encoder

K
Y
Fig.1 A Communication System with side information

In 1973, Slepian and Wolf presented a surprising

result [10] which states that if X and Y are two
correlated discrete alphabet random variables, X can
be encoded without access to Y at the encoder with the
same rate without losing any compression performance
with the case that X is encoded with access to Y . It
seems strange; here is a simple example to show how it
works.
Suppose X and Y are correlated integer number
from 0 to 7 such that the difference
between X and Y are no greater than 1. Without
compression, three bits are needed to encode X . But if
the decoder can access to Y , then the encoder can
group the eight numbers into four groups (co-sets):
Coset-1 {0, 4} Coset-2 {1, 5}
Coset-3 {2, 6} Coset-4 {3, 7 }
Without access to Y , X only needs to send the coset index using two bit to the decoder, for example,
coset-1 was set to decoder. The decoder knows Y , say
1, and then the decoder can know that the received
X must be 0, since it is from coset-1, if it is 4, the
difference between X and Y will be 3, which is greater
than 1.
The above results were established only for discrete
random variables. In 1976, Wyner and Ziv extended
the results of [10] to lossy distributed compression by
proving that under certain conditions [11], there are no
performance degradations for lossy compression with
side information available only at the decoder as
compared to lossy compression with side information

available at both the encoder and decoder. The results

established by [10] and [11] are only theoretical results,
Pradhan and Ramchandran [12], Aaron and Girod [13]
prescribed practical constructions for distributed
compression in an attempt to achieve the bounds
predicted by [10] and [11].
Distributed compression has its inborn advantages
in compressing data in wireless sensor networks. First,
distributed compression does not need to exchange
data among sensors when compressing their data,
which reduces the data flow in networks. Second, with
distributed compression, the data compression tasks
are distributed in the whole networks instead of some
particular nodes, which can prolong the average life of
the network. Otherwise, if data coding is centralized on
few nodes, those nodes may deplete their energy more
quickly than others. So we can adopt distributed
compression in data aggregation in wireless sensor
networks to prolong the life of the whole network.
In this paper, we proposed an improved Viterbi
algorithm for distributed source coding (VA-DSC) to
take the advantage of the known parity bits at the
decoder. When the algorithm is applied to recursive
systematic convolutional (RSC) and Turbo codes, it
can decrease both the decoding error probability and
the computation complexity. And a scheme of
applying distributed source coding to wireless sensor
networks was proposed. In our scheme, the quantized
data is encoded by a recursive systematic
convolutional (RSC) coder or turbo code. Only the
parity bits are transmitted to data gathering node,
piggybacked by a checksum of the original data. After
decoded with side information, the checksum of
decoded data is calculated and compared with the
received checksum. If the two checksums are the same,
we assume the information is decoded correctly,
otherwise, data gathering node requests the node to
retransmit its unencoded data, which ensures receiving
the data correctly as well as reducing the energy
consumption in the networks.
The rest of this paper is organized as follows.
Section 2 reviews two practical constructions of
distributed source coding, followed by the improved
Viterbi algorithm for distributed source coding (VADSC) in section 3. Section 4 presents the distributed
source coding in chain-type sensor network and
Section 5 presents the simulation results. Finally,
Section 6 gives the conclusion of the paper.

2. Practical constructions of distributed

coding review

Proceedings of the 2nd Int'l Conf. on Quality of Service in Heterogeneous Wired/Wireless Networks (QShine05)
0-7695-2423-0/05 $20.00 2005

IEEE

So far, most practical constructions of distributed

coding are based on channel coding technologies.
Error control code has the ability to correct transmitted
signal corrupted by channel noise. Since side
information Y is highly correlated with X , so we can
view the side information Y as an output of X being
transmitted through a channel and corrupted by the
channel. Here our task is to correct the corrupted Y to
get X . Fig. 2 gives an example of applying
convolutional code to distributed source coding. The
encoder is a N /( N  1) recursive systematic
convolutional (RSC) encoder [13]. Systematic
convolutional code means that the output of the
encoder includes exact the input bits and some parity
bits, for example, for a 2/3 systematic convolutional
encoder as shown in Fig. 3, if the input is 00, the
output should be 000 or 001, the first two bits are
exactly the input, and the last bit is parity bit. In this
case, the system will only transmit the parity bits, so
the compression ratio is 1 / N . In the decoder, the
corresponding bits from the side information Y are
inserted into the received parity bits to reconstruct the
corrupted version of the encoder output. The Viterbi
Algorithm is used to decode reconstructed bit stream to
get X , which is an estimation of X .

X
M bits

N/(N+1)
RSC
Encoder

X
M bits

Viterbi
Decoder

Parity
M/N bits
Y
M bits

Fig.2 Distributed coding using convolutional code

X0

Systematic bits

X1
D

D
Parity bit

Fig. 3 A 2/3 recursive systematic convolutional (RSC) encoder

Another kind of error control code applied to

distributed source coding is turbo code. Fig. 4 gives
the implementation of applying turbo code to
distributed source coding [13]. Turbo code consists of
two parallel concatenated recursive systematic
convolutional (RSC) encoders. The two RSC encoders
are separated by an interleaver. Only the parity bits of

the outputs of the two encoders are transmitted. Then

the compression ratio here is 2 / N . At the decoder, the
side information Y is used to reconstruct the channel
outputs. The outputs are fed to the turbo decoder,
which consists of two soft-input soft-output (SISO)
constituent decoders to decode X .

3. Improved viterbi algorithm

distributed source coding

In both convolutional code and turbo code, Viterbi

algorithm is the key algorithm for the decoder. As
described in previous section, in distributed source
coding, the possible corrupted bits are systematic
bits, which are uncertain; but the parity bits are exactly
known in the decoder, they are certain. An improved
Viterbi algorithm for distributed source coding (VADSC) is proposed in this section to take the advantage
of the certainty of the parity bits.
The Viterbi algorithm (VA) performs an efficient
maximum-likelihood (ML) decoding of finite state
signals observed in noise. Fig. 5 gives a trellis diagram
of a 4 states 2/3 RSC. From all the possible trellis
paths starting form state S0 and end at state S0, the
Viterbi algorithm find a path with minimum path cost
as the survival path.
For the RSC encoder shown as Fig. 3, there are 2
bits input each time, so from any states, the trellis
diagram has 4 (22) possible state transitions at next
time. An example of state transition in traditional VA
is shown as Fig. 6. As we all know, in distributed
source coding, channel coding methods are applied to
source coding, where the parity bits at the decoder is
exactly the same bits at the encoder, so in the decoder
of distributed source coding, we can take advantage of
the known parity bits. Fig. 7 shows an example of state
transition of the 2/3 RSC coder if the parity bits are
known. From state S0, if the parity bit is known as 1,
then the next state can not be state S0 or S4, because if
the next state is S0 or S4, the parity bit should be 0, so
the next state only can be S1 or S2. The same, from
state S3, if the parity bit is known as 0, the next state
only can be S4 or S7.
There are two benefits to take advantage of known
parity bits at decoder in distributed source coding. First,
the bit error probability of the decoded message will be
decreased. In VA-DSC, some of paths are gotten rid of,
which reduces the possibility of choosing the wrong
survival path. An extreme example is 1/2 RSC, if the
parity bits are known in decoder, then no matter how
many systematic bits are corrupted, the original
sequence can be decoded. But unfortunately, if we use
1/2 RSC in distributed coding, there is no compression

Proceedings of the 2nd Int'l Conf. on Quality of Service in Heterogeneous Wired/Wireless Networks (QShine05)
0-7695-2423-0/05 $20.00 2005

IEEE

for

Input X
M bits
N/(N+1)
RSC
encoder

M bits
Reconstruntion

M/N bits

SISO Decoder

Interleave

M bits
N/(N+1)
RSC
encoder

Interleav

DeInterleave

Decision

M bits
Reconstruntion

M/N bits

SISO Decoder

Y
M bits
Fig. 4 Distributed coding using turbo code

S0
S1

S2

S3
Fig 5. Trellis diagram of a 4 states 2/3 RSC

S0

S0

S1

S1

S2

S2

S3

S3

S4

S4

S5

S5

S6

S6

S7

S7

Fig. 6 State transition of the 2/3

RSC coder

at all. Here is a simple example shows that VA-DSC

has better performance than traditional VA. Suppose
the original message sequence is
{00 01 10 11 11 10}
After encoded with the RSC shown in Fig. 3, the
output becomes
{000 011 100 111 110 101}

Fig. 7 State transition of the 2/3 RSC

coder if parity bits are known

The parity bits {010101} are transmitted, at decoder,

suppose the side information is
{00 10 10 11 11 10}
The bold indicates the difference between encoded
information and side information. After reconstruction,
the sequence {000 101 100 111 110 101} is fed to

Proceedings of the 2nd Int'l Conf. on Quality of Service in Heterogeneous Wired/Wireless Networks (QShine05)
0-7695-2423-0/05 $20.00 2005

IEEE

decoder. If traditional VA is used, the decoded

sequence is
{00 10 10 11 01 10}
There are 3 error bits as shown in bold. But if VADSC is used, the exactly original message sequence
can be decoded.
Second, computation complexity will be reduced.
As we can see in Fig. 6 and Fig. 7, in traditional VA,
from each state, there are four possible transitions, but
in VA-DSC, the possible transitions are reduced to two.
So in forward process, the computation of VA-DSC is
only half of that of traditional VA. The trace back
process is the same in both algorithms, but in VA,
most of computation is spent in forward process. So
the computation complexity of VA-DSC is greatly
reduced.

4. Distributed source coding in chain-type

wireless sensor network
Chain-type wireless sensor networks are large scale
wireless sensor networks characterized by the
elongated chain-type network topology and sparsely
distributed sensor nodes. Such applications include
water quality and resource monitoring along some
major rivers that may extend tens to hundreds of miles
in length, marine habitat monitoring along coastal lines,
traffic and transportation monitoring along major
highways, etc. A generic 3-tier hierarchical
architecture was proposed as shown in figure 8. Tier-1
is common sensor nodes (SNs) which are grouped into
clusters at different strategic sites along an elongated
chain-type area. Each cluster has a cluster head node
(CHN). CHNs form the tier-2. Tier-3 is base stations
(BSs). SNs perform data sensing task and report to
local CHNs. CHNs aggregate the data streams from the
related SNs then forward to sink nodes BS. The
communication distance within a cluster may range
from a few dozen to a few hundred meters, while that
between clusters may range from a few hundred to a
few thousand meters depending on the applications.
Such hierarchical architecture can be expanded to
include more tiers according to practical needs. Since
the distances between clusters are relatively large, the
data form different clusters may not have much
correlation. So the data compression mainly focuses on
intra-cluster.
A cluster from a Chain-type Sensor network is
showed as Fig. 9, where node 0 is cluster head node
(CHN), node 1N are sensor nodes (SNs). Suppose
the information from the adjacent nodes and only the
information from the adjacent nodes are correlated.

That is to say node 0 and node 1 are correlated, but

node 0 and node 2 are not correlated.
BS

SN

CHN

Fig.8 The architecture of chain-type wireless sensor network

According to Slepian-Wolf coding theory, if X , Y are

two correlated sources, even though in the encoder end,
X can not access to Y , X can be encoded with a rate
of R X
H ( X | Y )  H X , where H X ! 0 and is
very small. In the decode end, with the presence of
Y as side information, X can be decoded with
arbitrarily small error probability.

IEEE

Fig. 9 A cluster from chain-type sensor networks

In the chain-type sensor network showed above, the

CHN (node 0) has its own sensed information, say X 0 ,
the information X 1 sensed by node 1 is correlated
to X 0 , so the information X 1 can be encoded at a rate of

R X 1 H ( X 1 | X 0 )  H X 1 at node 1 and sent to node 0.

At node 0, with X 0 as side information X 1 can be
decoded. With the same idea, information sensed by
node 2, X 2 can be encoded at a rate

R X 2 H ( X 2 | X 1 )  H X 2 and sent to node 0 through

node 1. At node 0, with the decoded X 1 as side
information, X 2 can be decoded. Therefore, all nodes

n(n 1,2,  , N ) can encode their information at a

rate of R Xn H ( X n | X n 1 )  H Xn and send them to
CHN. At CHN, with X n1 as side information X n can
be decoded. In this way, the amount of energy
consumption for processing and transmitting in CHNs
is reduced.
Above is theoretical analysis, based on which, the
nodes except the CHNs can encode their information at
a rate of R Xn H ( X n | X n 1 )  H Xn . In practical, its
to make the rate close to the theoretical bound. Here
RSC and Turbo code are used to construct practical

Proceedings of the 2nd Int'l Conf. on Quality of Service in Heterogeneous Wired/Wireless Networks (QShine05)
0-7695-2423-0/05 $20.00 2005

encoding and decoding scheme, and simulation results

are given.

5. Simulation results
To test our proposed VA-DSC algorithm, we
simulate both the decoding bit error rate and energy
efficiency in chain-type wireless sensor networks.
Both RSC and Turbo codes are used, for RSC, we use
8-state 2/3 recursive systematic convolutional encoder,
for the Turbo code, two 8-state 4/5 RSC encoders are
used as the constituent encoder.

the correlations between information of adjacent nodes.

The ration of variance of X and VZ is called
Correlation-SNR (CSNR) in [13], so
1
(3)
CSNR 10 log10 ( 2 )

5.1. Decoding bit error rate

For the decoding bit error rate, binary sequences are
used. we use two binary sequences as X and Y , which
have
symmetric
dependencies,
that
is
and
so
P( X Y ) 1  p
P( X z Y ) p ,

H(X |Y)

 p log 2 p  (1  p) log 2 (1  p) . For the

RSC, 10 4 sequence length L 1000 binary sequences

are tested; for Turbo code, 10 3 sequence length
L 10000 binary sequences are tested. The simulation
results of the decoding bit error rate (BER) are shown
as figure 10. From the results we can see when the
correlation is small, the performance of VA and VADSC are similar, for RSC, the VA-DSC is a little better
than VA, for Turbo code, the VA is a little better than
VA-DSC. And RSC performs better than Turbo code
when the correlation is small; this is because the rate of
turbo code is higher (4/5). When the correlation
increases, both RSC and Turbo codes with VA-DSC
algorithm perform much better than that with VA
algorithm. And the BER of Turbo code has a very
steep decrease, which conforms to the result of [13].

5.2. Energy efficiency in chain-type wireless

sensor networks
In our simulation system, the N in Fig. 9 is set to 9;
that is to say, including the CHN, there are total 10
nodes in each cluster. Each node
N i (i 0,1,  9) generates a sequence of i.i.d Gaussian
random variables X i 1 X i 2 , X i L per second, here L is
set to 400. To make the data correlated, we set X 0 to
be the zero mean variance one Gaussian random
sequence, for the other nodes

Xi

X i 1  VZ i (i 1,2,9)

(2)

where Z i is another zero mean variance one

V is a parameter to adjust

Gaussian random sequence;

Fig. 10 Decoding Bit Error Rate performance

Before actually encoding and decoding, the sequences

are quantized using 4-level Lloyd-Max scalar quantizer
[13]. Since both SRC and Turbo coding can not
guarantee 100% correct decoding, in our system, an
eight-bit checksum of original quantized information is
added at the end of parity bits. After the CHN receives
i-th nodes data, it use (i-1)-th nodes data (already
decoded successfully) as side information to decode
the i-th nodes information. The checksum of decoded
data is calculated and compared with the received
checksum. If the two checksums are the same, we
assume the information is decoded correctly, otherwise,
CHN request the node to retransmit its unencoded data.
This simulation focus on the total energy
consumption of the networks after distributed source
coding scheme is applied. We use energy model of
Mica motes, the power consumptions of the radio in
transmitting, receiving and sleep modes are 36mW,
14.4mW and 15 PW respectively [14]. Radio
bandwidth is 20kbps. One data gathering round per
minute; and 5000 rounds were run for the simulation.
Fig. 11 gives the energy efficiency of distributed
source coding. The energy consumption per bit is the
result of the total energy consumption of all nodes
divided by the total bits successfully received and
decoded by CHN. It shows that distributed coding (no
matter which method is used) is much more energy
efficient than raw data transmission. Also from the
result, we can find that both RSC and turbo code with
VA-DSC is more energy efficient than that with
traditional VA, that is because with VA-DSC, the error
decoding probability is reduced, so the retransmission
is reduced. The result shows that with VA, turbo code

Proceedings of the 2nd Int'l Conf. on Quality of Service in Heterogeneous Wired/Wireless Networks (QShine05)
0-7695-2423-0/05 $20.00 2005

IEEE

performs worse than RSC, two possible reasons here:

first is the code rate, Turbo code consists of two 4/5
RSC, in RSC code, the code rate is 2/3. When the code
rate increases, the decoding bit error rate increases.
The second reason is due to the interleave length,
Turbo code perform better with larger interleave length.
With VA-DSC, RSC and Turbo perform similarly, for
high C-SNR, turbo code with VA-DSC performs a
little better than RSC with VA-DSC.

[3] K. Sohrabi, J. Gao, V. Ailawadhi, and G.J. Pottie.

Protocols for self-organization of a wireless sensor network.
IEEE Personal Communications, Vol. 7, No. 5, Oct. 2000.
[4] G. Kulkarni, C. Schurgers, and M. Srivastava. Dynamic
link labels for energy efficient MAC headers in wireless
sensor networks. Sensors, 2002. Proceedings of IEEE,
Volume: 2, 12-14 June 2002.
[5] T.V. Dam and K. Langendoen. An adaptive energyefficient MAC protocol for wireless sensor networks. The
First ACM Conference on Embedded Networked Sensor
Systems (Sensys'03), Los Angeles, CA, USA. November,
2003.
[6] G. Lu, B. Krishnamachari, and C. Raghavendra. An
adaptive energy-efficient and low-latency MAC for data
gathering in sensor networks. The 4th International
Workshop on Algorithms for Wireless, Mobile, Ad Hoc and
Sensor Networks (WMAN 04), April 2004.
[7] W. Ye, J. Heidemann and D. Estrin, Medium access
control with coordinated adaptive sleeping for wireless
sensor networks, IEEE/ACM Trans. on Networks, Vol. 12,
No. 3, June 2004.

Fig. 11 Energy consumption performance

6. Conclusion
This paper gives a practical distributed source
coding in wireless sensor networks base on
convolutional code and turbo code which ensures
receiving the data correctly as well as reducing the
energy consumption in the networks. And an improved
Viterbi algorithm for distributed source coding (VADSC) is also proposed to take the advantage of the
known parity bits at the decoder. When the algorithm
is used to convolutional and Turbo codes, both the
decoding error probability and computation complexity
are decreased. The proposed methods are applied to
binary sequences and chain-type wireless sensor
networks. The simulation result shows that distributed
source coding in wireless sensor network is more
energy efficient. Also RSC and turbo code with VADSC perform better than RSC and turbo coding with
traditional VA in the simulation.

References
[1] 21 ideas for the 21st century, Business Week, pp. 78
167, Aug. 30,1999.
[2] Z. Xiong, A. Liveris, and S. Cheng, Distributed Source
Coding for Sensor Networks, IEEE Signal Processing
Magazine, Vol.21, Sept. 2004.

[8] J. Chou, D. Petrovic, and K. Ramchandran, A

distributed and adaptive signal processing approach to
reducing energy consumption in sensor networks, Proc.
IEEE INFOCOM, March 2003.
[9] A. Boulis, S. Ganeriwal, M. B. Srivastava,
Aggregation in sensor networks: an energy - accuracy
tradeoff, Elsevier Ad-hoc Networks Journal (special issue
on sensor network protocols and applications), 2003.
[10] D. Slepian and J. K. Wolf, Noiseless encoding of
correlated information sources, IEEE Trans. on Inform.
Theory, vol. IT-19, pp. 471480, July 1973.
[11] A. D. Wyner and J. Ziv, The rate-distortion function
for source coding with side information at the decoder,
IEEE Trans. on Inform. Theory, vol. IT-22, pp. 110,
January 1976.
[12] S. S. Pradhan and K. Ramchandran, Distributed source
coding using syndromes: Design and construction,
Proceedings of the Data Compression Conference (DCC),
March 1999.
[13] A. Aaron and B. Girod, "Compression with side
information using turbo codes," Proc. IEEE Data
Compression Conference, DCC-2002, Snowbird, UT, April
2002.
[14] RF Monolithics Inc., http://www.rfm.com/, ASH
Transceiver TR3000 Data Sheet.

Proceedings of the 2nd Int'l Conf. on Quality of Service in Heterogeneous Wired/Wireless Networks (QShine05)
0-7695-2423-0/05 $20.00 2005

IEEE