Arithmetic Coding

In arithmetic coding, the entire message is mapped to a small interval inside [0,1). The small
interval is then encoded as a binary pattern. Arithmetic coding is based on the fact that we can
have an infinite number of small intervals inside the half-open interval [0,1). Each of these small
intervals can represent one of the possible messages we can make using a finite set of symbols.

Encoding
To encode a message in arithmetic coding, we first need to assign the probability of occurrence
to each symbol. If we have M symbols in the alphabet (including the terminating symbol that we
need for decoding), the probabilities are P1, P2, . . . , PM, in which P1 + P2 + . . . + PM = 1.0.
In each iteration of the loop, we divide the current interval into M subintervals, in which the
length of each subinterval is proportional to the probability of the corresponding symbol. This is
done to uniformly scatter the messages in the interval [0,1). We also preserve the order of the
symbols in the new interval in each iteration.
P(a) = 0.4, P(b) = 0.5, P(*) = 0.1,
let us assume that our set of symbols is S = {A, B, *}, in which the asterisk is the terminating
symbol. We assign probability of occurrence for each symbol as below shows how we find the
interval and the code related to the short message "BBAB*".

We initialize the current interval to [0,1). In each iteration of the loop, we divide the current
interval into three subintervals according to the probability of each symbol occurring. We then
read the first symbol and choose the corresponding subinterval. We then set the current interval
to be the chosen interval. The process is repeated until all symbols are input. After reading each
symbol, the current interval is reduced until it becomes (0.685= 0.101011, 0.690 = 0.101100). We
encode the lower bound, 0.685 in binary, which is approximately (0.1011)2, but we keep only
the fractional part, (1011)2, as the code.

Decoding
Decoding is similar to encoding, but we exit the loop when the terminating symbol is output.
This is the reason we need the terminating symbol in the original message.