DATA COMPRESSION.

Uncompressed graphics, audio & video data require

large storage space and huge bandwidth. So
compression of digital video & audio data are
required. Data compression & coding are almost
synonymous. Text data requires least storing space,
image requires more, audio & video data requires still
more space. Data compression is needed for:
1. Large data volumes for secondary storage may
make system too expensive & sometimes
unfeasible.
2. Relatively slow storage media may not allow
data transmission from secondary storage
devices to output devices in real time.
3. BW limitation may not allow real time A/V
transmission over networks.

Coding Requirements:

To compare data storage & BW needs of different

visual media,(text, graphics, image), following
specifications are based on typical window size of
640 pixels X 480 pixels on screen.
1. For text representation, 2 bytes for each character
are used allowing some variants
Data compression technology:
It is an example of coding technique. Coding means
to code & transform the input bit stream to a new bit
stream based on some principles. The source data is
passed thru a adapt compressor (coder) & then stored
in a storage media like disk, CD, tape etc. It Is later
de-compressed or expanded in an expander
(decoder). The combination of coder-decoder is
called codec or compander.
The compression ratio (CR) = size of uncompressed
data set in bits/bytes: size of compressed data set in
bits/bytes

data
Compressor/
source Data
coder
storage
device

data Expander/
sink Decoder
Data Compression

I---------------------------I------------------------------I
Entropy Encoding Source Encoding
Lossless compression Lossy compression
Independent of data semantics Depends on data semantics
/\
1.Vector quantization
/ \
2.Differential coding
Repetitive Sequence Statistical encoding
3.Transform based coding
Run length encoding Color Book
Zero/blank replacement Morse code
Pattern substitute
Huffman coding
Arithmetic coding
The typical sequence of operations in data
compression for still A/V data streams are:
1. Image preparation: AD conversion &
generation of appropriate digital representation
of information & image is divided to blocks of
8X8 pixels & represented by fixed number of
pixels.
2. Picture processing: First step of compression
which makes use of compression algo.
3. Quantization: It processes the result of step 2,
specifies granularity of mapping of real nos. to
integers.
4. Encoding: Last step of image compression.

UNCOMPRESSED
IMAGE

PICTURE
PROCESSING
QUANTIZATION
IMAGE
PREPARATION

COMPRESSED ENCODING
IMAGE
Encoding:
1. Entropy encoding principle
Entropy is the total information content (TIC) of
any information object
= Useful information content (entropy) +
redundant information content (RIC)

In ideal compression, all redundant information is

removed & all useful information is retained. This
happens in ENTROPY ENCODING which is lossless
compression. Here, some RIC may be left but no
entropy should be LOST. In source encoding, it may
be lossless or lossy. Lossless encoding is reversible.

What is meant by data semantics?

The type of data, whether text, audio or video data is
not considered in entropy encoding. Only, the
sequence of bits is taken into consideration. Entropy
encoding is used for executable codes where every
bit of data needs to be regenerated on expansion.
However, in audio & video source encoding is used
which can afford to lose some entropy not recognized
by human senses. Higher CR is achieved with source
encoding.
Source coding principals
Signal predictability: A signal totally predictable
carries no information. A signal usually has two
components – signal predictability &
unpredictability, the latter requires compression. In
source coding, semantics of source signal MUST be
KNOWN.
Data Compression Technique: Entropy encoding: It
is of two types – repetitive sequence & statistical
encoding.
a) Repetitive Sequence – Oldest technique of data
compression. The sequence of repetitive bits or bytes
is represented by no. of repetition & some special
character. The general form if this technique is called
Run length En-coding – special case being zero/
blank character replacement.
If a character C is repeated in r time in the input data,
the sequence is represented by character C followed
by special character (C  r) and followed by number
of repetition. RLE is beneficial for 4 or more
characters where a considerable CR is achieved.
Ex: Sequence of characters: = A BB CCC DDDD
EEEEE ……J can be replaced by A BB CCC D4
E5 F6 ……..I 9 J10.
The original sequence is 64 characters, the
compressed one is 36 characters. So, CR = 64:36.
Blanking is a special case of RLE. Often, blank
spaces are encountered in a text document. In some
data streams, repeated character may have zero
amplitude of signal e.g. in a communication channel,
an audio may have periods of silence. It works by
specifying number of zeros/blanks.
S.P. = 000052.00 C.P. = 000034.00 Profit =
000018.00
S.P 452.00 C.P. 434.00 Profit 418.00
b) Statistical encoding: It includes
i) Code book
ii) Morse Code
iii) Pattern Substitution
iv) Huffman coding
v) Arithmetic coding.
It uses small code words that use few bits to encode
data patterns occurring frequently.
i) Code book – It is a table which forms central
data structure of statistical code word. The
code book sets the correspondence between
code words & data patterns represented by
code words. Same book may be used by he
coder as well as decoder. The book may be
static or dynamic.
a) Static – Book need not be transmitted with
data as in Morse code. In Morse code, two
values of bits i.e. occurrence & non-
occurrence are represented by dots &
dashes. Code book MUST follow
international standard. Both ends of
transmission channel use same code book.
 b) Dynamic – Statistics for one data set may
not match with another. To improve coding
efficiency, i.e. to increase the CR, each data
set must be scanned separately & a new
book be created for each separate data set.
So, a dynamic data set is transmitted with
data set so that coder & decoder use the
same dynamic code book.
Order of Statistical Model – A statistical model that
counts the occurrence of individual characters with
no reference to characters before them is called an
order-0 model. An order-1 model looks at the
probability of occurrence of a character with
reference to one character before it. An order-2
model looks at two previous characters before a
character. The size of a code book may increase
dramatically from 256 bytes to 64 K-Bytes moving
from order-0 code book to ordr-1 code book. Passing
the entire code-1 book from source to sink may
nullify the increased compression ratio offered by
code-1 statistical model. For this, there are three
main statistical encoding schemes:
a) Pattern Substitution
b) Huffman Coding
c) Arithmetic Coding

Pattern Substitution – The words Multimedia,

Information, Networking & Multimedia Information
Networking will frequenty occur in our discussion.
The following code book replaces these character
strings by shorter code words.
Code Book

Code word Character string Compression

Ratio
M Multimedia 5:1= 5
I Information 11:2= 5.5
N Networking 5:1= 5
MIN Multimedia 33:4=8.25
Information Networking
We see the CR of longer repeated patterns is higher.

Entropy calculation – If probability of an occurrence

of character X in a given data set is 1/16, entropy is
given by: Ebits(X) = -log2(Px) = -log2(1/16) = log216
= 4.
Character X should be encoded with 4 bit codeword.
 Input data
Sink
stream
Statistical
Model
Source Output data
stream

Probabilit
Statistical y Code Statistical
Analyzer Book Model
Information

Statistical Model

Huffman coding – It is widely used in image

compression. First, a statistical model is made of the
input data. This model gives the probability of
occurrence of each input symbol. Based on these
probabilities, code words are assigned to data
characters. The code is built using probabilities in
statistical model It is used in JPEG & MPEG for still
& moving image compression.

An example of Huffman code book

Huffman Encoded character Entropy bits Probability
Code word (C ) (E bits( C)) (Pc)
100 E 3 1/23
101 T 3 1/23
1100 A 4 1/24
01101111 X 8 1/28
0110111000 Y 11 1/211
01101110000 Z 11 1/211

.
. . Entropy E( c) = - log2( Pc)
Arithmetic coding

For the entropy formula E( c) = - log2( Pc) bits may

yield non ineger values for certain number of entropy
bits, Huffman coding can allocate only integer no. of
bits for each code word. So, in many cases, cde book
generated by H.C. will not provide ideal entropy
coding. Some redundant information will always be
there with along with useful information in output
data stream. Arithmetic coding allows
FRACTIONAL NUMBER OF BITS FOR
ENCODING INDIVIDUAL CHARACTERS IN
OUTPUT DATA STREAM. It does NOT assign
code words to individual characters. It generates a
coded string for entire data stream. As a new
character is read from the input data stream, it
modifies the output stream. Generation of coded
stream is done incrementally. This makes possible for
use fractional no. of bits for each character, It is
possible to add 3.5 bits for encoding a character in
input stream with probability of ½ 3.5.
SOURCE ENCODING
VECTOR QUANTIZATION.

B0 IV0 VP3 Vector Pattern

code word
B1 IV1 VP2
VP0 0
B2 VP1 1
IV2 VP8 X ……… 8 2 3
VP2 2

………………
BX IVX  VPX
VPX X

Expander

Code book

B0 VP3 Compander

It is similar to pattern substitution techniques. There

are two types – simple vector quantization & vector
quantization with error term.
The input data streams are divided to blocks called
input vectors. Compander uses code book to equate
or find a match for finite number of vector patterns
(VP)s to code words. If input data is an image, blocks
are usually 2X2 pixels (like vectors). Compander
observes the contents of each data block in the input
stream & finds closest matching vector pattern in the
code book. In the input, each data block is replaced
by a code word of matching vector pattern. In the fig;
the context of the first data block B0 is input vector
IV0 which matches closely with the vector pattern
VP3. So, the output stream of the first block is
replaced with 3 which is the code word for the vector
pattern VP3 in code book. The output stream has
3,2,8…& so on. To expand the encoded stream, code
book is used in reverse. However the context of B0
after companding will be VP3 which shall be close to
IV0 & NOT EXACTLY IV0. Hence, this is a lossy
compression.
VQ with error term: The loss of information in
simple VQ can b reduced or eliminated by including
error terms in the output stream. The error term is
equal to contents of input block minus matching
vector pattern. For input block B0, the codeword is 3
& error term E0 = IV0-VP3. VQ is not entropy
encoding but source encoding as E0 my not be zero.
 B0 IV0-VP3=E0 Vector Pattern
code word
B1 IV1-VP2=E1 B2 B1 B0
VP0 0
B2 IV2-VP8=E2 VP1 1
X ……… 8 2 3
VP2 2 E2 E1 E0
………………
BX IVX  VPX
VPX X

Expander

Code book

B0 VP3 Compander

Fig: Vector quantization with error term.

Fractal encoding: All pattern substitution based

computing techniques look for patterns in input data.
Another way of generating an image is by recursive
application of a single pattern called fractal. Fractals
need to be found out from images which is very
difficult & rather compute-intensive. But high CR =
1000:1 may be achieved.
Differential / Predictive coding
1. Rather than coding the entire value of sample,
only difference between sample value &
predicted value is encoded. The difference
 between predictive value & sample value is called
error term/ prediction difference.
2. In this type of encoding, data volume is reduced
& so it can be considered a method of data
compression.
3. It gives best result when individual samples are
large but difference between successive samples
is small. The difference can be coded with fewer
bits than the no. of bits required for coding the
full value of the sample. If value of a signal can
be predicted, it carries no information. So, real
information lies in the error term which is
encoded instead of encoding the full signal.
4. If no. of bits needed to encode full signal is B &
no. of bits needed to code sample error term is b,
then CR = B/b.

There are three types of differential encoding:

i) DPCM: Last sample value is taken as
predicted value of current sample. The error
term is difference between last & current
sample;
ii) Delta modulation: Error term is encoded with
single bit suitable for low frequency slow
changing signals. It is a special case of DPCM
where b=1.
iii) ADPCM: Predicted value of sample is
calculated from some previous sample based
on a prediction function.
Where is predictive encoding used?
1. For still images, during calculation of
differences between nearby pixels, edges are
represented by larger values & & areas with
similar luminance & chrominance
characterized by smaller values. A
homogeneous are is characterized by large
number of zeroes which can be further
compressed by RLE (run-length encoding).
2. For video, the use of relative coding in the
tome domain can lead to encoding of
differences from previous image. For
newscast, & video telephone application,
background remains same for a long time. So
difference between successive images is very
small leading to a large number of zeroes
which can be compressed by RLE.
TRANSFORM BASED ENCODING
1. MM signals vary in space & time called
spatial & temporal domains. Real life MM
signals vary in both the domains.
Sometimes, it is EASY TO REMOVE
REDUNDANCY by TRANSFORMING
the SIGNAL from SPATIAL domain to
SOME OTHER domain.
2. Original signal is sampled & transformed to
other domain. If abstract domain is
frequency domain, then the transform
would represent amplitude of various
frequency components. Compression can be
achieved by identifying those frequency
components which are more significant than
others. These more significant components
are coded with greater accuracy & less
significant ones are neglected. Some
transformations are Fourier, Cosine,
Hadamard, Harloev transforms.