Advanced Digital Communications

(EE5511)
MSc Module of Wireless Communication System

Dr. Qiang Ni
ECE, School of Eng & Design, Brunel University
E-mail: [email protected]
Homepage: http://people.brunel.ac.uk/~eestqqn/
Office: Howell Building H237
Dr Qiang Ni Brunel University 1/54
 Section 2:

Coding

Dr Qiang Ni Brunel University 2/54

 Source/Channel Coding

`Source Coding:
` Reduce number of bits required to transmit
information (Reduce Redundancy)

`Channel Coding:
` Protect information from errors caused by noise
between the transmitter and receiver (Add “Special”
Redundancy)

Dr Qiang Ni Brunel University 3/54

 Some General Concepts about Codes:
A variable length code assigns a bit string (codeword) of
variable length to every message value
e.g. a = 1, b = 01, c = 101, d = 011
What if you get the sequence of bits
1011 ?
Is it aba, ca, or, ad? (A failed code!?)
A uniquely decodable code is a variable length code in which
bit strings can always be uniquely decomposed into its
codewords:
e.g. a = 0, b = 110, c = 111, d = 10 (Example: 1100)
A prefix code is a variable length code in which no codeword
is a prefix of another word
The above example is also a prefix code
Dr Qiang Ni Brunel University 4/54
Code Example -
Huffman Codes (1)
Invented by Huffman as a class assignment in 1950.
Used in many, if not most compression algorithms
agzip, bzip, jpeg (as option), fax compression,…

Properties:
`Generates optimal prefix codes
`Cheap to generate codes
`Cheap to encode and decode

Dr Qiang Ni Brunel University 5/54

 Huffman Codes (2)

Huffman Algorithm
a Start with a forest of trees each consisting of a single
vertex corresponding to a message s with weight p(s)

a Repeat:
`Select two trees with minimum weight roots p1 and
p2
`Join into single tree by adding root with weight p1 +
p2

Dr Qiang Ni Brunel University 6/54

 Huffman Coding example

p(a) = .1, p(b) = .2, p(c ) = .2, p(d) = .5

a(.1) b(.2) c(.2) d(.5)

(.3) (.5) (1.0)
0 1
a(.1) b(.2) (.5) d(.5)
(.3) c(.2)
0 1
Step 1 (.3) c(.2)
a(.1) b(.2)
0 1
Step 2 a(.1) b(.2)
Step 3

Generate Codeword: a=000, b=001, c=01, d=1

Dr Qiang Ni Brunel University 7/54
 Source Coding Technology

Coding algorithms for images: JPEG, wavelet,

fractals, ...

Coding algorithms for audio/speech: MP3,

MPEG4, LPC, (AD)PCM, GSM codec, ...

Dr Qiang Ni Brunel University 8/54

 Channel Coding (1)

Definition : A method of encoding data in a communications

channel that adds patterns of redundancy into the
transmission path in order to lower the error rate.

Note 1: Such methods are widely used in wireless

communications.
Note 2: It is different from source coding, the channel
coding adds redundancy.

Dr Qiang Ni Brunel University 9/54

 Channel Coding (2)

aChannel Coding is to add r extra protection

bits to the k information bits transmission
which protect them from noise induced bit
errors, which is used for
¾ Error Detection and
¾ Error Correction
aNo Channel Coding System is 100% -
capabilities depending upon complexity

Dr Qiang Ni Brunel University 10/54

 Error Detection Process

aTransmitter
`For a given frame, an error-detecting code (check
bits) is calculated from data bits
`Check bits are appended to data bits
aReceiver
`Separates incoming frame into data bits and check
bits
`Calculates check bits from received data bits
`Compares calculated check bits against received
check bits
`Detected error occurs if mismatch

Dr Qiang Ni Brunel University 11/54

Dr Qiang Ni Brunel University 12/54
 Forward Error Correction

aTransmitter
`Forward error correction (FEC) encoder
maps each k-bit block into an n-bit block
codeword
`Codeword is transmitted
aReceiver
`Incoming signal is demodulated
`Block passed through an FEC decoder

Dr Qiang Ni Brunel University 13/54

Dr Qiang Ni Brunel University 14/54
 FEC Decoder Outcomes
aFour possibilities:
¾No errors present
`Codeword produced by decoder matches
original codeword
¾Decoder detects and corrects bit errors
¾Decoder detects but cannot correct bit
errors; reports uncorrectable error
¾Decoder detects no bit errors, though
errors are present
Dr Qiang Ni Brunel University 15/54
 Code Principles
aHamming distance - for 2 code words, it’s
defined as the number of bits in which they
differ:
` e.g., v1=011011; v2=110001; d(v1, v2) = 3

aCode rate – ratio of data bits to total bits

aCoding gain – the reduction in the required SNR

(Eb/N0) to achieve a specified BER of an error-
correcting coded system

Dr Qiang Ni Brunel University 16/54

 Error Detection/Correction

aThe Hamming distance d of a Block code

is the minimum distance between two
code words
aError Detection:
`Up to d-1 errors
aError Correction:
⎢ d − 1⎥
`Up to ⎢ 2 ⎥
⎣ ⎦

Dr Qiang Ni Brunel University 17/54

Dr Qiang Ni Brunel University 18/54
 Error Detection and Correction
Capability of a Code (1)

The minimum distance dm of a linear code is the key

property of a code that determines its capability to detect
and correct codes.
In order to examine the error control capability of a linear
code we can think of transmitted and received code words
as points in the n-dimensional space.
Each code word can be thought of as having a decoding
Sphere of radius t around it.
Each decoding sphere contains redundant words that are
at a distance of t or less away from the codeword at the
centre of the sphere.
Dr Qiang Ni Brunel University 19/54
 Error Detection and Correction
Capability of a Code (2)

There will usually be redundant words lying outside the

decoding spheres.
Let us consider the four ways the error-control capability
of a code with dmin = 7
(a) t = 3
t=3 t=3

c1 r1 r2 r3 r4 r5 r6 c2

Dr Qiang Ni Brunel University 20/54

 Error Detection and Correction
Capability of a Code (3)

(b) t = 2
t=2 t=2

c1 r1 r2 r3 r4 r5 r6 c2

(c)t=1
t=1 t=1

c1 r1 r2 r3 r4 r5 r6 c2

Dr Qiang Ni Brunel University 21/54

 Error Detection Capability of a Code

(d)

c1 r1 r2 r3 r4 r5 r6 c2

t 1

dm

d min ≥ t det
+1
t det
≤ d min − 1
Dr Qiang Ni Brunel University 22/54
 Error Correction Capability of a Code
(e)

c1 r1 r2 r3 r4 r5 r6 c2

t 1 t

dm

d min ≥ 2t corr
+1
⎢ d min − 1 ⎥
t corr
≤⎢ ⎥
⎣ 2 ⎦
Dr Qiang Ni Brunel University 23/54
 Code Example (1)

Example:
Message Encoder Code words
000 000000
001 001101
010 010011
011 011110
100 100110
101 101011
110 110101
111 111000

The minimum distance is 3 t=1

Dr Qiang Ni Brunel University 24/54
 Code Example (2)

t t
t=1
010011
011110
t t
t

111000 000000 001101

t
t
t

110101 100110 101011

Dr Qiang Ni Brunel University 25/54
 Error Detection/Correction Example
aIn noisy binary wireless transmission, the reliability of a
transmission can be increased by repeating a message for 3
times, i.e., transmitting 000 for a symbol 0 and 111 for a
symbol 1. The receiver should make a decision by choosing
the majority rule; that is, if at least two of the three detected
symbols are 0, the decision is 0 and so on. An error detection
is made if at least two bits are in error. Let Pe denote the
error probability of one bit (Pe = 10-4), and assume the bit
error is evenly distributed over the symbol.
Can you calculate the probability of making a
Wrong Decision - Pm?
Dr Qiang Ni Brunel University 26/54
 Channel Coding

aIn the following, we examine a

number of channel coding techniques:
`Parity (Error-detection code)
`Cyclical Redundancy Codes (CRC)
`Convolutional Codes

Dr Qiang Ni Brunel University 27/54

 Parity Codes

aInformation bits grouped into blocks of k bits

(commonly, k=7 bits)
ar=1 parity bits added to provide protection
aBinary value of parity bit set to ensure either
EVEN or ODD number of binary 1 in 8 bit block
aDecoder checks number of binary 1 in received
block - EVEN or ODD?
aError Detection ONLY!

Dr Qiang Ni Brunel University 28/54

 Parity Codes

aParity bit appended to a block of data

aEven parity
`Added bit ensures an even number of 1
aOdd parity
`Added bit ensures an odd number of 1
aExample, 7-bit character [1110001]
`Even parity [11100010]
`Odd parity [11100011]

Dr Qiang Ni Brunel University 29/54

 Polynomial Codes

aCan detect errors on large chunks of data

aHas low overhead
aMore robust than parity bit
aRequires the use of a “code polynomial”
`Example: x2 + 1

Dr Qiang Ni Brunel University 30/54

 Binary Polynomials

aAn 8 bit binary pattern 10011011 can be

represented as a binary polynomial

1 0 0 1 1 0 1 1
x 7 x6 x5 x4 x3 x2 x1 x0

Binary Polynomial = x7+ x4 +x3 +x1+1

Dr Qiang Ni Brunel University 31/54
 Cyclic Redundancy Check

a CRC: An example of polynomial codes

a Procedure (at the sender):
1. Let r be the degree of the code polynomial c(x). (Both
sender and the receiver know the code polynomial.)
Append r zero bits to the end of the message bit string.
(call the entire bit string S(x))
2. Divide S(x) by the code polynomial c(x) using modulo 2
division.
3. Subtract the remainder from S(x) using modulo 2
subtraction. (call resulting polynomial t(x).)
4. Transmit the check-summed message t(x).

Dr Qiang Ni Brunel University 32/54

 Background

s(x) = f(x)c(x) code polynomial + remainder

a entire bit string
a s(x) - remainder = f(x)c(x) = t(x)

`sender transmits t(x)

`note that t(x) is divisible by c(x)
`if the received sequence at the receiver is not divisible by
c(x), error has occurred

Dr Qiang Ni Brunel University 33/54

 Background

aHow Modulo 2 calculation works?

Addition and subtraction are identical to
EXCLUSIVE OR (XOR).
No carries
a take mudulo2 addition of 1010 and 0110
t(x) = 1 x x3 + 0 x x2 + 1 x x + 0 = x3 + x
+ e(x) = 0 x x3 + 1 x x2 + 1 x x + 0 = x2 + x
---------------------------------------------------------
1 x x3 + 1 x x2 + 0 x x + 0 = x3 + x2

Dr Qiang Ni Brunel University 34/54

 Background

aDefinition of Exclusive OR (XOR)

Exclusive OR
00 0
01 1
10 1
11 0

Dr Qiang Ni Brunel University 35/54

 Generating a CRC Example

Message: 1011 1 x x3 + 0 x x2 + 1 x x + 1
= x3 + x + 1

Code Polynomial c(x): Both the sender and the receiver know the
code polynomial, this polynomial is pre-set

c(x) =x2 + 1 (101)

Step 1: Compute S(x) entire bit string

r = 2 (r: degree of the code polynomial)

S(x) = 101100 (x5 + x3 + x2)

Dr Qiang Ni Brunel University 36/54

 Generating a CRC Example
Step 2: Modulo 2 divide : s(x)/c(x)

1001
101 101100
101
001
000
010
000
100
101
01 Remainder

Dr Qiang Ni Brunel University 37/54

 Generating a CRC Example
Step 3: Modulo 2 subtract the remainder from S(x)

101100
- 01
101101

Transmitted Message t(x)

x5 + x3 + x2 = (x2 + 1) (x3 + 1) + 1 s(x)=c(x)f(x)+reminder

x5 + x3 + x2 - 1 = (x2 + 1) (x3 + 1) t(x)=s(x)-reminder =c(x)f(x)

Transmit x5 + x3 + x2 + 1 (101101) : +1 and –1 are identical

Dr Qiang Ni Brunel University 38/54
 Decoding a CRC

aProcedure (at the receiver)

`Divide the received message by the code
polynomial c(x) using modulo 2 division. If the
remainder is zero, there is no error detected.

Dr Qiang Ni Brunel University 39/54

 Decoding a CRC Example

101101 Transmitted message

1011 Original message (if there are

no errors)
1001
101 101101
101
001
000
010
000
101
101
Remainder = 0
00
(No error detected)
Dr Qiang Ni Brunel University 40/54
 Decoding a CRC- If an error
When a bit error occurs, there is a large probability
that it will produce a polynomial that is not an even
multiple of the code polynomial, and thus errors
can usually be detected.
1000
101 101001
101
000
000
000
000
001
000
Remainder = 1
01
(Error detected)
Dr Qiang Ni Brunel University 41/54
 The limitation of detecting error

aAssume errors occurred, and the receiver

received t(x) + e(x).
aReceiver calculates [t(x) + e(x)]/c(x).
`t(x)/c(x); divisible (remainder of zero)
`e(x)/c(x); if errors (e(x)) correspond to
polynomials containing c(x) as a factor,
this will result zero remainder. (These
errors are not detected.) All the other
errors are detected.
Dr Qiang Ni Brunel University 42/54
 A Special Example
a In our example:
`CRC generator = 101 (code polynomial c(x))
`Message = 1011
`Remainder = 01
`Transmitted message = 101101 (T(x))

`101101 divided by 101 yields remainder 0

`If error occurs:
101101
+ 101
---------
101000

Bits 4 and 6 are flipped.

Dr Qiang Ni Brunel University 43/54
 Example discuss

- Then the received message 101000 is a mod 2

multiple of the CRC generator and its modulo 2
division yields remainder = 0.

- No error detected.

- If CRC generator divides error, CRC fails to

detect error.
- E.g. if error was 10001 = 101 x 101 (mod 2
multiplication)

Dr Qiang Ni Brunel University 44/54

 Choosing a CRC polynomial
aThe longer the polynomial, the smaller
the probability of undetected error

aCommonly standardized polynomial

examples:

(1) CRC-12: x12 + x11 + x3 + x2 + x1 + 1

(2) CRC-16: x16 + x15 + x2 + 1
(3) CRC-CCITT: x16 + x12 + x5 + 1

Dr Qiang Ni Brunel University 45/54

Dr Qiang Ni Brunel University 46/54
 Convolutional Codes (1)
a Very powerful and relative complex codes capable
of correcting multiple bit errors

a Convolutional codes are applied in applications that

require good performance with low implementation
cost (ideal for vendors).

a Convolutional codes have memory that utilizes

previous bits to encode or decode following bits

a Convolutional codes achieve good performance by

expanding their memory depth
Dr Qiang Ni Brunel University 47/54
 Convolutional Codes (2)

input bit

k bits (n,k,L) n bits message bits

(n,k,L)
encoder
encoder encoded bits
n(L+1) output bits

a Convolutional codes are denoted by (n,k,L), where L

is code/encoder Memory depth (number of
register stages?)
a Constraint length C=n(L+1) is defined as the
number of encoded bits a message bit can
influence.
Dr Qiang Ni Brunel University 48/54
 Encoder Structure

a A typical Convolutional Encoder is based on a 3 bit

shift register and 2 XOR gates
a Two output bits (v1 and v2) are generated for each
incoming information bit (xi)
a Encoder usually starts from a reset state and
encodes a fixed length block of information bits
before returning to its reset state
a Extra bits added at the end of the incoming bit
sequence to ensure a return to reset state

Dr Qiang Ni Brunel University 49/54

 Convolutional Coder Structure
3 Bit Shift Register
Xi
Incoming S1 S2 S3
Information Bit

XOR Bits
V1 = S1 ⊕ S 2 ⊕ S 3
V1 V2 V2 = S1 ⊕ S3
More details in the reference book: Digital Coms: fundamentals & applications
Dr Qiang Ni Brunel University 50/54
 (2,1,2) Code Tree Diagram
aThis is a tree representation
of(2,1,2) convolutional code.

aIf the input bit is a 0/1, its

associated branch work is found
by moving to the next right
branch in the upward/downward
direction.

aExample: if the input sequence

is 1 1 0 1 1
aThe output codeword sequence
is: 11 01 01 00 01
Dr Qiang Ni Brunel University 51/54
 Trellis Diagram
a

states
c

aWe found that the tree diagram is repeated at some stage, and totally
we have 4 different states, so we can obtain another diagram, called the
trellis.

aExample again: if the input sequence is 1 1 0 1 1, The output codeword

sequence is: 11 01 01 00 01
Dr Qiang Ni Brunel University 52/54
 ARQ: Automatic Repeat reQuest

aARQ is commonly used in data link

control and transport protocols
aRelies on using of an error detection
code (such as CRC)
¾ Error Control
¾ Flow Control
aHybrid ARQ and FEC
Dr Qiang Ni Brunel University 53/54
Dr Qiang Ni Brunel University 54/54