An introduction to coding theory

Adrish Banerjee

Department of Electrical Engineering

Indian Institute of Technology Kanpur
Kanpur, Uttar Pradesh
India

Jan. 23, 2017

Lecture #3B: Problem solving session-I

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Linear block code

Problem # 1: Consider a linear block code, C with parity check

matrix given by ⎡ ⎤
1001011
⎢ 0101110 ⎥
H=⎢ ⎣ 0010111 ⎦
⎥

1110010
What is (n, k) of C?

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Linear block code

Problem # 1: Consider a linear block code, C with parity check

matrix given by ⎡ ⎤
1001011
⎢ 0101110 ⎥
H=⎢ ⎣ 0010111 ⎦
⎥

1110010
What is (n, k) of C?
Solutions: Rank of H matrix is 3. So, n = 7, k = 7 − 3 = 4.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Linear block code
Problem # 2: Consider the following binary block code, C,

C = {000000, 110011, 011101, 111111}

Is C a linear block code? Justify your answer.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Linear block code

Problem # 2: Consider the following binary block code, C,

C = {000000, 110011, 011101, 111111}

Is C a linear block code? Justify your answer.

Solutions: No.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Linear block code
Problem # 2: Consider the following binary block code, C,

C = {000000, 110011, 011101, 111111}

Is C a linear block code? Justify your answer.

Solutions: No.
Sum of two codewords for a linear block code is a valid codeword.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Linear block code

Problem # 2: Consider the following binary block code, C,

C = {000000, 110011, 011101, 111111}

Is C a linear block code? Justify your answer.

Solutions: No.
Sum of two codewords for a linear block code is a valid codeword.
Let v0 = 000000, v1 = 110011, v2 = 011101, and v3 = 111111, then
v1 + v2 , v1 + v3 , v2 + v3 , and v1 + v2 + v3 must also be a valid
codeword.

v1 + v2 = 101110
v1 + v3 = 001100
v2 + v3 = 100010
v1 + v2 + v3 = 010001

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 2 (contd.)

Thus a linear block code should have the following codewords

C = {000000, 110011, 011101, 111111, 101110, 001100, 100010, 010001}

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 2 (contd.)

Thus a linear block code should have the following codewords

C = {000000, 110011, 011101, 111111, 101110, 001100, 100010, 010001}

This is a (6,3) linear binary code.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 2 (contd.)

Thus a linear block code should have the following codewords

C = {000000, 110011, 011101, 111111, 101110, 001100, 100010, 010001}

This is a (6,3) linear binary code.

One example of generator matrix for this code
⎡ ⎤
1 1 0 0 1 1
G=⎣ 0 1 1 1 0 1 ⎦
1 1 1 1 1 1

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 2 (contd.): Generator matrix in systematic

form
How to write the generator matrix in systematic form?
⎡ ⎤
1 1 0 0 1 1
G=⎣ 0 1 1 1 0 1 ⎦
1 1 1 1 1 1

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 2 (contd.): Generator matrix in systematic
form
How to write the generator matrix in systematic form?
⎡ ⎤
1 1 0 0 1 1
G=⎣ 0 1 1 1 0 1 ⎦
1 1 1 1 1 1
Row 3 → Row 3 + Row 1
⎡ ⎤
1 1 0 0 1 1
⎣
G= 0 1 1 1 0 1 ⎦
0 0 1 1 0 0

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 2 (contd.): Generator matrix in systematic

form
How to write the generator matrix in systematic form?
⎡ ⎤
1 1 0 0 1 1
G=⎣ 0 1 1 1 0 1 ⎦
1 1 1 1 1 1
Row 3 → Row 3 + Row 1
⎡ ⎤
1 1 0 0 1 1
G=⎣ 0 1 1 1 0 1 ⎦
0 0 1 1 0 0
Row 2 → Row 3 + Row 2
⎡ ⎤
1 1 0 0 1 1
⎣
G= 0 1 0 0 0 1 ⎦
0 0 1 1 0 0

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 2 (contd.): Generator matrix in systematic
form

Row 1 → Row 1 + Row 2

⎡ ⎤
1 0 0 0 1 0
G=⎣ 0 1 0 0 0 1 ⎦
0 0 1 1 0 0

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 2 (contd.): Generator matrix in systematic

form

Row 1 → Row 1 + Row 2

⎡ ⎤
1 0 0 0 1 0
G=⎣ 0 1 0 0 0 1 ⎦
0 0 1 1 0 0

Similarly parity check matrix in systematic form can be written as

⎡ ⎤
0 0 1 1 0 0
H=⎣ 1 0 0 0 1 0 ⎦
0 1 0 0 0 1

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Linear block code
Problem # 3: Let H be the parity check matrix of an (n,k) linear
code C that has both odd and even-weight codewords. Construct a
new linear code C1 with the following parity-check matrix
⎡ ⎤
0
⎢ 0 ⎥
⎢ ⎥
⎢ .. ⎥
⎢ . H ⎥
H1 = ⎢ ⎥
⎢ 0 ⎥
⎢ ⎥
⎣ ··· ··· ⎦
1 11 · · · 1

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Linear block code

Problem # 3: Let H be the parity check matrix of an (n,k) linear
code C that has both odd and even-weight codewords. Construct a
new linear code C1 with the following parity-check matrix
⎡ ⎤
0
⎢ 0 ⎥
⎢ ⎥
⎢ .. ⎥
⎢ . H ⎥
H1 = ⎢ ⎥
⎢ 0 ⎥
⎢ ⎥
⎣ ··· ··· ⎦
1 11 · · · 1

1 Show that C1 is an (n + 1, k) linear code.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Linear block code
Problem # 3: Let H be the parity check matrix of an (n,k) linear
code C that has both odd and even-weight codewords. Construct a
new linear code C1 with the following parity-check matrix
⎡ ⎤
0
⎢ 0 ⎥
⎢ ⎥
⎢ .. ⎥
⎢ . H ⎥
H1 = ⎢ ⎥
⎢ 0 ⎥
⎢ ⎥
⎣ ··· ··· ⎦
1 11 · · · 1

1 Show that C1 is an (n + 1, k) linear code.

2 Show that every codeword of C1 has even weight.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Linear block code

Problem # 3: Let H be the parity check matrix of an (n,k) linear
code C that has both odd and even-weight codewords. Construct a
new linear code C1 with the following parity-check matrix
⎡ ⎤
0
⎢ 0 ⎥
⎢ ⎥
⎢ .. ⎥
⎢ . H ⎥
H1 = ⎢ ⎥
⎢ 0 ⎥
⎢ ⎥
⎣ ··· ··· ⎦
1 11 · · · 1

1 Show that C1 is an (n + 1, k) linear code.

2 Show that every codeword of C1 has even weight.
3 Show that C1 can be obtained from C by adding an extra parity
check digit, denoted by v∞ to the left of each codeword v as follows

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Linear block code
Problem # 3: Let H be the parity check matrix of an (n,k) linear
code C that has both odd and even-weight codewords. Construct a
new linear code C1 with the following parity-check matrix
⎡ ⎤
0
⎢ 0 ⎥
⎢ ⎥
⎢ .. ⎥
⎢ . H ⎥
H1 = ⎢ ⎥
⎢ 0 ⎥
⎢ ⎥
⎣ ··· ··· ⎦
1 11 · · · 1

1 Show that C1 is an (n + 1, k) linear code.

2 Show that every codeword of C1 has even weight.
3 Show that C1 can be obtained from C by adding an extra parity
check digit, denoted by v∞ to the left of each codeword v as follows
1) if v has odd weight, then v∞ = 1, and

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Linear block code

Problem # 3: Let H be the parity check matrix of an (n,k) linear
code C that has both odd and even-weight codewords. Construct a
new linear code C1 with the following parity-check matrix
⎡ ⎤
0
⎢ 0 ⎥
⎢ ⎥
⎢ .. ⎥
⎢ . H ⎥
H1 = ⎢ ⎥
⎢ 0 ⎥
⎢ ⎥
⎣ ··· ··· ⎦
1 11 · · · 1

1 Show that C1 is an (n + 1, k) linear code.

2 Show that every codeword of C1 has even weight.
3 Show that C1 can be obtained from C by adding an extra parity
check digit, denoted by v∞ to the left of each codeword v as follows
1) if v has odd weight, then v∞ = 1, and
2) if v has even weight, then v∞ = 0
Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

The matrix H1 is an (n − k + 1) × (n + 1) matrix.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

The matrix H1 is an (n − k + 1) × (n + 1) matrix.

First we note that the n − k rows of H are linearly independent. It is
clear that the first (n − k) rows of H1 are also linearly independent.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

The matrix H1 is an (n − k + 1) × (n + 1) matrix.

First we note that the n − k rows of H are linearly independent. It is
clear that the first (n − k) rows of H1 are also linearly independent.
The last row of H1 has a “1” at its first position but other rows of
H1 have a “0” at their first position. Any linear combination
including the last row of H1 will never yield a zero vector.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

The matrix H1 is an (n − k + 1) × (n + 1) matrix.

First we note that the n − k rows of H are linearly independent. It is
clear that the first (n − k) rows of H1 are also linearly independent.
The last row of H1 has a “1” at its first position but other rows of
H1 have a “0” at their first position. Any linear combination
including the last row of H1 will never yield a zero vector.
Thus all the rows of H1 are linearly independent. Hence the row
space of H1 has dimension n-k+1.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

The matrix H1 is an (n − k + 1) × (n + 1) matrix.

First we note that the n − k rows of H are linearly independent. It is
clear that the first (n − k) rows of H1 are also linearly independent.
The last row of H1 has a “1” at its first position but other rows of
H1 have a “0” at their first position. Any linear combination
including the last row of H1 will never yield a zero vector.
Thus all the rows of H1 are linearly independent. Hence the row
space of H1 has dimension n-k+1.
The dimension of its null space, C1 , is then equal to

dim(C1 ) = (n + 1) − (n − k + 1) = k

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

The matrix H1 is an (n − k + 1) × (n + 1) matrix.

First we note that the n − k rows of H are linearly independent. It is
clear that the first (n − k) rows of H1 are also linearly independent.
The last row of H1 has a “1” at its first position but other rows of
H1 have a “0” at their first position. Any linear combination
including the last row of H1 will never yield a zero vector.
Thus all the rows of H1 are linearly independent. Hence the row
space of H1 has dimension n-k+1.
The dimension of its null space, C1 , is then equal to

dim(C1 ) = (n + 1) − (n − k + 1) = k

Hence C1 is an (n + 1, k) linear code.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

Show that every codeword of C1 has even weight.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

Show that every codeword of C1 has even weight.

Solution: The last row of H1 is an all-one vector.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

Show that every codeword of C1 has even weight.

Solution: The last row of H1 is an all-one vector.
The inner product of a vector with odd weight and the all-one vector
is “1” . Hence, for any odd weight vector v,

vH1T = 0

and v cannot be a code word in C1 .

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

Show that every codeword of C1 has even weight.

Solution: The last row of H1 is an all-one vector.
The inner product of a vector with odd weight and the all-one vector
is “1” . Hence, for any odd weight vector v,

vH1T = 0

and v cannot be a code word in C1 .

Therefore, C1 consists of only even-weight code words.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

Show that C1 can be obtained from C by adding an extra parity

check digit, denoted by v∞ to the left of each codeword v as follows

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

Show that C1 can be obtained from C by adding an extra parity

check digit, denoted by v∞ to the left of each codeword v as follows
1) if v has odd weight, then v∞ = 1, and

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

Show that C1 can be obtained from C by adding an extra parity

check digit, denoted by v∞ to the left of each codeword v as follows
1) if v has odd weight, then v∞ = 1, and
2) if v has even weight, then v∞ = 0

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

Show that C1 can be obtained from C by adding an extra parity

check digit, denoted by v∞ to the left of each codeword v as follows
1) if v has odd weight, then v∞ = 1, and
2) if v has even weight, then v∞ = 0
Solution: Let v be a code word in C. Then vH T = 0. Extend v by
adding a digit v∞ to its left.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

Show that C1 can be obtained from C by adding an extra parity

check digit, denoted by v∞ to the left of each codeword v as follows
1) if v has odd weight, then v∞ = 1, and
2) if v has even weight, then v∞ = 0
Solution: Let v be a code word in C. Then vH T = 0. Extend v by
adding a digit v∞ to its left.
This results in a vector of n+1 digits,

v1 = (v∞ , v) = (v∞ , v0 , v1 , · · · , vn−1 ).

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

Show that C1 can be obtained from C by adding an extra parity

check digit, denoted by v∞ to the left of each codeword v as follows
1) if v has odd weight, then v∞ = 1, and
2) if v has even weight, then v∞ = 0
Solution: Let v be a code word in C. Then vH T = 0. Extend v by
adding a digit v∞ to its left.
This results in a vector of n+1 digits,

v1 = (v∞ , v) = (v∞ , v0 , v1 , · · · , vn−1 ).

For v1 to be a vector in C1 , we must require that

v 1 HT
1 = 0

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

Note that the inner product of v1 with any of the first n-k rows of
H1 is 0.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

Note that the inner product of v1 with any of the first n-k rows of
H1 is 0.
The inner product of v1 with the last row of H1 is

v∞ + v0 + v1 + · · · + vn−1

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

Note that the inner product of v1 with any of the first n-k rows of
H1 is 0.
The inner product of v1 with the last row of H1 is

v∞ + v0 + v1 + · · · + vn−1

For this sum to be zero, we must require that v∞ = 1 if the vector v

has odd weight and v∞ = 0 if the vector v has even weight.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory

Problem # 3 (contd.)

Note that the inner product of v1 with any of the first n-k rows of
H1 is 0.
The inner product of v1 with the last row of H1 is

v∞ + v0 + v1 + · · · + vn−1

For this sum to be zero, we must require that v∞ = 1 if the vector v

has odd weight and v∞ = 0 if the vector v has even weight.
Therefore, any vector v1 formed as above is a codeword in C1 , there
are 2k such codewords.

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory
Problem # 3 (contd.)

Note that the inner product of v1 with any of the first n-k rows of
H1 is 0.
The inner product of v1 with the last row of H1 is

v∞ + v0 + v1 + · · · + vn−1

For this sum to be zero, we must require that v∞ = 1 if the vector v

has odd weight and v∞ = 0 if the vector v has even weight.
Therefore, any vector v1 formed as above is a codeword in C1 , there
are 2k such codewords.
The dimension of C1 is k, these 2k codewords are all the code words
of C1 .

Adrish Banerjee Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India
An introduction to coding theory