: iy eit STUDENT ID NO neon QS unveRstY: MULTIMEDIA UNIVERSITY FINAL EXAMINATION ‘TRIMESTER 3, 2018/2019 TIT3131 —- INFORMATION THEORY (All sections / Groups) 1 June 2019 9,00 am - 11.00 am (2 Hows) INSTRUCTIONS TO STUDENTS 1. This question paper consists of 6 pages with Five questions only. 2. Atfempt All questions. All questions carry equal marks and the distribution of the marks for each question is given. 3. Please print all your answers in the Answer Booklet provided.Tas INFORMATION THEOR) Lune 2019 QUESTION 1 (@) Classify the elements of digital communication system for ‘X° and ‘¥” in Table 1 with their respective goals. code word’. Y | Add redundancy to the source code word in order to correct errors introduce by 0 Camel encoder” Table t Elements. Goals x sour, oncader [2 marks] () A source emits one of 25 possible symbols from a Scrabble game. Figure 1 shows 20 available symbols and S similar vowels is pending at the next move SEH BORGO sett! Figure 1 ‘What is the expected vowels in the next move? Complete the following table and calculate the amount of information for all vowel symbols only. [6 marks] ke 1[2 3 5 7 9 | 10 8 [5 |S Ss 8, | 1S, | Sv | ep ae Ms Be Le : Py | \28| 2/25] 3125] 1p us| 1/25 [1s | [2K] 2724] 1104 [5] a7 7 7 7 : ii. Show the information rate of this source if one of the 25 symbols is emitted, once every nanosecond. [4 marks} (0) To = ta, Cul 3/28) ©) Is = log, ¢ 4715) = 3.0584 bits > 9 2214 bits G) Ty = Ing. 6%) . = 4 6424 vits Hos) = Fe (ABD + ae Clog, 12) + CA) T= Ig. U8) 3.058) +H (2-219) = £6454 wits = 3.659 bits Continued... OCHRE era PaaPTTTAAT ES TEESE TT ti) R= rH bits/see vo ots | i = 100000 o0043.5591 498 @eeeesyGembitslceemp —~ 1000. 000, 000 sk, kilo o> | milli 107% M, age 10° | iene 107 Gr, ga tot | vane \o* Te onlin |e eate , ara FORMATION TIBORY 12ane2019 QUESTION 2 (a) Explain whether code I and code Il in Table 2 are uniquely decodable using Kraft inequality. {2 marks} prhin prec ‘Symbols Coder Coden > stity att . 7 a ruaguclity 3 ic TF wntqualy ~ decoualne Cc 1 100 K awd D ir oul ye ead Tabie2 a Kur 7 142007) 42°F ta (©) A discrete memoryless source uses the symbols A,B, C, D,E, andF with the following probabilities 1s Qt oot oBt lated2 PD =H 0 2 Gxt OF PB)=003, 6. 0.3 PO=013, 4. 9. OF PCD)=0.22, gs) = 0.05 lig, Ces) + 08 bog, c070-00 + PUB) = 5x, 0% us tog (013) + 023 Lo, (1/0.399 + : Ck Io LUO) + O32 ngs (052) Pr) =022- 2.154] 7 i) Find the value of x and calculate the entropy of the source. [2 marks} ii) Construct a Huffiman for this code. {5 marks] iii) Catenlate the average length of the Hufliman code and give the redundaney of the code. (2 marks} iv) Decode the following bit string into a string of symbols with Huffman codes that gives a unique answer, Ui mark} siqutaqoob TCOKWH 36 CAFE Continued...P(Al=x005 | O Hufman ole 6 Symbol Mage tga at IL Cie EV sage @ F O22 on ost 8 53 td @ é ow ow 1% oa a 8 DD Dd Oe oan on ow Oi P(F)=032 © c O18 sme 0.710 A 0S, 0.08 a 0.03 Encode, Toe Sywival | cadt_word wont | Provesility E 00 z oa E Ol L ow o 10 2 Ot c 10 2 cs A {ito y 0-05 6 at eV on 20.32) + 2002%)+aCore) + 3C013)+ 460s) + & C002) average Length , c 2.24 bits [symbol HGS) = 0-82 lags 11032) 0-8 logy C024) Dad lage COD) 4 HB toy, CUO. 13) + 6.05 Loge CLL d05) + 0-03 Cg OM #08) = q.rst bits | symbol Podundony = b ~ Hb) = 9.99 —9.UKHI = 0 0324 — Wits] symbolne 2019 131 QUESTION 3 (@) A source X produces letters from a two symbol alphabet with the probability assignment P, (0) = 0.8 and P.(1)= 0.2. Each source letter is transmitted through a channel as shown in Figure 2 and the transition probabilities are indicated as below. POR Figure 2 i, From Figure 2, what is the value of x? Prepare the conditional probability P(y}x) and joint probability tables PG). [5 marks} ii, Write all the steps to show that I(Y.X) equal to 1(%Y). {5 marks] Q Describe TWO characteristic of efficient source encoding. [2 marks} Continued... TooKwat WeTa ORMATION THEORY tase 2019 QUESTION 4 Error-control codes are classified into block codes and convolutional oodes. Differentiate between these two codes, [4 marks} (0) A (7.4) linear block code has a coefficient matrix, P as below: ii, iti, iv. G) 0 1 1 1 Construct a generator matrix, G for the linear block code. [2 marks] Based on the generator matrix, @ constructed above, construct a parity check matrix, [2 marks] Construct the table of error pattern and their corresponding syndrome for the linear block code, [2 marks] Assume a codeword c = [010100 1] is sent and the codeword r= [0101041] is received Show how the error can be detected and corrected, [2 marks] : . 7 > LP) iJ cil) Ho CDyq) PY _ tel ta “Cie =7 100 1001 O10 oll _y 1s tee aa bal 0 lio 0010 tl oool Continued... TOOKWH 36Gi Syndrome ror pattern, e| 060 0000000 Loo 1000000 0 Io 0100000 oo} 0010000 Tot 0001 600 ee O\l Ooo0100 lio 0000010 | ot? 4 put 000000) “ o.rut > |10 Ol 01 00} Olo1 oll 0000010 cz vr ¢e 010100] = oO colon +@ = 0000010 OF an+0 v’ tot | J ps+ps ‘pt Beto 3 HP ty Vala ie +>Bystumati2 down Ee Mon —Systamati don ep) = D> LP +0) mod 14040? . > Clb) 2 mtb) gD) . 408 mad 14040) 4p Ci40 4D) 4aot my = D+ pF tb m=O 194° yt Cheba Oates i QUESTIONS p+ 4 p4 4 pF tpt CRMOONONND @) A generator polynomial of a (7,4) cyclic code is given by rap s@=1eD408 Assume that a message 0011 is sent through the system. Find the valid code words in both non-systematic and systematic form. [4 marks} (®) Given the convolutional code of path 1 and path 2 as follow. Puth 1 Whe whim, ms aerate gs = [0,0,1} input |} = [11,0] : aid, i, _lustrate the convolutional encoder corresponding to the codes. [2marks] Frth 2 : ii ‘What are the values of code rate and constraint length of the convolutional encoder? | {2 marks] rey K=]t) *3 iii, By using the code tee, race the output of the encoder for the message sequence 1011, [4 marks} (Note: The message is input from right to lei) 00 fn 10 of 4 End of Page TOORWH 616