STUDENT ID NO MULTIMEDIA Be i UNIVERSITY MULTIMEDIA UNIVERSITY FINAL EXAMINATION TRIMESTER 1, 2019/2020 TIT3131 — INFORMATION THEORY (All sections / Groups) 21 OCTOBER 2019 9.00 am — 11.00 am (2 Hours) INSTRUCTIONS TO STUDENTS 1. This question paper consists of 6 pages with Five questions only. 2. Attempt All questions. All questions carry equal marks and the distribution of the marks for each question is given. 3. Please print all your answers in the Answer Booklet provided.mmis__mformationTHgony _aiocroarraoi9 QUESTION 1 (@) Classify the elements of digital communication system for ‘X’ and ‘Y” in Table 1 with their respective output. Elements Output X | Analog symbol in waveform, suitable for transmission over the channel. ¥___ | Source code word. Table1 [2 marks] (b) A source emits one of 28 possible symbols from the letter ‘R’, “G’, “Band ‘A’, Find the value x in Table 2. Calculate the amount of information for all symbols. [5 marks} Number of characters | __ 2x 3x Sx. Letter [ere e B ou Probability (P)___| ‘Amount of Information | Jey), J Table2 ii, Refer to your answer in Table 2, write down your observation. What is the relation between the probability and the amount of information? {2 marks} ii, Show the information rate of this source if one of the 28 symbols is emitted, once every microsecond, [3 marks} Continued... CcTEO 216QUESTION 1, (0) Clmsity he lermens of cg commanicaton sytem for nd“ in Table 1 with their respective output. Elements Ouiput X —_ | Analog symbol in waveform, suitable for transmission over the channel. ¥ Source code word. Tabled [2 marks} Elemmnt X% + Modulator Elemine Y > Source amend Infomation THeom pom OF VIEW. Powis 9 campctrepesentatin of mesoe sgn Goat” Add redundancy Yo the sence code werd oder to conet errs (b) _ Asource emits one of 28 possible symbols from the letter ‘R’, ‘G’, ‘B’ and ‘A’. i, Find the value in Table 2, Calculate the amount of information for all symbols. [5 marks} ‘Number of characters 2x 3x & Se loiter Re rey RF tar ‘Probability (Px) 4g = | eae = any | ee= 21% [lore = by Amount of Information Iss) 2. FOTY 9.2224 1. o7y 1 aery LCS,) = log, CAD Table2 ii, Refer to your answer in Table 2, write down your observation. What is the relation between the probability and the amount of information? [2 marks} iii, Show the information rate of this source if one of the 28 symbols is emitted, once every microsecond, [3 marks} Ci) da t But Ant on * ob lyn = wed Cit) Wien we provetilty & Smallty, uy, amount OF infymetron vecowus igh ae suede, IL amount oP formation veld He ante ye pabelity of ocuur HawnCit) information ete R= rH r= once per wtarocecond, -\/4 ye (a . = 108s 1000 0005 HOS) = Cl ca-8 074) + CANA Ca.22) + GIA) L274) +61ML, 454) = 1.82G2 bits J eyo! Qe rH = 1.443 £ {000 000 ~ (994 dk its] seuru INFORMATION THEORY OCrOBER 2019 QUESTION 2 (2) Consider the following source with discrete memoryless source with source alphabet S = {So, $1, S2} where po = x, p= 2x, pz = 2x. Prove that H(S*) = 3 H(S). [4 marks} (©) A discrete memoryless source uses the symbols ‘8, ‘1’, ‘2, 3%, @, and ‘5? with the following probabilities. PCI") = 0.08, .07, 15, 17, 0.26, 27 i) Construct a Huffiman for this code. [5 marks] ii) Calculate the average length of the Huffman code and give the redundancy of the code. [2 marks} Decode “0100011110000”. [1 mark} cerEo SeiQUESTION 2 (2) Consider the following source with discrete memoryless source with source alphabet S = {So, $1, $2} where po= x, p1= 2x, p2= 2x. Prove that H(S*) = 3 H(S). [4 marks] Lt Wt ee = | wig. 1» Su | cg?) = 3x HESD = 3x 1522 we 09 = 4 bbb cow S| o $, G pawablitY Pe} 0-2 | Ob | ou exvory, HS |g ueag | oor | ovaeg | isa 923 9-3 aot Gaur , & | 480% | S60] ES] 6S,55} C89] SES | 85.1 S58] GSS probability Px | 0-008 | 0.016] 0-014 | 0-016 | 0.032 0.032 | 0.016 | 0.032 0.022 Extropy, H 0.0554 0.0959 0.02S3}0.02SS] 0.1584 | 0.1509] 0.0955 | o1seg| oct? SOUT , & | $So% {S508} SSS] 8550] 5,5] £552] 2,855. $8,8,| 25,8, probability, Pr. o.olb } 0-082) 0.052] 0.082 | oov4| 0.064 | 0.032 | 5. 64 | 0.064 Entropy, H 0.6996 osser | ost] 01684] 0.2638 | 0.2638 | giceg | o.rgaa | 0 2638 Gourw , & | 480% |$5oh] SS] 455] $5] 8,55] 2,551 £55) $55 probability, fe | em [P05] 052) 0952) pool] 0.064] 0.08 | ooby | o.0by Entropy, o.otss forse] ott | 0689] ort] 0.298] o1s9 | 0-238 | 0-638 Gass 0.0554 + C0015) + \acoicts) + F Com? 7 45b9F(b) A discrete memoryless source uses the symbols ‘$’, ‘1’, ‘2’, ‘3’, ‘4’, and ‘5’ with the following probabilities. @ G ® Q 0 i) Construct a Huffman for this code. [5 marks] ii) Calculate the average length of the Huffman code and give the redundancy of the code. [2 marks] iii) — Decode “01000111 100000". {1 mark] i) Huffinan — code Sypnidsl | Stage | tage Stage Suge IV stg V $ ot oat 0.30 rps —t—>0-6F Ce] 5 oa one 0.24 Po0.20 t8} Psous oa 4 Olt ° Son ty | Portes 7 [ec Leenm pone 1 0-08 [81 ous 2 o.0Foy Ending Table Leymbol Praabilty Caan wergat | 0.08 0000 4 2 0.0F 000 | L 3 0-15 00\ 3 4 OF WV 2 5 0.26 \o 2 ¢ 0. 2F él 2Gi) awoage unt , C= 08) +09 LH + 193) + UW.MEIt Corb) + (0-24)02) = 745 0-08 tog, C 170-08) + oF log, Ct/oor) + OS loon C 1 /0-18) + H(S) = 61} logy CL /O1F) tO. 4 Log, Cilo.26) + 0-07 [og 2 Cl/o 2%) 2.4208 redundancy = £ -HCs) = a-45 ~ 2.4208 o-04s iii) Decode “10000000 0! ooo) Il 49 0000 ¢ 2 4 5 |QUESTION 3 @ A source X produces letters from a two symbol alphabet with the probability assignment P, (0) = 0.5 and P, (1) = 0.5. Each source letter X is transmitted through a channel from a source X to the destination Y as shown in Figure 1. The transition probabilities are indicated as below. PCYAIK) a = ey, Figure 1 i, From Figure 1, what is the value of &? Prepare the conditional probability P(yfx) and joint probability tables P(y,x). [5 marks} ii, Write all the steps to show that I(¥,X) equal to 1(%,Y). [5 marks} (©) Explain whether code I and code II in Table 3 are uniquely decodable using Kraft inequality. (2 marks oan wst_ayefc odt Lene ‘Symbols | Codel_ UZ] Code It Sete EET 1 o En { 00 110 el 001 100 Ss Oi, iit Table3 : -> K oe att p44 200%) cols T “ -L : + x7) Kent: ? ae = 0. &t§ Continued... ceTES 46QUESTION 3 (@ A source X produces letters from a two symbol alphabet with the probability assignment P:(0) = 0.5 and P;(1)= 0.5. Each source letter Xis transmitted through channel from a source X to the destination Y as shown in Figure 1. The transition probabilities are indicated as below. POX) <> PLY |x) = eeu.) [Peo / Ply,2) = PLYIX) * PO) Figure 1 i, From Figure 1, what is the value of A? Prepare the conditional probability P(y/x) and joint probability tables P(y,x). [5 marks} ii. Write all the steps to show that I(Y,X) equal to IXY). [5 marks] Poy, | X,) Yi Yo Ys 0.19 0-23 € > 18h Gr) Wr 0.62 oe kK h, > tndd 4% 4 k= 1-0.19- 0.23 = 0.58 k2|- 0.4 - 9.02 = 0-56Conditnna | Probability PCylx) vy, Yo Us 1 6.19 0.23 0.58 vom 0.02 6.40 0.58 Tomt Probavility Table PLy, x) Vly,2) = PCY|X) ® pu) Wi Ya Ve pcx) x, 0.19 x 0.5 | 023 x05 0.58 x0-6 . = 0.095 |= ONG |= 0.24 Ob 0.02 X05 | 040K05 6-68 0-5 Ar . 0.5 =0.01 =0.% |=0.24 CD} 0.105 | 6315 0.58 | conditional probability — POY) rodyd= Peau) | ply) L, Ad u 0.0957/ 0.105 0.0) [0.\0% 1 }= 0.9048 = 0.0952 0-5] 0.318 0>0 (0-418 Wa |=o.3051 |= 0.6349 u 0.4 [ost oT o-c€ ° = 05 2 05ICY|X) = LCx,y) ui) HOY) — HOY] = HOO ~ HOY) \.3222 - 1.24%] = J = 0.4964 0 O14 = 6. O74] HOD) = 0-105 — log, (ifo.105) + 0-315 log, Ci/o. 31S) + 0-68 — oy, C1} 0-58) = |.3222 HC) <3 0-5 log, Cte e 4 HICK |X) = 0-095 toy, C1008) + O05 ag 0.28) + 0.74 log, C1 / 0-68) + 0.0 Ings (10-02) + 0.9 Ing, Lilo H+ om fog, (1 058) = 1.2481 HOY) = 6-095 loge © 1/ 0-908) + 0.01 logs CU 00950) + 0. 5 Log, C0. BEI + 0.9 tag, 1/0 6344) + 0-24 log, CUS) + 0.94 Loy, C105) > 0. Fastmimiai_______mrormanon pony _____21 actoner ate QUESTION 4 (@) What is systematic and non-systematie codes? [2 marks} (b) Identify whether the following statements in Table 4 is TRUE/FALSE. {2 marks} Property _ Statements TRUEFALSE, Linearity | A cyclic code is not a linear black code property Cyelic property | The sum of two code words is also a code| word. Tabled (©) Construct the generator matrix, g(D)=1+ D+ D* + D* in "systematic" form. [3 marks] @ Construct all the valid code words for the following messages in "non- systematic" form. Show your steps. [ ‘Message m(D) (D) ‘Code word, { 110 1+D tp 14D? +Di Jove 1101 1+ 0 +p 1D 11000 J [2 marks] (©) Consider the generator polynomial for a (8, 4) cyclic code generated by g(D)=1+ D+ D? + D'. Construct the valid code words for the following messages in "systematic" form. Show your steps. Message m(D) ) ‘Code word 0100 D ote +y tt Toil o10o 1100 14D it 1ED+P 10’ Cd) = (pt) CD) wod 1+D408 +D* D+) marks] ED! 7 = ps toeptpe —-PtDE4D TD ? ee : DE + pts p40 F pb Att te wo) = bo) + OF ak Pp +41 + pep to dl een cere 316QUESTION 4 (@) What is systematic and non-systematic codes? [2 marks] (b) Identify whether the following statements in Table 4 is TRUE/FALSE. [2 marks] Property ___ Statements TRUE/FALSE pear [ara dan wet aa fe FALSE Cyotie property | The sum oF two code words is also a code] Fy) F word, uy adic sit of « edt & abo 0 cle wn Table 4 CA) cystematiz — coco Blok codes in which «message bits are crans mitted in unaltered — fowm won systomati2 code Glouk aS When esa Wits are ‘dransmaitted tn QHyred form (c) Construct the generator matrix, g(D)=1+ D+ D* + D* in "systematic" form. [3 marks](©) Consider the generator polynomial for a (8, 4) cyclic code generated by g(D)=1+ D+ D* + D*. Construct the valid code words for the following messages in "systematic" form. Show your steps. Message m(D) tp tl ED + Dd. veo) = DE CID) md tpt? tpt PHO prep Biss td = OF + D> mod (4D+p? tpt v +d =o 4D wip) = v0) + (4 +05) = ptr + De +S Dave! tot D4) = D4 ot 4 vh4r v Pipa oe Ae eso pip) = Dt m(D) med g > Of + o' +p? +d Vth +e +p 2 Dt Citp +o to) mm 14040 4D Reo wy 2 = ptaph+ pt +o? mod (4D4DttD* 67 +1 =P tl wo) = Oo) + DT gntDy = D4 + Di+p>tp° +97 = [tpt pt ey? t ob thtQUESTION 5 (@) Construct the encoder circuit for the generator polynomial for a (7, 4) eyclic code generated by g(D)=1+D+D*. [4 marks] (b) . Construct a state diagram for a convolutional path gi(1,1,1] and gof1,1,0] from Table 5. mo | m | m | & | Current | Next Japut State_| State ooo fo 00 0 GSE ESE EN BNET SS NN RNG ~ RS OE a 10 | 01 1a ooo 0a oo oor Fe or_10 CO a nH [ 01 oa oa i Table 5 [4 marks] (©) Trace the path of the code tree in Figure 2 corresponds to the message sequence 1010, 1011, 1100 and 1110. (Note: The message is input from right to left) eee ees Lae eo | 10 ah ep iL | Figure 2 {4 marks} End of Page CCTEO 616(b) . Construct a state diagram for a convolutional path gi(1,1,1] and gof1,1,0] from Table 5. me] mm [om [a | & | Came] Nat Taput State_| State ; a a ingut O soll a a a 1 dashed ea eT Seo ot a [4 marks] (c) Trace the path of the code tree in Figure 2 corresponds to the message sequence 1010, 1011, 1100 and 1110. (Note: The message is input from right to left) Figure 2 {4 marks} \o\0 eadg ; FO OL [ONL