Scribd
Upload
31 views

Classification of Markov Chain

Uploaded by

21bit026
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
31 views

Classification of Markov Chain

Uploaded by

21bit026
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
You are on page 1/ 6
Classification of Markov chains. Definition (Homogeneous Markov chain) LON LE RCO ed eden cutee ce Rc Ru clad an ra COs ene nee , Definition (Regular Markov chain) A Markov chain is called a regular markov chain if its tpm is regular. eetu 0.75 0.25 0 0.56 0.31 0.13 Consider P= | 0 05 0.5], then P? = ]0.30 0.45 0.25] Therefore P is regular. naan PCa ae eeu ene Pane) (ee) Consider P=] 0 1 0|,thenP?=|0 1 0],P=| 0 1 0 001 0 0 1 | Since P, P?, P® have zeros at identical places, therefore P is not regular. Oat ecg Definition (Irreducible (ergodic) and reducible Markov chains) A Markov chain is said to be irreducible or ergodic Markov chain when pi? Se ee Rae} eee cue see aL Note- (i) In an irreducible Markov chain, every state can be reached from every other state in finite De ee ese et ae ea ace cra (ii) If a Markov chain is irreducible, then all the states are recurrent. (iii) A Markov chain may be ergodic but not regular. Set ecg etna a) Enya) le ae 0 0.5 0.5], then P? = 30 0.45 0.25] Therefore P is irreducible. en 045 0.35 0.20 Sere 0 015 085 0 jer p_|032 9 0 068 Consider P = eae) eee) aur) 0 064 036 0 (a a eZ) 0 050 050. 0 (OZ pe [Ow oP OD ro ys ay cn vee a CU ss ho) 0 051 049 0 |’ ci nwa) 0 0.50 050 0 030 0 0 070 CeCe a ee a Therefore we see that all even powers of P have zeros at identical places and all odd powers of P have zeros at identical places. Therefore P is not regular. But P is ergodic as every state can be reached ee een ae Oe ae cg era Three boys A, B and C are throwing a ball to each other. A always throws the ball to B and B always throws ball to C, but C is just as likely to throw the ball to B as to A. Find the tpm and show that (i) the Markov chain is irreducible and regular. and (ii) all the states are aperiodic Le o 1 0 can Ce eae ea) id a we ean i PnP a SUP ees neat Lae aes Oe ee ee oe EC cae ee (i) Since pQ) icc ello Oat ecg ry a (ee eres emer meres Pl=|1/4 1/4 1/2],P°=|1/4 1/2 1/4 SU, eee Ue BURR Cee esse (ii) Since pS > 0, pQ) > 0, py) > 0, SS Bee ee Re ace Pe eRe eek Oe Ee Poe Oat ecg enna 0 2/3 1/3 Dees CRUG Rane Cae me ed Ue ea UC o1/2 1/2 0 seal Reo aL a 0 2/3 1/3 Soln. Let P= ]1/2 0 1/2 eee eee ROCCO UR RC ee ee ey a Ciel PSererererrey ie peed ei errs Cree tind cei oie nod eed cy Since p\") > 0 for n = 2,3,4,...Therefore the period d; = ged {2,3,4,...} = 1 ie. all the states are peers Set ecg

You might also like