Eigen Systems Prepared by: Dr. Naresh Vasant\Afre nareshafré@gmiail-com Ph.D, M.Phil, M.Sc, JRF-NET, Mah-SET in Mathematics.Contents 1 Eigenvalues and Eigenvectors 11 Introduction 1.2 Eigenvalues 13. Bigenvectors 2 Diagonalization 2.1. Introduction 2.2 Similarity of matrices 2.3 Diagonalizable matrices 3 Cayley Hamilton theorem 3.1 Introduction 3.2 Theorem 4 Functions of square matrix 4.1 Introduetion 4.2. Using diagonalizability 4.3 Using Cayley-Hamilton theorem 5 Singular Value Decomposition 18. 19 a 29 29 29 36 36 36 401 Eigenvalues and Eigenvectors 1.1 Introduction Sometimes, the effect of multiplying a vector by a matrix is to produce simply a scalar multiple of that vector. For example, alll sh] We often’ want to know to what extent this phenomenon oceurs fora given, matrix. That is, we ask for all the instances of the equation)AX = AX, where Ais given matrix but we know neither the vector X northéscalar A. Because there are two unknowns on the right multiplied together, this is not a linear problom! If \ were fixed, and we only wanted to find all the appropriate X — vectors, it would be a standard linéar problem. Even easier is the case when x is known, and we only want-to.determiine A, each of these is a linear problem. But it is nonlinear if ) and X are both unknown. Therefore, we must expect to encounter somtie obstacles in solving this problem.1.2 Eigenvalues First we note that the vector 0 always solve the equation AX — AX. ‘We rule out that case as uninteresting. ‘The properly posed question is therefore For what scalars ) is there a nontrivial solution (X # 0) to the equation AX ‘This is known as the eigenvalue problem for the given matrix A. AX where the matrix A is given. Definition 1.1. Let A be any real square matrix. A number ) is an eigenvalue of A if the equation AX = AX is true for some nonzero vector X. ‘The vector X is an eigenvector associated with the eigenvalue A. In the example given in the introduction X= and X = (1,1) For notational convenience we will denote | xy | 2= (1,2, 5)" ts In matrix theory, some terminglogy has changed overtime. For example, eigenvalues havesbeen known as characteristic values, latent roots and eigenwerte. Proceduré to-find éigenvalues: If ) is an eigenvalue of A, then the equa- tion AX = \X\has a nontrivial solution. Consequently, the equation (A —\/)X = 0 has a nontrivial solution. Heriee, wé have Det(A — \/) = 0. So, for a given matrix A we are interested in finding the values of \ for which. Det(A — Al) = 0. Det(A — AJ) is called characteristic polynomial of A, which will be of degree n, if A is an n x n matrix. ‘The equation Det(A — M1) = 0 is called characteristic equation of A, solv-ing which gives n eigenvalues, Challenge 1.1. Find the 21 4-1 Solution: The characteristic equation of A is 2-A\ 1 aracteristic equation and the eigenvalues of the matrix A= Det(A—M) = 0. Next we expand the determinant and obtain: (2 d)(-1— ) 0. This leads to # — 4-6 20. ‘Therefore ” — \- 6 = 0 is the required characteristic equation of A solving it for A gives the cigenvalues of A Hence, the given matrix A has precisely two eigetivalues 4 = 3 and \ 7 00 Challenge 1.2. Find the eigenvalues of the matrix | —8 3 0 0 00 Solution: ‘The characteristic equation of A is Tar 0 oO 8 3-X 0 =0. 0 0 O-X On expanding we obtain: (7 — A)(3—A)(—-A) = 0. Hence, the given matrix A has threo eigenvalues \ = 7,3,0. Det(A - 1) =Another way to find eigenvalues for 2x 2 and 3 x 3 matrices: Note: Trace(A) denotes the sum of elements on the main diagonal of A 1. If Ais a2 x 2 matrix then the characteristic polynomial of A is given. by ¥? ~ Trace(A)A + Det(A) @) 2, If Ais a 3x 3 matrix then the characteristic polynomial of A is given by XS = Trace(A) + (An + Azo + Ass)A — Del(A) (2) Where Ajj is the deteminant of a 2 x 2 matrix obtained from A by deleting it row and i column. Solving Challenge 1.1 using this methiod. we get, ¥ — Trace(A)A + Det(A) = 4? — 1d + (—6). Which gives \? — \— 6 = 0 as the characteristic equation Challenge 1.3. Find:the eigenvalues of the matrix | 1 3 2 Solution! The characteristic polynomial is AS — Trace(A)\? + (Air + Azz + Ags) A ~ Det(A) == 5¥ + (4+ (-2) +6)A— SARS ON + 8-4. Solving A — 5? + 8d O gives 4 = 1,2,2 as the eigenvalues,ou Challenge 1.4. Find the eigenvalues of the matrix 10 Solution: The characteristic equation of A is 2? — Trace(A)A + Det(A) 041 Solving \? + 1=0 gives \ = i, i as the eigenvalues Observation: A real matrix may have complex eigenvalues ‘We will focus only on the matrices having real eigenvalues: More eigenvalues challenges in the noxt section.1.3 Eigenvectors In previous section we learned how to find the eigenvalues of the given matrix A. Here in this section, we are going to find the eigenvectors of matrix A by using its eigenvalues. i.e., we will solve AX = AX in which A and A(which we are able to compute now) are known, so this is a linear problem to.solve. 2 Challenge 1.5. Find the eigenvalues and eigenvectors of A — [ a ] Solution: ‘The characteristic polynomial is 0? — Trace(A)A + Del(A) =? — 4-5. Solving ? —4\ — 5 = 0 gives A= Now we fix one eigenvalue and find the corresponding eigenvector/s. Let A= 5 be fixed, we want to find a vector which satisfy AX\= X which is equivalent to (A —AI)X =0 IES) [2 JS] [ame ‘Therefore X = (1,2)"> (x1, 2x1)" = x, (1, 2)". So X; ~ (1,2)"is at eigenvector corresponding to A — 5. , —1 as the eigenvalues. a Note: Any nonzero multiple of X; is also an eigenvector of A comesponding to A=5. 1412 a]_[o 4 341} [nm] Lol’ Now fix A 22 zy oO hhich =| | which gives a = x 44|[n 0 BINS a ‘Therefore X = (xy, x2)? = (x1, -a1)? = 2(1,-1)*. So X = (1,—1)" is an eigenvector corresponding to 4 = consider (A ~ M)X = Dic., 7Observation: Any nonzero multiple of eigenvector corresponding to is again an eigenvector corresponding to the same A. 2 Challenge 1.6. Find the eigenvalues and eigenvectors of A = | 1 1 Solution: ‘The characteristic polynomial is 8 — Trace(A)A? + (Aq; + Agz + Ags)A — Det(A) == 6N + (3+3+5)A-6 = 6" 411-6. Solving 43 — 64 + 11 — 6 = 0 gives A = 1,2,3'88 the eigenvalues, For \ 2-1

Trace(A)X? + (Ain + Aaa + Ags) A — Det(A) SM +O + (64243)A46 = +6" +1146 Solving A* + 6A? + 11A + 6 = 0 gives A = —1, —2, —3 as the eigenvalues. ulOr As the given matrix is lower triangular, so the eigenvalues are the diagonal elements ie., 2, -3. are the eigenvalues of A For \=-1, a 0 consider (A + I)X = 0, a oO]. 3 0 a) x 0 2-2 0 || a|=|o 14 ts 0 ‘This implies that’ x, £42 ~ 213 = 0 and 2; = xy substituting in the first equation gives x3 = Srp ‘Therefore X = (1,2, 05)" = (2,2, 5x2)" =(03(1, 1,5)" Hence X; = (1, 1,5)" is an eigenvector ofA corresponding to 4 = —1 For \= 2 MN 0 0 a a sider (A +27) X = 0 2y-342 0 En 0 1 4-242] | as 0 10 0] [a [0 | 2-10{)\af=]o| 1 4 0p 0 ‘This implies that xy = 0 and 2m —.r2 = 0 which gives sry = 0 and there is no condition on 3, Therefore’X = (x1, 72,23)" = (0,0,25)" = 23(0,0, 1)". Hence Xz = (0,0, 1)7 is an eigenvector of A corresponding to \ = —2. For A= —3, -14+3 0 0 en 0 consider (A+3/)X=0,] 2 343 0 a) =| 0 1 4-243] | as o 15200)/a 0 20 0]/m|=|o 141] | os 0 ‘This implies that ‘t; + 41+ 75 = 0 and x, = 0 substituting in the first equation gives xs = —4ery Therefore X = (11, 2,23)" = (0,22, —4ra)” = x2(0,1, —4)" Hence X3 = (0,1,—4)? is an eigenvector of A corresponding to A. 2, —3. So the eigenvalues of A? + 5A — 2/ are (-1)? +5(-1) - 2, (-2)? +5(-2) — 2, (3)? +5(-3), 52 ie., the eigenvalues of A? + 5A — 2 are —6, —8, 8. ‘As the eigenvalues of A are —1 ‘We know that eigenvectors of A? + 5A — 2/ are same.as that of A. So eigenvectors of A? +54 — 21 are Xi = (04158), Xo = (0,0,1)? and X, = (0,1,-4)7. More Challenges: Q.1. Find the eigenvalues and-eigenvectors of the following matrices 8 -8 -2 fa -3 -2 3-41 AnswermAx I, 2,3and X, = (4,3,2)7, X = (3,2, )7, Xs = (2,1, 7 respectively 2a. a2 32 334 Answer: \ = 1,1,7 and eigenvectors are (1, 1,0) .(-1,0,1)7 corre- sponding to A= 1 and (1, 2,3)" corresponding to 4 = 7. 16Q.2. 122 212 221 Answer: \ = —1,~1,5 and eigenvectors are (—1, 1,0)", (-1,0, 1)" corresponding to 4 = —1 and (1,1,1)" corresponding to 4 23 4 02 001 Answer: \ = 1,2,2 and eigenvectors are (—7, 1,1)? corresponding to A= Land (1,0,0)" corresponding to \ = 2. 2.2 -3 2 1 -6 “1-20 Ans: \ = 5,—3,—3 and eigenvectotsiare (1,2, 1)" corresponding to A=5 and (—2,1,0)", (3,1,0)" corresponding to A = —3. 1/3 2/3 2/3 2/3 1/3 2/8 2/3 -2/3 1/3 Ans: \<1,1,SDand eigenvectors are (1,1,0)", (1,0,1)" correspond- ing toA-— Land (—1,1, 1)" corresponding to A= —1 ‘Two of thé eigenvalues of a 3 x 3 matrix are —1,2. If the determinant of the matrix is 4, find its third eigenvalue. Q3. Qa. 24 A= 03 ] , then find the eigenvalues and eigenvectors of A? 234 WA= [0 4 2 | , then find the cigenvalues of adj(A) and A*—2A+1. 003 72 Diagonalization 2.1 Introduction Diagonalizable matrices are of interest because diagonal matrices are espe- ally easy to handle. In this section we will try to express as A= PDP~ where D is diagonal matri he given matrix A and P is some invertible matrix. Not all matrices are expressible in such way but the matrices which HaSjsuch expression are called diagonalizable matrix, we will define diagonalizability formally ahead. Once we have such expression then we can easily find A" as A" = (PDP)(PDP>)... (PDP) = PD(P'P)D(P“!P)D--- D(P-!P)P-! = PD"P*) Similarly we will be able to compute e4, tan(A) and manymore functions of A which we will dis- cuss in detail, 182.2 Similarity of matrices Definition 2.1 (Similar). Two n x n matrices A and B are called similar if there exists an n x n invertible matrix P such that B = P~1AP. 2-3 10 For exampl is similar to B because 1-2 o 31 P= | | | works. To check B= P-MAP, we check that RB =yAP and observe that PB | ° 1? | — ap. 1-1 Some properties of similar matrices: If A and B are two similar matrices then the following holds: 1. Rank(A)—Rank(B). 2. Characteristic polynomial of A is same as characteristic polynomial of B. 3. Eigenvalues of A is same as eigenvalues of B, which implies that Det(A) = Det(B) and Tracé(A)-— Trace(B) 12 0 Challenge 21s the matrix A = is similar to B ? 03 12 Solution: As/we know that, If A and B are similar then Trace(A) Trace(B), here Trace(A) = 4 and Trace(B) = 5 not same ‘Therefore A and B are not similar a -16 Challenge 2.2. Is the matrix A is similar to B 0 6 02 Solution: No, as Det(A) = —6 and Det(B) = 6 not same. ‘Therefore A and B are not similar. 19In this section we saw some examples which tells us that there are matrices which are similar and there are matrices which are not similar, The next section focuses on the matrices which are similar to diagonal ma- trices to which we call as diagonalizable matrices. 202.3. Diagonalizable matrices Definition 2.2 (Diagonalizable). A square matrix A is said to be diagonal- izable if it is similar to diagonal matrix. i.e., if there exist a diagonal matrix D and an invertible matrix P(called modal matrix) such that A = PDP We state and use Necessary and sufficient condition for diagonal izability to answer whether the given matrix is diagonalizable. Definition 2.3 (Algebraic multiplicity). Let A be any square matrix and let \ be an eigenvalue of A then the algebraic multiplicity of \ is denoted as A.M(A) and is defined as A.M(A) = the number of times A appears as a root of the characteristic polynomial of A. Definition 2.4 (Geometric multiplicity), Let™A be any square matrix and et \ be an eigenvalue of A then thé-geometrie multiplicity of \ is denoted as G.M(X) and is defined as G.M(A) = the number of linearly independent eigenvectors associated with the cigenvalue Necessary nd sufficient Condition for diagonalizability: A square M(A) = G.M(A) for all the eigen- matrix A is diagonalizabledf and only if / values \ of A Some facts: 1. Every diagonal matrix A is diagonalizable. (since P = I and D = A works.) 2. Kigenvectors comesponding to distinet eigenvalues are linearly indepen- dent. 3. Necessary and sufficient condition and above point ensures that there are n linearly independent eigenves trix. tors for a diagonalizable m 214. The matrix D is obtained by placing the eigenvalues of matrix A along the diagonals and P is obtained by placing the eigenvectors as columns in the same order as that of the eigenvalues placed in D 5. A.M(A) > G.M(A) > 1, for any eigenvalue. , If A.M(A) = 1 then by above point G.M(A) = 1 7. Suppose matrix A has distinct eigenvalues i.e., A.M(X) — 1 for all the eigenvalues of A then G.M(A) = 1 for all the eigenvalues of A by above point. This implies that A.M(A) = G.M() = 1, forall the eigenvalues of A. So we can state the following If a matrix 4 has distinct eigenvalues then it is diagonalizable, converse of this statement is not te ‘We are now ready to face the challeniges related! to the diagonalizablity. ‘ck whether the matrix A 2 1 ~6 | is diago- Challenge 2.3. nalizable? If yes, find the diagonal matrix and modal matrix. Solution: The chatactetistic polynomial is Y® — Trace(A)A%+ (Ai: + Aap + Ags) A — Det(A) = X= (=I) + (12) + (3) + (-6)) 45 = 4? — 21 — 45 Solving AS + \? — 214 — 45 = 0 gives = —3,-3,5 he eigenvalues. ‘This implies that A..M(5) = 1 and A.M(3) =2 For 4 = —3, 243 2 x 0 consider (A +31). 143 m]= 1 2 043 rs 222 4 ~6]] a2 | =] 0], R:-2R, Ry + Ri gives -1 3 | [os 12-3] fx 0] 00 0 | |x o| 00 0 | [2s 0 ‘This implies that x, + 2x7 — 3x3 =Oie., x = 3x3 — 2x Therefore X »t3) = (823—2r, 22, 5)? = xa(—2, 1, 0)" +23(3, 0, 1)”. Hence X; = ( tod=-3, 7 and Xp = (3,0,1)" are cigenvectbts corresponding By construction X; and X2 are linearly independent, For A= 5, consider (A —51)X = 0, 2-5 2 3 x 2 1-5 -6 wy | = : -1 -2 0-5|] a 0 72 3] [a 0 2-4 ~6 | | x | =o )y Ri —7Rs, Ro + 2Rs gives -1 -2 -5 | 0 16 32 a _ | 0 | 0 -8 —16 0 |, Ri —2Ro gives [a 2 as “| Lo] o 0 0 0-8 0 -2 This implies chat x = —2r5 and —x; — 2x2 — dry = 0 substituting first ‘equation in second equation gives 1) = —n5. Therefore X = (11, 2,3)" = (-5, —215, 25)" = 25(-1,-2,1)" Hence X3 = (—1, ~2,1)7 is an eigenvector corresponding to A = 5. We obtained 2 linearly independent eigenvectors corresponding to 4 = —3 23and 1 eigenvector corresponding to 4 — 5, therefore G.M(—3) = 2 and G.) (3) =1 Hence by necessary and sufficient condition, A is diagonalizable. 50 0 1-23 D=]|0 -3 0 |,P=]-2 1 0 | are the required diagonal and 00 -3 1 01 modal matrix respectively. Observation: Matrix P and D are not unique. since we gould have placed the eigenvalues in the construction of matrix D in a different way, which will change P as well B -8 -2 Challenge 2.4. Check whether the matrix A=)| 4 —3 —2 | is diago- 3-4 1 nalizable? If yes, find the diagonal matrix and modal matrix. Solution: The characteristic polynomial is 8 — Trace(A)? + (Aj, + Ara Aly) A — Det(A) =~ (M4 ((-1) + 144 8)A—6 8 — 6A? 411A — 6. Solving 13 — 6? 4 11)~ 6 = 0 gives \ = 1,2,3 as the eigenvalues, As the eigenvaltes.are distinct, so A is diagonalizable. Now to find modal matrix we need eigenvectors corresponding to A = 1,2, 3. For \=1, 8-1 -8 -2 a consider (A — I)X 4-3-1 -2 |= , 3 -4 1-1] | a5 0 7-8 -2) [x 0 4-4 -2] | 2, | =| 0], Ri — Rp, Ro —$Rs gives 3-4 0 | | as 0 24o]fa 0 2} | a] =| 0], Ri~ Rs gives 0 Ty 0 oo offm 0 o 4 n 0 3 4.0 ry 0 ‘This implies that $22 = 2x3 ie., 2r2 = 323 and 3x, = 422, if we substitute ry — 2¢ then ry — 3¢ and x, — dt then X = (21,22, 25)" = (44, 3t, 20)" = e(4,3, 2)” Hence X; = (4,3,2)" is an eigenvector corresponding to. 1S For \= 2, g-2 -8 -a | fen 0 consider (A — 2/)X 4 3-2 2 |= : 3 1=2 3 6 -8 -2 ry 0 4 2 OR — 2Rs, Ro — 4Ry gives 30-4 -1 ry 0 oO ry 9 0 ay =] 0 3 = 0 ‘This implies-that 3x — 4x2 — 23 = 0 and xp = 2ry substituting in the first equation givesr, = 3:5 Therefore X = (21, 72,23)" = (3x5, 2x5, 23)? = 43(3,2,1)7. Hence Xy-= (3,2,1)7 is an eigenvector corresponding to d= 2 For, = 3, 8-3 -2 ry 0 consider (A—3/)X=0,] 4 3-3 -2 |] m |= : 3 4 1-3 rs 255-8 2] [x 0 4-6 -2] | a | =| 0 |, Ri —2Rs, Ra — $Rs gives 3-4 -2] | as 0 5-8 2] [nm 0 0 2 2 0 4 4-2] | as This implies that 5x, — 8x2 — Zrs = 0 and x» = ry substituting in the first equation gives 1 = 2x3 ‘Therefore X = (x1, 29,23)" = (2x5, .23,23)" = 43(2, 1,1) Hence X; = (2, 1,1)" is an eigenvector corresponding to X 100 432 D 0 2 0]|,P=|3 2 1 | are the required diagonal and modal 003 211 matrix respectively. - By referring to the problems solved in prévious section we observe that 1. Challenge 1.5 and Challenge 1.6 are diagonalizable as we have dis- tinct eigenvalues 2. Challenge 1.7.48 diagonalizable as A.M(-1) = 2 = G.M(-1) and AM(8) = 1 =G.11(8) 3. Challenge 1.8 is not diagonalizable as A.M(2) = 2 but G.M(2) =1 4, Challenge 1.9 is diagonalizable as A.M(1) = 3 know all diagonal matrices are diagonalizable. G.M(1). Infact we 26More challenges: Check whether the following matrices are diagonalizable? If it is, then find the diagonal matrix and modal matrix. 42 - 100 210 5 3 2 | Ans: D=]020)/,P=|111 241 005 421 210 0 2 1| Ans: \=2,2,2, Here A.M(2) =3 but)G.M(2) = 002 Hence the given matrix is not diagonalizable. 17 18 -6 -18 19 -6 | Ans: = 2,1,1\Here A.M(1) = 2 but G.M(1) =1 9 9 2 Hence the given matrix is not diagonalizable 2-11 202 -1 | Ans:\W=u/1,1. Here A.M(1) = 3 but G.M(1) 12 -1 Hence the given matrix is not diagonalizable. 2 100 20-1 121 }.Ans: D=|0 2 0],P 110 10 3 002 lol 5 00 1 5 1] Ans: \=5,5,3. Here A.M(5) = 2 but G.M(5) = 1 103 Hence the given matrix is not diagonalizable. aT1 20 10 23 Ans: D = 283 Cayley Hamilton theorem 3.1 Introduction We turn now to a remarkable theorem, named after Arthur Cay William Rowan Hamilton. This deals with the polynomials of the matrices. The theorem helps us to find A~ if exists and also giv for the higher degree polynomials of the matrix. 3.2. Theorem Theorem 3.1 (Cayley-Hamilton). Every square matrix satisfies its own characteristic polynomial, i.c., If p(A) is characteristic polynomial of a square matrix A then p(A) = 0. Challenge 3.1. Verify Cayley-Hamilton theorem for A and hence find A~ if exists. Solution: ‘The characteristic polynomial of A is given by AS — Trace(A}A? + (Ain An + Ags)A — Det(A) =A — 5 (<3) 5+ 5)A—3 = 5 HTAR3. ‘Therefore p(d) = X¥ — 547 ++7A— 3. ‘To verify Cayley-Hamilton theorem we need to check whether P(A) 184, To check A? — 54? + 74-31 0. For this we need values of A? and A’. 7 2 -2|)[ 7 2 -2 2% 8 8 A=] -6 -1 2 —6 -1 2 |=/-2 -7 8 6 - 6 2 - mM 8 WT 292 8 7 2 79 26 26 Ab=A=|-24 -7 8 6 -1 2 |=| -78 -25 26 [4 8 6 -1 78 26 25 79 26 26 % 8 8 Now A’ — 54? + 7A — 37 -78 -25 26 |-5|-24 -7 8 f+ 78 26-25 mM & 7 7 2-2 100 7 2|-3)o10 6 2 -1 001 79-125 +49-3 2640414 —26+40-14 000 =| -78+120-42 -25435-7 2-40+M P=] 0 0 0 78-120+42 26-40+ 14 254387 a) In the process of finding characteristic polynomiabof A, we found that Det(A) = 3, so AW exists, We verified that A* — 5A? + 7A ~ 31 = 0. Multiplying both side by A~* we get, A‘AS— 54-14? + TAA <3AC = Oe, A?-5A4 77-3471 = 0. Hence A* A? +5A 7H) 8 8 7 2 -2 100 =}/- ATO | +5 -Tlo10 8 7 oolChallenge 3.2. Verii and hence find A~ if exists, and A* Solution: The characteristic polynomial of A is given by A — Trace(A)A? + (Ai + Az + Aas)A — Det(A) M54 (84145)A—1 = M5 49A—3, ‘Therefore p(A) = 8 — 54? +9 — 3. To verify Cayley-Hamilton theorem we need to check whether 43 — 54? + 9A-I=0 For this we need values of A? and AS 1 2 -2/f 1 2 -2 sYo2 -4 w= 3.0 ]]-1 3 “of=;-4 7 2 0-21 0-2 2-8 f-1 12 -4)[w 2 -2 13 42-2 MaMA=|-4 7 2 30 9 wo [2-8 0-21 10-22 -3 -13 42-2 -1 12 -4 Now # —sAXp9A~ 7 =] -u 9 10 |-s|-4 7 2 | 4 10-22 -3 2-8 1 12 42 100 9]-1 3 0 |-|oi0 0-2 1 ool 13+549-1 4260418 2420-18 ] [00 0] >=] -11+20-9 9-35+27-1 1-10 |=|000 1-10 -22440-18 -3-5+9-1 00 °| In the process of finding characteristic polynomial of A, we found that Det(A) = 1, so A~! exists. 31We verified that A’ — 54? +94 —I — 0. Multiplying both side by A~* we get, A1AS—~5A7A? 4 947A — ATT = Oil, A?—5A491—-A1=0, Hence A? = —A?+5A—97 1 12 -4 1 2 -2 100 3.26 =-|-4 7 2 [+5] -1 3 0 |-9fo 1 0]=J1 pe 2-8 1 0-2 1 oon P25 ‘To find A* multiply the verified equation A® — 54? +9A — 1 =0 by A, we get A‘ = 545-94? +A 13 42 2 1 12 4 1 2 -l 9 Wl 4 7 2 1 0 10-22 2-8 1 0-2 —55 104 24 20-15 32 320 42 13 211 Challenge 3.3. Find the chatactotistic equation of the matrix A= | 0 1 0 112 and hence find the matrix represented by AS —5A7 4 7A®— 34° + A*—5AS 4 8A? 244 1. Solution: The characteristic polynomial of A is given by A’ — Trace(A)M + (Ai + Azz + Aas)A — Det(A) =MA 5M + (24342)A-3 =AP= 5? + 7A — 3, ‘Therefore \¥ — 5)? + 7A — 3 = 0 is the required characteristic equation TA — 3X8 + MAS BN? —2A4 1 by the characteristic polynomial AX — 5\? + 7A — 3, we get the quotient A +\ and the remainder 7 + A+ 1 So X8—5\7 +7" —3A° + M455 = 8? 2041 = (08 Now dividing the polynomial \* — 5. 5+ TA—3)OP + 32NFRHAFL In terms of A we have, AS —5AT+7A8 349 + At 5.4? 484? 244] = (AS— 5A? + TA—31)(A + A)+A?+A41. But by Cayley-Hamilton theorem A’ ~ 5A? 4-74 — 37 ‘Therefore AS — 5AT + 7A® — 34° + AY— 5A84+ 847-2441 =A? + AGT 211i}f2i1 544 ##=/o010 ol1o}=}o10 ri2}firi2 445 544 211] [100 855 Hence ?+At7 =] 0 1 0]+] 0 1 0|+{ 0 Ao }=]0 3 0 aa5] [112] [oor 558 12 Challenge 3.4. Apply Cayley-Hamiilton theorem to A [} 1 and deduce that AS = 6251 Solution: ‘The characteristic polynomial of A is given by AX — Trace(A)A + Det(Ay -0A-5 = 8. So by Cayley-Hamilton theorem A? — 51 = 0 ie., A? = 51 ‘Therefore A* = 547 = 6251 Challenge 3.5. Verify C ‘y-Hamilton theorem for A= | 20-1 and) hehce find A~? if exists. Solution: ‘The characteristic equation is M+ -5A-5 = 0. ‘To check whether A + A? — 5A — 51 =0 For this we need A? and AS. 3312 o0]fi2 0 500 A =/2 -1 0 2-1 =|050 00 -1 0 oo. fsoo]fi2 0 5 10 #=AA=]050]] 2-1 0 10 -5 foo1}loo ~1 0 5 10 0 500 A4A?-5A-51=| 10 -5 0 |+]o 5 5 0 0-1 001 oso -1 100 s]o10 ool 545-5-5 10-10 0 000 1-10 -58+54+5-5 0 000 0 0 -14145- 000 Multiply the verified equation A’ +4? — 5A — 57 = 0 by A“! (which exists as Det(A) = 5) we get, A? +A—5/-5A 1 S0ie, A? =1(A?+A-51) 500 a2 0 100 12 0 osafyf2-1 0 }-sjo10 10 oo oo -1 oo1 0-5 fi 2 0 1200 500 o fs]2 0 05 0 00 3 00 5 0 0 25More challenges: 2 1 Q.1. Find the characteristic equation of the matrix A 1 1 1-12 and verify Cayley-Hamilton theorem and hence, obtain A~ if exists 137 Q.2. Find the characteristic equation of the matrix A = | 4 263] and 124 AS — 2045 — 3447-443 90.42 — hence, find the matrix represented by 4’ B3A +1. cos 0 sit Q.3. Verify Cayley-Hamilton theorem for [ and obtain sin® cos At V2 4 Q.4. Verily Cayley-Hamilton theorem for AS)" | and evaluate 24% — 5AS—7A +61. 354 Functions of square matrix 4.1 Introduction In this section, we use the concepts which were discussed in the previous sections and apply it to find the function f(A) = A”, e4, tan(A), etc where A is given. 4.2 Using diagonalizability Ao 0» We note the following, if D is diagonal matrix then 0, 00 nN Mo 0 My 0 1 pn= ° 0 oo x eo 0 2 oo | &° 0 0 a) em tan(hs) 0 0 0 tanQy) 0 3. tan(D) = 2) 0 0 0 tan(a,) We know that if matrix A is diagonalizable then there exists an invertible =PDP> matrix P and an diagonal matrix D such that P-1AP = Dice, / 36So 1. AY =(PDP“)(PDP“)-.-(PDP-') = PDP 2et = 1+ Atha? 4 dA +. = PP? + PDP + SPD PT + EPD'P1 4. = PU + D+ kD +3 D%+..-)Pt = PeP P+ 3. c= Pc? P, where c is constant 4. sin A= P(sinD)P* 5. cos A = P(cos D)P-! 6. tan A = P(tan D)P*! So we keep f(A) = Pf(D)P~! in mind. 21 Challenge 4.1. If A= [ lo , find A Solution: The characteristic ¢quation of A is 2 — Trace(A)d + Det(A) = 4043 Solving \? — 4A +3 20 gives A = 1,3 as the eigenvalues For = ly 2- 0 comer A = 056, [7 rou He}-(3] 1 2-1 ey 0 ‘This imples that x1 = —x2 ‘Therefore X = (01, 22)" = (—x2, 22)" = xa(-1,1)" Hence X; = (—1,1)7 is an eigenvector corresponding to A = 1 For \=3, consider (A—31)X =0ie, |? 1 nm) _}° 1 2-3 zy 0 ‘This imples that sy = 22 37‘Therefore X = (a, 22)" = (x9,22)" = x9(1, 1)". Hence X2 = (1,1)" is an eigenvector corresponding to sop=|) °}.p=|t) | angptsa)*? 03 1 1a ‘Therefore 7 - 1 -11 4% = pp»p2=|—) 1 ° 5 1 1} {lo 3® la 143% -143% -1+3 143° 2 Challenge 4.2. If A= [ | » find e4 and 44 Solution: ‘The characteristic equation of A is \ — Trace(A) + Det(A) =N=3A42 Solving ? — 3A +2=0 gives A= 1,2 as the eigenvalues For A= 1 2 a consider (A- 1X -oierf?)* 2? ||] -]° 2 3-1|| x o ‘This imples that 2) ="Cr2 Therefore X 4 (04,23) = (—x2, 22)" = xa(-1,1)" Hence Xy<(=1,1)™is an eigenvector corresponding to 4 = 1 For A= 2, n|_[o 2] |x 0 ‘This imples that sry =r» ‘Therefore X = (11, 22)" = (x2, 22)" = xa(1, 1). Hence X; = (1, 1)" is an eigenvector corresponding to \ = w= [) o]m= [ot |] mars [Fi] 02 1 11 ‘Therefore consider (A — 2/)X = ie. [ 2 38Re ~ i [20 12] _ [10 6 *[ 12 20 6 10 -ld4 Challenge 4.3. If A = [ D4 | then prove that 3tam(A)-= (tan 3)A Solution: The characteristic equation is 2? — 0-9 = 0 which implies For \=3, consider (A —31)X = Oie. { 4 He]-[3] a 1-3] | m 0 ‘This imples that 7 = x2 Therefore X = (21, 22)" = (2, 22)” = x2(1, 1)". Hence X; = (1, 1)" is afi eigenvector corresponding to ct (:}-(] = ay(-2,1)" ‘Hence, Xp = (—2,1)" is an eigenvector corresponding to A sb-|) ‘)e-{t mes [2 ‘| 0 -3 1. -11 Therefore tan(A) = P tan(D)P—* _f1 -2] [tans 0 ],f 1 2 “tia 0 tan(-3) }F | -1 1 39 -14+3 4 consider (A31)X =0ic., 2 143 ‘This imples that 2) = —2o. ‘Therefore X = (21, 22)" = (—222,22)_1[ tan3 4tan3] asf -1 4 S| 2tan3 tan3 Sta. Hence 3 tan(A) = (tan 3)A. 4.3 Using Cayley-Hamilton theorem Previous method can be used only if the given matrix is diagorial that method has limitation. Cayley-Hamilton theorem can be used even if the matrix.is not diagonaliz- able By dividing f(X) = A", e’, tan(A), ete by the characteristic polynomial ch(A) of A, we got some quotient q(\) and the remainder r(A). Then f(A) = ch(A)q(A)+r(A) writing this in terms‘of A and applying Cayley- Hamilton theorem we get, f(A) = r(A) (since ch(A) = 0.) Now we know that either r(A) = 0 or dégree(r(A)) < degree(Ch(2)) ‘Therefore f(A) = a2A?-+a,A + ap/ if Ais of order 3x3 and f(A) = a, Ataol if A is of order 2x 2 ad -2 Challenge 4.4. 1f A= [0.2 1 |, find A™ 0 Solution: ‘The characteristic equation is AS — 247 + ((42) + (-1) +2) - (-2) =0 ive, MS — 2A? — \ | 2= 0 which gives A = 1,2, —1 as the eigenvalues. (Note that the given matrix is upper triangular, so eigenvalues the diag- ‘onal elements.) Hote f(A) = A! and A is of order 3 x 3 So lot A! = a,A? + a,A + aol, in terms of A the equation becomes N° = apd? + aA +a ‘The above equality is satisfied by the eigenvalues of A. 40Putting \— 1,2, —1 we get the equations a + ay +a) > 1 @) Aa + 2a, + ap = 2° (4), a2 — a; + 9 = (3) Adding equation 3 and equation 5 we get ay + a (6) Subtracting equation 3 and equation 5 we get a =0 (7) On resubstituting, we get a2 + a9 = 1 and 44y +a = 22% my) 224 «2!—1) a2") ST oes 0 0 2 3 so _ | ~M49 150 Challenge 4.5. If A — { ] prove that A%® = [ 150 1st | Solution: The characteristic equation is \? +24 +1 = 0 which implies A=-1, Here f(A) = A® and A is of order 2 x 2 So leb.A® — a,A + aof, in terms of \ the equation becomes N= ard + ap. ‘The above equality is satisfied by the eigenvalues of A. Putting A= —1, we get. = ay +a (8) aLSince the eigenvalue is repeated, so the eigenvalue will also satisfy the deriva- tive of 8 = aA + ao ie, 50M” = ay Putting A= —1, we get a = 50 (9) Solving both the equation gives ay = —49 and a, = —50. —100 —150 49 0 A® = aA + al = —50A ~ 497 = )-[* }- 150 200 0 49 M49 —150 150 151 Challenge 4.6. If A [; | find cos A Solution: The given matrix is upper triangular)'so the eigenvalues of the matrix A are the diagonal elements, fie. 7, 5 Let cos cos = aA + pI in terms of A the equation becomes nA + ao. ‘The above equation is satisfied by the eigenvalues of A Putting A= 7, we get ayn + a9 = —1 (10) Putting A = # we get 3 a =0 (11) On solving, we get a; — —2 and ay ‘Therefore cos A = 2A41= 42100 Challenge 4.7. If A=] 1 0 1 |, find A. o10 Solution: The characteristic equation is Md? 4 ((-1) 40+ 0)A-(-1) =Die, B-2-AF1=0 which gives A= 1,1,—1 as the eigenvalues Now as A is of order 3 x 3 Let A = aA? + aA + ao/ in terms of \ the equation becomes, °° = ag? + ad + ao. ‘The above equality is satisfied by the eigenvalues of A. Putting 4 = —1, we get a) — a, +9 (12) Putting \ = 1, we get ay +a; + a5 (13) Now as the eigen values \ = 1'is fepeated, so A = 1 will satify the derivative of = apd? + aA + ay = 048. BOA” = 2apd + ay. Putting A= 1, we get 50 = 2a; +4 (aa) Solving these equations, we get az = 25,a; = 0 and ay = ~24 loo 100 10 ‘Therefore A= 2547-247 = 25) 1 0 1] | 1 0 1]-24]/0 1 00 Hoe o1o0ollo1r08 100 100 100 =%/110]-2/010)=}2310 lol ool 25 01 43‘More Challenges: 23 JL IEA = 5 find A? @ —3 -4 l g2aua-| 7? 7 | ind sina 0 3/2 31 3A , find e and 54 ° [: | 445 Singular Value Decomposition Reference Applied Mathematies-IV Textbook by G. V. Kumbhojkar Theorem 5.1. A rectangular matrit Amn can be decomposed into the prod- uct of three matrices, an orthogonal matri: Umm, @ matrit Tyne, and the transpose of another orthogonal matrit Vaxn t¢., we have A=UydV" Since U and V are orthogonal matrices, we have U = Iq andeVVT = I, Moreover, the columns of U are orthonormal eigenvectors\of AAT and the columns of V are the orthonormal eigenvectors of A? A> is an m x n matrix whose 04; entries are the square roots of thé eigenvalues (this exists, because the eigenvalues of ATA are non-negativeyof ATA or AAT arranged in descending order and rest all entries zero The square roots of the eigenvalites of ATA or AAT are called sin- gular values. STEPS TO OBTAIN SINGULAR VALUE DE! A 3OMPOSITION OF 1. Compute A? A. and eigenvalues Ar, Az... An of ATA 2. Find:the singular values hy taking square roots of the eigenvalues o = \V% obtained in step 1 and construct > 3. Obtain orthonormal eigenvectors vy, v2, ..., ty of AA corresponding to the eigenvalues Ay, Az,..., An. The matrix V is a matrix whose columns are 01,02, 45 Un, 4. The columns of U is given by ty = EL Avy, ta = EAva,- thn = EAU: 5. The required SVD is A= U V7. 4544 Challenge 5.1. Find singular value decomposition of A = e381: Pear anomie stt=[ * 4] Solution: 1 4-3)f4a4 257 1. We first find ATA = = 43 |[-33 7 ‘The characteristic polynomial of ATA is \? — 50A + 576. falues of ATA are \; = 32 and Ay = 18 So the eigh 2. The singular values are o; = ¥32 = 4V2 and o; = V8 = 3y72. 4y2 Hence, 5 = va 0 0 3v2 3. Fix Ay = 32, consider [ATA — AIX = 0 heey [ATA — 327)X = 0 7 OT oO "|_| | this gives a= 2a, 7 -T}| mJ] Lo hence eigenvector is X; = (1, 1)7 Normalising X1, we get % = Y5(1, 1)" Fix Aj = 18, consider [ATA SA2I]X = 0 ice, [ Sere hence eigenvector 18 Xy = (—1, 1)7. 1 Normalising X3y we get v» = (1,1)? ‘ Bw 4\The first column of U is aayfs av 1 meade -= 3 2] 2a *]-[s] a ‘The second column of U is 44 w= tan=ay| 4 ‘ThereforeyV = | vfio Therefore, U = | } ol 4 2 bk stone | 4 4]-[1 °][4v2 ° |[ aoa 33 oijfo saj[a 4 Challenge 5.2. Find si sel ular value decomposition of A = | | 13 0 Solution: 3-1], io 0 2 3 11 1. We first find ATA=] 1 3 13 | 0 10 4 aide 242 ‘The characteristic polynomial of ATA is 22)? + 120. So, the eigenvalues of ATA are“, =12, Ay = 10 and 23 = 0. 2. The singular values are 0; = VT2 = 2V3 and 02 = VIO. Hence, 5 = | 23 ONG } 0 vida 3. Fix A: = 12, consider [A? A AyI]X = Oie., [ATA 12]X = 0 2 0 2) fn 0 0-2-4 | | 22 | =| 0 | this gives 2, = 25 and a, = 2g 2 & -10] | 2s 0 hence eigenvector is X; = (1,2,1)* Normalising X1, we get vy = e(1,2,1)? Fix Ay = 10, consider [ATA — AgI|X =O ie, [ATA — 10I]X =0 00 2)/a 0 00 4 | | 2 | =| 0 | this gives x3 =0 and x = 2, 24 -8| [a 0 a7hence eigenveetor is X2 = ( 1,0)", Normalising X2, we get vz = Jg(—2, 1,0)". Fix A; = 0, consider [ATA — \y IX = 0 ie., [ATA — O1)X =0 wo 2][n 0 0 10 4] | a | =| 0 | this gives 23 = Sar and 22 = 2m, 24 2] | as 0 hence eigenvector is Xs = (1,2, Normalising Xs, Therefore, V = 31 v= d= a Thereforey Ui 4 ys a Shh st 48Challenge 5.3. Find singular value decomposition of A= Solution: lol 2 1. We first find A?A = o1]= ri-ajyy, ‘The characteristic polynomial of ATA is 1? — 5A +6, So the eigenvalues of ATA are A; = 3 and Ay = 2. 2. The singular values are a; = V3 and 02 = V2 v3 0 Hence, 3 =| 0 y2 0 0 3. Fix = 3, consider [ATA — A I]X 2040, (ATA — 31)X [7 LoL pi pheonn =o 0 of La a hence eigenveetoisis X= (0, 1)". This Xy‘is already normalised, so vy = (0,1)". Fix @=2, consider [ATA — AIX = Oie., [ATA —2N)X =0 00 a oO = this gives x, = 0, [ot] ]- [amano hence eigenvector is X, = (1,0). This X, is already normalised, so v2 = (1,0)". ‘Therefore, v=] ° | 10 4. The first column of U is 49m=edn=z/0 1 B 1-1 ‘The second colump of U is: 1 1 ol J-s of 7¥ 1-1 Let the third column of U be us = (a,6,¢) Now as the columns of U are orthonormal, so 11-13 =0-gives a'+b—¢ = 0 and uy - us = 0 gives a + 0, we get Xs; = (=1,2,1), which on normalising gives uy = (— Therefore, U = 5. Hence [a 1 |= 50‘More Challenges: 1. Find the singular value decomposition of A = 6 Answer: 1 1 1 oF =| 4 t rij=|zZ° 4 1-1 o -1 0 Find the singular value decomposition of A = Answer: 12]_f[ 12} Find the singular value decomposition of A = Answer: Find the singulat yale decomposition of A Answer 4-3 tow) f4v2 43 5 OF 0 ao ingular value decomposition of A= 0 0 1a 11 le3.0 7. Find the singular value decomposition of A= ts sooree [2 ¢]-[ ][M 3] [I 45 References 1] Linear Algebra, Theory and application, Ward Cheney and David Kin- caid, 2] Schaum's outlines Linear Algebra, Fourth edition, Seymour Lipschutz and Mare Lipson. '3] Applied Mathematics IV,.G. V, Kumbhojkar. 52