Quadratic Forms
Definition A quadratic form on Rn is a real-valued function of
the form n n
Q ( x1 , x2 ,× × ×, xn ) = åå aij xi x j
i =1 j =i
j ³i
Example
Q ( x1 , x2 ) = a11 x12 + a12 x1 x2 + a22 x22
Q ( x1 , x2 , x3 ) = a11 x12 + a12 x1 x2 + a13 x1 x3 + a22 x22 + a23 x2 x3 + a33 x32
In matrix form
é 1 1 ù
ê a11 2
a12 ×× a1n
2 ú é x1 ù
ê1 1 úê x ú
Q ( x1 , x2 ,× × ×, xn ) = x T Ax = [x1 x2 × × xn ]ê 2 a12 a22 ×× a2 n ú ê 2 ú
2 úê × × ú
ê
ê ×× ×× ×× × × úê ú
ê1 a 1
a2 n
x
× × ann ú ë n û
êë 2 1n 2 úû
Definiteness
Definition Let A be an (n ´ n ) symmetric matrix. Then A is
(a) positive definite if x T Ax > 0 for all x ¹ 0 in R n .
(b) negative definite if x T Ax < 0 for all x ¹ 0 in R n .
(c) positive semi-definite if x T Ax ³ 0 for all x ¹ 0 in R n .
(d) negative semi-definite if x T Ax £ 0 for all x ¹ 0 in R n .
(e) indefinite if none of the above.
More on Definiteness
When n=1
Quadratic form:
Q ( x1 ) = ax12
The function is always positive for all x ¹ 0 if a > 0 . Thus A = [a] is
positive definite if a is positive.
The function is always negative for all x ¹ 0 if a < 0 . Thus A = [a] is
negative definite if a is negative.
Q ( x1 , x2 ) = a11 x12 + a12 x1 x2 + a22 x22
When n=2
Matrix form:
See the figures in the next few slides
Principal Minor
Definition Let A be an (n ´ n ) matrix. A (k ´ k ) sub-matrix of A
formed by deleting (n - k ) columns and the same (n - k ) rows
of A is called a kth order principal sub-matrix of A. The
determinant of a (k ´ k ) sub-matrix is called a kth order
principal minor.
Example
é a11 a12 a13 ù
A = êa21 a22 a23 ú
ê ú
êëa31 a32 a33 úû
Third order principal minor: n=3. k=3 – deleting no row/column
det ( A)
Second order principal minors: n=3, k=2 – deleting 1 column and row
a22 a23 a11 a13 a11 a12
= a22a33 - a23a32 = a11a33 - a13a31 = a11a22 - a12a21
a32 a33 a 31 a 33 a21 a22
First order principal minors: n=3, k=1 – deleting 2 columns and rows
det (a33 ) = a33 det (a22 ) = a22 det (a11 ) = a11
Leading Principal Minor
Definition Let A be an (n ´ n ) matrix. The kth order principal sub-
matrix of A obtained by deleting the last (n - k ) rows and the
last (n - k ) columns from A is called the kth order leading
principal sub-matrix of A denoted as Ak . The determinant of Ak
is called the kth order leading principal minor of A denoted as
Ak
Third-order leading principal minor – determinant of the original matrix
Second-order leading principal minor – deleting row 3 and column 3
a11 a12
= a11a22 - a12a21
a21 a22
First-order leading principal minor – deleting rows 2, 3 and columns 2, 3
det (a11 ) = a11
Condition for Positive Definiteness
Theorem Let A be an (n ´ n ) symmetric matrix. Then, A is
positive definite if and only if all n leading principal minors
are positive.
Condition for Negative Definiteness
Theorem Let A be an (n ´ n ) symmetric matrix. Then, A is
negative definite if and only if its n leading principal minors
alternate in signs as follows:
( k
sgn( Ak ) = sgn (- 1) ) k = 1,2,× × ×, n
i.e.,
A1 < 0, A2 > 0, A3 < 0,× × ×
Condition for Semi-Definiteness
Theorem Let A be an (n ´ n ) symmetric matrix. Then, A is
positive semi-definite if and only if all of its principal minors (not
just the leading principal minors) are non-negative.
Theorem Let A be an (n ´ n ) symmetric matrix. Then, A is
negative semi-definite if and only if every principal minor of
odd order is its 0 and every principal minor of even order is
0.
Example
æ 1 2 0ö 1 2 0
ç ÷ 1 2
A = ç 2 4 5÷ A1 = 1 A2 = =0 A3 = 2 4 5 = -25 < 0
ç 0 5 6÷ 2 4
è ø 0 5 6
A is indefinite
æ-1 1 0 ö -1 1 0
ç ÷ -1 1
B = ç 1 -1 0 ÷ B1 = -1 B2 = =0 B3 = 1 -1 0 = 0
1 -1
ç 0 0 - 2 ÷ø 0 0 -2
è
Furthermore, all other first order principal minors are non-positive,
and all other 2nd order principal minors are non-negative.
Therefore B is negative semi-definite.
æ 2 1 0ö 2 1 0
ç ÷ 2 1
C = ç 1 1 0÷ C2 = =1 C3 = 1 1 0 = 2
ç 0 0 2÷ C1 = 2 1 1
è ø 0 0 2
C is positive definite
Quadratic Form Restricted to Linear Space
Consider a general quadratic form of two variables.
éa bù é x1 ù
Q ( x1 , x2 ) = ax1 + 2bx1 x2 + cx2 = [x1 x2 ]ê
2 2
úê ú
ë b c û ë x2 û
(x1 , x2 ) is allowed to have values in the set that satisfies
Ax1 + Bx2 = 0
Rewriting the quadratic form as:
2
æ Bx2 ö aB 2 2bB 2 2 aB 2 - 2bAB + cA2 2
Qç - , x2 ÷ = 2 x2 - x2 + cx2 = 2
x2
è A ø A A A
Q is positive (negative) for all values of x2 if and only if
aB 2 - 2bAB + cA2 > (< )0
Note that
é 0 A Bù
aB 2 - 2bAB + cA2 = - det ê A a b ú
ê ú
êë B b c úû
Quadratic Form Restricted to Linear Space
Theorem Consider the quadratic form Q ( x ) = x Ax where x is
T
n-vector and A is an (n ´ n ) symmetric matrix. The variables in
the quadratic form are allowed to take values in the set Bx = 0
where B is an (m ´ n ) matrix representing m linear constraints.
Construct an ((m + n ) ´ (m + n )) matrix H as
é0 Bù
m ´n
H = ê m´Tm ú
(m + n )´(m + n )
êë B
n ´m
Aú
n ´n û
(a) If the last (n - m ) leading principal minors of H matrix
n
alternate in sign, with det H having the same sign as (- 1) ,
Q is negative definite on the constraint set Bx = 0.
Quadratic Form Restricted to Linear Space
(b) If the last (n - m ) leading principal minors, including det H ,
m
have the same sign as (- 1) , then Q is positive definite on the
constraint set Bx = 0 .
(c) If both of the conditions (a) and (b) are violated by non-
zero leading principal minors, then Q is indefinite on the
constraint set.
Example
Q ( x1 , x2 , x3 , x4 ) = x12 - x22 + x32 + x42 + 4 x2 x3 - 2 x1 x4
é1 0 0 - 1ù é x1 ù
ê 0 -1 2 0 ú ê x2 ú
= [x1 x2 x3 x4 ]ê úê ú
ê0 2 1 0 ú ê x3 ú
ê úê ú
ë- 1 0 0 1 û ë x4 û
x2 + x3 + x4 = 0 x1 - 9 x2 + x4 = 0
é0 1 1 1 ù
B=ê ú
ë1 - 9 0 4 û
Example
é0 0 0 1 1 1ù
ê0 0 1 -9 0 1 ú
ê ú
ê0 1 - 1 0 0 - 1ú
H =ê ú
ê1 - 9 0 - 1 2 0 ú
ê1 0 0 2 1 0ú
ê ú
ë1 1 - 1 0 0 1 û
Need to check the last (4-2)=2 leading principal minors: H itself and
é0 0 0 1 1ù
ê0 0 1 - 9 0ú
ê ú
H 5 = ê0 1 - 1 0 0 ú
ê ú
ê 1 - 9 0 - 1 2 ú
êë1 0 0 2 1úû
Need to check the signs of determinants of the two matrices.
If both are positive, Q is positive definite on the constraint set. If
determinant of H is positive while determinant of H5 is negative, Q is
negative definite on the constraint set.
