Short notes on Linear systems theory

by Sanand D
“Sometimes the truth isn’t good enough, sometimes people need more...” -Batman

————————————————————————————————————————-

1 Introduction
The various types of systems are
1. Linear/Non-linear
2. Continuous/Discrete
3. Time invariant/Time varying
4. Deterministic/Stochastic
5. Lumped/Distributed
6. Finite/Infinite dimensional
7. Autonomous/Non-autonomous
8. Dissipative/Lossless.
In here, we study linear systems which one must understand before studying more general non-
linear systems and complex systems occuring in many real life applications. Some examples of
linear systems are electrical circuits with linear elements (e.g., RLC circuits), spring-mass-damper
systems, linear filters and so on. One can study non-linear systems locally around an equilibrium
point by local linearization. A system is said to be linear if the superposition and the homogeneity
principles hold.
Study of systems mainly involve: modelling, analysis and synthesis and design. Modelling is
done using first principles i.e., using Euler-Lagrange equations, Kirchoff/Newton’s laws, conserva-
tion laws (e.g., Maxwell’s equation, heat equation, wave equation) etc. We consider state-space
models which can be obtained from n−th order odes by introducing auxiliary variables. The number
of states is given by the number of independent energy storing elements in the system. Sometimes,
linear systems can also be represented in the input-output form using a convolution operator e.g.,
signal processing systems in communication engineering. The other form of modeling is known as
system identification where one tries to estimate the system equations by probing the system with
known input signals and observing the corresponding output. Modeling allows us to represent a
physical system using mathematical expression and allows us to determine the system properties
using mathematical concepts. It allows us to predict the system response to various input signals.
Analysis and synthesis involves studying intrinsic system properties such as controllability,
observability, stabilizability, controllable/unobservable subspaces, controllability indices, poles and
zeros etc. Designing control systems involve: control, estimation and optimization. We consider
continuous time systems of the form
ẋ = Ax + Bu
y = Cx + Du (1)
and discrete time systems of the form
x(t + 1) = Ax(t) + Bu(t)
y(t) = Cx(t) + Du(t) (2)
where x ∈ Rn , u ∈ Rm , y ∈ Rp , A ∈ Rn×n , B ∈ Rn×m , C ∈ Rp×n , D ∈ Rp×m . In other words, these
are the models for systems under consideration, if A, B, C, D are time dependent, then the system
is called linear time varying system (LTV) whereas, if they are time independent, then it is called
linear time invariant (LTI) system. These are called non-autonomous systems i.e. systems with
inputs. The simplest example of LTI systems ´ t is given by an integrator system where the output
is the integration of the input i.e., y = 0 udt. Let x = y, then ẋ = u. This is a scalar LTI
system
´ t ´where A = 0, B = 1, C = 1 and D = 0. One can also consider a double integrator where
s
y = 0 0 udsdt. Let x1 = y and x2 = ẋ1 , then ẋ1 = x2 and ẋ2 = u. (Find A, B, C, D.) Autonomous
systems are of the for ẋ = Ax where there is no input (e.g., LC circuit with no current or voltage
sources). The advantages of state space models is that both LTI and LTV can be studied. For
LTI systems and signal processing system, transfer function approach is also applicable (using the
Laplace and Fourier transforms). State space models also allows us to study the internal dynamics
of a system.
In general, one can consider the following questions for linear systems

1. How to drive the state from an initial condition to the desired state, how to choose a control
input?

2. What is controllability and observability? How to check if a system is controllable/observable?

3. What is stabilizability? How to check stabilizability and how to make a system stable?

4. What is the optimal control input for a state transfer, e.g., least energy input?

5. How to compute the energy required for a state transfer?

6. How to estimate states from the measured outputs?

7. What is state feedback/output feedback?

8. How to build a state/output feedback controller?

9. How to control a system in the presence of disturbance and noise?

10. How to study non-linear systems locally using linear models?

We need linear algebra (refer short notes on linear algebra) and linear odes to address the above
questions.

2 Linear ODEs
Consider a linear ode p(D)y = u where p(D) is a polynomial involving differential operator D.
Consider a homogeneous ode p(D)y = 0 and let yh be a solution of this homogeneous ode. Let
yp be a particular solution of the ode p(D)yp = u (where yp depends on u). A general solution
can be written as y = yh + yp . This is true for both time varying and time invariant odes as long
as they are linear. This not only holds for scalar odes above but also holds for vector odes. By
using additional variables, scalar odes can be converted to vector odes which are in a first order
form (i.e. only first order derivatives are involved). Roots of the scalar ode and eigenvalues of the
corresponding matrix in the first order form are the same. (This can be shown using ideas from
Quotient spaces from linear algebra and the proof is given in the linear algebra notes. Moreover, the
matrix obtained in the first order form is a companion matrix. Thus, its characteristic polynomial
is same as the characteristic equation for ode again from linear algebra.)
For linear time invariant odes, one can also apply Laplace transform methods to find a solution.
One can write down a solution of linear odes explicitly. This is called as a closed form solution
which is the sum of the two terms as explained above. By the existence and uniqueness theorem
for odes, a unique solution always exists. The terms such as a solution of an ode, a trajectory, an
integral curve are all synonymous. We assume that the reader is familiar with definitions of linear
ode, impulse response, causality, LTI systems and transfer functions. Let u(t) = δ(t) be an input
to a system = Ax + Bu. The homogeneous solution is xh = eAt x(0) and a particular solution
´ tẋ A(t−τ
is xp (t) = 0 e ) Bδ(τ )dτ = eAt B. Thus x(t) = eAt x(0) + eAt B. Suppose u(t) = u ejωt . Then,
0
xp (t) = xm e . Substituting xp (t) in the system equation, xm = (jωI − A)−1 Bu0 and x(t) =
jωt

eAt x(0) + (jωI − A)−1 Bu0 ejωt . The first term represents the transient response and the second
term represents the steady state response. Let y(t) = Cx(t) be the output of the system. Hence,
the impulse response is CeAt B and the frequency response is C(jωI − A)−1 Bu0 ejωt (Kwakernaak
and Sivan).
As far as the existence and uniqueness of solutions of an ode is concerned we refer the reader
to any standard textbook on ode. It turns out that for the linear odes under consideration, there
exists a unique solution which can be written in a closed form using the state transition matrix as
shown in the next section.

3 State transition matrix

We show in this section that for linear ODEs, the state transition matrix determines the evolution
of a solution trajectory depending upon the initial conditions.

Definition 3.1. The state transition matrix for a linear homogeneous system (or autonomous
system i.e. a system without any inputs) ẋ = A(t)x is defined as
ˆ t ˆ t ˆ s1
Φ(t, t0 ) := I + A(s1 )ds1 + A(s1 ) A(s2 )ds2 ds1 + . . . . (3)
t0 t0 t0

The state transition matrix has the following properties:

d
1. dt Φ(t, t0 )
= A(t)Φ(t, t0 ), Φ(t0 , t0 ) = I.
Proof: From the definition, and the fundamental theorem of calculus,
ˆ t ˆ t ˆ s2
d
Φ(t, t0 ) = A(t) + A(t) A(s2 )ds2 + A(t) A(s2 ) A(s3 )ds3 ds2 + . . .
dt t0 t0 t0
= A(t)Φ(t, t0 )

2. x(t) = Φ(t, t0 )x(t0 ) is a solution trajectory.

Proof: ẋ(t) = Φ̇(t, t0 )x(t0 ) = A(t)Φ(t, t0 )x(t0 ) = A(t)x(t).

3. Φ(t, s)Φ(s, τ ) = Φ(t, τ ).

Proof: x(t) = Φ(t, s)x(s) and x(s) = Φ(s, τ )x(τ ). Thus x(t) = Φ(t, s)Φ(s, τ )x(τ ). Moreover,
x(t) = Φ(t, τ )x(τ ). Then by existence and uniqueness of solutions of ODEs, Φ(t, s)Φ(s, τ ) =
Φ(t, τ ).

4. Φ(t, τ )−1 = Φ(τ, t).

Proof: From the previous property, Φ(t, τ )Φ(τ, t) = Φ(t, t) = I. Therefore, Φ(t, τ )−1 =
Φ(τ, t).

For LTI systems, A(t) = A and therefore from the definition of the state transition matrix (STM),
Φ(t, t0 ) = eA(t−t0 ) . Thus, by Cayley-Hamilton theorem, we can express eA(t−t0 ) as a linear combi-
nation of I, A, . . . , An−1 with time dependent coefficients. Note that AeAt = eAt A.
 For non-homogeneous linear ODEs,
ˆ t
x(t) := Φ(t, t0 )x(t0 ) + Φ(t, τ )B(τ )u(τ )dτ. (4)
t0
ˆ t
⇒ ẋ(t) = A(t)Φ(t, t0 )x(t0 ) + A(t) Φ(t, τ )B(τ )u(τ )dτ + Φ(t, t)B(t)u(t)
t0
= A(t)x(t) + B(t)u(t).
Thus, the closed form expression for x(t) satisfies the ODE hence, it is a solution.
´ t Observe that a so-
lution is a sum of a homogeneous response Φ(t, t0 )x(t0 ) and a forced response t0 Φ(t, τ )B(τ )u(τ )dτ .

4 Change of basis and system representation

Let (A, B, C, D) be representation of a system w.r.t. a given basis for a state space.
ẋ = Ax + Bu, y = Cx + Du
Consider a new basis for the state space formed by columns of matrix T . (Basis for the input space
and the output space is kept the same.) Let x̄ be the state vector w.r.t. the new basis. Thus
x̄ = T −1 x (refer notes on Linear algebra).
x̄˙ = Āx̄ + B̄u, y = C̄ x̄ + D̄u
We need to find (Ā, B̄, C̄, D̄) in terms of (A, B, C, D).
x̄˙ = T −1 ẋ = T −1 (Ax + Bu) = T −1 (AT x̄ + Bu) = T −1 AT x + T −1 Bu
y = Cx + Du = CT x̄ + Du
Thus, Ā = T −1 AT, B̄ = T −1 B, C̄ = CT, D̄ = D.

4.1 Transfer matrix and realization

The transfer matrix for the above model is given by G(s) = C(sI − A)−1 B + D. When D = 0,
then transfer matrix is strictly proper. Constructing (A, B, C, D) from G(s) is called realization.
We now see how to construct a realization for a given transfer function. Write G(s) = Gsp (s) + D
i.e., write G(s) as a sum of a constant matrix and a transfer matrix in strictly proper form, D :=
lims→∞ G(s). Note that Gsp (s) = C(sI − A)−1 B where (sI − A)−1 = 1
Adj.(sI − A). Let
det(sI−A)
n
d(s) = s + α1 s n−1 + . . . + αn be the monic least common denominator of all entries of Gsp (s). Let
1
Gsp (s) = [N1 sn−1 + . . . + Nn−1 s + Nn ]. (5)
d(s)
where Gsp (s) ∈ Rp×k (s). Let
   
−α1 Ik×k −α2 Ik×k ··· −αn−1 Ik×k −αn Ik×k Ik×k

 Ik×k 0 ··· 0 0 


 0 
  
A=
 . . ··· . . ,B = 
  .  , C = N1
 N2 ··· Nn−1 Nn
 . . ··· . .   . 
0 0 ··· Ik×k 0 0
(6)
We now show that this (A, B, C) realizes Gsp (s). Let Z(s) = (sI −A)−1 B. Thus, (sI −A)Z(s) = B
i.e.,     
s + α1 Ik×k α2 Ik×k · · · αn−1 Ik×k αn Ik×k Z1 Ik×k
 −Ik×k sIk×k · · · 0 0   Z2   0 
    

 . . ··· . .  .  =  . .
    (7)
 . . ··· . .   .   . 
0 0 ··· −Ik×k sIk×k Zn 0
Therefore, Zn = 1s Zn−1 , Zn−1 = 1s Zn−2 , . . . , Z2 = 1s Z1 ⇒ Zk = 1
Z .
sk−1 1
Hence,
α2 αn−1 αn
(s + α1 + + . . . + n−2 + n−1 )Z1 = Ik×k (8)
s s s
d(s) sn−1
i.e., Z
sn−1 1
= Ik×k ⇒ Z1 = d(s) Ik×k . Thus,

sn−1 Ik×k
   
Z1
 Z2   sn−2 Ik×k 
  1  

 . = 
 d(s)  . .

 .   . 
Zn Ik×k
Now, CZ(s) = C(sI − A)−1 B = Gsp (s). It can be verified that CZ(s) gives (5) giving the desired
transfer matrix. This realization is called controller canonical realization of G(s).
Now suppose
 
−α1 Ik×k Ik×k 0 ··· 0  
N1

 −α2 Ik×k 0 0 ··· 0 
  N2 
 . . . ··· .     
A=  , B =  .  , C = Ik×k 0 ··· 0 0 . (9)

 . . . ··· . 


 . 

 −αn−1 Ik×k 0 0 ··· Ik×k 
Nn
−αn Ik×k 0 0 ··· 0
One can verify that this also realizes Gsp (s). This is called observer canonical realization.

5 Local linearization and feedback linearization

5.1 Local linearization
Consider a nonlinear system
ẋ = f (x, u), y = g(x, u). (10)
where there may or may not be an explicit time dependence.
Definition 5.1. An equilibrium point of a dynamical system of the form (10) is given by (x∗ , u∗ )
such that ẋ = f (x∗ , u∗ ) = 0.
Let u(t) = u∗ + δu(t), y ∗ = g(x∗ , u∗ ) and x(0) = x∗ + δx. Define δx(t) := x(t) − x∗ and
δy(t) := y(t) − y ∗ . Notice that
∂f ∂f
δ ẋ(t) = ẋ(t) = f (x∗ + δx(t), u∗ + δu(t)) = f (x∗ , u∗ ) + δx + δu + O(kδxk2 ) + O(kδuk2 ) (11)
∂x ∂u
∂g ∂g
δy(t) = g(x, u) − y ∗ = g(x∗ + δx(t), u∗ + δu(t)) − y ∗ = δx + δu + O(kδxk2 ) + O(kδuk2 ). (12)
∂x ∂u
Thus, one obtains a linearized model around an equilibrium point. Note that linearization is valid
if there are no eigenvalues of the Jacobian matrix ∂f ∂x having zero real parts.
Now let’s linearize around a trajectory. Let xsol , usol , y sol be a non constant solution trajectory
of (10). Let u(t) = usol (t) + δu(t), x(0) = xsol (0) + δx, δx(t) := x(t) − xsol (t), δy(t) := y(t) − y sol (t).
Therefore,
δ ẋ(t) = ẋ(t) − xsol˙ (t) = f (xsol (t) + δx(t), usol (t) + δu(t)) − f (xsol (t), usol (t))
∂f (xsol (t), usol (t)) ∂f (xsol (t), usol (t))
= δx + δu + O(kδxk2 ) + O(kδuk2 ) (13)
∂x ∂u
δy(t) = y(t) − y sol (t) = g(xsol (t) + δx(t), usol (t) + δu(t)) − g(xsol (t), usol (t))
∂g(xsol (t), usol (t)) ∂g(xsol (t), usol (t))
= δx + δu + O(kδxk2 ) + O(kδuk2 ) (14)
∂x ∂u
Thus, one obtains time varying matrices while linearizing along a trajectory.
5.2 Feedback linearization
The equations of motion of a general mechanical system can be written as

M (q)q̈ + B(q, q̇)q̇ + G(q) = F (15)

where q is a generalized position vector, M (q) > 0 is the mass matrix, B(q, q̇) represents cen-
trifugal/Coriolis/friction matrix, and G(q) is a vector due to conservative
  forces. Using F =
q
B(q, q̇)q̇ + G(q) + M (q)v, one obtains M (q)q̈ = M (q)v. Now using x = , one gets
q̇
   
0 I 0  
ẋ = x+ v, y = 0 I x.
0 0 I
 
Using v = −KP q − KD q̇ = − KP KD x, one obtains
 
0 I  
ẋ = x, y = 0 I x.
−KP −KD

Systems in strict feedback form: Consider a system in the following form called strict-feedback
form

ẋ1 = f1 (x1 ) + x2
ẋ2 = f2 (x1 , x2 ) + u. (16)

In this case, one cannot cancel the nonlinearity f1 (x1 ) using u. Consider a change of variable
z2 := f1 (x1 ) + x2 . Thus,

ẋ1 = z2
∂f ∂f (x1 )
ż2 = ẋ1 + ẋ2 = (f1 (x1 ) + x2 ) + f2 (x1 , x2 ) + u. (17)
∂x1 ∂x1
Now, we can cancel the nonlinearity using u and linearize the system.

6 Stability
We now study stability properties of linear systems in absence and presence of inputs. First we
study the internal stability of a system in the absence of inputs. Then, we consider bounded input
bounded output stability in the presence of inputs. This discussion is borrowed from Hespanha
([1]).

6.1 Internal stability

Definition 6.1. • An autonomous system ẋ = A(t)x is stable (marginally) if x(t) = Φ(t, t0 )x0
is uniformly bounded.

• It is asymptotically stable if it is stable and x(t) → 0 as t → ∞.

• Exponentially stable if in addition to stability, there exists c, λ > 0 such that kx(t)k ≤
ce−λ(t−t0 ) kx(t0 )k.

• Unstable if it is not marginally stable.

Theorem 6.2. Let ẋ = Ax. Then, the following statements are equivalent.
 1. All eigenvalues of A lie strictly in the LHP.

2. The system is asymptotically stable.

3. The system is exponentially stable.

4. For every positive definite matrix Q, there exists a unique positive definite matrix P such that
AT P + P A = −Q (Lyapunov equation).

5. There exists a positive definite matrix P such that AT P + P A < 0 (LMI).

Proof. Suppose 1 is satisfied. Therefore, eAt → 0 as t → ∞. Hence x(t) = eAt x(0) → 0 as t → ∞.

Thus 1 ⇒ 2. Suppose 1 is not satisfied. Then, eAt does not go to zero as t → ∞. Hence the system
is not asymptotically stable. Hence 2 ⇒ 1. Since exponential stable is stronger than asymptotic
stability, 3 ⇒ 2. We now show that 1 ⇒ 3.
We want to show that the system is exponentially stable i.e. there exists c > 0, γ > 0 ∈ R such
that kx(t)k ≤ ce−γ(t−t0 ) kx(t0 )k.
Since x(t) = eA(t−t0 ) x(t0 ),

kx(t)k ≤ keA(t−t0 ) kkx(t0 )k. (18)

Note that by definition, keA(t−t0 ) k = max{y(t),ky(t)k=1} keA(t−t0 ) y(t)k. Let ŷ(t) be such that
keA(t−t0 ) k = keA(t−t0 ) ŷ(t)k. For simplicity, assume that A has only one eigenvalue λ lying in the
left half plane. Therefore,
(t−t0 )2 λ(t−t0 ) (t−t0 )n−2 λ(t−t0 ) (t−t0 )n−1 λ(t−t0 )
 
ŷ1 (t)

eλ(t−t0 ) (t − t0 )eλ(t−t0 ) 2! e ... (n−2)! e (n−1)! e
(t−t0 )n−2 λ(t−t0 )  ŷ2 (t) 
eλ(t−t0 ) (t − t0 )eλ(t−t0 )


 0 ... 0 (n−2)! e

 .


A(t−t0 ) . . . ... . .  
e ŷ(t) =  .

 
. . . ... . .  
.

  
eλ(t−t0 ) (t − t0 )eλ(t−t0 )  
 0 0 0 ...  ŷn−1 (t) 
0 0 0 ... 0 eλ(t−t0 ) ŷn (t)

Let {v1 , . . . , vn } be the columns of eA(t−t0 ) . Hence,

keA(t−t0 ) ŷ(t)k = kŷ1 (t)v1 + . . . + ŷn (t)vn k ≤ |ŷ1 (t)|kv1 (t)k + . . . + |ŷn (t)|kvn (t)k (19)

Note that kŷ(t)k = 1, therefore |ŷi (t)| ≤ 1 for 1 ≤ i ≤ n. Hence the previous equation reduces to
the following

keA(t−t0 ) k = keA(t−t0 ) ŷ(t)k ≤ kv1 (t)k + . . . + kvn (t)k

s
p (t − t0 )n−1 2 λ(t−t0 )
= eλ(t−t0 ) + (12 + (t − t0 )2 )eλ(t−t0 ) + . . . + (12 + (t − t0 )2 + . . . + ( ) )e
(n − 1)!
= g(t − t0 )eλ(t−t0 ) (20)

Observe that |g(t − t0 )eλ(t−t0 ) | is bounded and goes to zero as t → ∞. Since g(t − t0 )eλ(t−t0 )
is decaying to zero, we can choose γ > 0 such that |g(t − t0 )eλ(t−t0 ) | ≤ e−γ(t−t0 ) for t ≥ t1
for some t1 ∈ R. Let c1 = supt≥0 |g(t − t0 )eλ(t−t0 ) |. Thus by choosing an appropriate c ∈ R,
|g(t − t0 )eλ(t−t0 ) | ≤ ce−γ(t−t0 ) for all t ≥ 0.
Hence, kx(t)k ≤ ce−γ(t−t0 ) kx(t0 )k. The general case for multiple Jordan blocks follows similarly.
´∞ T
Suppose the system has all eigenvalues strictly in the LHP. Let P = 0 eA t QeAt dt. Observe
that ˆ ∞ ˆ ∞
T T AT t At T
A P + PA = A e Qe dt + eA t QeAt Adt.
0 0
But d AT t QeAt ) T T
= AT eA t QeAt + eA t QeAt A. Therefore,
dt (e
ˆ ∞
T d AT t At T
A P + PA = (e Qe )dt = limt→∞ (eA t QeAt ) − Q = −Q.
0 dt

Thus, P satisfies the Lyapunov equation. Observe that P = P T and P > 0 since Q > 0. To
show that P is unique, let P̄ be another solution. Therefore, AT P + P A = AT P̄ + P̄ A ⇒ AT (P −
T T d T
P̄ ) + (P − P̄ )A = 0 ⇒ eA t AT (P − P̄ )eAt + eA t (P − P̄ )AeAt = 0 ⇒ dt (eA t (P − P̄ )eAt ) = 0.
T
Thus, eA t (P − P̄ )eAt remains constant with time, but it goes to zero as t → ∞. Therefore,
T
eA t (P − P̄ )eAt = 0 ⇒ P = P̄ . Hence, 1 ⇒ 4. Now 4 ⇒ 5 is trivial.
Finally, we show that 5 ⇒ 3. Let Q := −(AT P +P A) and v(t) = xT P x > 0. Now v̇ = xT (AT P +
P A)x = −xT Qx < 0. Thus, v is a decreasing signal and v(t) = xT (t)P x(t) < v(0) = xT (0)P x(0).

v(t)
v(t) = xT (t)P x(t) ≥ λmin (P )kx(t)k2 ⇒ kx(t)k2 ≤ (21)
λmin (P )

λmin (Q)
v̇ = −xT (t)Qx(t) ≤ −λmin (Q)kx(t)k2 ≤ − v(t). (22)
λmin (P )
Lemma 6.3 (comparison lemma). Let v(t) be a scalar signal such that v̇ ≤ µv(t), ∀t ≥ t0 , for
some µ ∈ R. Then, v(t) ≤ eµ(t−t0 ) v(t0 ).

Proof. Let s(t) := e−µ(t−t0 ) v(t). Thus,

ṡ = −µe−µ(t−t0 ) v(t) + e−µ(t−t0 ) v̇(t) ≤ −µe−µ(t−t0 ) v(t) + µe−µ(t−t0 ) v(t) = 0.

Therefore, s is nonincreasing and s(t) = e−µ(t−t0 ) v(t) ≤ s(t0 ) = v(t0 ) which proves the lemma.
λmin (Q)
− (t−t0 )
Now applying the comparison lemma to (22), v(t) ≤ e λmin (P ) v(t0 ). Since v(t) converges
to 0 exponentially, from (21), x(t) also converges to 0 exponentially.

Theorem 6.4. For the system ẋ = Ax, following are equivalent.

1. The system is stable.

2. Every eigenvalue of A has a real part strictly less than zero or equal to zero and eigenvalues
with zero real part have trivial Jordan structure (i.e., they are semi-simple).
Proof. Note that eAt is bounded iff the condition on the eigenvalues of A mentioned above is
satisfied. Therefore, 1 ⇔ 2.

Thus, to check stability, it is enough to find eigenvalues of the system matrix. From the
characteristic equation of the underlying system, one can find a matrix in companion form having
the same characteristic polynomial as the characteristic equation.
Remark 6.5. For stability/asymptotic stability of time varying systems, we must have Φ(t, t0 )
bounded for all t ≥ t0 or Φ(t, t0 ) → 0 as t → ∞. For exponential stability, Φ(t, t0 ) must go to zero
exponentially and so on.
Geometry of Lyapunov functions Any positive definite matrix P defines an energy func-
tions on the state space where the energy v(t) = xT P x. This defines ellipsoids which are energy
surfaces on the state space. Consider a set {x ∈ Rn |xT P x = c} where c ∈ R. This is an ellipsoid
which contains all states whose energy v(t) = xT P x = c. Thus, the entire state space is partitioned
as a union of these ellipsoids (energy surfaces). As x(t) evolves as a function of time (say ẋ = f (x)
is the underlying dynamical system), its energy varies accordingly since x(t) is varying as a function
of time. If v̇ = 0, then the rate of change of energy is zero. Thus, the energy of x(t) is constant.
Hence, it does not leave the energy surface (ellipsoid) on which it is already lying. If v̇ > 0, then
the rate of change is positive and energy of the state trajectory is increasing hence, it goes to a
higher energy surface. If v̇ < 0, then the trajectory moves to a lower energy surface. If v̇ ≤ 0 for all
points in the set {x ∈ Rn |xT P x ≤ c}, then once a trajectory enters this set, it can not leave this
set. This is the idea to prove invariance of a set or stability/asymptotic stability related arguments.

Consider a nonlinear autonomous system ẋ = f (x) and its linearization around the equilibrium
point x∗ .
Theorem 6.6 (local stability via linearization). Consider ẋ = f (x) where f is at least twice
differentiable on a closed ball B̄ ⊂ Rn around the equilibrium point x∗ . If the linearized system
∂f (x∗ )
δ ẋ = Aδx, A := ∂x is exponentially stable, then there is exists a ball B ⊂ Rn around x∗ and
constants c, λ > 0 such that for every solution x(t) of the nonlinear system that starts at x(t0 ) ∈ B,
kx(t) − x∗ k ≤ ce−λ(t−t0 ) kx(t0 ) − x∗ k, ∀t ≥ t0 . (23)
Proof. Let r(x) := f (x) − f (x∗ ) − Aδx. By Taylor’s formula, there exists d > 0 such that
kr(x)k ≤ dkδxk2 , ∀x ∈ B̄. (24)
Since, the linearized system is exponentially stable, there exists P > 0 such that AT P + P A = −I.
Let v(t) := δxT P δx.
v̇ = f (x)T P δx + δxT P f (x)
= (Aδx + r(x))T P δx + δxT P (Aδx + r(x))
= δxT (AT P + P A) + 2δxT P r(x)
= −kδxk2 + 2δxT P r(x)
≤ −kδxk2 + 2kP kkδxkkr(x)k. (25)
We want to make the rhs negative. For example,
1
−kδxk2 + 2kP kkδxkkr(x)k ≤ − kδxk2 .
2
To achieve this, let  > 0 be small enough such that the ellipsoid
E := {x ∈ Rn | (x − x∗ )T P (x − x∗ ) ≤ } (26)
satisfies the following properties
1. The ellipsoid E is contained in B̄. When x is inside this ellipsoid, (24) holds, hence,
x(t) ∈ E ⇒ v̇ ≤ −kδxk2 + 2dkP kkδxk3 = −(1 − 2dkP kkδxk)kδxk2 .
1 1
2. We further shrink  such that inside the ellipsoid E, (1 − 2dkP kkδxk) ≥ 2 ⇔ kδxk ≤ 4dkP k .
For this choice of , x(t) ∈ E ⇒ v̇ ≤ − 12 kδxk2 .
Once x(t) ∈ E, v(t) ≤ , v̇ ≤ 0. Thus, v cannot increase above , hence, it cannot exit E.
Note that
δxT P δx 1 1 δxT P δx 1 v(t)
δxT P δx ≤ kP kkδxk2 ⇒ kδxk2 ≥ ⇒ − kδxk2 ≤ − =− . (27)
kP k 2 2 kP k 2 kP k
v(t)
Now if x(0) ∈ B ⊂ E, by (27) v̇ ≤ − 12 kδxk2 ≤ − 12 kP k . Hence, by the comparison lemma, v and
∗
consequently δx = x − x decrease to zero exponentially.

Theorem 6.7 (unstability via linearization). Consider ẋ = f (x) where f is twice differentiable
(x∗ )
and f (x∗ ) = 0. If the linearized system δ ẋ = Aδx, A := ∂f∂x is unstable, then there are solutions
that start arbitrarily close to x∗ , but do not converge to this point as t → ∞.
6.2 Input-output stability
Note that the forced output of a linear system due to the input u(t) is given by
ˆ t
yf (t) = C(t)Φ(t, τ )B(τ )u(τ )dτ + D(t)u(t).
0

Definition 6.8. A system is said to be uniformly BIBO stable if there exists a finite constant g
such that
supt∈[0,∞] kyf (t)k ≤ gsupt∈[0,∞] ku(t)k (28)

Theorem 6.9 (time domain characterization). Following statements are equivalent.

1. The system is uniformly BIBO stable.

2. Every entry of D(.) is uniformly bounded and

ˆ t
supt≥0 |gij (t, τ )|dτ < ∞ (29)
0

for every entry gij (t, τ ) of g(t, τ ) = C(t)Φ(t, τ )B(τ ).

Proof. Sketch: Clearly, 2 ⇒ 1 since, taking sup over the rhs, we can bound the forced response
yf (t). On the other hand, if 2 is not satisfied, then we cannot bound yf (t) over all bounded inputs
as there will be bounded inputs corresponding to unbounded entries in either D(.) or g(t, τ ) which
make the forced response yf (t) unbounded.

Theorem 6.10 (Frequency domain characterization). Following statements are equivalent for LTI
systems.

1. The system is uniformly BIBO stable.

2. Every pole of every entry of the transfer function of the system has a strictly negative real
part.

Proof. Sketch: The second statement in the time-domain characterization is equivalent to the
statement that every entry Gij (s) of the transfer matrix G(s) corresponds to either an exponentially
decaying term in the time domain.

Note that exponential/asymptotic or internal stability implies BIBO stability but the converse
need not be true.

Example 6.11. Consider ẋ = Ax + Bu, y = Cx where

   
1 0 0  
A= ,B = ,C = 1 1 .
0 −2 1

In this case eAt becomes unbounded, hence the system is not internally stable. But CeAt B = e−2t
which is bounded, hence the system is BIBO stable.

7 Controllability
We now formalize one of the central properties of control system which is controllability. The
other one being observability. We also study an equally important notion of stabilizability and pole
placement. We will see that control problems are basically least norm problems.
Definition 7.1. A system ẋ = A(t)x(t) + B(t)u(t) is said to be controllable if for any initial
condition x(t0 ) ∈ Rn (or Cn ) and for any final condition xf ∈ Rn (or Cn ), there exists an input
u(t) which drives the state from x(t0 ) to xf in finite time t ∈ R. If t0 = 0 and x(0) = 0 in the
above case and xf is arbitrary, then we say that the system is reachable. Moreover, if xf = 0 and
x(t0 ) is arbitrary in the above case, then we say that the system is 0−controllable.
Consider the following equation for linear systems
ˆ t
x(t) = Φ(t, t0 )x(t0 ) + Φ(t, τ )B(τ )u(τ )dτ (30)
t0
ˆ t
⇒ x(t) − Φ(t, t0 )x(t0 ) = Φ(t, τ )B(τ )u(τ )dτ (31)
t0
´t
Thus, the system if controllable ⇔ the range space of t0 Φ(t, τ )B(τ )u(τ )dτ spans the entire state
space. Note that if the system is controllable, then we have a least norm problem (please refer my
linear algebra notes). There could be multiple u(t)s driving the state from the initial to the final
condition and we would like to find the input with least norm (or least energy). It is clear that for
reachability and 0−controllability, we have the same equivalent condition. Hence all these notions
are equivalent for continuous time systems. For discrete time systems, state transition matrix is
not invertible if the system matrix A is non-invertible. For discrete time systems, these notions are
equivalent when A is invertible. If A is not invertible for discrete time systems, then the system
cannot be controllable or reachable. But it can be 0−controllable.
Theorem 7.2. Following statements are equivalent for LTI systems:
1. (A, B) is controllable.
´t T
2. The Controllability Gramian Wt (A, B) := t0 eA(t−τ ) BB T eA (t−τ ) dτ > 0 for some t > t0 .

3. The Controllability matrix C(A, B) = B AB . . . An−1 B has rank n (rank test).

 

4. [λI − A B] has rank n for all λ ∈ C (PBH test).

´t T
Proof. (1 ⇔ 2) Suppose the Controllability Gramian t0 eA(t−τ ) BB T eA (t−τ ) dτ > 0. Since x(t) =
´t T
eA(t−t0 ) x(t0 ) + t0 eA(t−τ ) Bu(τ )dτ , choose u(τ ) = B T eA (t−τ ) η. Thus, x(t) = eA(t−t0 ) x(t0 ) +
Wt (A, B)η and η = Wt (A, B)−1 (x(t) − eA(t−t0 ) x(t0 )). This shows that 2 ⇒ 1.
Suppose (A,´ B) is controllable, hence for all x(t) ∈ Rn , there exists u(t) such that x(t) =
´ t A(t−τ
A(t−t ) t A(t−τ ) A(t−t ) ) Bu(τ )dτ . Since, x(t) −
e 0 x(t0 ) + t0 e Bu(τ )dτ . Thus, x(t) − e 0 x(t0 ) = t0 e
´ t ´ t
eA(t−t0 ) x(t0 ) lies in the range space of t0 eA(t−τ ) Bdτ , the rank of t0 eA(t−τ ) Bdτ must be n for some
´ t A(t−τ ) T
t > t0 . Therefore, the rank of t0 e BB T eA (t−τ ) dτ must also be n for t > t0 and 1 ⇒ 2. (One
´t T
can also show ∼ 2 ⇒∼ 1 as follows. Suppose the Controllability Gramian t0 eA(t−τ ) BB T eA (t−τ ) dτ
T
is singular i.e. there exists v such that v T Wt´(A, B)v = 0 i.e. v T eA(t−τ ) BB T eA (t−τ ) v = 0. Hence,
t
v T eA(t−τ ) B = 0. Thus, the range space of t0 eA(t−τ ) Bu(τ )dτ is not equal to Rn since it has a
non-trivial kernel. Hence, (A, B) is uncontrollable.)
´t T
Suppose the Controllability Gramian t0 eA(t−τ ) BB T eA (t−τ ) dτ is singular i.e., there exists v
T
such that v T Wt (A, B)v = 0 i.e. v T eA(t−τ ) BB T eA (t−τ ) v = 0. Hence, v T eA(t−τ ) B = 0. Let ρ = t−τ .
´ t−t T
Thus, Wt (A, B) = 0 0 eAρ BB T eA ρ dρ and f (ρ) = v T eAρ B = 0 for all ρ ∈ [0, t − t0 ]. At ρ = 0,
f (0) = v T B = 0. Since this is a differentiable function of ρ, differentiatingf (ρ) n−1 times at ρ = 0,
we obtain v T AB = . . . = v T An−1 B = 0. Thus, the rank of C(A, B) = B AB . . . An−1 B
is less than n since there exists a non-trivial kernel. Conversely, if C(A, B) is rank deficient, then
Wt (A, B) singular since by Cayley-Hamilton theorem, eAρ can be expressed as a linear combination
of I, A, . . . , An−1 . Hence, (2 ⇔ 3).
 Suppose [λI − A B] loses rank at λ ∈ C. Therefore, v ∗ [λI − A B] = 0. Thus, v ∗ belongs to
the left kernel of C(A, B) hence C(A, B) is not full row rank. Conversely, suppose v ∗ C(A, B) = 0.
Thus, v ∗ B = 0 and v ∗ Ai ∈ ker(B) for i = 1, . . . , n − 1. Consider < v ∗ , v ∗ A, . . . , v ∗ An−1 >, an
A−invariant subspace contained in ker(B). Since the subspace is A−invariant, it must contain an
eigenvector of A. Therefore, there exists w∗ ∈< v ∗ , v ∗ A, . . . , v ∗ An−1 > such that w∗ A = λA and
w∗ B = 0. Hence, [λI − A B] is rank deficient. Therefore, (3 ⇔ 4).
´t
Corollary 7.3. For LTV systems, (A(t), B(t)) is controllable ⇔ t0 Φ(t, τ )B(t)B T (t)ΦT (t, τ )dτ > 0
for some t > t0 .

Theorem 7.4. Controllability of (A, B) remains invariant under a change of basis.

Proof. Follows
 from the rank test of the
 Controllability matrix. Suppose the Controllability matrix
C(A, B) = B  AB . . . An−1 B has rank n. Let Ā = T  AT and B̄ = T −1 B. Consider
−1

T −1 C(A, B) = T −1 B T −1 AT T −1 B . . . T −1 An−1 T T −1 B = C(Ā, B̄). Thus, C(Ā, B̄) has

rank n. Reversing the arguments, one can show that if C(Ā, B̄) has rank n, then C(A, B) has rank
n.

Theorem 7.5. (Invariance under feedback) (A, B) is controllable ⇔ (A − BF, B) is controllable.

Proof. Follows by applying PBH test to (A, B) and (A − BF, B). Assuming that the matrix for
PBH test is rank deficient for one system, one can show that the corresponding matrix for other
system is also rank deficient.

Now we show that for controllable systems, one can do an arbitrary pole placement to have
the desired system poles. Controllability is unaffected by the previous theorem. The following
lemma and the pole placement theorem is taken from Wonham (Linear Multi-variable control: A
geometric approach).

Lemma 7.6. Let 0 6= b ∈ column span of B. If (A, B) is controllable, then there exists a feedback
matrix F such that (A + BF, b) is controllable.

Proof. Let b1 = b. Let X1 be a cyclic subspace generated by b1 under the action of A i.e.
X1 =< b1 , Ab1 , . . . , An1 −1 b1 > where {b1 , Ab1 , . . . , An1 −1 b1 } forms a linearly independent set. Let
x1 = b1 , x2 = Ax1 + b1 , . . . , xn1 = Axn1 −1 + b1 . Clearly, xi s are linearly independent since the
set {b1 , Ab1 , . . . , An1 −1 b1 } is linearly independent. Choose b2 ∈ / X1 and generate a cyclic subspace
X2 =< b2 , Ab2 , . . . , A n2 −1 b2 > where {b2 , Ab2 , . . . , A n2 −1 b2 } forms a linearly independent set. De-
fine xn1 +1 = Axn1 + b2 , xn1 +2 = Axn1 +1 + b2 and so on. Thus, {x1 , . . . , xn1 +n2 } is a basis for
X1 + X2 . Continuing this way we obtain a basis {x1 , . . . , xn } for the entire state space. (Since
(A, B) is controllable, rank of C(A, B) is n. Therefore, the above procedure generates the entire
state space.)
Let xi+1 = Axi + b̂i (1 ≤ i ≤ n − 1). Let b̂i = Bηi . Define a linear map F on the state space
such that F (xi ) = ηi (1 ≤ i ≤ n). Thus, xi+1 = Axi + b̂i = Axi + Bηi = Axi + BF xi = (A + BF )xi .
Therefore, {x1 , (A + BF )x1 , . . . , (A + BF )n−1 x1 } generate the entire state space where x1 = b1 .

Theorem 7.7 (Pole placement). (A, B) is controllable ⇔ for every symmetric set Λ of complex
numbers, there exists a feedback matrix F such that eigenvalues of (A + BF ) are given by Λ.

Proof. (⇒) Suppose we have a single input system and (A, B) is controllable. Therefore, B = b
is a cyclic vector. Suppose pA (s) = sn − an sn−1 − . . . − a2 s − a1 . Let Λ = {λ1 , . . . , λn } and
(s − λ1 ).(s − λ2 ) . . . (s − λn ) = sn − ân sn−1 − . . . − â2 s − â1 the desired characteristic polynomial.
By change of basis (since (A, b) is controllable, span{b, Ab, . . . , An−1 b} = Rn , choose v1 = An−1 b −
an An−2 b − . . . − a2 b, v2 = An−2 b − an An−3 b − . . . − a3 b, . . . , vn−1 = Ab − an b, vn = b as a new
basis), we may assume that (A, b) is in the following companion form
   
0 1 ... 0 0
 0 0 ... 0   . 
   
A=  . . ... .  ,b =  . .
 
 . . ... 1   0 
a1 a2 . . . an 1
 
Choosing f = â1 − a1 â2 − a2 . . . ân − an , A + bf has the desired eigenvalues.
For multi-input systems, from the previous lemma, there exists F1 and b ∈ column span of B
i.e. b = Bη such that (A + BF1 , b) is cyclic. Thus by the single input case above, there exists f¯
such that A + BF1 + bf¯ = A + BF1 + Bη f¯ = A + B(F1 + η f¯) = A + BF has the desired eigenvalues.
(⇐) Choose Λ to be a set of n real and distinct numbers {λ1 , . . . , λn } such that λi is not an
eigenvalue of A (1 ≤ i ≤ n). Choose F such that {λ1 , . . . , λn } are eigenvalues of (A + BF ) and
{x1 , . . . , xn } are the corresponding eigenvectors.

(A + BF )xi = λi xi ⇒ (λi I − A)xi = BF xi ⇒ xi = (λi I − A)−1 BF xi

((λi I − A)) is invertible for (1 ≤ i ≤ n) since λi is not an eigenvalue of A. Note that (λi I − A)−1
is an analytic function since λi is not an eigenvalue of A and hence it is a power series expansion.
By Cayley-Hamilton theorem, this power series can be expressed as follows:
n
X
(λi I − A)−1 = pj (λi )Aj−1 (32)
j=1
Pn
Thus, xi = (λi I − A)−1 BF xi = j=1 pj (λi )Aj−1 BF xi . Therefore, the basis vectors xi ∈ column
 
span of C(A, B) = B AB . . . An−1 B which implies that rank of C(A, B) is n. Hence,
(A, B) is controllable.
   
λ 0 1 0
Example 7.8 (MIMO pole placement). Let A = ,B = . Following the proof
0 λ 0 1
of Lemma 7.6, X1 =< b1 > and X2 =< b2 >. X = X1 ⊕ X2 . Moreover, a basis for X is
x1 = b1 , x2 = Ax1 + b2 . Note that in the notation of Lemma 7.6, x2 = Ax1 + b̂1 = Ax1 + Bη1 where
b̂1 = b2 and b2 = Be2 . Hence, η1 = e2 and F x1 = e2 . Let F x2 = 0. Thus,
      
λ 0 1 0 0 0 λ 0
A + BF = + = . (33)
0 λ 0 1 1 0 1 λ

Clearly, (A + BF, b1 ) is cyclic. Now we can do an pole placement using f1 such that A + BF + b1 f1
has the desired characteristic polynomial. (A + BF + b1 f1 = A + BF + Be1 f1 = A + B(F + e1 f1 ) =
A + BK).
   
λ 1 0 0 0 0
 0 λ 0 0   1 0 
Example 7.9 (MIMO pole placement). Let A =   0 0 λ 1  and B =  0 0 . Suppose
  

0 0 0 λ 0 1
we want to place poles at arbitrary locations. Following the proof of Lemma 7.6, X1 =< b1 , Ab1 >
and X2 =< b2 , Ab2 >. X = X1 ⊕ X2 . Moreover, a basis for X is x1 = b1 , x2 = Ax1 + b1 , x3 =
Ax2 + b2 , x4 = Ax3 + b2 . Note that xi+1 = Axi + b̂i 1 ≤ i ≤ 3 where b̂1 = b1 , b̂2 = b2 , b̂3 = b2 .
Furthermore, b1 = Be1 and b2 = Be2 . Hence, η1 = e1 , η2 = e2 , η3 = e2 . Recall that F was defined
by its action of xi s i.e.,

F x1 = η1 = e1 , F x2 = η2 = e2 , F x3 = η3 = e2 , F x4 = η (34)
where η is some vector whose value will be specified later. Now we find F by its action on old basis
vectors e1 , . . . , e4 of X since we have (A, B) in the standard basis representation.

x1 = b1 = e2 , x2 = Ax1 + b1 = Ae2 + e2 = e1 + (λ + 1)e2 , (35)

x3 = Ax2 + b2 = A(e1 + (λ + 1)e2 ) + e4 = (2λ + 1)e1 + λ(λ + 1)e2 + e4 (36)
x4 = Ax3 + b2 = A((2λ + 1)e1 + λ(λ + 1)e2 + e4 ) + e4 = (2λ + 1)λe1 + λ(λ + 1)Ae2 + Ae4 + e4
= (λ(2λ + 1) + λ(λ + 1))e1 + λ2 (λ + 1)e2 + e3 + (λ + 1)e4
= λ(3λ + 2)e1 + λ2 (λ + 1)e2 + e3 + (λ + 1)e4 (37)

Therefore, from previous four equations,

F x1 = F e2 = e1 , F x2 = F (e1 + (λ + 1)e2 ) = e2 ⇒ F e1 = −(λ + 1)e1 + e2 (38)

F (x3 ) = F ((2λ + 1)e1 + λ(λ + 1)e2 + e4 ) = e2 ⇒
−(2λ + 1)(λ + 1)e1 + (2λ + 1)e2 + λ(λ + 1)e1 + F e4 = e2 ⇒
F e4 = (λ + 1)2 e1 − 2λe2 (39)
2
F (x4 ) = F (λ(3λ + 2)e1 + λ (λ + 1)e2 + e3 + (λ + 1)e4 ) = η ⇒
−λ2 (3λ + 2)e1 + λ2 (λ + 1)e1 + F e3 + (λ + 1)(λ2 e1 + e2 ) = η (40)

−(λ + 1) 1 0 (λ + 1)2
 
Choose η such that F e3 = 0. Thus F = and
1 0 0 −2λ
   
λ 1 0 0 0 0 
−(λ + 1) 1 0 (λ + 1)2

 0 λ 0 0 
+ 1 0 
 
A + BF = 
 0 0 λ 1   0 0  1 0 0 −2λ
0 0 0 λ 0 1
   
λ 1 0 0 0 0 0 0
 0 λ 0 0   −(λ + 1)
  1 0 (λ + 1)2 
= 
 0 + 
0 λ 1   0 0 0 0 
0 0 0 λ 1 0 0 −2λ
 
λ 1 0 0
 −(λ + 1) λ + 1 0 (λ + 1)2 
=  . (41)
 0 0 λ 1 
1 0 0 −λ

One can check that (A + BF, b1 ) = (A + BF, e2 ) is cyclic. Now we can use SISO trick for pole
placement.

We give the following algorithm for MIMO pole placement based on Lemma 7.6.

1. Select b = b1 i.e. the first column of B and generate a maximal cyclic subspace X1 . Thus
x1 = b1 , . . . , xn1 = Axn1 −1 + b1 .

2. Select b2 i.e. the second column of B and generate X2 . xn1 +1 = Axn1 + b2 , . . . , xn1 +n2 =
Axn1 +n2 −1 + b2 and so on. Thus, we obtain a basis {x1 , . . . , xn } for X.

3. b1 = Be1 , b2 = Be2 , . . . , bk = Bek , xi+1 = Axi + b̂i . For i = 1, . . . , n1 − 1 b̂i = b1 = Be1 . For
i = n1 , . . . , n1 + n2 − 1, b̂i = b2 = Be2 and so on.

4. Define F : X → U (where U is an input space) as follows. For i = 1, . . . , n1 − 1, F xi = e1 .

For i = n1 , . . . , n1 + n2 − 1, F xi = e2 and so on and finally F xn = η where η can be defined
to ease computations.
 5. Find representation of F in standard basis (assuming that A, B were given in a standard
basis). Then (A + BF ) is cyclic w.r.t. b1 and we are in SISO case.

Remark 7.10. Suppose one wants to drive the state from x(t0 ) to x(tf ) = xf and hold it there
for all t ≥ tf . For a controllable system, one needs to choose u|[t0 ,tf ] such that x(tf ) = xf . Then,
for all t ≥ tf , u(t) must be chosen such that ẋ = Ax + Bu = 0. Since x(t) = xf for all t ≥ tf ,
0 = Axf + Bu(t) i.e., choose u(t) such that Bu(t) = −Axf . This implies that Axf must lie in the
column span of B if one wants to hold the state at xf for all t ≥ tf .

Some canonical forms:

Lemma 7.11 (Wonham). Let B be a subspace with minimal polynomial β w.r.t. A. Then, there
exists b ∈ B with minimal polynomial β.

Proof. Let B1 = B + AB + . . . + An−1 B be of dimension m. Restrict A to the subspace B1

and denote this restriction by AB1 (an m × m matrix). Now the minimal polynomial of B w.r.t.
A is the same as the minimal polynomial of the A−invariant subspace B1 which is the minimal
polynomial of AB1 .
Suppose b1 ∈ B generates the maximal cyclic subspace Bb1 among all vectors in B. Let Bb2
be the second maximal cyclic subspace for some b2 ∈ B and so on. Using similar arguments used
in linear algebra in the derivation of Rational canonical form, one can show that the minimal
polynomial of b2 divides the minimal polynomial of b1 and so on. Let B1 = Bb1 ⊕ . . . ⊕ Bbk ,
bi ∈ B. This gives a cyclic decomposition of B1 and consequently that of AB1 . Let mb1 be the
minimal polynomial of b1 . Thus, mb1 (AB1 ) = 0. We need to show that B1 = Bb1 ⊕ . . . ⊕ Bbk
gives the Rational canonical decomposition of AB1 . Let B1 = Bv1 ⊕ . . . ⊕ Bvl be a decomposition
which gives the Rational canonical form where vi ∈ B1 and l ≤ k. Clearly, dim(Bv1 ) ≥ dim(Bb1 )
and the minimal polynomial of v1 is same as the minimal polynomial of AB1 which is β(s). But
mb1 (AB1 ) = 0 ⇒ β(s)|mb1 (s) which implies that dim(Bv1 ) = dim(Bb1 ). Thus, b1 ∈ B generates
the maximal cyclic subspace of B1 . Therefore, b = b1 gives the required vector.

Theorem 7.12 (Wonham). Suppose (A, B) is controllable. Let α1 , . . . , αk be the invariant factors
of A. Then, there exists A−invariant subspaces Xi ⊆ X and vectors bi ∈ column span of B such
that

• X = X1 ⊕ . . . ⊕ Xk .

• A restricted to Xi is cyclic with minimal polynomial αi .

• If Bi =< b1 > + . . . + < bi > (1 ≤ i ≤ k), then < Bi , ABi , . . . , An−1 Bi >= X1 ⊕ . . . ⊕ Xi .

Proof. Since (A, B) is controllable, the minimal polynomial of B is same as the minimal polynomial
of A. By the previous lemma, there exists b1 ∈ B such that the minimal polynomial of b1 is same
as the minimal polynomial of B which is mA . Let X1 =< b1 , Ab1 , . . . , An−1 b1 >. Since X1 is
maximaly cyclic, X = X1 ⊕ Y1 where AY1 ⊂ Y1 . Let Q be a projection on Y1 along X1 . Since
both X1 and Y1 are A−invariant, QA = AQ and (I − Q)A = A(I − Q). Therefore,

Y1 = QX = Q < B, AB, . . . , An−1 B >=< QB, AQB, . . . , An−1 QB >

By the previous lemma and controllability, there exists b2 ∈ B such that Qb2 under the action of
A forms a maximal cyclic subspace X2 ⊂ Y1 = X2 ⊕ Y2 . Note that b2 ∈ X1 ⊕ X2 . Furthermore,
B2 =< b1 > + < b2 > and < B2 , AB2 , . . . , An−1 B2 >= X1 ⊕ X2 . Continuing this way, one obtains
that if Bi =< b1 > + . . . + < bi > (1 ≤ i ≤ k), then < Bi , ABi , . . . , An−1 Bi >= X1 ⊕ . . . ⊕ Xi .
 It is clear that the basis used in the proof above to convert A into Rational
 canonical form
is {b1 , Ab1 , . . . , b2 , Ab2 , . . . , bk , Abk , . . . , Ank −1 bk }. Let B = b̂1 b̂2 . . . b̂m . Recall that un-


der a change of basis, B is mapped to T −1 B. Choose B1 = b1 b2 . . . bm and define

 

T −1 such that T −1 b̂i = bi for 1 ≤ i ≤ k and extend it to an n × n invertible map. Thus,

under
 this change of basis, we may assume that A gets transformed to T −1 AT and T −1 B =
b1 b2 . . . bm . Denote the transformed matrices by (A, B) pair itself by abusing notation.
Now choose {b1 , Ab1 , . . . , b2 , Ab2 , . . . , bk , Abk , . . . , Ank −1 bk } as a basis. With respect to this basis,
   
A1 0 . . . 0 b11 b12 . . . b1n ∗
 0 A2 . . . 0   0 b22 . . . b2n ∗ 
   
A= .  . ... . ,B =  
 . . ... . . 
. (42)
 . . ... .   . . ... . . 
0 0 . . . Ak 0 0 . . . bkk ∗
Note
 that in general, one can obtain various canonical forms for controllable (A, B) pairs. Let
B = b1 b2 . . . bm . We may consider {b1 , Ab1 , . . . , b2 , Ab2 , . . . , } as a basis for X. X1 =<
b1 , Ab1 , . . . > is a cyclic subspace generated by b1 . If b2 ∈
/ X1 adjoin b2 to previous linearly
i
independent set. Keep adjoining A b2 until one gets a linearly independent set and so on. This
basis gives a canonical form for (A, B).
One can choose another basis as follows. Consider {b1 , . . . , bm }, the linearly independent
columns of B. Now act A on this set and adjoin linearly independent vectors. Act A again
on the set and adjoin newly generated linearly independent elements and so on. This also gener-
ates a basis for X and w.r.t. this basis, one gets a different canonical form for (A, B). We will
revisit the significance of subspaces mentioned above in Section 13.2.
Remark 7.13. Suppose the basis obtained in the previous method to generate a basis for the state
space is {b1 , Ab1 , . . . , Ak1 −1 b1 , b2 , Ab2 , . . . , Ak2 −1 b2 , . . . , Akm −1 bm }. Then {k1 , . . . , km } are called
controllability indices of an (A, B) pair. They are invariants of a control system.
Open loop control using the Controllability Gramian and the least input energy:
We now obtain an expression for the minimum input energy required for a state transfer from x(0) to
T
xf in time t. Let ul (τ ) = B T eA (t−τ ) η. One can check that choosing η = Wt (A, B)−1 (xf −eAt x(0)),
T
ul (τ ) = B T eA (t−τ ) Wt (A, B)−1 (xf −eAt x(0)) does the state transfer from x(0) to xf in time t. Thus,
the energy required for this transfer is
ˆ t
kul (τ )k2 dτ = (xf − eAt x(0))T Wt (A, B)−1 Wt (A, B)Wt (A, B)−1 (xf − eAt x(0))
0
= (xf − eAt x(0))T Wt (A, B)−1 (xf − eAt x(0)). (43)
Let û(τ ) be any other input doing the same state transfer. We show that (û − ul ) ⊥ ul . Since both
inputs does the same state transfer,
ˆ t
eA(t−τ ) B(û(τ ) − ul (τ ))dτ = 0
0
ˆ t
⇒ ηT eA(t−τ ) B(û(τ ) − ul (τ ))dτ = 0
0
ˆ t
T
⇒ (B T eA (t−τ ) η)T (û(τ ) − ul (τ ))dτ = 0
0
ˆ t
⇒ uT
l (τ )(û(τ ) − ul (τ ))dτ = 0.
0
Now using above relation and using
ˆ t
2
kûk = û(τ )T û(τ )dτ = kû − ul + ul k2 = kul k2 + kû − ul k2 ( using the orthogonality relation above)
0
Thus, kûk2 ≥ kul k2 and ul is the least energy input.

Now we study some properties of the Controllability Gramian when (A, B) is controllable.
ˆ t ˆ t
T T T
Wt (A, B) = eA(t−τ ) BB T eA (t−τ ) dτ = eAt ( e−Aτ BB T e−A τ dτ )eA t
0 0
d
⇒ Wt (A, B) = AWt (A, B) + BB T + Wt (A, B)AT ( differential Lyapunov equation).
dt
Suppose A is stable, i.e. all eigenvalues of A lie strictly in the LHP. Then,
ˆ ∞ ˆ ∞
T T
W∞ (A, B) = eA(t−τ ) BB T eA (t−τ ) dτ = eAρ BB T eA ρ dρ
0
ˆ ∞ 0
T d Aρ T AT ρ
⇒ AW∞ (A, B) + W∞ (A, B)A = (e BB e )dρ = −BB T
0 dρ
Thus, W∞ (A, B) satisfies a Lyapunov equation. Moreover, the solution to this Lyapunov equation
is unique. Suppose W̄ is some other solution of the Lyapunov equation.
Tt
A(W∞ (A, B) − W̄ ) + (W∞ (A, B) − W̄ )AT = 0 ⇒ eAt (A(W∞ (A, B) − W̄ ) + (W∞ (A, B) − W̄ )AT )eA =0
ˆ ∞
d At AT t d At T
⇒ (e (W∞ (A, B) − W̄ )e ) = 0 ⇒ (e (W∞ (A, B) − W̄ )eA t )dt = 0
dt 0 dt

Thus, evaluating the integral at boundary points, eAt vanishes at ∞, hence W∞ (A, B) = W̄ .

The following result is from Hespanha (Theorem 12.4).

Theorem 7.14. Suppose all eigenvalues of A lie strictly in the LHP. Then, (A, B) is controllable ⇔
there exists a unique positive definite solution W to the Lyapunov equation AW + W AT = −BB T .
Proof. If (A, B) is controllable and A stable, we saw above that the Controllability Gramian satisfies
the Lyapunov equation and the solution is unique. Conversely, assume that a unique positive
definite solution exists for the Lyapunov equation. We need to show that (A, B) is controllable.
Suppose (A, B) is not controllable. Therefore, by PBH test, there exists w such that w∗ A = λw∗
and w∗ B = 0. This implies that

w∗ (AW + W AT )w = (λ + λ∗ )w∗ W w = −w∗ BB T w = 0

Since A is stable, Re(λ) < 0. Thus, w∗ W w = 0 contradicting positive definiteness of W . Hence,

(A, B) is controllable.

Observe that eigenvectors of A and −µI − A are the same. One can choose µ > 0 such that
eigenvalues of −µI − A are strictly in the LHP. By the PBH test, (−µI − A, B) is controllable ⇔
(A, B) is controllable. Thus, from the above theorem, there exists W > 0 such that

(−µI − A)W + W (−µI − A)T = −BB T ⇒ AW + W AT − BB T = −2µW (44)

Let P = W −1 > 0. Therefore, from the previous equation,

1
P A + AT P − P BB T P = −2µP ⇒ P (A − BK) + (A − BK)T P = −2µP (K := B T P ). (45)
2
Thus, from Lyapunov stability test, A − BK is stable and a state feedback control u = −Kx makes
the system asymptotically stable.
Controllable subspace and stabilizability: Suppose rank(C(A, B)) < n. Then, the column
span of C(A, B) forms a controllable subspace of the state space X. If x(0) and xf both lie in the
controllable subspace, it is possible to find an input which does the required state transfer in finite
time. But if either one of them lies outside the controllable subspace, it is impossible to do the
required state transfer.
Observe that by construction, the controllable subspace if A−invariant. Thus, extending a basis
for controllable subspace to the entire state space, we get the following canonical form for (A, B)
   
A1 A12 B1
A= ,B = .
0 A2 0
where (A1 , B1 ) is controllable and eigenvalues of A1 are controllable. Observe that A1 above is
obtained by restricting A to the A−invariant controllable subspace and B1 is the representation of
B on this subspace. Note that if the rank of the Controllability matrix is n1 < n, then n1 is the
dimension of this subspace. Therefore, by construction (A1 , B1 ) is controllable. Note that A2 forms
an uncontrollable part of the system and eigenvalues of A2 are uncontrollable. Observe that the
left eigenvectors corresponding to uncontrollable eigenvalues belong to the left kernel of B. Since
(A1 , B1 ) is controllable, one can change eigenvalues of A1 using appropriate feedback. Eigenvalues
of A2 can not be changed under any feedback. Note that if eigenvalues of A2 lie in the LHP, then
using a proper feedback, eigenvalues of (A + BF ) can be made to lie in the LHP. Observe that
uncontrollable states need not form an A−invariant subspace.
Definition 7.15. A system (A, B) is said to be stabilizable, if for every initial condition x(0), there
exists an input u(.) such that limt→∞ x(t) = 0.
Theorem 7.16. Following are equivalent:
1. (A, B) is stabilizable.
2. Uncontrollable eigenvalues of A are stable.
3. rank([λI − A B]) = n for all λ ∈ C such that Re(λ) ≥ 0.
Proof. 1 ⇔ 2 is clear. 2 ⇒ 3 and 3 ⇒ uncontrollable eigenvalues are in the LHP, hence, stable.
Thus, 3 ⇒ 2.

Note that by pole placement theorem, controllability implies stabilizability. The following result
is from Hespanha (Theorem 14.4).
Theorem 7.17. (A, B) is stabilizable ⇔ there exists a positive definite solution P to the Lyapunov
matrix inequality AP + P AT − BB T < 0.
Proof. (⇒) Suppose (A, B) is in the following form after a change of basis
   
A1 A12 B1
A= ,B = .
0 A2 0
Since, (A1 , B1 ) is controllable, by Equation (44), there exists positive definite P1 such that A1 P1 +
P 1 AT T
1 −B1 B1 = −Q1 < 0. Moreover, since A2 is stable, there exists P2 > 0 such that A2 P2 +P2 AT
2 =
P1 0
−Q2 < 0. Let P = , ρ > 0. Thus, AP + P AT − BB T =
0 ρP2
 T
B1 B1T 0
        
A1 A12 P1 0 P1 0 A1 0 Q1 −ρA12 P2
+ − =− .
0 A2 0 ρP2 0 ρP2 AT12 A2
T 0 0 −ρP2 AT12 ρQ2
By making ρ sufficiently small, the RHS can be made negative definite.
(⇐) Let v ∗ be a left eigenvector of A associated with an unstable eigenvalue λ. Thus, Re(λ) > 0
v ∗ (AP + P AT − BB T )v < 0 ⇒ 2Re(λ)v ∗ P v − kv ∗ Bk2 < 0 ⇒ kv ∗ Bk2 > 0. (46)
Therefore, v ∗ ∈
/ kerB. Hence, (A, B) is stabilizable by PBH test for stabilizability.
 Let K = 21 B T P −1 where P > satisfies the Lyapunov matrix inequality above.

1 1
AP + P AT − BB T = (A − BB T P −1 )P + P (A − BB T P −1 )T = (A − BK)P + P (A − BK)T < 0.
2 2
Multiplying on the left and right by Q = P −1 , we get a Lyapunov inequality hence, (A − BK) must
be stable. Thus, for stabilizable systems, there exists a feedback law u = −Kx which asymptotically
stabilizes the system.

8 Observability
We show that the observability problem is least squares problem. We also give a canonical Kalman
decomposition of a linear system into controllable/uncontrollable and observable/unobservable
parts.

Definition 8.1. Consider a linear system ẋ = Ax + Bu and y = Cx + Du. We say that a system
is observable if it is possible to uniquely determine the state x(t) from the knowledge of the output
y(t) and the input u(t).

For simplicity, assume that D = 0 and y = Cx. Thus, taking derivatives on both sides of the
output equation,

y = Cx
d d
y = C x = CAx + CBu
dt dt
.
.
d n−1 d d
( ) y = C( )n−1 x = CAn−1 x + CAn−2 Bu + . . . + CB( )n−2 u
dt dt dt
      
y C 0 ... 0 0 u
d d
dt y  CA  CB ... 0 0  dt u
       
   

 .  = 
  . x + 
  . . . . . . 
 . 
 (47)
 .   .   . ... . .   . 
d n−1 CAn−1 CAn−2 B . . . CB 0 d n−1
( dt ) y ( dt ) u
 
C

 CA 

Let O(C, A) := 
 .  be the Observability matrix and let

 . 
CAn−1
    
y 0 ... 0 0 u
d d
dt y CB ... 0 0  dt u
     
   
z :=  . − . ... . .   . 
    
 .   . ... . .   . 
d n−1 CAn−2 B . . . CB 0 d n−1
( dt ) y ( dt ) u

Thus, we have a linear equation O(C, A)x = z. Observe that this is a Least squares problem.
Observe that by construction, z ∈ Im(O(C, A)). Thus, this system of linear equations has a
solution. The solution is unique ⇔ O(C, A) has full column rank. Thus, a system is observable ⇔
O(C, A) has full column rank.
Assuming that O(C, A) has full column rank, O(C, A)T O(C, A)x = O(C, A)T z and
x = (O(C, A)T O(C, A))−1 O(C, A)T z.
Lemma 8.2. O(C, A) is observable ⇔ (AT , C T ) is controllable

Proof. Follows from the rank test above for observability and the corresponding rank test for
controllability.

Observe that if O(C, A) is not full column rank, then it is not possible to uniquely determine
x which implies unobservability. Ker(O(C, A)) = ker(C)∩ker(CA) ∩ . . . ∩ker(CAn−1 ) forms unob-
servable states. If a state x is unobservable, then Cx = CAx = . . . = CAn−1 x = 0. Hence, by
Cayley-Hamilton theorem, CAi x = 0 for all i ∈ N∪0. Thus, if Ō = ker(O(C, A)) is the unobservable
subspace, then AŌ ⊆ Ō. Therefore, the unobservable subspace is A−invariant.
Consider a following equation for y with D = 0 (for simplicity)
ˆ t
y(t) = Cx(t) = CeAt x(0) + CeA(t−τ ) Bu(τ )dτ
0
ˆ T ˆ T ˆ T ˆ t
T T T
⇒ eA t C T y(t)dt = eA t C T CeAt dtx(0) + eA t C T CeA(t−τ ) Bu(τ )dτ dt. (48)
0 0 0 0
´T T
Define WT (C, A) = 0 eA t C T CeAt dt as the Observability Gramian. Thus, from the previous
equation, one can uniquely determine x(0) from the knowledge of outputs and inputs ⇔ WT (C, A)
is positive definite (hence invertible). From x(0), one can determine x(t) using the closed form
expression for x(t).

Theorem 8.3. Following statements are equivalent.

1. (C, A) is observable.

2. O(C, A) has full column rank equal to n (rank test).

3. The Observability Gramian is positive definite.

 
C
4. has full column rank for all λ ∈ C (PBH).
λI − A
Proof. 1 ⇔ 2 and 1 ⇔ 3 is already proved above. 2 ⇔ 4 follows using the same arguments used to
prove the corresponding equivalence for controllability.

Note that in presence of noise, taking derivatives of the output would increase the effect of
noise. Determining the state using the Observability Gramian would work better in the presence
of noise. Moreover, it works for LTV too.
´t
Corollary 8.4. For LTV systems, (C(t), A(t)) is observable ⇔ t0 ΦT (t, τ )C T (t)C(t)Φ(t, τ )dτ > 0
for some t > t0 .

Theorem 8.5. Observability is invariant under a change of basis.

Theorem 8.6. (C, A) is observable ⇔ (C, A + KC) is observable.

Proof. Follows from the PBH test.

Theorem 8.7. (C, A) is observable ⇔ for every symmetric set Λ of complex numbers, there exists
a feedback matrix K such that eigenvalues of (A + KC) are given by Λ.

Proof. Follows from (C, A), (AT , C T ) duality (Lemma 8.2) and the corresponding controllability
theorem for pole placement.
 Thus, one use a feedback matrix such that all eigenvalues of the system matrix lie in the LHP
i.e. (A − KC) has eigenvalues in the LHP.
´T T
Recall that WT (C, A) = 0 eA t C T CeAt dt.
ˆ t ˆ t
y(t) = CeAt x(0) + CeA(t−τ ) Bu(τ )dτ = CeAt x(0) + CeAt e−Aτ Bu(τ )dτ
0 0
ˆ t
T T T
therefore, y T (t) = xT (0)eA t C T + ( uT (τ )B T e−A τ dτ )eA t C T
0
´t T
Let z(t) = 0 e−Aτ Bu(τ )dτ . Thus, y(t) = CeAt x(0) + CeAt z(t) and y T (t) = xT (0)eA t C T +
T
z T (t)eA t C T .
T T
⇒ y T (t)y(t) = xT (0)eA t C T CeAt x(0) + z T (t)eA t C T CeAt x(0)
T T
+xT (0)eA t C T CeAt z(t) + z T (t)eA t C T CeAt z(t)
ˆ T ˆ T
T T T T
⇒ y (t)y(t)dt = (xT (0)eA t C T CeAt x(0) + 2z T (t)eA t C T CeAt x(0) + z T (t)eA t C T CeAt z(t))dt
0 0
ˆ T
T T T
= x (0)WT (C, A)x(0) + (2z T (t)eA t C T CeAt x(0) + z T (t)eA t C T CeAt z(t))dt(49)
0

d T
Note that dT WT (C, A) = eA t C T CeAt . Therefore,
ˆ T ˆ T
d d
y T (t)y(t)dt = xT (0)WT (C, A)x(0) + (2z T (t) WT (C, A)x(0) + z T (t) WT (C, A)z(t))dt(50)
0 0 dT dT

This gives as expression for the output energy in terms of the Observability Gramian.
Suppose all eigenvalues of A lie strictly in the LHP. Thus, one can show using similar arguments
used for the Controllability Gramian that W∞ (C, A) satisfies the Lyapunov equation AT W∞ (C, A)+
W∞ (C, A)A = −C T C.

Theorem 8.8. Suppose all eigenvalues of A lie strictly in the LHP. Then, (C, A) is observable ⇔
there exists a unique positive definite solution W to the Lyapunov equation AT W + W A = −C T C.

Proof. Follows from Theorem 7.14 and Lemma 8.2.

Suppose (C, A) is not observable. We saw that the unobservable subspace is A−invariant and
belongs to the kernel of C. Choose a basis for the unobservable subspace and extend it to a basis
for the entire state space. By arranging the basis vectors such that vectors in the kernel of C are
placed after the vectors which are not in the kernel of C, we get a following decomposition for A
and C ,

 
A1 0  
A= ,C = C1 0 .
A21 A2

Note that (C1 , A1 ) is observable.

Detectability: This property is dual of the stabilizability property just like observability is dual
to controllability.

Definition 8.9. A system ẋ = Ax + Bu y = Cx + Du is said to be detectable if the unobservable

states converge to zero as t → ∞.
 Observe that (C, A) is detectable ⇔ (AT , C T ) is stabilizable. Thus one can show that (C, A) is
detectable ⇔ unobservable eigenvalues of A lie in the LHP. Moreover, there is an analogous PBH
test for detectability and an analogous Lyapunov test for detectability.
Kalman decomposition: Let T = [Vc Vc̄ ] where columns of Vc form a basis for the controllable
subspace and column of Vc̄ form uncontrollable states. With T as a similarly transform, it leads to
the following form
   
Ac A∗ Bc  
A= ,B = , C = Cc Cc̄ .
0 Ac̄ 0
 
xc
Let x = . Thus,
xc̄

ẋc = Ac xc + A∗ xc̄ + Bc u(t)

ẋc̄ = Ac̄ xc̄

Now further subdivide both Vc and Vc̄ into observableand unobservable  states i.e. Vc = [Vco Vcō ]
xco
 xcō 
and Vc̄ = [Vc̄o Vc̄ō ]. Thus, T = [Vco Vcō Vc̄o Vc̄ō ] and x = 
 xc̄o .


xc̄ō
With this similarity transform, recalling the observability decomposition,
         
ẋco Aco 0 xco A∗o 0 xc̄o Bco
ẋc = = + + u
ẋcō Ac∗ Acō xcō A∗∗ A∗ō xc̄ō Bcō
    
ẋc̄o Ac̄o 0 xc̄o
ẋc̄ = = (51)
ẋc̄ō Ac̄∗ Ac̄ō xc̄ō

Thus, we have a following form for (A, B, C),

   
Aco 0 A∗o 0 Bco
 Ac∗ Acō A∗∗ A∗ō   Bcō   
A= ,B =   , C = Cco 0 Cc̄o 0 . (52)
 0 0 Ac̄o 0   0 
0 0 Ac̄∗ Ac̄ō 0

It follows from the Kalman decomposition that the transfer function of a system only captures the
controllable and observable part of the system. This can be verified by substituting (A, B, C) triple
above in G(s) = C(sI − A)−1 B. Thus, G(s) = C(sI − A)−1 B = G(s) = Cco (sI − Aco )−1 Bco .

9 State estimation
The feedback law u = −Kx can not be implemented when the state x is not directly accessible.
For observable systems, one can uniquely determine the state vector. However, to implement a
feedback law, we need x(t) for all time steps. Building an observer which estimates the state vector
asymptotically at each time step lets one implement a feedback law. If (C, A) is unobservable but
detectable, it is possible to estimate x from the system output up to an error e → 0 as t → ∞.
Consider a following model for an observer

x̂˙ = Ax̂ + Bu, e = x − x̂

ė = ẋ − x̂˙ = A(x − x̂) = Ae
Thus, if A is asymptotically stable, then the open loop system above estimates states with e → 0.
When A is not stable, it is still possible to estimate x by using a closed loop estimator using an
output feedback as follows:

x̂˙ = Ax̂ + Bu + L(y − ŷ), ŷ = C x̂, e = x − x̂ (53)

ė = ẋ − x̂˙ = A(x − x̂) − LC(x − x̂) = (A − LC)e (54)

Theorem 9.1. If there exists L such that A − LC has eigenvalues in the LHP, then the state
estimation error converges to 0 exponentially for all input signals u.

Proof. Follows from the equations above.

For observable and or detectable systems, it is always possible to choose L such that eigenvalues
of (A − LC) lie strictly in the LHP.

Suppose we want to implement a state feedback law using state estimates x̂ i.e. u = −K x̂.

ẋ = Ax + Bu = Ax − BK x̂ = Ax − BK(x − e) = (A − BK)x + BKe (55)

Thus, from Equations (54) and (55),

    
ẋ A − BK BK x
= ( Separation Principle) (56)
ė 0 A − LC e

The above equation is known as separation principal where one can design a stable feedback gain
K and an output injection gain L independently such that the closed loop system is stable and the
state estimation error converges to zero. (Eigenvalues of the matrix above are given by the union
of eigenvalues of (A − BK) and (A − LC). If (A, B) is controllable and (C, A) is observable, one
can choose K and L such that both x and e converge to zero at infinity. Moreover, if (A, B) is
stabilizable and (C, A) is detectable, one can still construct an observer such that x and e converge
to zero at infinity.) This is also called stabilization through output feedback. This determines the
observer design to implement state feedback laws.
The dynamics of the state estimate x̂ under the feedback u = −K x̂ are as follows:

x̂˙ = Ax̂ + Bu + LC(x − x̂) = Ax̂ − BK x̂ + LC(x − x̂) = (A − LC − BK)x̂ + LCx (57)

This is called a full order observer.

Reduced order observer: The observer used above is called a full order observer. Note that the
output y = Cx (assuming D = 0) already contains some information about the state. Thus, one
can use this fact and build a reduced order observer or a minimal order observer. Suppose by a
change of basis, (A, B, C) can be brought into a following form
   
A11 A12 B1  
A= ,B = ,C = I 0
A21 A22 B2

Let (Ā, B̄, C̄) be a matrix representation in the old basis. Thus, A = T −1 ĀT , B = T −1 B̄ and
C = C̄T . Let T = [T1 T2 ]. Thus [C̄T1 C̄T2 ] = [I 0]. (Therefore,
 T1 is the right inverse of C̄
x1
and T2 is orthogonal complement of rows of C̄.) Let x = be partition of x according to the
x2
partition [T1 T2 ]. Observe that y = [I 0]x = x1 . Thus, we need to estimate x2 .

ẋ1 = A11 x1 + A12 x2 + B1 u ⇒ A12 x2 = ẋ1 − A11 x1 − B1 u (58)

ẋ2 = A21 x1 + A22 x2 + B2 u (59)
Since y = x1 , x1 is observable. Thus, from Equation (58), A12 x2 is observable. Define a state
estimator x̂2 for x2 as follows

x̂˙ 2 = A21 x1 + A22 x̂2 + L(ẋ1 − A11 x1 − B1 u − A12 x̂2 ) + B2 u = A21 x1 + A22 x̂2 + LA12 (x2 − x̂2 ) + B2 u(60)
⇒ ẋ2 − x̂˙ 2 = A22 (x2 − x̂2 ) − LA12 (x2 − x̂2 ) = (A22 − LA12 )(x2 − x̂2 )(61)

We need to show that (A12 , A22 ) form  an observable

 pair. Suppose, (A12 , A22 ) is not observ-
A12
able. Thus, there exists λ̂ such that is rank deficient i.e., there exists v such that
λ̂I − A22
 
    I 0   
A12 C C 0
v = 0. Note that = λ̂I1 − A11
 −A12 . Hence, =
λ̂I − A22 λ̂I − A λ̂I − A v
−A21 λ̂I2 − A22
 
I 0  
 λ̂I1 − A11 0
−A12  = 0 which contradicts the observability of (C, A). Hence, (A12 , A22 )
v
−A21 λ̂I2 − A22
 
x1
is observable. Thus, we can design L such that A22 − LA12 is stable and T x = T gives an
x̂2
asymptotic state estimate in the old basis. This gives a minimal order observer.

10 Feedback
Feedback can be an output feedback or a state feedback for state space models. Moreover, it can
be static or dynamic. Static feedback does not involve any dynamics. Note that with a static
output feedback u = Ky = KCx, due to the presence of the matrix C, one cannot do an arbitrary
pole placement. This is a limitation of static output feedback. However, using an output feedback
and (A, B, C, D) matrices, one can build an observer/state estimator (observer based controller)
as shown above which asymptotically estimates states and allows us to implement state feedback
laws/arbitrary pole placement with an error converging to zero. Note that static feedback can be
identified with P (proportional) controller.
Dynamic feedback controller involves dynamics in its system model. Assuming that (A, B)
is stabilizable and (C, A) is detectable (where ẋ = Ax + Bu, y = Cx), one can design a linear
dynamic output feedback controller

ż = F z + Gy, u = Lz + M y (62)

such that the closed loop matrix  

A + BM C BL
GC F
is Hurwitz. Note that if (A, B) is uncontrollable, then the above dynamic output feedback
 controller

A1 A12
won’t stabilize the system as we can bring A in block upper triangular form and
0 A2
 
B1
B = . Using the dynamic output feedback, only eigenvalues of A1 can be altered. Thus,
0
if the uncontrollable eigenvalues are unstable, one cannot stabilize the system by either state or
output feedback. Moreover, if y = Cx + Du, then let ȳ = y − Du = Cx. Let ȳ be the input to
the dynamic controller and the above arguments work.
With a dynamic output feedback, if (C, A) is observable, then one can reconstruct states x(t)
using an observer based controller. Thus, for controllable and observable systems, one can do
arbitrary pole placement using a dynamic output feedback. (For input output models, proportional,
integral and derivative feedback (P, I, D, PI, PID) is used based on the observed output to get the
 ´
desired characteristics.) Note that integral control where u = y is an example of dynamic output
feedback. The controller equations are ż = y and u = z. For derivative control, u = ẏ which also
forms a dynamic output feedback.
For controllable systems, with a static state feedback, one can do arbitrary pole placement
(assuming that states are accessible i.e., y = x). Dynamic state feedback can be understood using
dynamic output feedback via the relation y = x.
Derivative feedback and DAEs: Dynamic output/state feedback involving derivatives of out-
puts/states turns the system into an differential algebraic system as shown in the following example.
Example 10.1. Consider a double integrator
      
ẋ1 0 1 x1 0
= + u
ẋ2 0 0 x2 1
 
  x1
y = 0 1 .
x2
Suppose u = ẋ1 + v which is a dynamic state feedback. Thus, ẋ2 − ẋ1 = v. We can write the state
equations as follows:
       
1 0 ẋ1 0 1 x1 0
= + v.
−1 1 ẋ2 0 0 x2 1
This is of the for E ẋ = Ax + Bu. In this example E is invertible hence, using E −1 , one can
bring this to standard state space form. However, E may not always be invertible. Such systems
are called differential algebraic systems. The solutions of these systems involve use of impulses or
distributions (Dirac delta functions and its derivatives in a distributional sense).
Consider a derivative control u = K ẏ = KC ẋ. Therefore, ẋ = Ax + BKC ẋ ⇒ (I − BKC)ẋ =
Ax. Suppose B = e1 , C = eT 1 and K = 1. Then, I − BKC is not  invertible and one
 obtains
 a
A1 A12 B1
differential algebraic system. Suppose (A, B) is uncontrollable and ,B= . Let
0 A2 0
u = K ẏ = KC ẋ + v. Therefore,
     
A1 A12 B1 B1  
ẋ = x+ (KC ẋ + v) ⇒ (I − KC1 KC2 )ẋ = Ax + Bv
0 A2 0 0
     
I1 − B1 KC1 −B1 KC2 A1 A12 B1
⇒ ẋ = x+ v.
0 I2 0 A2 0
Thus, the controllability and stabilizability properties do not change under the derivative feedback
and the system may turn into DAE. 1
To model transformers, we need to use differential algebraic equations (DAEs) as there is an
algebraic relation between the primary and secondary currents/voltages. DAEs arise quite naturally
in many practical systems. Dynamic output/state feedback involving derivatives of outputs/states
is not used in general as it may make the system more complicated.
To implement feedback laws using frequency domain methods, loop shaping techniques are
used where one uses appropriate blocks of transfer matrices to get the desired closed loop transfer
function. Loop shaping techniques are applicable for both feedback and feed forward control.

11 Well-posed interconnection
Consider an interconnection of a system and a controller with following dynamics
ẋs = A1 xs + B1 u1 , ys = C1 xs + D1 u1 (63)
ẋc = A2 xc + B2 u2 , yc = C2 xc + D2 u2 . (64)
1
Thanks to Pragada Shivaramkrishna for this issue.
The interconnection implies that u2 = ys and u1 = yc . Therefore, one obtains

u2 = ys = C1 xs + D1 u1 = C1 xs + D1 (C2 xc + D2 u2 ) = C1 xs + D1 C2 xc + D1 D2 u2
⇒ u2 = (I − D1 D2 )−1 (C1 xs + D1 C2 xc ) (65)

provided that (I − D1 D2 )−1 exists. Notice that

u1 = yc = C2 xc + D2 u2 = C2 xc + D2 (C1 xs + D1 u1 ) = C2 xc + D2 C1 xs + D2 D1 u1
⇒ u1 = (I − D2 D1 )−1 (C2 xc + D2 C1 xs ) (66)

provided that (I − D2 D1 )−1 exists.

Observe that (I −D1 D2 )−1 exists ⇔ (I −D2 D1 )−1 exists. Suppose (I −D1 D2 )−1 does not exist.
Hence, there exists v 6= 0 such that (I − D1 D2 )v = 0 i.e., v = D1 D2 v. Therefore, D2 v = D2 D1 D2 v
i.e, (I − D2 D1 )D2 v = 0 and D2 v 6= 0 because if D2 v = 0, then (I − D1 D2 )v 6= 0. Thus, (I − D2 D1 )
is not invertible. Similarly, one can show the converse as well. An interconnection is said to be
well posed if (I − D1 D2 )−1 exists. This indicates that u1 and u2 are uniquely determined. The
dynamics of the interconnected system is given by

A1 + B1 (I − D2 D1 )−1 D2 C1 B1 (I − D2 D1 )−1 C2
    
ẋs xs
= .
ẋc B2 (I − D1 D2 )−1 C1 A2 + B2 (I − D1 D2 )−1 D1 C2 xc

An interconnection which is not well posed is called ill posed. It turns out that if either the system
or the controller is strictly proper (either D1 = 0 or D2 = 0), then the interconnection is always
well posed. It follows that interconnection of the system with an observer based controller is well
posed since D2 = 0.

12 Applications/Design examples
Example 12.1 (Feedback linearization and a stabilizing control law). Consider an oscillating
pendulum θ̈ + gl sin(θ) = 0. Find a stabilizing control law u for the system θ̈ + gl sin(θ) = u which
drives the pendulum from an arbitrary initial position with an arbitrary initial velocity to θ = 0 and
θ̇ = 0 asymptotically.
Observe that introducing state variables x1 = θ, x2 = θ̇, this can be converted into a first order
system.
g
ẋ1 = x2 , ẋ2 = − sin(θ) + u
l
(If we consider θ̇ as a variable of interest, this system can be modeled as a first order system where
the order of the ode is equal to the number of energy storing elements which is one (mass) in this
case.)
Using u to for feedback linearization, choose u = gl sin(θ) + v. Thus we have the following linear
system:
 ˙      
x1 0 1 x1 0
= +
x2 0 0 x2 v

Clearly, the above system is controllable (by the rank test). Thus, an arbitrary pole placement is
possible. Choose v = k1 x1 + k2 x2 . Thus we have,
 ˙    
x1 0 1 x1
=
x2 k1 k2 x2

Now choose k1 , k2 such that the eigenvalues of the feedback matrix lie in the LHP.
 One can consider other practical examples of non-linear systems too which are feedback lin-
earizable and controllable after feedback linearization. Find out such systems in robotics, power
electronics or systems in other engineering domains and find appropriate control laws for their
designs. Build observers for these systems to implement feedback laws. Do simulations for these
models.
Example 12.2 (Tracking). Consider a SISO system (A, b, c) which is both controllable and observ-
able. Find u such that y tracks a constant reference signal which is unit step. Let u = −Kx + v
such that v(t) = 1 for t ≥ 0. Thus,
ˆ t ˆ t
(A−BK)t (A−BK)(t−τ ) (A−BK)t
y(t) = Ce x(0) + Ce Bv(τ )dτ = Ce x(0) + Ce(A−BK)(t−τ ) Bdτ
0 0
ˆ t
limt→∞ y(t) = limt→∞ {Ce(A−BK)t x(0) + Ce(A−BK)t ( e−(A−BK)(τ ) dτ )B}
0
´t
Since (A − BK) has eigenvalues strictly in the LHP, it is invertible. Hence, 0 e−(A−BK)(τ ) dτ =
−(A − BK)−1 (e−(A−BK)t − I). Thus,
limt→∞ y(t) = limt→∞ {−C(A − BK)−1 e(A−BK)t (e−(A−BK)t − I)B}
= limt→∞ {−C(A − BK)−1 (I − e(A−BK)t )B}
= −c(A − bK)−1 b (67)
Thus, we normalize v by the factor above so that y(t) tracks the unit step signal. (Observe that
the transfer function for the closed loop system is G(s) = c(sI − A + bK)−1 b. Thus G(0) =
−c(A − bK)−1 b.)
Example 12.3 (Feedback linearization and tracking). Consider Example 1. Suppose y(t) = θ(t).
Suppose we want to design a feedback law such that θ(t) tracks the position θ = π and θ̇ = 0.
Then one can use feedback linearization to convert the non-linear system to a linear system which
is controllable. Then one can use Example 2 to design an appropriate input signal.
One can combine the above methods to obtain methods which will give a strategy to track
reference signals for more general feedback linearizable non-linear systems. Construct/find such
examples in applications. One can consider an example of robot arm manipulator which is feedback
linearizable. One can have a network of single input systems where each system could be linear or
feedback linearizable. Then one can choose an appropriate input for each of these systems.
Example 12.4. Suppose we want to build an observer to implemented state feedback laws. If the
output is a non-linear function of states, linear observers won’t work. If the output is a linear
function of states, still there are difficulties due to non-linearities in the state equation and one
needs to use techniques from non linear control theory. However, there are feedback linearizable
systems for which it is possible to build linear observers to implement state feedback laws. They are
of the following form
ẋ1 = x2
.
.
ẋn−1 = xn
ẋn = f (x1 , . . . , xn ) + u
y = x1 .
Thus, all states can be derived from the output equation. Thus, one can implement state feedback
laws for such systems. Find more example of systems where it is possible to build an observer using
tools from linear systems theory.
13 Disturbance decoupling problem
Consider a following model for LTI system where q represents either disturbance or noise (Wonham).
ẋ = Ax + Bu + Sq (68)
y = Cx (69)
We say that a system in the form above is disturbance decoupled w.r.t. the pair (q, y) if for each
initial state x(0) ∈ X, the output y(t) is the same for every q. Note that with a state feedback
u = Kx,
ˆ t
(A+BK)t
ẋ = (A + BK)x + Sq ⇒ x(t) = e x(0) + e(A+BK)(t−τ ) Sq(τ )dτ
0
ˆ t
⇒ y(t) = Ce(A+BK)t x(0) + C e(A+BK)(t−τ ) Sq(τ )dτ
0

Hence, disturbance decoupling means that find u = Kx such that the forced response yf (t) =
´t
C 0 e(A+BK)(t−τ ) Sq(τ )dτ = 0. In other words, we want to find K such that the transfer function
from q to y is identically zero. Thus, we want T (s) = YQ(s)
(s)
= C(sI − (A + BK))−1 S = 0. Let K =
Ker(C) and S = Im(S). Let V =< A + BK, S > be a subspace generated by columns of S under
´ t (A+BK)(t−τ
the action of A + BK. It is clear from Cayley-Hamilton theorem that 0 e ) Sq(τ )dτ lies

in the subspace < A + BK, S >. This is called disturbance decoupling problem (DDP). Thus DDP
is solved ⇔ there exists K such that < A + BK, S >= V ⊂ K.
Observe that V is (A + BK)−invariant from construction.
Definition 13.1. (Wonham) A subspace W ⊂ X is said to be (A, B)−invariant, if there exists a
matrix K such that (A + BK)W ⊂ W. We denote the class of (A, B)−invariant subspaces of X
by γ(A, B, X).
It is clear that all A−invariant subspaces are (A, B)−invariant by choosing K = 0.
Lemma 13.2. Let W ⊂ X and B = Im(B). Then W ∈ γ(A, B, X) ⇔ AW ⊂ W + B.
Proof. (⇒) Suppose W ∈ γ(A, B, X). Therefore, there exists K such that (A + BK)W ∈ W. Let
w1 ∈ W. Therefore, (A + BK)w1 = w2 ∈ W ⇒ Aw1 = w2 − BKw1 ∈ W + B.
(⇐) Suppose AW ⊂ W + B. Let {w1 , . . . , wl } be a basis for W. Therefore, for each wi 1 ≤ i ≤ l,
there exists vi ∈ W and ηi such that Awi = vi − Bηi . Define a linear map K̂ on W such that
K̂wi = ηi (1 ≤ i ≤ l). Extending the basis for W to a basis for X, we can extend K̂ to a linear
map K on X. Then (A + BK)wi = Awi + BKwi = vi − Bηi + Bηi ∈ W. Thus, W is (A + BK)
invariant.

Lemma 13.3. The class of subspaces γ(A, B, X) is closed under subspace addition.
Proof. Follows from the above lemma.

Lemma 13.4. The class of subspaces γ(A, B, K) has a maximal element V ∗ .

Proof. Let W1 , W2 ∈ γ(A, B, K). Then, by the previous lemma, W1 + W2 ∈ γ(A, B, K). Consider
W3 ∈ γ(A, B, K) and obtain W1 + W2 + W3 ∈ γ(A, B, K) such that W1 + W2 ⊂ W1 + W2 + W3 .
Continuing this procedure, we obtain a chain of subspaces of higher dimensions. Since, K is finite
dimensional, this process terminates after finite steps and we no longer generate a subspace of
higher dimension. This gives a maximal element V ∗ .

Theorem 13.5. DDP is solvable ⇔ < A + BK, S >= V ⊆ V ∗ = sup(γ(A, B, K)).

Proof. (⇒) DDP is solvable ⇒ there exists K such that < A + BK, S >= V ⊂ K ⇒ < A +
BK, S >= V ⊆ V ∗ = sup(γ(A, B, K)). (⇐) Obvious.
13.1 Output stabilization problem (OSP)

ẋ = Ax + Bu, y = Cx

Find if possible a feedback matrix K such that y(t) = Ce(A+BK)t → 0 as t → ∞. From the
previous DDP problem, find V ∗ = sup(γ(A, B, Ker(C) = K)). Let (A + BF )V ∗ ⊆ V ∗ . If all
unstable modes of (A + BF, B) lie in the kernel of C, then this solves OSP by choosing u = F x.
Consider the case when there are unstable modes of A + BF lying outside ker(C). If these unstable
modes are controllable, then they can be made to lie in the LHP by an appropriate state feedback
i.e. for ẋ = (A + BF )x + Bv, use v = Kx such unstable (but controllable) eigenvalues of (A + BF )
are shifted to the LHP. Thus, OSP is solved. Thus, OSP is solvable ⇔ (unstable modes of A)
⊆ < A, B > +V ∗ . (Union of subspaces is not a subspace in general. Hence we take the sum of
subspaces to get a bigger subspace containing both the smaller subspaces.)
Uncontrollable modes of A are unaffected under state feedback. If they lie in the kernel of C or
are stable, OSP can be solved by shifting all controllable modes to the LHP. If there are unstable
uncontrollable modes of A lying outside the kernel of C, then OSP is not solvable.

13.2 Controllability subspaces

Recall that we had seen different canonical forms of (A, B) pairs by generating cyclic subspaces
using columns of B and so on (discussion after Theorem 6.11).

Definition 13.6. Let B1 ⊆ B where B is a subspace generated by column span of B and B1 is

a subspace of B. A subspace C(A, B1 ) generated by the action of A on B1 is called controllability
subspace.

Observe that by construction, controllability subspaces are A−invariant. Thus when A is re-
stricted to C(A, B1 ), arbitrary pole placement is possible for restriction of A on this subspace. A
collection of controllability subspaces forming a class of controllability subspaces of the state space
X is denoted by C(A, B, X). By choosing elements in B, we can generate different controllability
subspaces. It is clear from the construction of controllability subspaces that the class C(A, B, X) is
closed under subspace addition (just like the class γ(A, B, X) of (A, B)−invariant subspaces) and
it contains a unique maximal element.

Remark 13.7. Consider a controllability subspace C(A, B1 ) such that the initial condition x0 ∈
C(A, B1 ). Let {b1 , . . . , bk } be a basis for B1 . Let B1 = [b1 , . . . , bk ]. Taking action of A on this
set and taking only linearly independent vectors, generate a basis for C(A, B1 ) and extend it to a
basis of X. Let A1 be the restriction of A to C(A, B1 ). Clearly, (A1 , B1 ) is controllable and using
a feedback F1 , we can do arbitrary pole placement for (A1 + B1 F1 , B1 ). Note that (A, B1 ) need not
be controllable and w.r.t. the chosen basis for X, we have an upper triangular decomposition of
(A, B1 ) into the controllable and uncontrollable part. Since the initial condition x0 ∈ C(A, B1 ), the
component of the intial condition in uncontrollable part of the state space is zero. This implies that
the evolution of x(t) remains inside the controllable subspace C(A, B1 ) if x0 ∈ C(A, B1 ). This is
an important property of controllability subspaces.

DDP with stability (DDPS): We want to solve DDP using a state feedback such that the
feedback matrix is stable. For an (A, B)−invariant subspace W, we know from the definition that
there exists K such that (A + BK)W ⊂ W. Let F(W) denote the set of matrices K such that
(A + BK) has eigenvalues strictly in the LHP. Consider a class Γ(A, B, K) ⊂ γ(A, B, K) such that
if W ∈ Γ(A, B, K), then F(W) 6= ∅. Thus, we have the following natural analogue of the DDP
result.

Theorem 13.8. DDPS is solvable ⇔ < A + BK, S >= V ⊆ V ∗ = sup(Γ(A, B, K)).

14 Markov parameters, Minimal realizations, Transfer functions
Note that (A − KC, B) need not be controllable and (C, A − BF ) need not be observable in general.

Definition 14.1. A realization (A, B, C, D) of a transfer function G(s) (i.e. C(sI − A)−1 B + D =
G(s)) is said to be minimal if there does not exist a realization of a smaller order.

Definition 14.2. Two state space systems are said to be zero state equivalent if they realize the
same transfer function.
ti i ti
Observe that (sI − A)−1 = L(eAt ) = L{ ∞ −(i+1) . Therefore,
P
i=0 i! A }. Note that L(P
i! ) = s
−1 ∞ −(i+1) i −1 ∞ −(i+1) CAi B.
P
(sI − A) = i=0 s A . Hence, G(s) = C(sI − A) B + D = D + i=0 s

Definition 14.3. CAi B for i ≥ 0 are called Markov parameters of a realization (A, B, C, D).

Observe that the impulse response g(t) is given by

d d di di
g(t) = L −1 (G(s)) = CeAt B + Dδ(t), g(t) = CAeAt B + Dδ(t), . . . i g(t) = CAi eAt B + i Dδ(t)
dt dt dt dt
d
Thus, taking the derivatives from the right as t → 0+ , g(0+ ) = CB, dt g(t)|t=0+ = CAB and so on.

Theorem 14.4. Two realizations (A, B, C, D) and (Ā, B̄, C̄, D̄) are zero state equivalent ⇔ they
have the same Markov parameters and D = D̄.

Proof. (⇒) Zero state equivalence implies that D = D̄. Moreover, since G(s) = Ḡ(s), Markov
parameters must be the same. Conversely, if two realizations have the same Markov parameters
and D = D̄, then they realize the same transfer function hence, are zero state equivalent.

Theorem 14.5. A realization is minimal ⇔ it is controllable and observable.

Proof. (⇒) Suppose a minimal realization is either uncontrollable or unobservable. Thus, by

Kalman decomposition, one can find a smaller order realization that realizes the same transfer
function. This contradicts minimality.
(⇐) Suppose a realization (A, B, C, D) is controllable and observable but not minimal. Let
(Ā, B̄, C̄, D̄) be a realization of order n̄ < n. Let C = C(A, B) and O = O(C, A) be the control-
lability and observability matrices of (A, B, C, D) and let C̄ = C(Ā, B̄) and Ō = O(C̄, Ā) be the
controllability and observability matrices of (Ā, B̄, C̄, D̄). Note that since both C and O have rank
n, OTOCC T has rank n since it is a product of two positive definite matrices. Therefore, OC has
rank n because if the rank of OC is less than n, then OTOCC T will not have rank equal to n.
Similarly, ŌC̄ has rank n̄ < n.

CAB . . . CAn−1 B C̄ ĀB̄ . . . C̄ Ān−1 B̄

   
CB C̄ B̄
 CAB
 CA2 B . . . CAn B  
 C̄ ĀB̄
 C̄ Ā2 B̄ . . . C̄ Ān B̄  
OC =   . . ... .  , ŌC̄ = 
  . . ... . .

 . . ... .   . . ... . 
CA n−1 n
B CA B . . . CA 2n−2 B C̄ Ā n−1 n
B̄ C̄ Ā B̄ . . . C̄ Ā 2n−2 B̄

Since (A, B, C, D) and (Ā, B̄, C̄, D̄) realize the same transfer function, they have the same Markov
parameters. Therefore, OC = ŌC̄. But rank(ŌC̄) = n̄ < n = rank(OC), which is a contradiction.
Therefore, the realization (A, B, C, D) is minimal.

Corollary 14.6. (A−KC, B) is controllable ⇔ (C, A−KC, B) is minimal. Similarly, (C, A−BF )
is observable ⇔ (C, A − BF, B) is minimal.

Proof. Follows from the theorem above.

Corollary 14.7. OC remains invariant under a change of basis.
Proof. By the proof previous theorem, OC = ŌC̄ where Ō and C̄ are observability and controlla-
bility matrices respectively in a different basis.

Observe that the controllability/observability matrix and the corresponding Gramian matrices
do not remain invariant under a change of basis. However, their rank remains invariant.

Suppose a matrix A is full column rank. Then AT A is positive definite, hence invertible. Let
Al = (AT A)−1 AT . Then Al A = I and Al is called the left inverse of A. Similarly, if A is full row
rank, then AAT > 0 and Ar = AT (AAT )−1 is called the right inverse of A i.e. AAr = I.
Theorem 14.8. All minimal realizations of a transfer function are algebraically equivalent.
Proof. Let (A, B, C, D) and (Ā, B̄, C̄, D̄) be two minimal realizations. By the previous theorem,
they are both controllable and observable and OC = ŌC̄. Note that C and C̄ have full row rank
and O, Ō have full column rank. Let T = C̄C r .

OC = ŌC̄ ⇒ OCC r = ŌC̄C r ⇒ Ol O = (Ol Ō)(C̄C r ) = I ⇒ T −1 = (Ol Ō) (70)

l l −1
O OC = C = O ŌC̄ = T C̄ (71)
r r
OCC = O = ŌC̄C = ŌT. (72)

Thus, from previous two equations, T −1 B̄ = B and C̄T = C. Since OC = ŌC̄ and both realizations
have the same Markov parameters, OAC = ŌĀC̄ ⇒ Ol OACC r = (Ol Ō)Ā(C̄C r ). This implies
that A = T −1 ĀT . Thus, T provides a similarity transform between the two realizations and D = D̄
since they realize the same transfer function.

System norms: Substituting s = jω in the transfer function G(s), we obtain the frequency
response for an LTI system. The 2−norm is given by
ˆ ∞ ˆ ∞
1 1
kGk2 := ( 2
kG(jω)kF dω) = (
2 trace(G(jω)G∗ (jω))dω) 2 (73)
−∞ −∞

If G(s) is stable, then from Parseval’s theorem for Fourier transforms,

ˆ ∞ ˆ ∞
1 T 1
2
kGk2 = ( kg(t)kF dt) ) = (
2 trace(CeAt BB T eA t C T )dt) 2 . (74)
−∞ −∞

Similarly, the infinity norm of G is defined as

kGk∞ := supω σ1 (G(jω)). (75)

where σ1 is the maximum singular value of G(jω). Note that infinity norm is finite ⇔ G(s) has
no poles on the purely imaginary axis. For 2−norm to be finite, apart from the poles condition,
G(s) also has to be strictly proper. (We refer the reader to Doyle, Francis and Tannerbaum for
details.) System norms are useful in robust control to design controllers which minimize the effect
of the disturbance on the output by minimizing the norm of the corresponding transfer function
(from the disturbance to the output).
Matrix-fraction Descriptions (MFDs) Given a matrix transfer function G(s), one can write
−1 −1
G(s) = NR (s)DR (s) or G(s) = DL (s)NL (s). This is called right and left matrix fraction respec-
tively and it is not unique. Kailath shows that it is possible to obtain a controllable realization for
right MFDs and observable realization for left MFDs. For more on this topic, we refer the reader
to Kailath (Chapter 6). MFD is useful for design problems where one wants desired transfer func-
tions/loop transfer functions. Using MFDs, one can shape the transfer function of an underlying
system by choosing compensators with appropriate MFDs so that one obtains a desired transfer
function.
15 Poles-zeros and Smith McMillan form, Transmission zeros, In-
variant zeros, System inverse
Refer Hespanha. Applications of Invariant zeros: Modeling, detection and correction of cyber
attacks (Refer works of Pasqualetti, Fawzi)
G(s) = C(sI − A)−1 B + D = pA1(s) (CAdj(sI − A)B + DpA (s)). One can write G(s) = pA1(s) N (s)
and from the Smith form Σ(s) of N (s) where Σ = diag(σi (s)) such that σi |σi+1 , we obtain the
Smith-McMillan form SMG of G(s), where the nonzero diagonal entries of SMG are ψnii such that
ni σi ni+1 σi+1
ψi = pA and ψi+1 = pA . Therefore, ni |ni+1 and ψi+1 |ψi . Roots of Πi ni (s) are called
P
zeros of G(s)
and roots of Πi ψi (s) are called poles of G(s). The McMillan degree is given by i deg(ψi ).
Lemma 15.1. Poles of G(s) ⊆ eigenvalues of A.
1
Proof. Follows from the fact that G(s) = pA (s) (CAdj(sI − A)B + DpA (s)).

For minimal realizations, poles of G(s) = eigenvalues of A.

Theorem 15.2. Let SMG (s) be the Smith-McMillan form for G(s). Then the order n of a minimal
realization of G(s) is equal to the McMillan degree of G(s).
Proof. Let (A, B, C, D) be a minimal realization of G(s). Thus, poles of G(s) are equal to eigen-
values of A. Suppose G(s) has n number poles (and by definition this is the McMillan degree).
Therefore, the order of a realization must be greater than or equal to n in general for any arbitrary
realization. But for minimal realizations, the order of realization is equal to the number of poles
since there are no pole zero cancellations. The number of poles of G(s) is equal to its McMillan
degree.
 
sI − A B
The Rosenbrock system matrix is given by P (s) = . Suppose it has rank r.
−C D
Invariant zeros are complex numbers where P (s) loses its rank. Transmission zeros ⊆ Invariant
zeros and both of them block certain input signals with a frequency given by transmission/invariant
zero and an appropriate initial condition. When Transmission zeros = Invariant zeros?
The Rosenbrock system matrix can be factored as
I (sI − A)−1 B
    
sI − A B sI − A 0
P (s) = =
−C D −C I 0 G(s)

where G(s) is the transfer function. It is clear from this factorization that the rank of P (s) drops at
transmission zeros of G(s). Moreover, if there are pole-zero cancellations, then there are eigenvalues
of A which are not poles of G(s). At these complex numbers, P (s) again loses rank. Thus, if there
are no pole-zero cancellations i.e., for minimal realizations, transmission and invariant zeros are
the same.
Example 15.3. Consider the following system
   
1 0 1  
A= ,B = ,C = 1 0 ,D = 0
0 2 1
 
s−1 0 1
P (s) =  0 s−2 1 .
−1 0 0

Rank(P (s)) = 3. At s = 2, P (s) loses its rank. Thus, there is an invariant zero at s = 2. Note
1
that G(s) = s−1 . Therefore, there are no transmission zeros hence the set of transmission zeros
= ∅ ⊂ the set of invariant
 zeros.

If we choose C = 1 1 in the above example, then there are no invariant zeros.
 Suppose there is a transmission zero at z0 then, G(z0 ) is rank deficient i.e., there exists u0 such
that G(z0 )u0 = 0. Suppose x0 = (z0 I − A)−1 Bu0 . Let u = ez0 t u0 . We will see that the output is
zero for this input and the intial condition x0 .
ˆ t ˆ t
At −Aτ z0 τ At
x(t) = e (x0 + e e Bu0 dτ = e (x0 + e(z0 I−A)τ dτ Bu0
0 0
At −1 (z0 I−A)t
= e (x0 + (z0 I − A) (e − I)Bu0 )
At −At z0 t −1
At
= e x0 + e e e (z0 I − A) Bu0 − eAt (z0 I − A)−1 Bu0
= eAt x0 + ez0 t x0 − eAt x0 = ez0 t x0
y(t) = Cez0 t x0 = ez0 t Cx0 = ez0 t G(z0 )u0 = 0.

Thus, for the initial condition x0 = (z0 I − A)−1 Bu0 , a change in the input u(t) of the form u(t) +
ez0 t u0 goes undetected at the output. Therefore, malicious attacks made at system transmission
zeros go undetected.
Note that if there is an invariant zero at z0 , then P (z0 ) is rank deficient i.e., there exists x0 , u0
such that
    
x0 zI − A B x0
P (z0 ) = =0
−u0 −C D −u0
⇒ (z0 I − A)x0 − Bu0 = 0 ⇒ x0 = (z0 I − A)−1 Bu0 , and − Cx0 − Du0 = 0.

Let u(t) = ez0 t u0 and x(0) = x0 . Thus, from the arguments seen previously, x(t) = ez0 t x0 and
y(t) = Cez0 t x0 + Dez0 t u0 = 0. For a connection between the set of invariant zeros and detection
and correction of cyber attacks, we refer the reader to [11], [12]. Absence of invariant zeros implies
that attacks are not possible. Attacks can be made on states via B matrix and on the output via
D matrix. We show below that invariant zeros are invariant under feedback.
Consider the Rosenbrock system matrix P (s). By a unimodular transformation,
    
sI − A B I 0 sI − A − BK B
=
−C D −K I −C − DK D

Similarly, one can show that

    
I L sI − A B sI − A − LC B − LD
= .
0 I −C D −C D

Thus, invariant zeros do not change under a state feedback/output feedback as pre and post mul-
tiplication by unimodular matrices do not affect invariant zeros of P (s). Note that the Smith form
of P (s) gives all invariant zeros of P (s). These are precisely those complex numbers where P (s)
loses rank.
 
sI − A B
Remark 15.4. Observe that the Rosenbrock system matrix P (s) = captures all
−C D
information about the underlying linear dynamical system.

• The controllability and stabilizability property can be checked using the PBH test from the
block matrices in the first row of P (s).

• The observability and detectability property can be checked using the PBH test from the block
matrices in the first column of P (s).

• Invariant zeros are given by the zeros of the Smith form associated with P (s).
 • Taking Schur complement w.r.t. the block (sI − A), we obtain the transfer function G(s) =
C(sI − A)−1 B + D from which one can deduce the impulse response, frequency response,
Markov parameters, poles and transmission zeros. (From the knowledge of the transfer func-
tion, on can draw Nyquist/Bode plots too.)

• The state transition matrix can be found from the inverse Laplace transform of (sI − A)−1 .

Definition 15.5. A system (A, B, C, D) with transfer function G(s) is said to be invertible if there
exists a system (Â, B̂, Ĉ, D̂) with a proper transfer function Ĝ(s) such that GĜ = ĜG = I.

For an inverse to exist, #inputs = #outputs.

Theorem 15.6. A system inverse exists ⇔ D is invertible

Proof. Refer Hespanha. If D is invertible, (Â, B̂, Ĉ, D̂) can be constructed explicitly using D−1 and
system equations. If D is not invertible, we get a contradiction by choosing v such that Dv = 0.

Theorem 15.7. A system inverse exists ⇔ the Rosenbrock system matrix is invertible for almost
all complex numbers.

Proof. Suppose that if z0 is not an eigenvalue of A and not an invariant zero of the Rosenbrock
system matrix P (s). Therefore,

I (z0 I − A)−1 B
     
z0 I − A B I 0 z0 I − A 0
= .
−C D −C(z0 I − A)−1 I 0 C(z0 I − A)−1 B + D 0 I

Thus, P (z0 ) is invertible ⇔ G(z0 ) = C(z0 I − A)−1 B + D is invertible ⇔ D is invertible. Therefore,

the statement follows from the previous theorem.

16 LQR/LQG
In this section, we elaborate a little on some topics from Hespanha.

Definition 16.1 (Feedback invariants). A functional H(x, u) that involves system’s input and state
is called a feedback invariant if computed along solution of the system, it depends only on the initial
condition x(0) and not on the input signal u.

16.1 Finite horizon LQR

Proposition 16.2. For every symmetric matrix P (t), the following functional
ˆ t
T
H(x, u) := x (t)P (t)x(t) − (Ax + Bu)T P (t)x + xT P (t)(Ax + Bu) + xT Ṗ (t)xdt
0

is a feedback invariant.

Proof. H can be rewritten as

ˆ t
T
H(x, u) = x (t)P (t)x(t) − ẋT P x + xT P ẋ + xT Ṗ xdt
0
ˆ t
d T
= xT (t)P (t)x(t) − (x P x)dt
0 dt
= x (t)P (t)x(t) + x (0)P (0)x(0) − xT (t)P (t)x(t)
T T

Thus, H(x, u) = xT (0)P (0)x(0) along the state trajectory.

 LQR problem formulation: We want to make states to go the origin in finite time. How-
ever, there is a cost attached to input signals and deviation of the state from the origin. This is
mathematically formulated as follows:

Minimizeu {JLQR (x, u)} subject to ẋ = Ax + Bu, (76)

where
ˆ tf
JLQR (x, u) := xT (tf )F (tf )x(tf ) + xT (t)Qx(t) + uT (t)Ru(t) + 2xT (t)N u(t)dt (77)
0

such that Q ≥ 0 and R > 0. Note that the term xT (tf )F (tf )x(tf ) represents the cost associated
with the deviation of x(tf ) from the origin. The terms inside the integral give cost attached to
inputs and deviation of x(t) from the origin along the state trajectory from t = 0 to t = tf .
Using the feedback invariant obtained above, we want to represent JLQR (x, u) as
ˆ tf
JLQR (x, u) = H(x, u) + Λ(x, u)dt (78)
0

where minu Λ(x, u) = 0. In this case, u(t) = argminu Λ(x, u) minimizes JLQR and the minimum
cost is equal to H(x, u). Note that there are infinitely many choices for a feedback invariant
depending on P (0) and P (t). We will see below that there is a specific feedback invariant for which
the corresponding P (t) which satisfies a matrix differential equation (differential Riccati equation
(DRE)) which will be our choice of feedback invariant to find minu {JLQR (x, u)}.
ˆ tf
T
JLQR = x (tf )F (tf )x(tf ) + xT (t)Qx(t) + uT (t)Ru(t) + 2xT (t)N u(t)dt
0
ˆ tf
T
= H(x, u) + x (tf )F (tf )x(tf ) + xT (t)Qx(t) + uT (t)Ru(t) + 2xT (t)N u(t)dt − H(x, u)
0
ˆ tf
= H(x, u) + xT (tf )F (tf )x(tf ) − xT (tf )P (tf )x(tf ) + (xT (t)Qx(t) + uT (t)Ru(t) + 2xT (t)N u(t) +
0
(Ax + Bu)T P x + xT P (Ax + Bu) + xT Ṗ x)dt
= H(x, u) + xT (tf )F (tf )x(tf ) − xT (tf )P (tf )x(tf ) +
ˆ tf
xT (AT P + P A + Q + Ṗ )x + uT Ru + 2uT (t)(B T P + N T )xdt (79)
0

Let K := R−1 (B T P + N T ).

(uT + xT K T )R(u + Kx) = uT Ru + xT (P B + N )R−1 (B T P + N T )x + 2uT (B T P + N T )x (80)

Adding and subtracting the term xT (P B + N )R−1 (B T P + N T )x in Equation (79) and using Equa-
tion (80),

JLQR = H(x, u) + xT (tf )F (tf )x(tf ) − xT (tf )P (tf )x(tf ) +

ˆ tf
xT (AT P + P A + Q + Ṗ − (P B + N )R−1 (B T P + N T ))x + (uT + xT K T )R(u + Kx)dt
0

If we find P (t) such that AT P (t) + P (t)A + Q + Ṗ (t) − (P (t)B + N )R−1 (B T P (t) + N T ) = 0 and
P (tf ) = F (tf ), then Λ(x, u) = (uT + xT K T )R(u + Kx) which has a minimum equal to zero when
u = −Kx. This leads to a closed loop system ẋ = (A − BR−1 (B T P (t) + N T ))x. Therefore, choose
the feedback invariant among all possible choices such that P (t) satisfies the differential Riccati
equation with boundary conditions.
Theorem 16.3. Assume that there exists a symmetric solution to differential Riccati equation
AT P (t) + P (t)A + Q + Ṗ (t) − (P (t)B + N )R−1 (B T P (t) + N T ) = 0 satisfying the boundary condition
P (tf ) = F (tf ).Then the feedback law u(t) = −Kx(t) where K := R−1 (B T P + N T ) minimizes
JLQR (x, u).

Proof. Proof: Follows from the equations above.

If states are not directly accessible, we use a state estimator using an output feedback to im-
plement the feedback law of LQR.

Recall that we had considered an optimization problem on linear systems earlier as follows:
ˆ t
minimizeu uT (τ )u(τ )dτ
0
ˆ t
At
subject to ẋ = Ax + Bu, xf = e x(0) + eA(t−τ ) Bu(τ )dτ. (81)
0

We solved this optimization problem using tools from linear algebra. Cost functions of the above
problem and the LQR problem are different. Substituting Q = 0, F (tf ) = 0, N = 0 and R = I in the
LQR problem, we get the same cost function as above but the constraints are still different as there
is an additional constraint on (81). Therefore, clearly the optimal solutions to the two problems
are also different as there are different cost functions and different constraints. Both problems can
be solved independently using two different approaches as we have seen. However, they are closely
related to one another. They can be solved using Hamilton-Jacobi-Bellman equations (HJB) from
optimal control. Note further that for the LQR problem, we only gave sufficient conditions for an
optimal solution. HJB theory says that these conditions are also necessary for an optimal solution.
Infinite horizon LQR We refer the reader to Hespanha. The above approach is inspired from
Hespanha’s infinite horizon LQR approach.

16.2 Minimum energy estimator (MEE)

If states are not directly available for measurement, we need a state estimator/observer to imple-
ment feedback laws for LQR. Moreover, there could be noise and measurement errors. Thus we
need a minimum energy estimator (MEE) using an output feedback. When noise and measure-
ment errors are modeled as zero mean white noise Gaussian stochastic processes, we have an LQG
problem (a cousin of LQR). Separation principal holds here as well.
Consider the following model (Hespanha)

ẋ = Ax + Bu + B̄d, y = Cx + n (82)

where d represents disturbance and n represents measurement noise with both of them unknown.
Thus any estimate of x would work for sufficiently large d and n. Thus, previous state estimator or
observer does not work in the presence of disturbance and noise. MEE consists of finding a state
trajectory

x̄˙ = Ax̄ + Bu + B̄d, y = C x̄ + n (83)

that starts at rest at t → −∞ and is consistent with the past measured output y and control input
u for the least amount of noise n and disturbance d, measured by
ˆ t
JM EE := (n(τ )T Qn(τ ) + d(τ )T Rd(τ ))dτ (84)
−∞
where Q, R > 0. Once x̄ is found, the minimum energy state estimate x̂ is the most recent value of
x̄ i.e. x̂(t) = x̄(t). The MEE problem is solved by minimizing the quadratic cost
ˆ t
JM EE := ((C x̄ − y)T Q((C x̄ − y)) + dT Rd)dτ (85)
−∞

by appropriately choosing d(.). Note that the dynamic constraint on x̄ is given by Equation (83).

Proposition 16.4. Suppose u(.) and y(.) are given up to some time t > 0. For every symmetric
matrix P , differentiable signal β : (−∞, t] → Rn and a scalar H0 that does not depend on (d, x̄),
the functional
ˆ t
H(x̄, d) := H0 + ((x̄˙ − β̇)T P (x̄ − β) + (x̄ − β)T P (x̄˙ − β̇))dτ − (x̄(t) − β(t))T P (x̄(t) − β(t))(86)
−∞

is a feedback invariant for (83) if limτ →−∞ (x̄(τ ) − β(τ )) = 0.

Proof.
ˆ t
d(x̄(τ ) − β(τ ))T P (x̄(τ ) − β(τ ))
H(x̄, d) := H0 + dτ − (x̄(t) − β(t))T P (x̄(t) − β(t))
−∞ dτ
= H0

since limτ →−∞ (x̄(τ ) − β(τ )) = 0.

Add and subtract the feedback invariant term to JM EE . Thus,

ˆ t
T
JM EE := H(x̄, d) − H0 + (x̄(t) − β(t)) P (x̄(t) − β(t)) + ((C x̄ − y)T Q((C x̄ − y)) + dT Rd)dτ
−∞
ˆ t
− ((x̄˙ − β̇)T P (x̄ − β) + (x̄ − β)T P (x̄˙ − β̇))dτ
−∞
ˆ t
T
= H(x̄, d) − H0 + (x̄(t) − β(t)) P (x̄(t) − β(t)) + ((C x̄ − y)T Q((C x̄ − y)) + dT Rd)dτ
−∞
ˆ t
− ((Ax̄ + Bu + B̄d − β̇)T P (x̄ − β) + (x̄ − β)T P (Ax̄ + Bu + B̄d − β̇))dτ (87)
−∞
ˆ t
= H(x̄, d) − H0 + (x̄(t) − β(t))T P (x̄(t) − β(t)) + ((C x̄ − y)T Q((C x̄ − y)) + dT Rd)dτ −
−∞
ˆ t
[x̄T AT P x̄ + uT B T P x̄ + dT B̄ T P x̄ − β̇ T P x̄ − x̄T AT P β − uT B T P β − dT B̄ T P β + β̇ T P β
−∞
+x̄T P Ax̄ + x̄T P Bu + x̄T P B̄d − x̄T P β̇ − β T P Ax̄ − β T P Bu − β T P B̄d + β T P β̇]dτ
ˆ t
= H(x̄, d) − H0 + (x̄(t) − β(t))T P (x̄(t) − β(t)) + (x̄T C T QC x̄ + y T Qy − 2x̄T C T Qy + dT Rd)dτ
−∞
ˆ t
− [x̄T (AT P + P A)x̄ + 2x̄T P Bu + 2dT B̄ T P x̄ − 2x̄T P β̇ − 2x̄T AT P β − 2β T P Bu −
−∞
2dT B̄ T P β + 2β T P β̇]dτ
ˆ t
T
= H(x̄, d) − H0 + (x̄(t) − β(t)) P (x̄(t) − β(t)) + [x̄T (−AT P − P A + C T QC)x̄ + y T Qy +
−∞
−2x̄T (−AT P β + P Bu + C T Qy − P β̇) + 2β T P (Bu − β̇) + dT Rd − 2dT B̄ T P (x̄ − β)]dτ. (88)
Now, we complete the square in terms involving d:

dT Rd − 2dT B̄ T P (x̄ − β) = dT Rd − 2dT B̄ T P (x̄ − β) + (x̄ − β)T P B̄R−1 B̄ T P (x̄ − β) −

(x̄ − β)T P B̄R−1 B̄ T P (x̄ − β)
= dT Rd − 2dT RR−1 B̄ T P (x̄ − β) + (x̄ − β)T P B̄R−1 B̄ T P (x̄ − β) −
(x̄ − β)T P B̄R−1 B̄ T P (x̄ − β)
= (d − R−1 B̄ T P (x̄ − β))T R(d − R−1 B̄ T P (x̄ − β)) −
(x̄ − β)T P B̄R−1 B̄ T P (x̄ − β) (89)

Substituting (89) in (88),

ˆ t
T
JM EE = H(x̄, d) − H0 + (x̄(t) − β(t)) P (x̄(t) − β(t)) + [x̄T (−AT P − P A + C T QC − P B̄R−1 B̄ T P )x̄
−∞
−2x̄T (−AT P β − P B̄R−1 B̄ T P β + P Bu + C T Qy − P β̇) + y T Qy + 2β T P (Bu − β̇) − β T P B̄R−1 B̄ T P β
+(d − R−1 B̄ T P (x̄ − β))T R(d − R−1 B̄ T P (x̄ − β))]dτ. (90)

Choose
1. P such that −AT P − P A + C T QC − P B̄R−1 B̄ T P = 0.
2. β such that −AT P β −P B̄R−1 B̄ T P β +P Bu+C T Qy −P β̇ = 0. Using −AT P −P B̄R−1 B̄ T P =
P A − C T QC,

P A − C T QC + P Bu + C T Qy − P β̇ = 0
⇒ β̇ = (A − P −1 C T QC)β + Bu + P −1 C T Qy (91)
´t
3. H0 = −∞ y
T Qy + 2β T P (Bu − β̇) − β T P B̄R−1 B̄ T P β.
Therefore,
ˆ t
T
JM EE = H(x̄, d) + (x̄(t) − β(t)) P (x̄(t) − β(t)) + (d − R−1 B̄ T P (x̄ − β))T R(d − R−1 B̄ T P (x̄ − β))]dτ.(92)
−∞

Thus, JM EE can be minimized by choosing d = R−1 B̄ T P (x̄ − β) and x̄(t) = β(t).

We need to show that limτ →−∞ β(τ ) = 0. We need to solve Equation (91) backwards in time.
Observe that as τ → −∞, u(τ ) = 0 and y(τ ) = 0. Thus, we eventually have an “autonomous
system” with time running backwards as after a finite time running back, both u and y become ar-
bitrarily small. Thus, limτ →−∞ β(τ ) = 0 if −(A − P −1 C T QC) is asymptotically stable. (The minus
−1 T
sign is obtained when we take the limit limτ →−∞ of the state transition matrix e(A−P C QC)τ .)
Moreover, we need to show that limt→∞ (x̄(τ ) − β(τ )) = 0. Note that at time t, x̄(t) = β(t).
Using equations for x̄˙ and β̇ and d,

x̄˙ − β̇ = (A + B̄R−1 B̄ T P )(x̄ − β) + P −1 C T Q(Cβ − y). (93)

Solving this in backward time and using the fact that limτ →−∞ y(τ ) = 0 and limτ →−∞ β(τ ) = 0,
limt→∞ (x̄(τ ) − β(τ )) = 0 if −(A + B̄R−1 B̄ T P ) is asymptotically stable. The minimum value of
JM EE is given by H(x̄, d) = H0 . This proves the following theorem (Hespanha)
Theorem 16.5. Assume that there exists a symmetric solution P to the ARE: −AT P − P A +
C T QC − P B̄R−1 B̄ T P = 0 such that matrices −(A + B̄R−1 B̄ T P ) and −(A − P −1 C T QC) are
asymptotically stable. Then MEE estimator for (83) is given by

x̂˙ = (A − LC)x̂ + Bu + Ly, L := P −1 C T Q, (94)

where d = R−1 B̄ T P (x̄ − β).

Proof. Follows from the discussion above.

This is a deterministic estimator. Let e = x − x̂. Then ė = ẋ − x̂˙ = (A − LC)e + B̄d − Ln where
A − LC is asymptotically stable. Thus, we have BIBO stability of MEE estimator w.r.t. d and n
which ensures that the estimate x̂ doesn’t diverge from x. In the absence of both d and n, we have
asymptotically converging estimates.

16.3 LQG (Kalman-Bucy filter)

Suppose d and n are known and both are uncorrelated zero mean Gaussian white-noise stochastic
processes with covariance matrices

E[d(t)dT (t)] = δ(t − τ )R−1 , E[n(t)nT (t)] = δ(t − τ )Q−1 , Q, R > 0. (95)

Then the MEE estimate x̂ minimizes JLQG := limt→∞ E[kx(t) − x̂(t)k2 ] which is same as the trace
of E(x(t) − x̂(t))(x(t) − x̂(t))T . The MEE estimator for the above models of d and n is also called
Kalman-Bucy filter.
The proof of the following result is taken from Murray (Lecture notes).
Theorem 16.6 (Kalman-Bucy filter). Let ẋ = Ax + Bu + B̄d, y = Cx + n where d and n are
uncorrelated zero mean Gaussian white-noise stochastic processes with covariance matrices

E[d(t)dT (t)] = δ(t − τ )R−1 , E[n(t)nT (t)] = δ(t − τ )Q−1 , Q, R > 0. (96)

The optimal estimator has the following form of a linear observer

x̂˙ = Ax̂ + Bu + L(t)(y − C x̂) (97)

where L(t) = P (t)C T Q and P (t) = E{(x − x̂)(x − x̂)T } satisfies

Ṗ (t) = AP + P AT − P C T QCP + B̄R−1 B̄ T , P (0) = E{x(0)x(0)T } (98)

Proof. Let e = x − x̂ and ξ = B̄d − Ln. Therefore,

ė = (A − LC)e + ξ
⇒ Ṗ (t) = (A − LC)P + P (A − LC)T + B̄R−1 B̄ T + LQ−1 LT . (99)

For the explanation of the last equation, we refer the reader to Appendix, Equation (108) (This
derivation is taken from Friedland: Section 10.7, derivation of equation (10.58)).

Ṗ (t) = AP + P AT − LCP − P C T LT + B̄R−1 B̄ T + LQ−1 LT

= AP + P AT + B̄R−1 B̄ T + (LQ−1 − P C T )Q(LQ−1 − P C T )T − P C T QCP

We need to find L such that P (t) is as small as possible. Since (LQ−1 −P C T )Q(LQ−1 −P C T )T ≥ 0,
to make P (t) small, the positive term in the expression for Ṗ can be made zero. Thus, Ṗ (t) =
AP + P AT − P C T QCP + B̄R−1 B̄ T and P (0) = E{x(0)x(0)T }.

Suppose we design a state feedback controller u = −Kx that solves an LQR problem and
constructed an LQG/MEE state estimator x̂˙ = (A − LC)x̂ + Bu + Ly. If states are not available,
we may use u = −K x̂. Therefore,

x̂˙ = (A − LC − BK)x̂ + Ly
ẋ = (A − BK)x + BKe + B̄d
ė = (A − LC)e + B̄d − Ln
 ˙        
x A − BK BK x B̄ 0
= + d+ n. (100)
e 0 A − LC e B̄ −L
Thus, the separation principle holds for LQG problem as well. By choosing K and L, the closed loop
system can be made asymptotically stable. This ensures that both x and e do not diverge. Note that
the separation principle says that we can design optimal estimator and optimal regulator separately,
and combining them together solves an optimal regulation problem using output feedback.

16.4 Tracking problem

Consider an LTI system ẋ = Ax + Bu, y = Cx + Du where we want the output y to track a
reference signal r corresponding to an equilibrium point (x∗ , u∗ ) of the system i.e.

0 = ẋ∗ = Ax∗ + Bu∗ , r = Cx∗ + Du∗

−x∗
    
−A B 0
= (101)
−C D u∗ r

• If the number of outputs is greater than the number of inputs, we have an overdetermined
system of equations and it may or may not have a solution.

• If the number of outputs is equal to the number of inputs, then the matrix in the previous
equation is the Rosenbrock system matrix evaluated at s = 0. Thus, if the origin is not an
invariant zero, then there is a unique equilibrium point.

• If the number of outputs is less than the number of inputs, we have an under-determined
−x∗
 
system of equations and it has multiple solutions. In general, we may write =
u∗
 
Fx
r.
Fu
Let x̄ = x − x∗ , ū = u − u∗ and ȳ = y − r. Consider the following cost functional
ˆ ∞
JLQR = (ȳ T Qȳ + ρūT Rū)dt, ρ > 0. (102)
0

Note that Q penalizes the deviation from the reference signal and R penalizes deviation from the
equilibrium input. Observe that

x̄˙ = ẋ = Ax + Bu = Ax + Bu − Ax∗ − Bu∗ = Ax̄ + B ū

ȳ = Cx + Du − r = Cx + Du − Cx∗ − Du∗ = C x̄ + Dū

Thus, we now have a regulator problem and the optimal solution is given by ū = −K x̄. Thus
u = ū + u∗ = −K(x − x∗ ) + u∗ .
We showed that the tracking problem can be transformed into an equivalent LQR (regulator)
problem. If states are not available, then to implement feedback laws, we need an estimator
(Hespanha). In the presence of noise and disturbance, we need a linear quadratic estimator to
estimate states x̂ and we use u = −K x̂.
Tracking non constant reference signals: Suppose we want x(t) to track a trajectory r(t).
Suppose (A, B) is controllable and there exists u∗ (t) such that ṙ = Ar(t) + Bu∗ (t). Let x̄(t) =
x(t) − r(t) and ū = u − u∗ . Therefore,

x̄˙ = Ax(t) + Bu(t) − Ar(t) − Bu∗ (t) = Ax̄(t) + B ū

and one needs to choose ū such that the above system is asymptotically stable. For output tracking,
suppose there exists x∗ , u∗ such that ẋ∗ = Ax∗ + Bu∗ and r(t) = Cx∗ + Du∗ . Then, using
x̄ = x − x∗ , ȳ = y − y ∗ and ū = u − u∗ , one can solve the problem. We refer the reader to ([7]) for
building controllers to track non constant reference signals.
17 Appendix
The following definitions are from Papoulis and Pillai [15].

Definition 17.1. A random variable is a function from a sample space to C or R. (A sample space
is the set of all possible events/outcomes of an experiment.)

All random variables have a probability density function (distribution) which allows us to
compute probabilities of different events. A normal or Gaussian distribution is the one where
−(x−µ)2
the probability density function f is of the form of a Gaussian f (x) = √ 1 e 2σ 2 where µ is
2πσ 2
the mean and σ 2 is the variance.

Definition 17.2. A random variable X is said to be zero mean if its expectation E(X) = 0, for a
random vector X̄, zero mean ⇒ E(X̄) = 0.

Definition 17.3. Two random variables X and Y are said to be uncorrelated if their covariance
C(X, Y ) = E(XY ) − E(X)E(Y ) = 0. For random vectors X̄, Ȳ , they are said to be uncorrelated
if their covariance matrix C(X̄, Ȳ ) = E(X̄ Ȳ T ) − E(X̄)E(Ȳ )T is diagonal.

Definition 17.4. A function x(t, ξ) is said to be a stochastic process if for a fixed t0 , x(t0 , ξ) is a
random variable.

A stochastic process x(t) is said to be white noise if its values x(ti ) and x(tj ) are uncorrelated
for every ti 6= tj . A white noise Gaussian stochastic process is the one where each random variable
is zero mean and Gaussian with any two random variables being uncorrelated.
A correlation matrix is given by Rx (t, τ ) = E{x(t)xT (τ )}.
Consider a following system

ẋ = Ax + F v, y = Cx

where v is a white Gaussian noise. Thus, E(v) = 0. Let Φ(t, t0 ) be the state transition matrix.
Thus,
ˆ t
x(t) = Φ(t, t0 )x(t0 ) + Φ(t, λ)F (λ)v(λ)dλ.
t0

Then,
ˆ τ
T T T
x(t)x (τ ) = Φ(t, t0 )x(t0 )x (t0 )Φ (τ, t0 ) + Φ(t, t0 )x(t0 ){ Φ(τ, λ)F (λ)v(λ)dλ}T +
t0
ˆ t ˆ tˆ τ
T T
Φ(t, λ)F (λ)v(λ)dλ.x (t0 )Φ (τ, t0 ) + Φ(t, λ)F (λ)v(λ)v T (ξ)F T (ξ)ΦT (τ, ξ)dξdλ. (103)
t0 t0 t0

Taking expectation on both sides in the previous equation and observing that E(v) = 0,
ˆ tˆ τ
Rx (t, τ ) = Φ(t, t0 )E(x(t0 )xT (t0 ))ΦT (τ, t0 ) + Φ(t, λ)F (λ)E(v(λ)v T (ξ))F T (ξ)ΦT (τ, ξ)dξdλ.(104)
t0 t0

Let E(v(λ)v T (ξ)) = Qδ(λ − ξ) and E(x(t0 )xT (t0 )) = P (t0 ). Thus,
ˆ tˆ τ
Rx (t, τ ) = Φ(t, t0 )P (t0 )ΦT (τ, t0 ) + Φ(t, λ)F (λ)Qδ(λ − ξ)F T (ξ)ΦT (τ, ξ)dξdλ
t0 t0
ˆ t
T
= Φ(t, t0 )P (t0 )Φ (τ, t0 ) + Φ(t, λ)F (λ)QF T (λ)ΦT (τ, λ)dλ (105)
t0
Observe that
ΦT (τ, λ) = [Φ(τ, t)Φ(t, λ)]T = ΦT (t, λ)ΦT (τ, t). (106)
Substituting (106) in (105),
ˆ t
T
Rx (t, τ ) = Φ(t, t0 )P (t0 )Φ (τ, t0 ) + Φ(t, λ)F (λ)QF T (λ)ΦT (t, λ)ΦT (τ, t)dλ
t0
= Rx (t, t)ΦT (τ, t) (107)
Let P (t) := E(x(t)xT (t)) = Rx (t, t). Therefore,
ˆ t
d d
Rx (t, t) = (Φ(t, t0 )P (t0 )ΦT (t, t0 ) + Φ(t, λ)F (λ)QF T (λ)ΦT (t, λ)dλ)
dt dt t0
⇒ Ṗ (t) = AP + P AT + F QF T (108)
where the last equality follows by using differentiation under the integral sign.
Loop shaping: The idea behind loop shaping is to manipulate block transfer functions such that
one obtains a desired transfer function (which gives desired characteristics such as time response,
stability etc.). Using the Nyquist criterion for stability of a closed loop transfer function using
an open loop transfer, one manipulates open loop transfer function to obtain a desired frequency
response. This method is used frequently in design problems. MFDs are used for loop shaping.
We refer the reader to Doyle, Francis and Tannerbaum, Skogestad and Postlaithwaite for further
details. While implementing feedback laws using state estimators (e.g. linear quadratic estimator
(LQE)), open loop gain gets affected. It is shown in Hespanha (Chapter 24) that one can recover
open loop gain if certain conditions are satisfied. This is called loop transfer recovery (LTR).
Kalman equality:
GT H = 0, K = R−1 B T P, L̂(s) = K(sI − A)−1 B, L̂T (−s) = −B T (sI + AT )−1 K T ,
T̂ (s) = G(sI − A)−1 B + H, T̂ T (−s) = −B T (sI + AT )−1 GT + H T
T̂ T (−s)T̂ (s) = −B T (sI + AT )−1 GT G(sI − A)−1 B + H T H (109)
T T −1 T T T −1 T
A P + P A + G G − P BR B P = 0 ⇒ A P + sP − sP + P A + G G − P BR B P =0
pre multiplying by -B T (sI + AT )−1 and post multiplying by (sI − A)−1 B,
T −1
T
⇒ −B (sI + A ) ((sI + A )P − P (sI − A) + GT G − P BR−1 B T P )(sI − A)−1 B = 0
T

⇒ −B T P (sI − A)−1 B + B T (sI + AT )−1 P B − B T (sI + AT )−1 GT G(sI − A)−1 B +

B T (sI + AT )−1 P BR−1 B T P (sI − A)−1 B = 0.
using RK = B T P and L̂(s) = −K(sI − A)−1 B,
−RL̂(s) − L̂T (−s)R + T̂ T (−s)T̂ (s) − H T H − L̂T (−s)RL̂(s) = 0
⇒ RL̂(s) + L̂T (−s)R + L̂T (−s)RL̂(s) = T̂ T (−s)T̂ (s) − H T H
⇒ RL̂(s) + L̂T (−s)R + L̂T (−s)RL̂(s) + R = R + T̂ T (−s)T̂ (s) − H T H
⇒ (1 + L̂T (−s))R(1 + L̂(s)) = R + T̂ T (−s)T̂ (s) − H T H (110)
Further reading: We recommend the reader to read Hespanha [1]. For abstract linear algebraic
approach and geometric approach using subspaces for LTI systems, we refer the reader to Wonham
[2]. Hespanha [1] and Brockett [3] consider both LTI and LTV systems. For a detailed realization
theory and matrix fraction descriptions and examples, we refer the reader to Kailath [4]. For
numerical aspects in systems and control one may refer to the lecture notes by Van Dooren [13].
For loop shaping and frequency domain approach, refer [9], [10]. For Kalman filter, stochastic
processes and more, refer [5]. For linear and optimal control, [8], [14] and [14] in particular for
robust control. Skogestad and Postlaithwaite [9] also discuss robust control in detail. For model
order reduction and large scale systems refer Antoulas [16].
Acknowledgments
Thanks to Shauvik Das for an extremely careful reading of the document and also for his use-
ful comments. Thanks to Pragada Shivaramakrishna for many useful discussions and comments.
Thanks to Lieut. CDR. Vamsi, Moduga Vijay Babu, Noble Tawra and Arpita Sikder for their
useful comments.

References
[1] J. Hespanha, Linear systems theory, Princeton university press, 2009.

[2] W.M. Wonham, Linear Multi-variable Control A Geometric Approach, third edition, Springer-
Verlag, 1985.

[3] R. W. Brockett, Finite Dimensional Linear Systems, John Wiley and Sons, Inc., 1970.

[4] T. Kailath, Linear systems, Prentice hall inc. 1980.

[5] B. Friedland, Control system design : An introduction to state-space methods, Dover publica-
tions inc., 1986.

[6] R. Murray, Optimization-Based Control : Lecture notes

[7] A. Astolfi, Tracking and Regulation in Linear Systems, Encyclopedia of Systems and Control
Springer-Verlag London, 2014.

[8] H. Kwakernaak, R. Sivan, Linear Optimal Control Systems, Willey Interscience, 1972.

[9] S. Skogestad, I. Postlaithwaite, Multi-variable Feedback Control Analysis and design, John Wil-
ley and sons, 2007.

[10] J. Doyle, B. Francis, A. Tannerbaum, Feedback Control Theory, Macmillan Publishing Co.,
1990.

[11] F. Pasqualetti, A Systems and Control Perspective on Privacy, Safety, and Security in Large-
Scale Cyber-Physical Systems, Presentation slides from Disc summerschool talk, the Hague, the
Netherlands, 2017.

[12] H. Fawzi, P. Tabuada, S. Diggavi, Secure state-estimation for dynamical systems under active
adversaries, IEEE Transaction on Automatic Control, vol. 59, no.6, pp. 1454-1467, 2014.

[13] P. M. Van Dooren Numerical Linear Algebra for Signals Systems and Control, Lecture notes
for Graduate School in Systems and Control, 2003.

[14] K. Zhou, J. Doyle, K. Glover, Robust and Optimal Control, Prentice hall, 1996.

[15] A. Papoulis, U. Pillai, Probability, Random Variables and Stochastic Processes, fourth edition,
Tata-McGraw hill, 2002.

[16] A. Antoulas, Approximation of Large-Scale Dynamical Systems, Advances in Design and Con-
trol, SIAM, 2005.

“There is no charge for awesomeness...” -Po (Dragon warrior)