523 M1380: Adaptive Control Systems

Lecture 4: On-Line Parameter Estimation

Spring 2005

In many applications, plant (model) structure may be known, but its parameters
may be unknown and time-varying due to change in operation conditions, aging of
equipment, etc. Thus, the off-line parameter estimation is inefficient.
On-line estimation schemes refer to those estimation schemes that provide frequent estimates of plant parameters by properly processing the plant I/O data on-line.
The essential idea behind is the comparison of the observed system response y(t) with
the output of a parameterized model y (, t ) , whose structure is the same as that of
plant model. Then, (t) is adjusted continuous so that y (, t ) approaches y(t) as t increases. (Under certain input conditions, y being close to y implies that (t) is close
to the unknown .)
*

The on-line estimations procedure, therefore, involves 3 steps:

Step 1: Select an appropriate plant parameterization.
Step 2: Select an adaptive law for generating or updating (t).
Step 3: Design the plant input so that (t) approaches as t .
*

Remark 4.0.1: In adaptive control, where the convergence of (t) to is usually not
*

one of the objectives, the first two steps are the most important ones.

4.1 SCALAR EXAMPLE: ONE UNKNOWN PARAMETER

In this section, a scalar example is used to illustrate the importance of the normalized in identification.
4.1.1 Estimation without Normalization
Consider the plant
y (t ) = u (t ),

(4.1.1)

where is unknown, and y(t) and u(t) are measurable. If u and y are measured in a
*

noise-free manner, then

(t ) =

y (t )
, u (t ) 0 .
u (t )

However, disadvantages lie in:

(i)

numerical problem when u(t) 0, and

(ii)

noise effect of measurements of y(t) and u(t) will cause wrong estimations.

Remedy is possible to use a recursive (on-line), division-free scheme.

Alternatively, let y (t ) be the estimation value of y(t) in the form of
y (t ) = (t )u (t ) . Define

~
1 = y y = u u = u,

~
where = , and define
1
1
J ( ) = 12 = ( y u ) 2 .
2
2
Then, adjustment of is trying to minimize J(), which naturally leads to the gradient method as:
& = J () = [ y u ]u = 1u , (0) = 0 ,
where > 0 is a scaling constant. For stability analysis of the estimator, construct the
following Lyapunov function candidate:
~
1 ~2
V ( ) =
,
2

~
~
~&
subject to = & = 1u , then V& ( ) = 12 0, which implies that L (or L ).
Since

V& ()d = V0 V = 12 ()d < ,

0

we have 1 L2 L . Now, assuming that u L, we have & = 1u L2 L . Ad~&

~
ditionally, if we assume that u& L , then it follows that & 1 = u u& L . By

~&
Barbalats Lemma, 1(t) 0 as t and hence (t ) 0 as t .
On the other hand, note that
t

u 2 ( ) d ~
~
(t ) = e 0
(0).

~
Therefore, (t ) converges to zero if and only if

t + T0
t

u 2 ()d 0T0 , t 0 ,

for some 0, T0 > 0, which is referred to as persistence excitation (PE).

4.2.1 Estimation with Normalization

Consider the plant (4.1.1) again and assume u and y are piecewise continuous but not

necessarily bounded, and is to be estimated. Let y u , 1 y y = y u ,

where (t) is estimation at t.

Minimization problem: min J () = min 12 ( y u ) 2 is ill posed because

u, y L .

An alternative is y u , y =

chosen so that

u
m

y
m

, u=

u
m

where m 2 = 1 + ns2 and ns is

L . A straightforward choice of ns is ns u such that

m 2 = 1 + u 2 and thus, u , y L . Thus, y u , and 1 y y = y u .

Minimization problem: min J () = min 12 ( y u ) 2 = min 2 m1 2 ( y u ) 2

is well-posed.

Using the gradient method, we obtain

& = 1u , > 0,
or
u
& = 1 2 = u ,
m
with

1
m2

, where is called the normalized estimation error. Note that

~
~
1
u
u
= 2 = 2 =
,
m
m
m

~&
~
~
where , which implies that = & = u 2 . Construct
~2
~

V ( ) =
.
2
Then,
~
V& = 2u 2 = 2m2 0
~
~
~
such that , L , m L2. Because u , L , if follows that = mu and m

L . Hence we have & = r m u L L2 . Since

~&
~
d
(m) = u u& ,
dt

if we assume that u& L , then

d
dt

(m) L , which together with m L2 implies

~&
that m(t) 0 as t by Barbalats Lemma. Then, (t ) 0 as t .
Remark 4.2.1: Despite y, u may be unbounded, adaptive law can still be designed

such that L and & L L2 .

4.2 NORMALIZED ADPTIIVE LAWS BASED ON SPR-LYAPUNOV DESIGN

APPRAOCH

Consider an SISO plant as follows:

x& = Ax + Bu,
y = C T x.
Two kinds of parameterizations: (i) y = T + 0 , or (ii) y = W ( s )T + 0 , where
= [bn1 , bn2 , K, b1 , b0 , an1 , an2 , K, a1 , a0 ]T , = b , b = [0, T ]T ,
u
= H1 ( s ) ,
y

u
= H (s) ,
y

and W(s) is one strictly proper transfer function with stable poles, stable zero (i.e.,
minimum phase) and relative-degree one. A general model is the form of
z = W (s )T , which is linear parameter model or linear regression model.
1

Choose L(s) so that L (s) is a proper stable transfer function and W(s)L(s) is a
proper SPR transfer function. Then
z = W ( s ) L( s )T ,

= L1 ( s ) ,

with its estimate as z = W ( s ) L( s ) T so that the estimation error is 1 z z and

normalized estimation error is
= 1 W ( s ) L( s) ns2 ,
where ns is the normalizing signal such that

L , m 2 = 1 + ns2 .
m
(Note that the example of ns is ns2 = T , or ns2 = T P , where P > 0.) Thus,
~
= W ( s) L( s)( T ns2 ) ,
~
where . Now, let (Ac, Bc, Cc) be the state space representation of W(s)L(s),

then

~
e& = Ac e + Bc ( T ns2 ),
= CcT e,
and
W ( s ) L( s ) = CcT ( sI Ac ) 1 Bc .
Since W(s)L(s) is SPR, there is a matrix Pc > 0 such that

Pc Ac + AcT Pc = qq T Lc ,
Pc Bc = Cc ,
for some vector q, matrix Lc > 0, and a small constant > 0, by either KYL Lemma (if
(Ac, Bc, Cc) is minimal) or MKY Lemma (if (Ac, Bc, Cc) is nonminimal). To design the
adaptive law, construct a Lyapunov like function
~
~
~
e T Pc e T 1
V ( , e) =
+
,
2
2
where > 0 such that
~
~
~&
1

V& = eT qq T e e T Lc e + eT Pc Bc ( T ns2 ) + T 1
2
2
~
~&

1
= eT qq T e e T Lc e 2 ns2 + T 1 ( ),
2
2

and hence, the adaptive law is selected as follows:

& = .

(4.2.1)

~
~&
i.e., = . Obviously, e, , , L , and , ns L2 , so that m L2 and

& = (m ) L2 ,
m

which is independent of the boundedness of . We summarize the property of this

design by the following theorem:
Theorem 4.2.1: The SPR-Lyapunov adaptive law guarantees that
(i)

, L ,

(ii) , ns , & L2 ,

independent of the boundedness properties of .

Remark 4.2.1: For stable plants, L . Then, & L and hence 0, 1 0 as t

by Barbalats Lemma provided ns2 = T , or ns2 = T P .

One important property of the adaptive law is the convergence of to the un

known vector . Such a property is achieved for a special class vector signals described by the following definition.
Definition 4.2.1 (Persistence of Excitation (PE)): A piecewise continuous signal
n

vector : R + R n is PE in R with a level of excitation 0 > 0, if there are constants 1, T0 > 0 such that
1I

1
T0

t + T0
t

()T ()d 0 I , t 0 .

Remark 4.2.2: is PE if and only if 1

1
T0

t +T0
t

[q T ()]2 d 0 , t 0 , where q

is any constant vector in R with q = 1 .

n

Before we guarantee the convergence of to by the PE condition, useful

lemmas are introduced as follows.
Lemma 4.2.1 (Uniformly Complete Observability (UCO) with Output Injection):
nl

Assume that there exits constants > 0, kv 0, such that for all t0 0, K(t) R

sat-

isfies

t0

nl

Then (C, A + KC ), where C R

T

t0 + v

K () d kv
2

nn

,AR

, is a UCO pair if and only if (C, A +

KC ) is a UCO pair.

n
Lemma 4.2.2: If w: [0, ) R is PE, w& L , and H(s) is a stable, minimum phase,

proper rational transfer function, then w1 = H(s)w is PE.

Lemma 4.2.3: Consdier
Y&1 = AcY1 Bc TY2
Y&2 = 0
y0 = CcTY1

where Ac is a stable matrix, (Cc, Ac) is observable, and L. If f defined as

f CcT ( sI Ac ) 1 Bc

satisfies

2I

1
T0

t + T0

f ()Tf ()d 1I , t 0

for some constant 1, 2, T0 > 0, then the system above is UCO.

Now the convergence of to by the PE condition is proved in the following.

Corollary 4.2.1: If ns , , & L , and is PE, then the former adaptive law guar

antees (t) exponentially fast.

Proof: Consider
~
e& = Ac e + Bc ( T ns2 )

~&
=

= Cc e

(4.2.2)

that describe the stability properties of the adaptive law. In proving the Theorem 4.1.1,
we have also shown that the time derivative of
& = J () = ,
where = T > 0 and Pc = PcT > 0 satisfies

V& 2
for some constant > 0. Defining
A BcCcT ns2
A(t ) = c
T
Cc

Bc T
T
, C = [Cc
0

0]T , P =

1 Pc
2 0

0
1

we rewire (4.2.2) as
x& = A(t ) x, = C T x
and express the above Lyapunov-like function V and its derivative as
V = x T Px
V& = x T ( PA + AT P + P& ) x x TCC T x = 2

where P& = 0 . This implies that

PA(t ) + AT (t ) P + C (t )C T (t ) 0.
Using Theorem 3.3.4 (See Lecture 3), we can establish the equilibrium xe = 0 (i.e., ee
~
= 0 and e = 0) is u.a.s., equivalent e.s. provided (C, A) is a UCO pair.
Since the (C, A) and (C, A + KC ) have the same UCO property, where
T

B n2
K c s ,

7

is bounded (see Lemma 4.1.1). We can therefore establish that (4.2.1) is UCO by
showing that is a UCO pair. The system corresponding to (C, A + KC ) is as
T

Y&1 = AcY1 Bc TY2

Y&2 = 0
y0 = CcTY1

Because is PE and CcT ( sI Ac ) 1 Bc is stable and minimum-phase and & L , it

follows that

f () CcT e Ac ( ) Bc ()d
t

is also a SPR (refer to Lemma 4.1.2); therefore, there exists constants 1, 2, T0 > 0
such that
2I

1
T0

t + T0
t

f ()Tf ()d 1I , t 0

We can conclude that (C, A + KC ) is UCO (see Lemma 4.1.3) which implies (C, A) is
~
UCO. Hence, we conclude that the equilibrium xe = 0 (i.e., ee = 0 and e = 0) is e.s. in
T

the large.
1

If W(s) is minimum phase, one may choose L(s) = W (s) leading to W(s)L(s) = 1.
Then,
=

~T
1

=

.
2
m
m2

Consider
~ T 1~
~

V ( ) =
2
so that V& = 2 m 2 provided the adaptive law is chosen
& = .

4.3 NORMALIZED GRADIETN ALGOIRHTM

4.3.1 The Gradient Algorithm Based on Instantaneous Cost

Consider the following cost functions:

J () =

2 m 2 ( z T ) 2
=
(Quadratic cost function),
2
2m 2

where
8

z T
m2

is the normalized estimation error based on the estimate of . We have

& = J () = , (Instantaneous adaptive Law)
i.e., (t) is chosen at each time t to minimize the square of the error. The performance
of the instantaneous adaptive law is summarized as follows.
Theorem 4.3.1: The instantaneous adaptive law guaranteed that
(i)

, ns , , & L ,

(ii) , ns , & L2 ,

independent of the boundedness of the signal vector and

(iii) if ns, L and is PE, then exponentially fast.

Proof: From the adaptive law, we have

~&
= .

(4.3.1)

We choose the Lyapunov-like function

V=

~ T 1~

.
2

Then along the trajectory of the adaptive law, we have

~
V& = T = 2 m 2 0.

(4.3.2)

(4.3.3)

~
Hence, V , L , which implies that , m L . In addition, we establish from the
properties of V, V& that m L2 , which implies that , ns L2 . Now, from the
adaptive law, we have

~& &
= m
m
which together with

L and m L2 I L implies that & L2 I L and the

proof of (i) and (ii) is complete.

The proof for (iii) is given now. The parameter error equation may be written as
~&
~
= A(t )
~
y0 = C T (t )

where

(4.3.4)

A(t ) =

T
T
T
,
C
(
t
)
=

,
m2
m

y0 = m

This system is analyzed using the Lyapunov-like function (4.3.2) that led to (4.3.3)
along the trajectory of this adaptive law. We need to establish that the equilibrium
~
e = 0 of (4.3.4) is e.s. We achieve that by using Theorem 3.3.4 (See Lecture 3) as
1

follows. Let P = , then

V=

~T ~
P
2

and
~
~
~
1~
V& = T ( PA + AT P + P& ) = TCC T
2
where P& = 0 . This implies that
PA(t ) + AT (t ) P + 2C (t )C T (t ) 0
~
According to Theorem 3.3.4, e = 0 is e.s. provided (C, A) is UCO. Using Lemma

4.2.1, we have that (C, A) is UCO if (C, A + KC ) is UCO for some that satisfies the
T

condition of Lemma 4.2.1. We choose

K =

leading to A + KC = 0. We consider the following system that corresponds to the pair

T

(C, A + KC ), i.e.,
T

Y& = 0
y0 = C T (t )Y =

T
m

(4.3.5)

The observability grammian of (4.3.5) is given by

N (t , t + T ) =

t +T
t

()()
d
m 2 ()

Because is PE and m 1 is bounded, it follows that immediately that the grammian

matrix N(t, t + T) is positive definite for some T > 0 and for all t 0, which implies
that (4.3.5) is UCO which in turn implies that (C, A) is UCO; thus, the proof is com

plete.
4.2.1 The Gradient Algorithm Based on Integral Cost

Consider another cost functions:

10

J () =

1 t ( t ) 2
e
(t , )m 2 ()d (Integral cost function),

0
2

where
(t , ) =

z () T (t )()
, (t , t ) =
m 2 ( )

is the normalized estimation error at time based on the estimate (t) of at time
t . We have
t
z () T (t )()
()d , (Integral adaptive law)
& = J () = e ( t )
0
m 2 ()

i.e. (t) is chosen at each time t to minimize the integral square of the errors on all
past data that are discounted exponentially. That is,
& = [ R(t ) + Q(t )]

R& = R + 2 ,
m
z
Q& = Q 2 ,
m
T

R(0) = 0,

R R n n

Q(0) = 0,

Q Rn

The performance of the integral adaptive law is summarized as follows.

Theorem 4.3.2: The integral adaptive law guarantees that
(i)

, ns , , & L ,

(ii) , ns , & L2 ,
(iii) lim & (t ) = 0 , and
t

(iv) if ns, L, and is PE, then converges exponentially to .

*

Proof: Because

L , it follows that R, Q L and, hence, behaves as a LTV

system with bounded input. Substituting for z = T in the differential equation for
Q, we verify that
t

Q = e

(t )

()T ()
d = R(t ) ,
2
m

and hence,
~&
~
& = = R(t ) .

(4.3.6)

Consider the Lyapunov-like function

~ T 1~
~

V ( ) =
2
11

(4.3.7)

such that
~
~
V& = T R (t ) 0 .

(4.3.8)

~
T
Since R(t) = R (t) 0, t 0, it follows that V, L ,
1~
~ ~ 1
( T R ) 2 = R 2 L2 .

~T
~
From = m 2 and ,

L , we conclude that and m, therefore, ns L.

From (4.3.6), we have

1
1~
& R 2 R 2 L2 L

1~
which together with R L and R 2 L2 L imply that & L2 L . Since

~& &
~&
, R L , it follows from (4.3.6) that && L , which together with L2 , implies
that
~
lim & (t ) = lim R (t ) (t ) = 0 .
t

To show that m L2, we proceed as follows. We have

~
~
~ ~ ~
d ~T
R(t ) = 2 m 2 2 T RR T R .
dt
so that

t
0

t~
t~
~ ~
~
~
2 m 2 d = T R + 2 T RR d + T R d .
0

1~
~
~
Because T (t ) R(t ) (t ) 0 as t 0 , and R 2 L2 , it follows that

lim 2 m 2 d = 2 m 2 d < .
t

i.e. m L2 .
The proof for (iv) is given now. In proving (i) to (iii), we have shown (4.3.8)
from (4.3.7). From the differential equation on R, we have
t

R (t ) = e ( t )
0

Because is PE and is bounded, we have

12

()T ()
d
m 2 ()

R (t ) =

t
t T0

(t )

0e T0

T
t T0
()T ()
( t ) ( ) ( )
d +
e
d
0
m 2 ()
m 2 ()

t
t T0

()T ()d

1e T0 I
for any t T0, where 1 = 00T0 , 0 = sup t

1
m2 ( t )

and 0, T0 > 0 are constants given

by the definition of PE. Hence,

~
~
~ ~
V& = T R(t ) 1e T0 T 21 min ()e T0 V
for any t T0, which implies that
V (t ) e (t T0 )V (T0 ), t T0
where = 21 min ( )e T0 . Using
2 min ( )V 2 max ( )V

we have that
max ( ) ~
( t T )
(T0 ) e 2 0
min ( )

~
( t T )
(t ) 2 max ( )V (t0 )e 2 0

Thus, (t) converges exponentially to with a rate of

*

Remark 4.3.1: & = J () 0 as t without any additional condition on

and m.
Remark 4.3.2: (t) converges to a trajectory that minimizes the integral asymptoti-

cally with time. Furthermore, if ns , L , and is PE, then

~
(t )

(t t 0 )
max ( ) ~
, t T0 ,
(t0 ) e 2
min ( )

where = 21e T0 min () , 1 = 0T00 , 0 = sup t

1
m2 ( t )

and 0, T0 are the constants

defining PE of . Hence, larger 0 and larger min () will guarantee faster convergence of (t ) to zero.

4.4 NORMALIZED LEAST SQUARES

Consider a simple plant:

y = u + d n ,
where dn is a noise disturbance. Consider the following two approaches:
13

Bad approach:

(t ) =

y ( )
d ( )
= + n
u ()
u ( )

for some < t for which u() 0.

Better approach:

J () =

1 t
( y () (t )u ())2 d,
2 0

J () = y ()u ()d + (t ) u 2 ()d = 0,

t
(t ) = u 2 ()d

t
0

y ()u ()d (Least-squares estimate).

Example 4.4.1: u(t) 1, t 0 and dn has a zero average value,

1 t
1 t

y
(

)
u
(

)
d

+
lim
d n ()d = .
t t 0
t t 0

lim (t ) = lim
t

For general linear model, z = T , the estimate z of z and the normalized estimation error are generated as: z = T , and
=
J () =

z z z T
2
=
, m2 = 1 + ns ,
m2
m2

1 t ( t ) ( z () T (t )()) 2
1
e
d + e t ( 0 )T Q0 ( 0 ) ,
2

2 0
m ()
2

where Q0 > 0, 0, 0 = (0). Because

z
m

, m L , J() is a convex function of

over R at each time t. Hence, any local minimum is also global and satisfies
J ((t )) = 0 , t 0,
i.e.
J () = e t Q0 ((t ) 0 )

1 t ( t ) z () T (t )()
e
()d = 0
2 0
m 2 ( )

which yields the so-called non-recursive least-squares algorithm:

t

z ()()
(t ) = P(t ) e t Q00 + e ( t )
d ,
0
m 2 ()

where
1

()T ()
P(t ) = e t Q0 + e ( t )
d .
0
m 2 ( )

14

Because Q0 = Q0T > 0 , and () () is positive semi-definite, P(t) exists at each time
T

t. Then

d
d

PP 1 = P& P 1 + P P 1 = 0
dt
dt

We can show that

1
P& = P P 2 P , P (0) = P0 = Q0 ,
m
T

& = P

(Continuous-time recursive least-squares algorithm).

Supplementary: Why & = P for Least-squares?

Since
t

z ()()
d ,
(t ) = P(t ) e t Q00 + e ( t )
0
m 2 ()

it follows that
t

P 1 (t )(t ) = e t Q00 + e ( t )
0

()T ()
d ,
2
m ( )

Hence,
t
()T ()
(t )T (t )

P& 1 (t )(t ) + P 1 (t )& (t ) = e t Q00 e ( t )

d

+
.
0
m 2 ( )
m 2 (t )

Recall that
(t ) (t )
P& 1 (t ) = P 1 (t ) +
.
m 2 (t )
T

Therefore,
(t ) (t )
P (t )& (t ) = P& 1 (t )(t ) ( P 1 (t )(t )) +

m 2 (t )
T

(t )T (t )
()T ()
1

(
t
)

(
P
(
t
)

(
t
))
+

= P 1 (t ) +
m 2 (t )
m 2 ()

( z z )(t ) 1 (t )
=
= 2 (t ) = (t )(t ),
m 2 (t )
m (t )
and hence, the least square is given by
& (t ) = P(t )(t )(t ).
4.4.1 Pure Least-Squares Algorithm

Set = 0, we have
15

P P &
, = P ,
P& =
m2
T

where P is usually called the covariance matrix. In terms of the P , we have

d 1 T
P = 2 ,
dt
m
1

which implies that P may grow without bound. In the matrix case, this means P may
become arbitrarily small and slow down adaptation in some directions. This is the
so-called covariance wind-up problem that constitutes one of the main drawbacks of
the pure least-squares algorithm.
Theorem 4.4.1: The pure least-squares algorithm guarantees that
(i)

, ns , , & , P L ,

(ii) , ns , & L2 ,
(iii) limt (t ) = , where is a constant vector, and
(iv) if ns, L, and is PE, then (t) converges to * as t .
Proof: We have that P& 0 , i.e. P(t) P0. Because P(t) is nonincreasing and bounded

from below (i.e., P(t) = P (t) 0, t 0), it has a limit, i.e.,

T

limt P(t ) = P .
where P = P T 0 is a constant matrix. Let us now consider
T~
d 1~
&
1 & 1~
1~
+ = 0,
( P ) = P PP + P =
dt
m2
~
~
1~
1~
Hence, P 1 (t ) (t ) = P0 (0), and therefore, (t ) = P (t ) P0 (0), and
~
1~
lim (t ) = P P0 (0),
t

which implies that

1~
lim (t ) = + P P0 (0) .
t

~
~
~
Because P(t) P0 and (t ) = P(t ) P01 (0) , we have , L , which, together with

~T

L , implies that m = m L and hence , ns L . Let us now consider the

function. Consider
~T 1~
P
,
V=
2
The time derivative of along the trajectory of this adaptive law is given by
16

~T T~

2m2
2m2
~T
2 2
&
V = +
=

m
+
=

0
2m 2
2
2
which implies that V L, m L2; therefore, , ns L2, we have. From the adaptive law, we have

& P
m
m
Because P,

, m L and m L2, we have & L2 L , which completes the

proof (i), (ii), and (iii).

The proof of (iv) is given now. In proving (i) to (iii), we have shown that satisfies
the following equation
~
1~
(t ) = P (t ) P0 (0),

We now show that P(t) 0 as t when satisfies the PE condition. Because P 1

satisfies
d 1 T
P = 2
dt
m
using the condition that is PE, i.e.,

t +T
t

()T ()d 0T0 I

for some constant 0, T0 > 0, it follows that

P 1 (t ) P 1 (0) =

T
t
()T ()
T
d n0 0 0 I 1 0 0 I
2
0
m ()
m
m
T0
t

Therefore
T
t
T
t
P 1 (t ) P 1 (0) + 1 0 0 I 1 0 0 I , t T0
T0 m
T0 m

which implies that

1

t
t

P (t ) 1 0 0 I = 1 0T0 m I , t T0
m

T0
T0

Since P(t) 0 for all t 0 and the right-hand side of the above inequality goes to zero
asymptotically, we can conclude that P(t) 0 as t as. Hence the proof of (iv) is

complete.

Remark 4.4.1: Convergence rate of (t) to is not guaranteed to be exponential

even when is PE. In fact,

17

P (t )

m
~
P01m ~
I , (t )
(0) , t T0 ,
(t T0 ) 0
(t T0 ) 0

~
where m = sup t m 2 (t ) . i.e. (t ) is guaranteed to converge to zero with a speed of
1/t.
4.4.2 Pure Least-Squares with Covariance Resetting

To avoid covariance wind-up problem, we modify least-squares by incorporating covariance resetting mechanism:
& = P,
P P
P& =
, P(t r+ ) = P0 = 0 I ,
m2
T

where tr is the time for which min ( P (t )) 1 , and 0 > 1 > 0 are some design scalars
such that P (t ) 1I , t 0 .
Theorem 4.4.2: The pure least-squares algorithm with covariance resetting has the

following properties:
(i)

, ns , , & L ,

(ii) , ns , & L2 ,
(iii) if ns, L, and is PE, then (t) converges to exponentially fast.
*

Proof: The covariance matrix P(t) has elements that are discontinuous functions of

time due to the resetting. At the discontinuity or resetting point tr, P(tr+ ) = P0 = 0 I ,
therefore, P 1 (tr+ ) = 01I . Between discontinuities,

d
dt

P 1 (t ) 0 , i.e., P (t2)

P (t1) 0, t2 t1 0 such that tr [t1, t2], which implies that

P 1 (t ) 01I , t 0. On the other hand, because of resetting, P (t ) 1I , t 0. .
Hence, we have 0 I P (t ) 1I , 11I P 1 (t ) 01I , t 0.
Let us consider the function
~T 1~
P
V=
2
Since is a bounded positive definite symmetric matrix, it follows that V is decrescent
~
and radially unbounded in the space of . Along the trajectory of this adaptive law,
we have

18

1 ~ d ( P 1 ) ~ ~ T 1~&
1 ~ d ( P 1 ) ~
+ P = 2 m 2 + T

V& = T
dt
dt
2
2
Between the resetting points, it follows
1 (T ) 2
2m2
V& = 2 m 2 +
=

0
2 m2
2
t [t1, t2], where [t1, t2] is any interval in [0, ) for which tr [t1, t2]. On the other
hand, at the discontinuity of P, we have
V (tr+ ) V (tr ) =

~
1 ~ T 1 +
[ P (tr ) P 1 (tr )].
2

Because P 1 (tr+ ) = 01I , P 1 (tr ) 01I , it follows that

V (tr+ ) V (tr ) 0
which implies that V is a nonincreasing function of time for all t 0. Hence, V L
and limt V(t) = V. Since the points of discontinuities form a set of measure zero, it
~
follows that m, L2 . From V L, and 11I P 1 01I , we have L ,
which implies that , m L . Using m L I L2 and 0 I P (t ) 1I , we have
& L I L2 and the proof of (i) and (ii) is, therefore, complete.
The proof of (iii) is similar to the proof of Theorem 4.4.1 (iii) and is omitted.
4.4.3. Modified Least-Squares with Forgetting Factor

When > 0, the problem of P(t) becoming arbitrarily small in some directions no
longer exist. But P(t) may grow without bound since P& > 0 for P > 0 . Thus,
modification is the following:
& = P

PT P
P

m2
P& =
0,

if P(t ) P0
otherwise

where P (0) = P0 > 0 .

4.5 NORMALIZED ADAPTIVE LAWS WITH PROJECTION

Consider the linear parametric model:

z = W (s )T , R n .
Sometimes it could be advisable to design adaptive laws that are constrained to search
19

for estimates of in the set where is located. The advantages lie in:
*

(i)

to speed up convergence,

(ii) to reduce large transients when (t) is far away from , and
*

(iii) to constrain (t) such that it always satisfies certain properties t (Con-

strained parameter estimation).

4.5.1 Gradient Algorithm with Projection

Let us start with the gradient method as follows:

min J ()
subject to S
where S is a convex with smooth boundary almost everywhere. Let S be given by
S = { R n | g () 0},
where g : R R a smooth function. The solution of the constrained minimization
n

problem follows from the gradient projection method is given by:

J ,
if Int( S ) or if ( S ) and (J )T g 0

& = Pr(J )
gg T

J ,
otherwise
J

g T g

or
,
if Int( S ) or if ( S ) and ()T g 0

& = Pr(+) =
gg T

,
otherwise

g T g

where (0) S..

Theorem 4.5.1: The gradient projection adaptive law retain all the properties that are

established in the absence of projection, and in addition guarantees that S ,

t 0 provided (0) S and S.
*

Brief Proof: Whenever (S ) , & g 0 , which implies that & points either in-

side or along the tangent plane of (S ) at point . Because (0) S , it follows that
(t) will never leave S, t 0 .
Next, the difference of the adaptive law, when projection is applied, lies in the
additional term:
Q=

g g T
J .
g T g
20

To show that this additional term will not try to make V& more positive, we see that
~ T g g T
~ T 1

J ,
Q = g T g
0 ,
otherwise

if ( S ) and (J )T g > 0

~
and that Tg = ( )T g 0 when (S ) because S is convex. Therefore,

~
( T g )(J ) T g
~T 1

0,
Q=
g T g
0 ,
otherwise

if ( S ) and (J ) T g > 0

~
In other words, the additional term T 1Q in V& introduced by projection can only
make V& more negative.

4.5.2 Least-Squares with Projection

Consider the adaptive law as follows:

P,
if Int( S ) or ( S ) and ( P)T g 0

& = Pr ( P) =
gg T
otherwise
P P g T g P,

where (0) S , S = R n | g () 0 , and

PT P
P
,
&
P=
m2
0,

if Int( S ) or ( S ) and ( P)T g 0

otherwise

where P(0) = P0 > 0 .

4.6 BILINEAR PARAMETRIC MODEL

As shown in Lecture 2, a certain class of plants can be parameterized in terms of their

desired controller parameters that are related to the plant parameters via a Diophantine
equation. Such parameterizations and their related estimation problem arise in direct
MRAC. Recall that the bilinear parametric model means
z = W ( s )[ (T + z0 )]

where is an unknown constant; z, , z0 are signals that can be measured and is a

*

known proper transfer function with stable poles.

For simplifying, we assume sgn( ) is given here.
*

4.6.1 SPR-Lyapunov Design

We rewrite the bilinear parametric model in the form

21

z = W ( s ) L( s )[ (T + z1 )]

where z1 = L1 ( s ) z0 , = L1 ( s ) and L(s) is chosen so that L is proper and stable

and WL is proper and SPR. The estimate z of z and the normalized estimation error
are generated as
z = W ( s ) L( s )[(T + z1 )]
= z z W ( s ) L( s )ns2

where ns is designed to satisfy

z1
, L , m 2 = 1 + ns2
m m

and (t), (t) are the estimates of at time t, respectively. Letting

~
~
= , =
it follows that
= W ( s ) L( s )[T ~
z1 T ns2 ]

~
~T which implies that
Now T T = T
~
~ n 2 ], = T + z
= W ( s ) L( s )[ T
s
1

A minimal state representation is given by

~z T n 2 )
e& = Ac e + Bc (T
1
s

= CcT e

(4.6.1)

where CcT ( sI Ac ) 1 Bc = W ( s ) L( s ) is SPR. The adaptive law is now developed by

considering the Lyapunov-like function
~ T 1~ ~ 2
~

e T Pc e
+
+
V ( , e) =
2
2
2

where Pc = PcT > 0 satisfies the algebraic equations implied by the KYL Lemma, and
= T > 0 , > 0. Along the trajectory of the adaptive law, we have
~
~
~& ~
eT qq T e T
~
&
V& =
e Lc e T ~
2 ns2 + T 1 +

2
2

where > 0, Lc = LTc > 0. Since = sgn( ) , it follows that by choosing

~& &
= = sgn( )

& = & =
we have
22

(4.6.2)

e T qq T e T
V& =
e Lc e 2 ns2 0
2
2

The performance of this adaptive law is summarized in the following.

Theorem 4.6.1: This SPR-Lyapunov adaptive law guarantees that:
(i)

, , L ,

(ii) , ns , & , & L2 ,

*
(iii) If , & L, is PE and L2, then (t) converges to as t .

(iv) If L2, converges to a constant independent of the properties of .

Brief Proof: The proof of (i) and (ii) follows directly form the properties of by fol-

lowing the same procedure as the linear parametric model case and is left as an exercised for the students. The proof of (iii) is establish by using the results of Corollary
treated as an external input
4.2.1 to show the homogeneous part of (4.6.1) with ~
L2 and Ac is
together with the equation of (4.6.2) form an e.s. system. Since ~

~
stable, it follows that e(t ), (t ) 0 as t . The proof of (iv) follows from ,
L2 and the inequality
1

2
2

2
2
0 & () d 0 ()() d 0 ()d 0 ()d <

which implies & L1 . Hence, we conclude that has (t) a limit .

Remark 4.6.1: The lack of convergence of to is due to L2. If, however, are

such that is PE, then we can establish by following the same approach as in the proof
in Corollary 4.2.1 that converge to zero exponentially fast. For L2, the vector [ ,
T

] can not be PE even when is PE.

T

4.6.2 Gradient Algorithm

For the gradient method, we rewritten the model as

z = (T + z1 )

where z1 = W ( s ) z0 , = W ( s ) . The estimate z of z and the normalized estimation

error are generated as
z = (T + z1 )
=

z z z (T + z1 )
=
m2
m2
23

where ns is designed to satisfy

z1
, L , m 2 = 1 + ns2
m m

We consider the cost function

2 m 2 ( z T + z1 ) 2
J=
=
2
2m 2

Using the gradient method, we obtain

& = 1

& =

which indicates that a implementable form

& = 1 sgn( )

& =

for

1 =

= sgn( )

n+1

Remark 4.6.2: Strictly speaking, J is not a convex function of , over R

because

of the dependence of on . Let us, however, ignore this dependence and treat as an
independent function of time.
The performance of this adaptive law is summarized in the following.
Theorem 4.6.2: This instantaneous adaptive law guarantees that:
(i)

, ns , , & , & L ,

(ii) , ns , & , & L2 ,

(iii) If ns, L, is PE and L2, then (t) converges to as t .
*

(iv) If L2, converges to a constant independent of the properties of .

The proof from that of the linear parametric model and Theorem 4.6.1 and is left as an
exercise for the students.
The extension of the integral adaptive law and least-squares algorithms to the
bilinear parametric model is more complicated and difficult to the implement due to

the appearance of the unknown in the adaptive laws. This problem is avoided the

knowledge of a lower bound for in addition to sgn( ). (The detailed introduction is

24

given in [Ioannou & Sun, 1996].)

4.7 HYBRID ADAPTIVE LAWS

For gradient algorithm, we have

z z
& = , = 2 ,
m

where z = *T is the output of the linear parametric model and z = T . Then,

k +1 = k +

t k +1
tk

()()d, 0 = (0) , k = (t k ), k = 1,2,K

Theorem 4.7.1: Let m, Ts = (tk+1 tk), and be chosen such that

T
1, m 1,
m2
and

2 Ts m r for r > 0 , where m = max () .

Then the hybrid adaptive law guarantees that
(i) k l ,
(ii) k l2 , , m L L2 , where k = k +1 k ,
(iii) If , m L and is PE, then k as k exponentially fast.
~
~
Proof: Let V (k ) = kT 1k

~
where k = k . We have
~
~
~
V (k ) = (2 k + k )T 1 k

where V (k ) = V (k + 1) V (k ) , which implies that

T
t k +1
t k +1
~ T t k +1
V (k ) = 2 k ()()d + ()()d ()()d
tk

tk
tk
~T
Because m 2 = k (t ) and m (= max () ) , we have
2

V (k ) 2

t k +1
tk

t k +1
()
d .
()m ()d + m ()m()
t
m()
k
2

Using Schwartz inequality, we can establish that

2

t k +1
t k +1 ( )
t k +1
()
d
()m()
d 2 ()m 2 ()d
t
tk
tk
m ( )
m()
k

Ts

t k +1
tk

such that
25

2 ()m 2 ()d

V (k ) = (2 Ts m )

t k +1
tk

2 ()m 2 ()d

So, if 2 Ts m > r > 0 , then V (k ) 0 , which implies that V (k ) is a nonincreasing

~
function and thus the boundedness of V (k ) , k and k follows. Hence, we have

t k +1
0

2 ()m 2 ()d

V (0) V (k + 1)
(2 Ts m )

which yields that

lim V (k + 1) exists, and m L L2 L L2 since m 1

Similarly, we can obtain

Tk k Ts m

t k +1
tk

2 ()m 2 ()d

such that

T
k

k =1

Ts m

2 ()m 2 ()d <

which implies k l2 , and thus completes the proof of (i) and (ii).
The proof of (iii) is given now. From the proof of (1) and (ii), we have
V (k ) = V (k + 1) V (k ) = (2 Ts m )

t k +1
tk

2 ()m 2 ()d

which implies
n 1

V (k + n) V (k ) = (2 Ts m )
i =1

t k +i +1
t k +i

2 ()m 2 ()d

(4.7.1)

for any integer n. We now write

t k +i +1
t k +i

~
[ kT+1()]2
d
()m ()d =
t k +i
m 2 ( )
~T
~
~ T
2
t k +i +1 [ ( ) + (
k
k +1 k ) ( )]
d
=
t k +i
m 2 ()
2

t k +i +1

Using ( x + y ) 2 12 x 2 y 2 , we write

Since

t k +i +1
t k +i

( )
m ( )

2 ()m 2 ()d

~
~
~ T
2
t k +i +1 [(
1 t k +i+1 [ kT+1()]2
k +1 k ) ( )]
d

d
t k +i
2 t k +i
m 2 ()
m 2 ()

is bounded, we have

t k +i +1
t k +i

~
~
~
~
[( k +1 k )T ()]2
d cTs k + i k
2
m ( )

where
26

(4.7.2)

()
m 2 ()
2

c = sup

From the hybrid adaptive algorithm, we have

t k +i
~
~
k + i k = ()()d, i = 1, 2, K , n
tk

Using the Schwartz inequality and the boundedness of

( )
m ( )

t k +n
~
~ 2
k + i k ciTs 2 ()m 2 ()d,
tk

(4.7.3)

Using (4.7.2) and (4.7.3), we have

t k +i +1
t k +i

~
t k +n
1 t k +i +1 [ kT+1()]2
()m ()d
d c 2iTs2 2 ()m 2 ()d
2
t k +n
2 t k +i
m ( )
2

which leads to

t k +n
tk

n 1

2 ()m 2 ()d =
i =0

t k +i +1

2 ( ) m 2 ( ) d

t k +i

~
t k +n

1 t k +i+1 [ kT+1()]2

d c 2iTs2 2 ()m 2 ()d
2
t
t
k +n
m ()
i = 0 2 k +i

n 1

and hence it follows that

t k +n
tk

~
~ T t k +i+1 [ kT+1()]2 ~
1
dk
()m ()d
k
2[1 + n(n 1)c 2Ts2 / 2] t k +i
m 2 ()
2

(4.7.4)

Since is PE and 1 m < , there exist constants 2 , 1 , T0 > 0 such that

2 I

t + T0
t

~
[ kT+1()]2
d 1 I
m 2 ()

for any t. Hence, for any integer k, n where n satisfies nTs T0 we have
~
~ T t k +i +1 [ kT+1()]2 ~
~ ~ V (k )
k
1
dk 2 kT k
2
t k +i
m
m ( )
Using (4.7.4) and (4.7.5) in (4.7.1), we obtain the following inequality:

V ( k + n) V ( k ) =

(2 Ts m )1
V (k )
2[1 + n(n 1)c 2Ts2 / 2]

hold for any n with nTs T0. Now it follows that

V (k + n) V (k )
with
=1

(2 Ts m )1
<1
2[1 + n(n 1)c 2Ts2 / 2]
27

(4.7.5)

Therefore,

V (kn) = V ((k 1)n + n) V ((k 1)n) 2V ((k 2)n) L kV (0)

or
~
V (kn)
V ( 0)
kn

( )k
m
m

~
yields that kn 0 exponentially, which together with the property of the hybrid
~
~
adaptive algorithm (i.e., k +1 k ), implies k converges to exponentially. The

proof of (iii) is complete.

ASSIGNING READING

[1] Chapter 4 in [Ioannou & Sun, 1996].

[2] Chapter 2 in [Sastry & Bodson, 1989].

28