System Identification, Lecture 1

Kristiaan Pelckmans (IT/UU, 2338)

Course code: 1RT880, Report code: 61800 - Spring 2011
F, FRI Uppsala University, Information Technology

19 January 2011

SI-2011 K. Pelckmans Jan.-March, 2011

 Lecture 1

• Course Overview.

• System Identification in a Nutshell.

• Applications.

• Simple Example.

• Course Outline.

SI-2011 K. Pelckmans Jan.-March, 2011 1

 Course Organisation

Part I.: Basics.

• 7 Lectures.

• 4 Exercise Sessions.

• 5 Computer Labs (Report Mandatory, 0.5ECTS).

• 1 Laboratory Session (Report Mandatory, 0.5ECTS).

→ Written Exam (March 8. 8am-1pm, 5ECTS).

Part II. Advanced.

• 5 Lectures.

• Projects.

→ Presentation + Report project (4ECTS).

SI-2011 K. Pelckmans Jan.-March, 2011 2

 Part I.: Basics

SISO:

(i) Overview.

(ii) Least Squares Rulez.

(iii) Models & Representations.

(iv) Stochastic Setup.

(v) Prediction Error Methods.

(vi) Model Selection and Validation.

(vii) Recursive Identification.

SI-2011 K. Pelckmans Jan.-March, 2011 3

 Problem Solving Sessions

1. Aspects of Least Squares.

2. Statistical Aspects: what can go wrong with OLS?

3. Prediction Error Methods.

4. Recursive Identification.

5 Computer Labs:

1. Least Squares Estimation: do’s and don’ts.

2. Timeseries Modeling.

3. Recursive Identification.

4. The System Identification Toolbox.

5. MIMO: Kalman Filter and Subspace ID.

SI-2011 K. Pelckmans Jan.-March, 2011 4

 Part I.: Advanced

MIMO:

(i) State Space Models.

(ii) Realization Theory.

(iii) Subspace Identification.

(iv) Design of Experiments.

(v) Perspectives.

SI-2011 K. Pelckmans Jan.-March, 2011 5

 Projects:

• Identification of an industrial Petrochemical plant

• Identification of an Acoustic Impulse Response

• Identification of Financial Stock Markets

• Identification of a Multimedia stream

• *.*

SI-2011 K. Pelckmans Jan.-March, 2011 6

 Course Material

• Book: ”System Identification”, T. Söderström, P. Stoica,

Prentice-Hall, 1989 1

• Lecture Notes: Available from next week in lectures, or online.

• Slides: available at lectures

• Solutions excercises.

1
see [Link] ...

SI-2011 K. Pelckmans Jan.-March, 2011 7

 In order to pass the course, I need to have for each of the
candidates:

1. Attendance of the lab. session, as well as a filled out copy of

the lab report.

2. A filled out report of the computer sessions.

3. successful exam.

4. A project report.

5. A successful presentation of the project (possibly shared

amongst partners in the group).

SI-2011 K. Pelckmans Jan.-March, 2011 8

 Prerequisites

• Linear algebra and statistical techniques.

• 120 ECTS credits.

• Courses Signals and systems, Automatic control I, Automatic

control II.

• Ph.D. student.

SI-2011 K. Pelckmans Jan.-March, 2011 9

 System

System (S): A defined part of the real world. Interaction

with the environment are described by input signals, output
signals and disturbances.

Dynamical System: A system with a memory, i.e. the input

value at time t will influence the output signal at the future, i.e.
t0 > t.

Figure 1: Schematic picture of a system

SI-2011 K. Pelckmans Jan.-March, 2011 10

 Figure 2: A System and A Model

SI-2011 K. Pelckmans Jan.-March, 2011 11

 Ex.: A Stirred Tank

Figure 3: A Stirred Tank

SI-2011 K. Pelckmans Jan.-March, 2011 12

 Ex.: Speech

SI-2011 K. Pelckmans Jan.-March, 2011 13

 Ex. and...

• Stock (Shock) Market

SI-2011 K. Pelckmans Jan.-March, 2011 14

• Acoustic Noise Cancellation Headset (Adaptive filtering)

SI-2011 K. Pelckmans Jan.-March, 2011 15

• Evolution of the Temperature in the world

SI-2011 K. Pelckmans Jan.-March, 2011 16

• Construction (Strength)

SI-2011 K. Pelckmans Jan.-March, 2011 17

• Robots (Mechanical, Operational, Intellectual)

SI-2011 K. Pelckmans Jan.-March, 2011 18

• Social Behavior of Crowd (gossip)

SI-2011 K. Pelckmans Jan.-March, 2011 19

• A human protein-protein interaction network

SI-2011 K. Pelckmans Jan.-March, 2011 20

 Models

Model (M): A description of a system. The model should

capture the essential behavior of the system.
Systems Models
Complex Approximative (Idealization)
Examine real Models can answer
system is costly many questions.

SI-2011 K. Pelckmans Jan.-March, 2011 21

 Applications

• Process Design. Ex. Designing new cars, planes, ... .

• Control Design.
1. Simple regulators
2. Simple models, optimal regulators,
3. sophisticated models.

• Prediction. Ex. Forecast the weather, Predict the Stock

market.

• Signal Processing. Ex. Acoustic Echo Cancellation.

• Simulation. Ex. Train new nuclear plant operators, try new

operating strategies.

• Fault Detection. Ex. VISA.

SI-2011 K. Pelckmans Jan.-March, 2011 22

 Type of Models

• Mental, intuitive or verbal. Ex. Driving a car.

• Graphs and Tables. Ex. Bode plots and step responses.

• Math. models. Ex. Differential and Difference equations.

SI-2011 K. Pelckmans Jan.-March, 2011 23

 Mathematical Models

• Analytical Models Basic laws from physics (...) are used to

describe the behavior of a phenomenon (system).

– Know the physics.

– Yields physical Interpretation
– Quite general models. Often Nonlinear

• System Identification

– Black-Box models (Konfektionsmodeller) ”Choose a

standard model and tune the parameters (...) to the
data”.
∗ Easy to construct and use.
∗ Less general. Linear (-ized)
– Grey-Box models (Skräddarsydda Modellerer) ”Derrive the
model from laws and tune ’some’ parameters to data”.
∗ Combines Analytical models and black-box identification.

SI-2011 K. Pelckmans Jan.-March, 2011 24

 Figure 4: White-, Black- and Grey-Box Models

SI-2011 K. Pelckmans Jan.-March, 2011 25

 Examples of Models

• Nonlinear vs. Linear (superposition principle):

”The net response at a given place and time caused by
two or more stimuli is the sum of the responses which
would have been caused by each stimulus individually.”
(Wiki)

• Time-continuous versus Time-discrete

• Deterministic versus Stochastic

SI-2011 K. Pelckmans Jan.-March, 2011 26

 System Identification (SI)

Def. System Identification is the study of Modeling dynamic

Systems from experimental data.

• Statistics, Systems Theory, Numerical Algebra.

• System Identification is art as much as science.

• Software available (MATLAB)

• – Estimation (Gauss (1809)),

– Modern System Identification (Åström and Bohlin (1965),
Ho and Kalman (1966)),
– Recent System Identification (L. Ljung, 1977-1978)
– Textbooks (Ljung 1987, Söderström and Stoica, 1989).

SI-2011 K. Pelckmans Jan.-March, 2011 27

 The System Identification Procedure

1. Collect Data. If possible choose the input signal such that

the data is maximally informative. Display data, and try to
get some intuition about the problem at hand.

2. Choose Model Structure. Use application knowledge and

engineering intuition. Most important and most difficult step
(don’t estimate what you know already)

3. Choose Identification Approach. How would a good model

look like?

4. Do. Choose best model in model structure (Optimization or

estimation)

5. Model Validation. Is the model good enough for our purpose?

SI-2011 K. Pelckmans Jan.-March, 2011 28

SI-2011 K. Pelckmans Jan.-March, 2011 29
 Typical Problems to Answer

• How to design the experiment. How much data samples to

collect?

• How to choose the model structure?

• How to deal with noise?

• How to measure the quality of a model?

• What is the purpose of the model?

• How do we handle nonlinear and time-varying effects?

SI-2011 K. Pelckmans Jan.-March, 2011 30

 System Identification Methods

• Non-parametric Methods. The results are (only) curves,

tables, etc. These methods are simple to apply. They give
basic information about e.g. time delay, and time constants
of the system.

• Parametric Methods (SI) The results are values of the

parameters in the model. These may provide better accuracy
(more information), but are often computationally more
demanding.

SI-2011 K. Pelckmans Jan.-March, 2011 31

 Course Outline

SISO:

(i) Overview.

(ii) Least Squares Rulez.

(iii) Models & Representations.

(iv) Stochastic Setup.

(v) Prediction Error Methods.

(vi) Model Selection and Validation.

(vii) Recursive Identification.

SI-2011 K. Pelckmans Jan.-March, 2011 32

 Conclusion

• System identification is the art of building mathematical

models of dynamical systems using experimental data. It is
an iterative procedure.
– A real system is often very complex. A model is merely a
good approximation.
– Data contain often noise, individual measurements are
unreliable.

• Analytical methods versus system identification (white-,

black-, grey box)

• Non-parametric versus Parametric Methods

• Procedure: (a) Collect data, (b) Choose Model Structure,

(c) Determine the best model within a structure, (d) Model
validation.

SI-2011 K. Pelckmans Jan.-March, 2011 33

 An example

Identify a hairdryer: air is fanned through a tube and heated

at the inlet. Input u(t): power of the heating device. Output
y(t): air temperature.

>> load dryer2

>> z2 = [y2(1:300) u2(1:300)];
>> idplot(z2, 200:300, 0.08)

SI-2011 K. Pelckmans Jan.-March, 2011 34

 Nonparametric Modeling

>> z2 = dtrend(z2);
>> ir = cra(z2);
>> stepr = cumsum(ir);
>> plot(stepr)

SI-2011 K. Pelckmans Jan.-March, 2011 35

 Parametric modeling:

y(t) + a1y(t − 1) + a2y(t − 2) = b1u(t − 3) + b2u(t − 4)

>> model = arx(z2, [2 2 3]);

>> model = sett(model,0.08);
>> u = dtrend(u2(800:900));
>> y = dtrend(y2(800:900));
>> yh = idsim(u,model);
>> plot([yh y]);

SI-2011 K. Pelckmans Jan.-March, 2011 36

 Pole-zero plot of the model:

>> zpth = th2zp(model);

>> zpplot(zpth);

SI-2011 K. Pelckmans Jan.-March, 2011 37

 Compare the transfer functions obtained from from non- and
parametric methods:

>> gth = th2ff(model);

>> gs = spa(z2); gs = sett(gs,0.08);
>> bodeplot([gs gth]);

SI-2011 K. Pelckmans Jan.-March, 2011 38