Systems & Control: Foundations & Applications

Alexander B. Kurzhanski
Pravin Varaiya

Dynamics and
Control of
Trajectory Tubes
Theory and Computation
Systems & Control: Foundations & Applications

Series Editor
Tamer Başar, University of Illinois at Urbana-Champaign, Urbana, IL,
USA

Editorial Board
Karl Johan Åström, Lund University of Technology, Lund, Sweden
Han-Fu Chen, Academia Sinica, Beijing, China
Bill Helton, University of California, San Diego, CA, USA
Alberto Isidori, Sapienza University of Rome, Rome, Italy
Miroslav Krstic, University of California, San Diego, CA, USA
H. Vincent Poor, Princeton University, Princeton, NJ, USA
Mete Soner, ETH Zürich, Zürich, Switzerland;
Swiss Finance Institute, Zürich, Switzerland
Roberto Tempo, CNR-IEIIT, Politecnico di Torino, Italy

More information about this series at http://www.springer.com/series/4895

Alexander B. Kurzhanski • Pravin Varaiya

Dynamics and Control

of Trajectory Tubes
Theory and Computation
Alexander B. Kurzhanski Pravin Varaiya
Faculty of Computational Mathematics Electrical Engineering
and Cybernetics and Computer Sciences
Moscow State (Lomonosov) University of California, Berkeley
University Berkeley, CA, USA
Moscow, Russia
Electrical Engineering
and Computer Sciences
University of California, Berkeley
Berkeley, CA, USA

ISSN 2324-9749 ISSN 2324-9757 (electronic)

ISBN 978-3-319-10276-4 ISBN 978-3-319-10277-1 (eBook)
DOI 10.1007/978-3-319-10277-1
Springer Cham Heidelberg New York Dordrecht London
Library of Congress Control Number: 2014948336

Mathematics Subject Classification (2010): 34A60, 34K35, 35E10, 35K25, 37N35

© Springer International Publishing Switzerland 2014

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.birkhauser-science.com)

Preface

This book presents a systematic treatment of theoretical methods and computation

schemes for solving problems in dynamics and control. The mathematical models
investigated here are motivated by processes that emerge in many applied areas. The
book is aimed at graduate students, researchers, and practitioners in control theory
with applications and computational realizations.
The emphasis is on issues of reachability, feedback control synthesis under
complex state constraints, hard or double bounds on controls, and performance in
finite continuous time. Also given is a concise description of problems in guaranteed
state estimation, output feedback control, and hybrid dynamics using methods of
this book. Although its focus is on systems with linear structure, the text also
indicates how the suggested approaches apply to systems with nonlinearity and
nonconvexity.
Of primary concern is the problem of system reachability, complemented by the
problem of solvability, within a given class of controls. This leads to two basic
questions: find the terminal states that the system can reach from a given initial state
within a specified time, using all possible controls, and find the initial states from
which the system can reach a given terminal (“target”) set in specified time, using
all possible controls. The answer to these questions takes the form of bundles of
controlled trajectories—the reachability tubes—that emanate either forward in time
from a given starting set (the forward reachability tube) or backward in time from a
given target set (the solvability or backward reachability tube). The cross-sections
(“cuts”) of these tubes are the reachability (“reach”) sets. Backward reach sets are
important in designing feedback (closed-loop) controls.
We are thus led to dealing with trajectory tubes—set-valued functions that are
the main elements of the required solutions. These tubes are also key in describ-
ing systems operating under uncertainty, in the form of unknown but bounded
parameters in the system model, inputs, and measurement errors. The mathematical
properties of the solutions, such as non-differentiability, may also lead to set-valued

v
vi Preface

functions. Hence we have to conduct the investigation of feedback control system

dynamics taking into account the set-valued nature of the functions involved. This
is achieved using the Hamilton–Jacobi formalism in conjunction with the Dynamic
Programming approach and studying value functions for appropriate problems of
dynamic optimization. These value functions turn out to be the solutions to related
types of the HJB (Hamilton–Jacobi–Bellman) equation with appropriate boundary
conditions. The level sets of such value functions, evolving in time, describe the
tubes we need.
In addition to theoretical rigor, the solutions to modern control problems
must be effectively computable. That is, the results must be available within
prescribed time, perhaps on-line, the set-valued trajectory must be calculated with a
guaranteed accuracy, and the computation procedures should be able to handle high-
dimensional systems. This may require parallelization of solutions and distributed
computation. The computation approaches presented here meet such needs. They
are based on ellipsoidal calculus introduced in [174], successfully extended and
applied in [181, 182], and complemented by appropriate software tools [132].
The book is divided into eleven chapters. Chapter 1 provides an exposition of
problems in control theory for linear systems with solutions given in the specific
forms needed for transition to computable operations over trajectory tubes.
Chapter 2 is focused on how to solve. The main theoretical tools—Hamiltonian
methods and Dynamic Programming techniques—as applied to problems of reach-
ability and target control synthesis are described. These are reduced to optimization
problems, whose solution is given by an equation of the HJB type. In its turn, the
solution to the HJB equation produces value functions whose level sets are the
desired reach sets. Usually, attention is needed in interpreting the solution to the HJB
equation. However, for “linear-convex” systems (with convex constraints on the
controls and convex starting or terminal sets) the value functions are convex in the
state variables and usually unique. These functions are directionally differentiable
along any direction in the state space. Hence there is no need for subtle definitions of
generalized viscosity solutions since for equations used here they may be expressed
through conventional classical arguments. Moreover, direct integration of the HJB
equation is avoided by calculating the exact value function through duality methods
of convex analysis. Such a procedure is also used to apply a verification theorem
to the concrete HJB equation, confirming that the calculated value function is its
solution. This also allows us to derive the feedback control directly from the HJB
equation. The derived control is in general a set-valued function and its substitution
in the system equation produces a differential inclusion. Effective computation of
trajectories that steer the system from point to point arrives through an intersection
of colliding tubes—the forward reach tube from a given starting point and the
backward reach tube from a given terminal point. The final section describes a
solution to the problem of time-optimal target control.
Chapter 3 indicates how to calculate the solutions given as set-valued functions.
The approach develops external and internal ellipsoidal-valued functions that
approximate the reachability tubes. Parametrized families of ellipsoids are designed,
whose intersection produces an external bound and whose union produces an
Preface vii

internal bound. Increasing the number of approximating ellipsoids in the limit

yields the exact reach set. The next move is to calculate the synthesizing feedback
target control. As already mentioned, such a feedback strategy may be calculated
directly from the HJB equation. However, this control strategy, being a nonlinear
function of time and state, would require an appropriate existence theorem for the
synthesized nonlinear differential inclusion. To avoid this possible difficulty, it is
convenient to use the “aiming” rule of Krasovski. Namely, calculating the backward
reachability set in advance, one should design the feedback control such that it keeps
the trajectory within the backward reach tube, following this “bridge” until it reaches
the terminal target set. The computation scheme for the aiming rule is as follows:
with given starting point in “time-space” that lies within or beyond the backward
reach set, one selects an internal ellipsoidal approximation of the backward set
that either contains this point or is the closest to it. This aiming strategy may be
calculated explicitly or with minimal computational burden.
Chapter 4 offers examples of problems solved by ellipsoidal methods and
illustrated graphically. Though attractive and effective, ellipsoidal solutions may
run into some degenerate cases that are more common for internal approximations.
These potential computational problems are especially evident in large systems
where the dimension of controls is much smaller than the system dimension. Ways
of regularizing such situations are discussed in a section on high-dimensional
systems. The described tools have proved effective in computational experiments.
The treatment of high-dimensional systems indicated here is reached through a
parallelization of solutions, which is a natural extension of the suggested schemes.
The first part of Chap. 5 is devoted to nonlinearity and nonconvexity. The general
approach formulated here is applicable to nonlinear systems and presented in the
form of a comparison principle. The idea is to approximate the available HJB
equation from above or below through relations that ensure guaranteed upper or
lower solution estimates by operating with functions simpler than in the original
equation. This approach does not depend on the type of exact solution to the HJB
equation, whether classic or generalized. At the same time, the comparison principle
allows one to develop a deductive approach to the ellipsoidal calculus of this book
in contrast with the inductive approach of Chap. 3. These topics are followed by
examples of reachability for nonlinear systems and calculation of the set of points
reachable within a time interval, which is a nonconvex union of convex sets. The
second part of the chapter deals with the application of ellipsoidal methods to
systems with non-ellipsoidal constraints, such as boxes and zonotopes (symmetrical
polyhedra).
The chapters that follow cover a variety of useful properties and specific
problems related to implementing the proposed approach. Chapter 6 emphasizes
the role of double constraints—the simultaneous presence of both a hard bound
and an integral bound on the controls. Solutions to such problems are needed for
treating impulse controls investigated in this chapter. Impulse inputs were previously
studied as open-loop controls with few special results on feedback solutions [23,36].
A theory of feedback impulse control is presented here for systems of arbitrary
dimension, with solutions in terms of quasi-variational inequalities of the HJB
viii Preface

type. However, impulse inputs are ideal elements and their physical realization by
“ordinary” bounded functions may be achieved through approximation by double-
bounded controls which are functions whose bound tends to infinity.
Chapter 7 is concerned with dynamics and control of systems under state
constraints. Previously discussed problems are now subject to additional “viability”
restrictions on the state variables. The description covers solution approaches to
problems of reachability (forward and backward) and emphasizes specifics of
related mathematical techniques including ellipsoidal approximations. The case
of linear systems with convex hard bounds on controls and state coordinates are
worked out in detail.
The contents of Chaps. 1–7 indicate that one of the main items in treating
considered problems are trajectory tubes and the means of their calculation. The
same is true for the rest of the chapters. So Chap. 8 begins with fundamentals of
a general vision—the theory of trajectory tubes with models of their evolutionary
dynamics. Indicated results, together with considerations of the previous chapter,
are further applied to closed-loop control under state constraints, with techniques
borrowed from both Hamiltonian approach and duality theory of nonlinear analysis.
This brings forward the discussion to complex state constraints in the form of
obstacle problems, wherein constrained trajectories must simultaneously lie within
one set and outside another.
The next two chapters consider uncertainty, which is inherent in realistic
problems of control. These chapters may serve as an introduction to a more thorough
description of uncertainty.
Chapter 9 is a concise explanation of the theory of guaranteed state estimation,
also known as the set-membership bounding approach to external disturbances in
estimation models. In contrast with conventional descriptions, the present exposition
involves Hamiltonian methods and is applicable to nonlinear systems. Dynamic
estimation of system trajectories under unknown but bounded errors is also formu-
lated as a problem with state constraints, which now are not known in advance, but
arrive online, in real time. In the linear case the proposed deterministic “filtering”
equations demonstrate connections and differences when compared with stochastic
(Kalman) filtering. Both continuous and discrete measurements are considered.
The results of Chap. 9 have a natural application to problems of output feedback
control under unknown but bounded disturbances in system and measurement
inputs. These problems are addressed in Chap. 10. The solutions introduced there
are based on the notions of generalized state and information tubes which describe
the overall system dynamics. For the linear case with convex constraints on controls
and uncertainties items, computation schemes based on ellipsoidal approximations
are presented. Several examples are worked out.
Finally, Chap. 11 is confined to verification problems and hybrid systems, with
a description of exact solutions and ellipsoidal schemes for their computation. The
discussion is accompanied by some examples. Special attention is given to possible
involvement of impulse inputs in formal mathematical models of hybrid systems.
The aim of this chapter is to emphasize the applicability of techniques presented in
this book to the investigation of hybrid systems.
Preface ix

References to prior literature and research results related to this book are given
in the introduction to each chapter.
The mathematical level of the book presumes reasonable knowledge of advanced
calculus, linear algebra and differential equations, as well as basics of variational
analysis with optimization methods and computational mathematics.
Throughout the previous years we had the pleasant opportunity for useful
discussions on the topics arranged in this book with K. Åström, J.-P. Aubin,
J. Baras, T. Başar, F. Chernousko, P. Kokotovic, A. Krener, A.A. Kurzhanskiy,
Yu. Ledyayev, G. Leitmann, A. Lindquist, J. Lygeros, M. Milanese, I. Mitchell,
S. Mitter, A. Rantzer, J. Sousa, C. Tomlin, I. Vályi, and V. Veliov. Their valuable
comments helped to crystallize the contents.
We thank A. Daryin, M. Gusev, T. Filippova, I. Rublev, and P. Tochilin for
reading parts of the manuscript, their useful comments, and contributed illustrations.
Our thanks surely goes to the authors of illustrations for the examples of this
book. Their names are indicated throughout the text.
The authors are grateful to the US national Science Foundation, the first author is
also grateful to the Russian Foundation for Basic Research for the support of their
work.

Moscow, Russia Alexander B. Kurzhanski

Berkeley, CA, USA Pravin Varaiya
Contents

1 Linear Control Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 The Controlled System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Control and State Constraints: Open-Loop
and Closed-Loop Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 Constraints on Control and State . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.2 Open-Loop and Closed-Loop Control . . . . . . . . . . . . . . . . . . . . . 8
1.3 Optimal Control with Norm-Minimal Cost:
The Controllability Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3.1 Minimum Energy Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3.2 Minimum Magnitude Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3.3 Controllability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.3.4 The Finite-Dimensional Moment Problem . . . . . . . . . . . . . . . . 17
1.4 The Reachability Problem: Time-Optimal Control . . . . . . . . . . . . . . . . . 18
1.4.1 Reachability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4.2 Calculating Controls for the Boundary
of Reach Set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.4.3 Time-Optimal Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.5 Optimal Open-Loop Control for Linear-Convex
Systems: The Maximum Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.5.1 The Necessary Conditions of Optimality . . . . . . . . . . . . . . . . . . 25
1.5.2 Degenerate Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.5.3 Sufficiency Conditions for Optimality . . . . . . . . . . . . . . . . . . . . . 32
1.5.4 A Geometrical Interpretation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1.5.5 Strictly Convex Constraints on Control . . . . . . . . . . . . . . . . . . . 38
1.6 Duality Methods of Convex Analysis in Problems
of Optimal Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
1.6.1 The Primal and Dual Optimization Problems . . . . . . . . . . . . . 41
1.6.2 General Remark: Feedforward Controls. . . . . . . . . . . . . . . . . . . 46

xi
xii Contents

2 The Dynamic Programming Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.1 The Dynamic Programming Equation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.2 The Linear-Quadratic Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.3 Reachability Through the HJB Equation: Hard Bounds. . . . . . . . . . . . 58
2.3.1 Forward Reachability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.3.2 Backward Reachability or the Solvability Problem . . . . . . . 61
2.4 Reachability for Linear-Convex Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.4.1 Forward Reachability: Calculating the Value Functions . . 64
2.4.2 The Conjugate of the Value Function V0 . . . . . . . . . . . . . . . . . . 67
2.4.3 The Value Function (Backward) . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.5 Colliding Tubes: Calculating All Reachable Points . . . . . . . . . . . . . . . . 70
2.6 The Closed-Loop Target Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.7 Reachability Within an Interval. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2.8 Dynamic Programming: Time-Optimal Control . . . . . . . . . . . . . . . . . . . . 83
3 Ellipsoidal Techniques: Reachability and Control Synthesis . . . . . . . . . . 87
3.1 Linear Systems Under Ellipsoidal Constraints. . . . . . . . . . . . . . . . . . . . . . 88
3.2 Ellipsoidal Approximation of Reach Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.3 Recurrent Relations: External Approximations . . . . . . . . . . . . . . . . . . . . . 94
3.4 The Evolution of Approximating Ellipsoids . . . . . . . . . . . . . . . . . . . . . . . . 102
3.5 The Ellipsoidal Maximum Principle and the Reachability Tube . . . 107
3.6 Example 3.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.7 Reachability Sets: Internal Approximations . . . . . . . . . . . . . . . . . . . . . . . . 114
3.8 Example 3.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.9 Reachability Tubes: Recurrent Relations—Internal
Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.10 Backward Reachability: Ellipsoidal Approximations . . . . . . . . . . . . . . 128
3.11 The Problem of Control Synthesis: Solution Through
Internal Ellipsoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
3.12 Internal Approximations: The Second Scheme . . . . . . . . . . . . . . . . . . . . . 141
4 Solution Examples on Ellipsoidal Methods: Computation
in High Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4.1 The Multiple Integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4.1.1 Computational Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
4.2 A Planar Motion Under Newton’s Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
4.2.1 Computational Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
4.3 Damping Oscillations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
4.3.1 Calming Down a Chain of Springs in Finite Time . . . . . . . . 168
4.4 Computation in High-Dimensional Systems.
Degeneracy and Regularization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.4.1 Computation: The Problem of Degeneracy. . . . . . . . . . . . . . . . 182
4.4.2 Regularizing the Ellipsoidal Estimates . . . . . . . . . . . . . . . . . . . . 186
4.4.3 Regularizing the Estimate for the Reachability Tube . . . . . 189
4.4.4 Efficient Computation of Orthogonal Matrix S . . . . . . . . . . . 193
4.4.5 Parallel Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Contents xiii

5 The Comparison Principle: Nonlinearity and Nonconvexity . . . . . . . . . . 197

5.1 The Comparison Principle for the HJB Equation . . . . . . . . . . . . . . . . . . . 198
5.1.1 Principal Propositions for Comparison Principle . . . . . . . . . 198
5.1.2 A Deductive Approach to Ellipsoidal Calculus . . . . . . . . . . . 202
5.2 Calculation of Nonconvex Reachability Sets . . . . . . . . . . . . . . . . . . . . . . . 208
5.3 Applications of Comparison Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5.3.1 Forward Reachability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5.3.2 Systems with Hamiltonians Independent of the State. . . . . 215
5.3.3 A Bilinear System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
5.3.4 External Ellipsoids for the Unicycle: Reachability . . . . . . . 222
5.4 Ellipsoidal Methods for Non-ellipsoidal Constraints . . . . . . . . . . . . . . . 229
5.4.1 Degenerate Ellipsoids: Box-Valued Constraints . . . . . . . . . . 229
5.4.2 Integrals of Box-Valued Functions . . . . . . . . . . . . . . . . . . . . . . . . 231
5.4.3 Reachability Tubes for Box-Valued
Constraints: External Approximations. . . . . . . . . . . . . . . . . . . . . 235
5.4.4 Reach Tubes for Box-Valued Constraints:
Internal Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
5.5 Ellipsoidal Methods for Zonotopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
5.5.1 Zonotopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
5.5.2 Internal Ellipsoidal Tubes for a Zonotope . . . . . . . . . . . . . . . . . 243
5.5.3 External Ellipsoidal Tubes for a Zonotope . . . . . . . . . . . . . . . . 246
6 Impulse Controls and Double Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
6.1 The Problem of Impulse Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
6.1.1 Open-Loop Impulse Control: The Value Function . . . . . . . . 255
6.1.2 Closed-Loop Impulse Control: The HJB
Variational Inequality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
6.2 Realizable Approximation of Impulse Controls . . . . . . . . . . . . . . . . . . . . 266
6.2.1 The Realistic Approximation Problem . . . . . . . . . . . . . . . . . . . . 266
6.2.2 The Approximating Motions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
7 Dynamics and Control Under State Constraints . . . . . . . . . . . . . . . . . . . . . . . . 275
7.1 State-Constrained Reachability and Feedback . . . . . . . . . . . . . . . . . . . . . . 275
7.1.1 The Reachability Problem Under State Constraints . . . . . . 276
7.1.2 Comparison Principle Under State Constraints . . . . . . . . . . . 281
7.1.3 Linear-Convex Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
7.2 State-Constrained Control: Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
7.2.1 The Modified Maximum Principle. . . . . . . . . . . . . . . . . . . . . . . . . 289
7.2.2 External Ellipsoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
7.2.3 Generalized Multipliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
7.2.4 An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
7.2.5 Specifics: Helpful Facts for State-Constrained
Control Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
xiv Contents

8 Trajectory Tubes State-Constrained Feedback Control. . . . . . . . . . . . . . . . 311

8.1 The Theory of Trajectory Tubes: Set-Valued Evolution
Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
8.1.1 Trajectory Tubes and the Generalized
Dynamic System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
8.1.2 Some Basic Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
8.1.3 The Set-Valued Evolution Equation . . . . . . . . . . . . . . . . . . . . . . . 315
8.1.4 The Funnel Equations: Specific Cases . . . . . . . . . . . . . . . . . . . . . 316
8.1.5 Evolution Equation Under Relaxed Conditions . . . . . . . . . . . 319
8.2 Viability Tubes and Their Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
8.2.1 The Evolution Equation as a Generalized
Partial Differential Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
8.2.2 Viability Through Parameterization . . . . . . . . . . . . . . . . . . . . . . . 322
8.3 Control Synthesis Under State Constraints: Viable Solutions. . . . . . 325
8.3.1 The Problem of State-Constrained
Closed-Loop Control: Backward Reachability
Under Viability Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
8.3.2 State-Constrained Closed-Loop Control. . . . . . . . . . . . . . . . . . . 328
8.3.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
8.4 Obstacle Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
8.4.1 Complementary Convex State Constraints:
The Obstacle Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
8.4.2 The Obstacle Problem and the Reach-Evasion Set. . . . . . . . 335
8.4.3 Obstacle Problem: An Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
8.4.4 Closed-Loop Control for the Obstacle Problem. . . . . . . . . . . 338
9 Guaranteed State Estimation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
9.1 Set-Membership State Estimation: The Problem.
The Information Tube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
9.2 Hamiltonian Techniques for Set-Membership State Estimation . . . 344
9.2.1 Calculating Information Tubes: The HJB Equation. . . . . . . 344
9.2.2 Comparison Principle for HJB Equations . . . . . . . . . . . . . . . . . 346
9.2.3 Linear Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
9.3 Ellipsoidal Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
9.3.1 Version AE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
9.3.2 Version-BE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
9.3.3 Example: Information Set for a Linear System . . . . . . . . . . . 354
9.4 Discrete Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
9.4.1 Continuous Dynamics Under Discrete
Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
9.4.2 Discrete Dynamics and Observations. . . . . . . . . . . . . . . . . . . . . . 358
9.5 Viability Tubes: The Linear-Quadratic Approximation . . . . . . . . . . . . 363
Contents xv

9.6 Information Tubes vs Stochastic Filtering Equations.

Discontinuous Measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
9.6.1 Set-Valued Tubes Through Stochastic Approximations . . 367
9.6.2 The Singular Perturbation Approach:
Discontinuous Measurements in Continuous Time . . . . . . . 370
10 Uncertain Systems: Output Feedback Control . . . . . . . . . . . . . . . . . . . . . . . . . . 371
10.1 The Problem of OFC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
10.2 The System: The Generalized State and the Rigorous
Problem Formulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
10.3 The Overall Solution Scheme: General Case . . . . . . . . . . . . . . . . . . . . . . . 375
10.4 Guaranteed State Estimation Revisited: Information
Sets and Information States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
10.5 Feedback Control in the Information Space . . . . . . . . . . . . . . . . . . . . . . . . 381
10.5.1 Control of Information Tubes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
10.5.2 The Solution Scheme for Problem C . . . . . . . . . . . . . . . . . . . . . 383
10.5.3 From Problem C to Problem Copt . . . . . . . . . . . . . . . . . . . . . . . . 385
10.6 More Detailed Solution: Linear Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
10.6.1 The “Linear-Convex” Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
10.6.2 The Computable Solution: Ellipsoidal Approximations . . 388
10.7 Separation of Estimation and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
10.8 Some Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
11 Verification: Hybrid Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
11.1 Verification Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
11.1.1 The Problems and the Solution Schemes . . . . . . . . . . . . . . . . . . 396
11.1.2 Ellipsoidal Techniques for Verification Problems . . . . . . . . . 398
11.2 Hybrid Dynamics and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
11.2.1 The Hybrid System and the Reachability Problem . . . . . . . 401
11.2.2 Value Functions: Ellipsoidal Approximation . . . . . . . . . . . . . . 407
11.3 Verification of Hybrid Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
11.3.1 Verification: Problems and Solutions . . . . . . . . . . . . . . . . . . . . . . 417
11.3.2 Ellipsoidal Methods for Verification . . . . . . . . . . . . . . . . . . . . . . . 418
11.4 Impulse Controls in Hybrid System Models . . . . . . . . . . . . . . . . . . . . . . . . 419
11.4.1 Hybrid Systems with Resets in Both Model
and System States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
11.4.2 Two Simple Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
11.4.3 Impulse Controls in Hybrid Systems . . . . . . . . . . . . . . . . . . . . . . 423
11.4.4 More Complicated Example: The
Three-Dimensional Bouncing Ball . . . . . . . . . . . . . . . . . . . . . . . . 425

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
Notations

Rn —the n-dimensional Euclidean vector space, R1 D R

Rnm — the linear space of n  m-matrices

hx; yi D x 0 y—the scalar (inner) product of vectors x; y 2 Rn ; with prime as the

transpose

kxkM D hx; Myi1=2 ; M D M 0 > 0

kxk D kxkI ; I —the identity (unit) matrix

coQ; convQ—the convex hull of set Q

compRn —the variety of all compact subsets Q  Rn

convRn —the variety of all convex compact subsets Q  Rn

d.x; Q/ D inffkx  yk j y 2 Q:g—the distance of point x from set Q

h.P; Q/ D maxfhC .P; Q/; hC .Q; P /:g—the Hausdorff distance between sets
P; Q

hC .P; Q/ D maxfd.x; Q/ j x 2 P g—the Hausdorff semi-distance

G.t; s/—the fundamental transition matrix for a linear homogeneous system

xP D Ax

xvii
xviii Notations

int X —interior of set X

N
I.ljB/—indicator function of set B: I.ljB/ D 0 if l 2 B, I.ljB/ D 1 if x 2B

¡.l j Q/ D supfhl; xi j x 2 Qg—the support function of set Q at point l 2 Rn

graph Y .t / D fft; yg 2 T  Rn j t 2 T D Œt; ª; y 2 Y .t /g

epif .x/ D f.x; y/ 2 Rn  R j y > f .x/g—epigraph of function f

Domf D fx W f .x/ < 1; g—effective domain of f

co®.l/—closed convex hull of ®

@®.l/—subdifferential of convex function ® W Rn ! R at point l

f  .l/ D supx fhl; xi  f .x/g—Fenchel conjugate of function f .x/

Vx —partial derivative in x of function V

C m ŒT —the space of m-dimensional continuous vector functions f W T ! Rm

Lm
p ŒT —the space of m-dimensional vector functions f W T ! R integrable with
m

power p

Vm ŒT —space of m-dimensional vector functions of bounded variation on the

interval T D Œt0 ; ª

VarU.t /; t 2 T —total variation of function U 2 Vm ŒT ; over the interval t 2 T

kf ./kLp —the norm of function f ./ in infinite dimensional space LŒT 

Chapter 1
Linear Control Systems

Abstract This chapter gives an exposition of control theory for linear systems
with emphasis on items and techniques given in a form appropriate for topics in
forthcoming chapters. It introduces problems of reachability and optimal target
control under constraints, as well as time-optimal control. Indicated are solution
approaches to open-loop control that involve the moment problem, the maximum
principle, and the duality methods of convex analysis.

Keywords Control theory • Linear systems • Open-loop control • Reachability •

Controllability • Maximum principle • Convex analysis

Among the mathematical models of controlled systems considered in this book

priority is given to those with linear structure [38, 40, 105, 120, 191, 245, 274]. This
class of systems allows detailed analysis with solutions in a form appropriate for
computational schemes that yield complete solutions to specific problems. In this
chapter we describe standard models and indicate the difference between the design
of open-loop control and closed-loop control strategies that use on-line measure-
ments. We treat problems of dynamic optimization, that is of optimal control. We
indicate several basic open-loop control problems with solution techniques centered
around necessary and sufficient conditions of optimality in two main forms: the
Maximum Principle and duality methods of convex analysis. We start with the
system model.

1.1 The Controlled System

A standard mathematical representation of a controlled system is an ordinary

differential equation (ODE) of the form

P / D f .t; x; u/;
x.t (1.1)

© Springer International Publishing Switzerland 2014 1

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__1
2 1 Linear Control Systems

in which t is time, the vector x 2 Rn is the state, and the vector u 2 Rp is the
control. The function f .t; x; u/ is defined in a domain

D D T  D  P ; t 2 T; x 2 D ; u 2 P ;

in which it is assumed to be continuous. The time interval T may be bounded as

T D Œt0 ; t1 , or unbounded as T D Œt0 ; C1/ or T D .1; t1 . The control set P
may be a compact subset of Rp or it may coincide with Rp . D is an open set that
may coincide with Rn .
The function f .t; x; u/ is assumed to satisfy a condition that guarantees unique-
ness and extendibility of solutions of (1.1) within the domain D. One such condition
is: the partial derivatives fxi .t; x; u/; i D 1; : : : ; n; are continuous in x uniformly
for t 2 T; u 2 P and the inequality

hx; f .t; x; u/i  k.1 C hx; xi/

holds for some k > 0 for all .t; x; u/ 2 D.

The book concentrates on systems with linear structure of the form

P / D A.t /x.t / C B.t /u.t / C v.t /;

x.t (1.2)

with x.t / 2 Rn ; u.t / 2 Rp as before, and with v.t / 2 Rn as a disturbance or

external forcing term. The n  n matrix function A.t / 2 Rnn and the n  p matrix
function B.t / 2 Rnp are assumed to be continuous in t , whereas u.t /; v.t / are
assumed to be Lebesgue-integrable or piecewise right-continuous in t 2 T D
Œt0 ; C1/.1 Under these conditions the solution to (1.1) exists, is unique, and is
extendible within D D T  Rn  P for any initial position ft0 ; x 0 g; x.t0 / D x 0 .
Given an initial position ft0 ; x 0 g, the unique solution x.t / D x.t; t0 ; x 0 / is given
by the integral formula
Z t
x.t / D G.t; t0 /x 0 C G.t; s/.B.s/u.s/ C v.s//ds: (1.3)
t0

Here G.t; s/ is the fundamental transition matrix solution of the homogeneous

equation (1.2). That is, the matrix function G.t; s/ satisfies the equation

@
G.t; s/ D A.t /G.t; s/; G.s; s/ D I (1.4)
@t

1
We use the terms measurable and integrable for Lebesgue-measurable and Lebesgue-integrable
functions.
1.1 The Controlled System 3

in t , as well as the matrix adjoint equation

@
G.t; s/ D G.t; s/A.s/; G.t; t / D I; (1.5)
@s
in s.
G.t; s/ is nonsingular for all t; s and has the following properties:

G 1 .t; s/ D G.s; t /; so that G.t; s/G.s; t / D I (invertibility formula); and

G.t; £/G.£; s/ D G.t; s/ .usually s  £  t; superposition formula):

An integral formula for the adjoint equation, similar to (1.3), is given in (1.80).
When A.t /  A is constant, G.t; s/ D exp.A.t  s//; in which exp A is the
matrix exponential.
Formula (1.3) may be checked by direct substitution. The existence of a solution
to system (1.2) is thus a standard property of linear differential equations (see [38,
105, 120, 274]).
Exercise 1.1.1. Check formula (1.3).
System (1.2) can be put into a simpler form. Transforming the state by

z.t / D G.t0 ; t /x.t /; (1.6)

differentiating, and substituting x.t / for z.t / using (1.1), we come to

zP.t / D G.t0 ; t /B.t /u.t / C G.t0 ; t /v.t /; (1.7)

with

z0 D z.t0 / D G.t0 ; t0 /x 0 D x 0 : (1.8)

Thus there is a one-to-one correspondence (1.6) between the solutions x.t / and
z.t / to Eqs. (1.2) and (1.7), respectively. Their initial values are related through (1.8).
The state z.t / satisfies a particularly simple version of (1.2),

zP.t / D B0 .t /u.t / C v0 .t /; z.t0 / D x 0 ; (1.9)

with

B0 .t / D G.t0 ; t /B.t /; v0 .t / D G.t0 ; t /v.t /:

Thus we may consider system (1.9) rather than (1.2). In other words, in the notation
of (1.2) we may take A.t /  0 with no loss of generality. Note however that the
matrix function B.t / is now time-variant (even when (1.2) is time-invariant).
4 1 Linear Control Systems

One should realize, however, that the transformation (1.6) allows us to take
A.t /  0 only within the time range ft  t0 g, a different t0 leads to a different
transformation (1.6). A similar result is obtained by the substitution

z.t / D G.t1 ; t / x.t /:

With this choice of state the original system again gives A.t /  0, but this is correct
only for ft  t1 g.
We shall sometimes make use of these transformations to demonstrate some
basic techniques with a simpler notation. The reader will always be able to return to
A.t / 6 0 as an exercise.
A shrewd reader may have now realized that there should exist a transformation
of the state that takes a given linear homogeneous equation

xP D A.t /x; (1.10)

into the linear system

zP D A0 .t /z; (1.11)

for any preassigned matrix A0 .t / with the same initial condition, x.t0 / D z.t0 /.
Exercise 1.1.2. Indicate the transformation that converts Eq. (1.10) into (1.11) and
vice versa.
Among the equations of interest are linear matrix differential equations of the
form

P / D A.t /X.t / C X.t /A1 .t / C V .t /;

X.t (1.12)

in which the state X.t / and input V .t / are n  n square matrices.

Introduce two transition matrices G.t; s/ and G1 .t; s/

@ @
G.t; s/ D A.t /G.t; s/; G.s; s/ D I I G1 .t; s/ D A1 .t /G1 .t; s/;
@t @t
G1 .s; s/ D I: (1.13)

Assume X.t0 / D X 0 : Then an integral representation formula (similar to (1.3)) for

Eq. (1.12) is true, namely,
Z t
X.t / D G.t; t0 /X 0 G1 .t0 ; t / C G.t; s/V .s/G1 .s; t /ds: (1.14)
t0
1.2 Control and State Constraints: Open-Loop and Closed-Loop Control 5

For A1 .t / D A0 .t / we have G1 .t0 ; t / D G0 .t; t0 / and

Z t
X.t / D G.t; t0 /X 0 G0 .t; t0 / C G.t; s/V .s/G0 .t; s/ds: (1.15)
t0

Exercise 1.1.3. (a) Check formula (1.14) by substitution.

(b) Solve Exercise 1.1.2 for Eq. (1.12).
Also within the scope of this book are quasilinear systems of the form

P / D A.t /x.t / C B.t /®.t; u/ C v.t /;

x.t (1.16)

with ®.t; u/ continuous in ft; ug 2 T  Rp and convex in u.

More complicated systems that can be treated with the techniques developed in
this book are bilinear systems of the form

P / D A.t; u/x.t / C v.t /;

x.t u 2 U; (1.17)

with A.t; u/ D A.t /U C D.t /, with matrix of controls U 2 Rnn and continuous
matrices of coefficients A.t /; D.t / 2 Rnn . In a simple case we may have A.t; u/ D
A.t /u C D.t / with a scalar u.

1.2 Control and State Constraints: Open-Loop

and Closed-Loop Control

One objective of this book is to develop solutions for systems subject to constraints
on the control and state. The values u of the control may be restricted by various
types of bounds. Here are some typical types.

1.2.1 Constraints on Control and State

Hard bounds also known as geometrical or magnitude bounds are constraints of

the form

u 2 P .t /; (1.18)

in which P .t / is a Hausdorff-continuous set-valued function P W T ! convRp .

The symbol convRn denotes the class of closed, convex sets in Rn , while compRn
denotes the class of compact, convex sets in Rn . In the sequel, unless specially
remarked, we shall suppose that P .t / has a nonempty interior in Rp for all t:
6 1 Linear Control Systems

Hard bounds may also be specified as the level set (at level ) of a function
®.t; u/:

U .t / D fu W ®.t; u/  g:

Suppose ®.t; u/ is continuous in t; u and convex in u. Then its level set P .t / will
be closed and convex. As indicated in [237], for U .t / 6D ; to be bounded, hence
compact, for any , it is necessary and sufficient that
 
0 2 int Dom® .t; / : (1.19)

Here

® .t; l/ D supfhl; ui  ®.t; u/ j u 2 Rp g

is the (“Fenchel”) conjugate of ®.t; u/ in the variable u; and

Dom® .t; / D fl W ® .t; l/ < 1g;

is the effective domain of the function ® .t; l/ for fixed t , see [237] (interiorQ
denotes the collection of all interior points of a set Q). Under condition (1.19) the
level sets P .t / will be bounded, ®.t; u/ ! 1 as hu; ui ! 1; and in the definition
of ® .t; l/ the operation of “sup” may be replaced by “max,” see [72]. Under the
given conditions the convex compact set-valued function P .t / will be Hausdorff-
continuous.
In practice, hard bounds represent limits imposed by equipment or considerations
of safety on a control variable such as voltage, force, or torque. Linear control syn-
thesis techniques, like those based on frequency domain methods, which formulate
the control as a linear function of the state, cannot satisfy hard bounds.
Integral bounds, also known as soft bounds, are constraints of the form
Z t1
jju.t /jjq dt  ;  > 0; q  1; (1.20)
t0

in which jjujj is a norm and q is usually an integer. A more general integral bound
is expressed as
Z t1
®.t; u.t //dt  ; (1.21)
t0

 
in which ®.t; u/ is continuous in t; u and convex in u, with int Dom® .t; /
6D ;.
1.2 Control and State Constraints: Open-Loop and Closed-Loop Control 7

Among the integral bounds one may single out

Z t1
ku.t /kdt  ;  > 0; (1.22)
t0

which coincides with (1.20) for q D 1, and

Z t1 X
k 
kd i u.t /=dt i k dt  ;  > 0: (1.23)
t0 iD0

Constraints (1.22), (1.23) may be satisfied by controls u that weakly converge to

generalized functions: delta functions u D •.t  £ / and their derivatives such as
u D •.i/ .t  £ /.
Integral bounds can represent limits on energy consumption, using (1.20) with
q D 2. A delta-function control, u D •.t  £ /, is an impulse control that causes an
instantaneous change in the state, x.£  0/ ¤ x.£ C 0/, as can be seen from (1.3).
The constraint (1.23) (with k D 1) is used to restrict the instantaneous change in
state.
Joint bounds (or double bounds) combine hard and soft bounds. For example, u
must simultaneously satisfy two constraints (hard and soft):
Z t1
u.t / 2 P .t /; t 2 Œt0 ; t1 ; jju.s/jj2 ds  2 ; or (1.24)
t0
Z t1
hu.t /; u.t /i   ; t 2 Œt0 ; t1 ;
2
hu.s/; u.s/i1=2 ds  2 : (1.25)
t0

Constraint (1.25) can be used to approximate an impulse control by increasing 2

while keeping  fixed.
Constraints may also be imposed on the state space variables. Taking z.t / D
H.t /x.t /, with H.t / 2 Rmn continuous, the constraint may be

z.t / 2 Z .t /; t 2 Tz ;

in which Tz may comprise a discrete set of points, Tz D f£1 ; : : : £k g  T , or it may

be an interval such as T D Tz .
The set-valued function Z .t / 2 convRm is assumed to be piecewise absolutely
continuous in t in the Hausdorff metric (see [76]). Note that the solutions to
system (1.2) are piecewise absolutely continuous functions (with discontinuities
corresponding to impulse controls), hence differentiable almost everywhere. Thus
the properties of the bounding function Z .t / match those of the system solutions.
Also common are integral bounds on the state trajectories of the form
Z t1
®.t; x/dt  ; (1.26)
t0
8 1 Linear Control Systems

with ®.t; x/ being convex in x; or mixed integral bounds on both control and state
of the form
Z t1
®.t; x; u/dt  ; (1.27)
t0

in which ®.t; x; u/ is continuous in t and convex in x; u.

In addition to these common constraints we will also meet in this book some
nonstandard constraints that are less common, but which are very relevant for our
problems. Such, for example, are bounds that define obstacle problems of Chap. 8,
Sect. 8.4, when the system trajectory must lie beyond one convex set and within
another at the same time, or state constraints that are due to on-line measurements
under unknown but bounded noise in Chap. 9.

1.2.2 Open-Loop and Closed-Loop Control

In addition to constraints, one needs to specify the class of functions from which the
controls are to be selected. The appropriate class of functions should be considered
from both the mathematical and control design points of view.
Mathematically, we will consider classes that are functions of time only, u D
u.t /, and those that are functions of both time and state, u D u.t; x/. The two
classes are called open-loop and closed-loop controls, respectively.
An open-loop control, u D u.t /, is required to be measurable. For each open-loop
control, the solution to (1.1) reduces to solving the equation

xP D f .t; x; u.t //;

which, in the linear case, is immediate through the integral formula (1.3). Thus,
substituting an open-loop control u.t / in (1.1) or (1.2) gives a unique solution xŒt  D
x.t; t0 ; x 0 / for any starting position ft0 ; x 0 g; x.t0 / D x 0 . From the point of view of
control design, taking u D u.t / as a function of time only means that the selection
of the control action for each time t is fixed, implanted in the system design, and
cannot be changed throughout the process, which lasts from t0 to t1 : The class of
open-loop controls is denoted by UO .
A closed-loop control, u D u.t; x/, depends on both t and x. The function u.t; x/
may be nonlinear, even discontinuous, in x. The class of admissible closed-loop
controls u.t; x/ must however be so restricted that nonlinear differential equations
of type

P / D f .t; x; u.t; x.t ///;

x.t
1.2 Control and State Constraints: Open-Loop and Closed-Loop Control 9

or

P / D A.t /x.t / C B.t /u.t; x.t // C v.t /;

x.t (1.28)

would have a solution in some reasonable sense. That is, there should be an existence
theorem ensuring that Eq. (1.28) can be solved.
From the point of view of design and implementation, a closed-loop control
selects the control action u.t; x/ at each time t depending on the value of the state
x.t /. For on-line control (that is, when the value of the control action u.t; x.t //
is calculated in real time), the design presupposes that the state x.t / is exactly
measured for all t and continuously communicated to the control device. A control
of this form leads to a feedback loop (also “closed loop”), hence the name closed-
loop or feedback control. They are also referred to as synthesized controls in
contrast with controls of class UO which are known as control programs or open-
loop controls. The class of closed-loop controls to be used, complemented with an
existence theorem, is denoted by UC .
If the disturbance term v.t / is fixed, there is no essential difference between UO
and UC , as the following exercise indicates.
Exercise 1.2.1. Suppose the disturbance v.t / in (1.28) is fixed. Let xŒt  D
x.t; t0 ; x 0 / be a given solution of (1.28) under a closed-loop control g.t; x/, i.e.,

P / D A.t /x.t / C B.t /g.t; x.t // C v.t /:

x.t

Show that there is a corresponding open-loop control u.t / such that xŒt  is also the
solution of

P / D A.t /x.t / C B.t /u.t / C v.t /:

x.t

Show also that the corresponding open-loop control depends on the disturbance, i.e.,
if the disturbance input v.t / changes, so does the open-loop control in order to keep
the same solution xŒt .
Thus the difference in system performance achieved by using open-loop and closed-
loop controls becomes apparent for systems subject to unknown disturbances. The
control of systems with unknown disturbances is mentioned in Chap. 10.
In practice, exact measurement of the state vector x may not be possible. The
measurement may be incomplete because only a part of the vector x or a function of
x can be measured, or because the measurements can be taken only at discrete time
instants, or because the measurements may be corrupted by unknown disturbances
(“noise”). Consideration of such situations will bring us to new classes of feedback
control.
In many cases treated later we allow the closed-loop control u D U .t; x/
to be a set-valued map, with values U .t; x/ 2 compRp , measurable in t and
10 1 Linear Control Systems

upper-semicontinuous in x [7, 8, 48, 238]. Upon substituting u D U .t; x/, Eq. (1.2)
then becomes a nonlinear differential inclusion,

P / 2 A.t /x.t / C B.t /U .t; x/ C v.t /;

x.t (1.29)

whose solution exists and in general is set-valued [75, 76]. The class of such set-
valued controls is denoted by UCS .
In the next section we consider several problems of optimal control within the
class of open-loop controls UO . These are the “norm-minimal” controls that can be
obtained through simple considerations.

1.3 Optimal Control with Norm-Minimal Cost:

The Controllability Property

The moment problem is a simple way of presenting the two-point boundary problem
of control. It naturally allows to present the optimality criterion as one of minimizing
the norm of the control u.t / in an appropriate functional space.
Consider system (1.2) on a finite time interval T D Œt; ª. We start with one of
the simplest problems of optimal control.

1.3.1 Minimum Energy Control

Problem 1.3.1. Given system (1.2) and two points, xŒt  D x, xŒª D x .1/ , find the
optimal control that moves the system trajectory xŒs D x.s; t; x/ from x to x .1/
with minimum cost
Z ª 
.p/
J .t; x/ D min hu.s/; N.s/u.s/ids j u./ 2 L2 .T / ; (1.30)
t

in which the continuous p  p matrix N.s/ is positive definite, N.s/ D N 0 .s/ > 0.
We shall solve this problem applying basic Hilbert space techniques. Using
formula (1.3) the boundary constraints xŒt  D x; xŒª D x .1/ may be rewritten
in the form
Z ª Z ª
G.ª; s/B.s/u.s/ds D x .1/
 G.ª; t /x  G.ª; s/v.s/ds D c: (1.31)
t t
1.3 Optimal Control with Norm-Minimal Cost: The Controllability Property 11

Define the n  p matrix D.s/ D G.ª; s/B.s/ with rows d .i/ ; i D 1; : : : ; n.

For a p-dimensional row d./ denote
Z ª
hd 0 ./; u./iN D d.s/N.s/u.s/ds:
t

.p/
to be an inner product in L2 .T /—the space of square-integrable Rp -valued
.p/
functions. Consider both d .i/ ./ and u./ as elements of the Hilbert space L2 .T /
space, with norm
Z ª 1=2
ku./kN D hu.s/; N.s/u.s/ids :
t

The cost of a control u is the square of its norm and Problem 1.3.1 is to find
the element of a Hilbert space with minimum norm that satisfies certain linear
constraints.2 We can thus reformulate Problem 1.3.1.
Problem 1.3.1-A: Minimize hu./; u./iN under the constraints

hD 0 ./; u./i D hN 1 ./D 0 ./; u./iN D c: (1.32)

We assume that the n functions d .i/ ./ are linearly independent, which means
that for any n-dimensional row-vector œ, œD.t / D 0, a.e. for t 2 T , only if œ D 0.
Now consider controls of the form u.s/ D N 1 .s/D 0 .s/l; for some l 2 Rn , and
substitute in (1.32), to get
Z ª
W .ª; t /l D c; W .ª; t / D D.s/N 1 .s/D 0 .s/ds:
t

The Gramian or Gram matrix W .ª; t / is nonsingular if and only if the functions
d .i/ are linearly independent [82]. As we shall observe later in this section the
determinant jW .ª; t /j 6D 0 (for any matrix N.s/ D N 0 .s/ > 0) ensures that
system (1.2) is controllable (see below, Lemma 1.3.1).
The particular control

u0 .s/ D N 1 .s/D 0 .s/l 0 ; l 0 D W 1 .ª; t /c; (1.33)

satisfies the constraint (1.32). We now show that u0 .s/ minimizes the cost (1.30).

.p/
2
We could also regard functions d .i / .s/ as elements of Lq  ŒT , q   1. Later, while dealing
with impulse controls, it will be more convenient to treat functions d .i / .s/ as elements of the
space C .p/ ŒT .
12 1 Linear Control Systems

Indeed, suppose u.s/ satisfies (1.32), but u.s/ 6D u0 .s/. Then ue .s/ D u.s/ 
u .s/ 6D 0 satisfies
0

hN 1 ./D 0 ./; ue ./iN D hN 1 ./D 0 ./; N./ue ./i D hD 0 ./; u.s/  u0 .s/i D 0;

and

hu./; N./u./i D hu0 ./ C ue ./; N./.u0 ./ C ue .//i

D hu0 ./; N./u0 ./i C hue ./; N./ue ./i
 hu0 ./; N./u0 ./i;

since hu0 ./; N./ue ./i  0 due to (1.33). This gives the next result.
Theorem 1.3.1. The optimal control for Problem 1.3.1 is given by the continuous
function

u0 .s/ D N 1 .s/D 0 .s/W 1 .ª; t /c:

Remark 1.3.1. If x .1/ D 0; v.s/  0, then c D G.ª; t /x, so that the optimal
control

u0 .s/ D N 1 D 0 .s/W 1 .ª; t /G.ª; t /x; (1.34)

is linear in x. This property will be important later in treating problems of closed-

loop control.
.p/
We have just solved the simplest case of the moment problem for Lq with q D 2
(see Theorem 1.3.4 below).

1.3.2 Minimum Magnitude Control

Problem 1.3.2. Given system (1.2), starting position xŒt  D x, and convex compact
terminal set M and terminal time ª, find the optimal control that moves the system
trajectory xŒs D x.s; t; x/ from x to some point xŒª 2 M with minimum cost

J .t; x/ D min ess sup fku.s/k; s 2 T g; (1.35)

in which kuk is a norm in Rp .

If u.s/ is a continuous or even a piecewise right- or left-continuous function, we
may substitute “esssup” by “sup.”
1.3 Optimal Control with Norm-Minimal Cost: The Controllability Property 13

From formula (1.3) the terminal constraint is

Z ª Z ª
G.ª; s/B.s/u.s/ds C G.ª; t /x C G.ª; s/v.s/ds D xŒª 2 M : (1.36)
t t

Again take D.s/ D G.ª; s/B.s/. We treat its rows—the p-dimensional functions
.p/
d .i/ ./—as elements of L1 D L1 ŒT  and treat the control u./ as an element of its
.p/
conjugate space L1 D L1 ŒT . Denote the bilinear functional
Z ª
hu./; d./i D hd.s/; u.s/ids
t

and norms
Z ª
kd./kL1 D kd.s/kds; ku./kL1 D ess supfku.s/k j s 2 T g:
t

Here the finite-dimensional norms ku.s/k and kd.s/k are conjugate.

Let us now take

B1 D fu./ W ku./kL1  1g;

and find the smallest  D 0 for which there exists u./ 2 B1 that satisfies (1.36).
Since M is convex and compact, x.ª/ 2 M is equivalent to the system of
inequalities

hl; x.ª/i  ¡.l j M / D maxfhl; xi j x 2 M g; 8l 2 Rn :

So the necessary and sufficient condition for u./ to satisfy (1.36) is

Z ª
l 0 G.ª; s/B.s/u.s/ds C hl; c  i  ¡.l j M /; 8l 2 Rn ; (1.37)
t

in which
Z ª
c  D G.ª; t /x C G.ª; s/v.s/ds:
t

Hence, for some u./ 2 B1 to satisfy (1.36), it is necessary and sufficient that
Z ª 
min l 0 D.s/u.s/ds j u./ 2 B1 C hl; c  i  ¡.l j M /; 8l 2 Rn ;
u t
14 1 Linear Control Systems

or
Z ª
 minfl 0 D.s/u.s/ j ku.s/k  1gds C hl; c  i  ¡.l j M /g; 8l 2 Rn ;
t

from which it follows that

 
hl; c  i  ¡.l j M /
  0 D max : (1.38)
l kl 0 D./kL1

Under the assumption jW .ª; s/j ¤ 0, (with N.s/  I /—the controllability

property—the maximizer in (1.38) is l 0 ¤ 0.
Because the numerator and denominator in (1.38) are homogeneous, if l 0 is a
maximizer, so is ’l 0 ; 8’ > 0. Hence 0 may be found by solving
 ˇZ 
ˇ ª
0 D max hl; c  i  ¡.l j M / ˇˇ kl 0 D.s/kds D 1 ; (1.39)
l t

or
Z ˇ 
ª ˇ
.0 /1 D min kl 0 D.s/kds ˇˇ hl; c  i  ¡.l j M / D 1 : (1.40)
l t

We are thus led to the next result whose proof follows from (1.40) and the definition
of the norms involved.
Theorem 1.3.2. The minimum magnitude control u0 that solves Problem 1.3.2
satisfies the maximum condition
0 0 0
maxfl 0 D.s/u j kuk  0 g D l 0 D.s/u0 .s/ D 0 kl 0 D.s/k; s 2 T; (1.41)

in which l 0 is the maximizer in (1.39) or the minimizer in (1.40).

The maximum condition is a particular case of the maximum principle discussed
further in more detail in Sect. 1.5.
A not uninteresting problem is the following:
Problem 1.3.2-A Find largest  D  for which the inclusion (1.36) holds for all
u./ 2 B1 .
In order to fulfill the requirement of Problem 1.3.2 it is necessary and sufficient
that
Z ª 
max l D.s/u.s/ds j u./ 2 B1 C hl; c  i  ¡.l j M /; 8l 2 Rn ;
0
u t
1.3 Optimal Control with Norm-Minimal Cost: The Controllability Property 15

or
Z ª
 maxfl 0 D.s/u j kuk  1gds C hl; c  i  ¡.l j M /; 8l 2 Rn ;
t

from which
 
¡.l j M /  hl; c  i
 D min : (1.42)
l kl 0 D./kL1

(As before, kuk is the Rp -dimensional norm conjugate to the norm used in
kl 0 D.s/k.)
Theorem 1.3.3. The solution to Problem 1.3.2-A is given by formula (1.42).
Exercise 1.3.1. Solve Problem 1.3.1 with minimum cost
Z ª ˇ 
ˇ
J .t; x/ D min ˇ
.u.s/; N.s/u.s// ds ˇ u./ 2 Lq .T / ; 1 < q < 1; q 6D 2:
q=2 .p/
t
(1.43)
instead of (1.30).
Observe that the existence of solutions to Problems 1.3.1, 1.3.2 (and as one may
check, also to (1.43)) requires the solvability of Eq. (1.32) for any vector c D
.c1 ; : : : ; cn /0 . This is guaranteed if and only if the functions d .i/ ./; i D 1; : : : ; n;
which are the rows of the n  p matrix

D.s/ D G.ª; s/B.s/

are linearly independent over T D Œt; ª:

Let 1=q C 1=q  D 1.
Definition 1.3.1. An array of elements d .i/ ./ 2 Lq  ŒT ; q  2 Œ1; 1 is said to be
.p/
P
linearly independent if l 0 D.s/ D i li d .i/ .s/ D 0 a.e. implies l D 0.
Lemma 1.3.1. The functions d .i/ ./ are linearly independent if and only if the
Rª
symmetric matrix W .ª; t / D t D.s/D 0 .s/ds with elements
Z ª
Wij D hd .i/ .s/; d .j / .s/ids
t

is nonsingular, i.e., its determinant jW .ª; t /j 6D 0:

W .ª; t / is known as the Gram matrix.
16 1 Linear Control Systems

Exercise 1.3.2. Prove Lemma 1.3.1.

Lemma 1.3.1 leads to the following conclusion.
Lemma 1.3.2. Equation (1.32) is solvable for any vector c within the class of
.p/
functions u./ 2 Lq ŒT ; q 2 Œ1; 1 if and only if jW .ª; t /j 6D 0:

1.3.3 Controllability

The notion of controllability which characterizes the solvability of the two-point

boundary control problem for any pair of points was introduced by Kalman [107,
109]. Having stimulated intensive research on the structure of time-invariant linear
systems [274], it became one of the basic concepts in control theory for any type of
system.
The solvability of (1.32) for any c means that there exists a control that transfers
the system (1.2) from any given starting position ft; xg; xŒt  D x to any given
terminal position fª; x 1 g; xŒª D x 1 . (Since c is defined by (1.31), by choosing
appropriate vectors x; x 1 , we can get any c.)
Definition 1.3.2. System (1.2) is said to be controllable over Œt; ª if given any
positions ft; xg and fª; x 1 g, there exists a control u.s/; s 2 Œt; ª which steers the
system from ft; xg to fª; x 1 g. The system is completely controllable over Œt; ª if
it is completely controllable over any subinterval Œt1 ; t2  Œt; ª, t1 < t2 .
The notion of controllability deals with the existence of controls that solve certain
boundary problems [109]. It is applicable to a broad class of systems, allowing many
versions.
Corollary 1.3.1. System (1.2) is controllable over Œt; ª iff jW .ª; t /j 6D 0 and it is
completely controllable iff jW .ª; t /j 6D 0; 8Œt1 ; t2  Œt; ª, t1 < t2 .
The Gram matrix W .ª; t / is also known as the controllability matrix for
system (1.2).
Remark 1.3.2. (i) In the case of constant matrices A.t /  A; B.t / D B the
property of complete controllability is equivalent to the condition that the set of
column vectors in the array

B; AB; : : : ; An1 B

contains a subset of n linearly independent vectors. In other words, the matrix

ŒB; AB; : : : ; An1 B has full rank n.
1.3 Optimal Control with Norm-Minimal Cost: The Controllability Property 17

(ii) A stronger version of condition (i) requires

b .i/ ; Ab .i/ ; : : : ; An1 b .i/

to be linearly independent for each i D 1; : : : ; p; : This is the case of strong

complete controllability.
In the time-varying case the stronger version requires complete controllability
of (1.2) with B.t / replaced by b .i/ .t /.
Exercise 1.3.3. Prove the assertion of Remark 1.3.2(i).
Remark 1.3.3. From Corollary 1.3.1 it follows that the controllability condition for
.p/
controls in Lq ŒT ; q 2 Œ1; 1 is the same for all q. However, when solving
an optimization problem of type Exercise 1.3.1 within the class of controls from
L1.p/ ŒT , with q D 1, the minimum may not be attained in this class but is attained
in a broader class that includes generalized delta functions.
Exercise 1.3.4. For the one-dimensional system

xP D ax C u;

consider the problem of finding a control that moves the state from xŒt  D x 0 6D 0
to x.ª/ D x .1/ D 0 with minimum cost
Z ª
.1/
minf ju.s/jds j u./ 2 L1 ŒT g:
t

Show that if a ¤ 0 the minimum is not attained, but is attained in the broader class
that includes delta functions.
An interesting issue is therefore to solve an optimization problem like Exercise
1.3.1, but with u./ D d U./=dt being the generalized derivative of U (in the sense
of the theory of distributions, [242]) and with cost as the total variation VarU./
of the function U.t / over the interval T . The solution, which may include delta
functions and its derivatives, is considered later in a separate section.

1.3.4 The Finite-Dimensional Moment Problem

.p/
Consider Eq. (1.32) with D.s/ D G.ª; s/B.s/ having rows d .i/ ./ 2 Lq ŒT ; T D
.p/ .p/
Œt; ª: Let kd./kq denote the norm in the Lq ŒT . The classical finite-dimensional
.p/
moment problem seeks solutions u./ 2 Lq ŒT  of (1.32) with minimum norm [1].
18 1 Linear Control Systems

Theorem 1.3.4. Suppose the d .i/ ./ 2 Lq  ŒT ; q  2 Œ1; 1/; i D 1; : : : ; n; are

.p/

.p/ .p/
linearly independent. Then (1.32) is solvable by some u./ 2 Lq ŒT ; ku./kq 
, if and only if

Z !1=q 
ª
0 0 q  =2
 hD .s/l; D .s/li ds  hl; ci; 8l 2 Rn :
t

Here 1=q C 1=q  D 1:

Theorem 1.3.4 suggests a scheme to calculate norm-minimal solutions to the two-
point boundary control problem for linear systems by reducing it to the moment
problem. This approach, which also applies to impulse controls, was brought into
linear control theory by Krasovski [120]. Equivalent results are also obtained
through the techniques of convex analysis that are used in this book.
So far we considered the problem of reaching a particular state. We now pass to
the problem of finding all the states reachable from a given starting point through
all possible controls, restricted by a given bound.

1.4 The Reachability Problem: Time-Optimal Control

1.4.1 Reachability

This is one of the important elements in analyzing system performance, namely,

what states may be reached in given time by available controls?
Consider system (1.2) with v.s/  0,

P / D A.t /x.t / C B.t /u;

x.t (1.44)

and control constraint (1.18): u.s/ 2 P .s/; s 2 Œt; 1/.

Definition 1.4.1. The reach set X Œª D X .ª; t0 ; x 0 / of system (1.44) at given
time ª, from position ft0 ; x 0 g, is the set of all points x for each of which
there exists a trajectory xŒs D x.s; t0 ; x 0 /, generated by a control subject to the
given constraint (1.18), that transfers the system from position ft0 ; x 0 g to position
fª; xg; x D xŒª:

X .ª; t0 ; x 0 / D fx W 9u./ 2 P ./ such that x.ª; t; x 0 / D xg:

The reach set X Œª D X .ª; t0 ; X 0 / from set-valued position ft0 ; X 0 g at given time
ª is the union
[
X .ª; t0 ; X 0 / D fX .ª; t0 ; x 0 / j x 0 2 X 0 g:

The reach set is also known as the attainability set.

1.4 The Reachability Problem: Time-Optimal Control 19

Lemma 1.4.1. With X 0 and P .s/; s 2 T D Œt0 ; ª, convex and compact, the reach
set X λ is also convex and compact.
Proof. Consider two points x 0 2 X λ reached through control u0 .s/ 2 P .s/, and
x 00 2 X Œª, reached through u00 .s/ 2 P .s/; s 2 Œt0 ; ª. The convexity of X 0 and
P .s/ implies that x .’/ D ’x 0 C .1  ’x 00 / ; ’ 2 Œ0; 1 is reached through u.’/ .s/ D
’u0 .s/C.1’/u00 .s/ 2 P .s/ with initial condition x .’/ Œt0  D ’x 0 Œt0 C.1’/x 00 Œt0 :
This shows convexity. Boundedness of X λ is obvious. That X λ is closed follows
.p/
from the fact that the set of functions u./ 2 P ./ is weakly compact in L2 , so if
x is reached through u ./ and x ! x, then x is reached by a weak limit u./ of
n .n/ n

fu.n/ ./g. t
u
Problem 1.4.1. Find the reach set X λ.
Since X Œª D X .ª; t0 ; x 0 / is convex and compact, we shall describe it through its
support function

¡.l j X Œª/ D maxfhl; xi j x 2 X Œªg

D maxfhl; xŒªi j u./ 2 P ./; xŒt0  D x 0 2 X 0 g: (1.45)

Using formula (1.3), rewrite (1.45) as

Z ª 
¡.l j X Œª/ D max l 0 G.ª; s/B.s/u.s/ds C l 0 G.ª; t /x 0 j u 2 P .s/; x 0 2 X 0
t0
Z ª
D maxfl 0 G.ª; s/B.s/u j u 2 P .s/gds C maxfl 0 G.ª; t0 /x 0 j x 0 2 X 0 g
t0
Z ª
D ¡.B 0 .s/G 0 .ª; s/l j P .s//ds C ¡.G 0 .ª; t0 /l j X 0 /: (1.46)
t0

Introducing the column vector §Œt  D §.t; ª; l/ D G 0 .ª; t /l as the solution to the
adjoint equation

P / D A0 .t /§.t /; §Œª D l;

§.t (1.47)

we may express the support function of X .ª; t0 ; X 0 / in the following form.

Lemma 1.4.2. The reach set from set-valued position ft0 ; X 0 g is given by the
formula
Z ª
¡.l j X .ª; t0 ; X 0 // D ¡.B 0 .s/§Œs j P .s//ds C ¡.§Œt0  j X 0 /: (1.48)
t0

The reach set X Œt  D X .t; t0 ; X 0 / regarded as a set-valued function of t , has the

following interpretation. Consider the differential inclusion

xP 2 A.t /x C B.t /P .t /;
20 1 Linear Control Systems

which is a differential equation with set-valued right-hand side [7, 75]. From a
starting position ft0 ; X 0 g it has a set-valued solution X Œt  D X .t; t0 ; X 0 / which
is nothing else than the reach set X Œt . This set X Œt  may be represented as the set-
valued integral
Z t
X Œt  D G.t; t0 /X 0 C G.t; s/B.s/P .s/ds:
t0

The set-valued integral above may be interpreted in the sense of either Riemann or
Lebesgue.

1.4.2 Calculating Controls for the Boundary of Reach Set

The support functions in (1.46), (1.48) yield the boundary @X λ of the set X λ. We
may also need to find a control that leads to each point of this boundary. We will
do that by calculating, for each direction l, the points of the boundary that are the
points of support xl for X λ in the direction l. For any l, xl is the solution to the
optimization problem

xl D arg maxfhl; xi j x 2 X λg;

maxfhl; xi j x 2 X Œªg D ¡.l j X Œª/

Z ª
D ¡.B 0 .s/G 0 .ª; s/l j P .s//ds C ¡.G 0 .ª; t0 /l j X 0 /:
t0
(1.49)
Remark 1.4.1. By considering all directions l 2 R ; one gets all the points of
n

support of X Œª. Since, with l fixed, all the vectors ’l; ’ > 0, give the same point
of support xl , it suffices to deal only with vectors l of unit norm, hl; li D 1.
The calculation of xl decomposes into separate calculations of the two terms
in (1.49). The second term, which gives the appropriate initial condition, is obtained
through a convex optimization problem. The first term, which gives the control ul .t /
corresponding to xl from formula (1.46) is considered next.
Lemma 1.4.3. The point of support xl for X λ in direction l satisfies rela-
tion (1.49). The control ul .t / that steers the system to point xl satisfies the maximum
condition

l 0 G.ª; s/B.s/ul .s/ D maxfl 0 G.ª; s/B.s/u j u 2 P .s/g D ¡.l j G.ª; s/B.s/P .s//:
(1.50)
1.4 The Reachability Problem: Time-Optimal Control 21

Fig. 1.1 The hyperplane H.l/ supports X at xl and X  H .l/ (left). Inner and outer
approximation to X (right)

Exercise 1.4.1. Prove formula (1.50) and also the assertions stated in Remark 1.4.1.
Definition (1.49) of xl has the following geometric interpretation. Since

¡l D ¡.l j X Œª/ D hl; xl i  hl; xi; x 2 X D X Œª;

the hyperplane H.l/ D fx j hl; xi D ¡l g with normal l supports X at the point xl .

See Fig. 1.1.
Observe that the reach set X is contained in the closed half-space

H .l/ D fx j hl; xi  ¡l g:

Thus we have the inner and outer approximation to X λ of the following exercise.
Exercise 1.4.2. Let l1 ;    ; lk be nonzero vectors. Let ¡i D ¡.li j X Œª/, and
xi D xli . Show that

cofx1 ;    ; xk g X λ \kiD1 H .li /:

Here cofx1 ;    ; xk g denotes the convex hull of fx1 ;    ; xk g.

The reach sets may be of course calculated under constraints other than (1.18).
Problem 1.4.2. Find reach set X Œª D X .ª; t0 ; x 0 / of Problem 1.4.1 under the
integral constraint
Z ª  q1
ku./kLq D hu.s/; u.s/i q=2
ds  ;  > 0; q 2 .1; 1/; (1.51)
t0

with q integer, instead of constraint (1.18).

22 1 Linear Control Systems

Since

¡.l j X Œª/ D maxfhl; xŒªi j ku./kLq  ; xŒt0  D x 0 g;

u

we have
Z ª  1
 q
¡.l j X Œª/ D  .§0 Œt B.t /B 0 .t /§Œt /q =2 dt C §0 Œt0 x 0 ; (1.52)
t0

where 1=q  C1=q D 1; so that q  D q.q1/1 : Recall that §Œt  D .G 1 .t; ª//0 l D
G 0 .ª; t /l:
To obtain (1.52) we used Hoelder’s inequality with d./ 2 Lq  ; u./ 2 Lq 3
Z ª
hu.s/; d.s/ids  ku./kL .p/ kd./kL .p/ ;
t0
q q


 
1=q
with equality if ku.s/kq  ’kd.s/kq , u.s/ D ’kd.s/kq q d.s/, for some
constant ’ > 0.
Lemma 1.4.4. The reach set X λ under constraint (1.51) is given by
formula (1.52).
In particular, if q D 2, then also q  D 2, we come to the next result.
Corollary 1.4.1. With q D 2 one has (§Œt  is a column vector)
Z ª 1=2
¡.l j X Œª/ D  §0 Œt B.t /B 0 .t /§Œt dt C §0 Œt0 x 0 D hl; W .ª; t0 /li1=2 C l 0 c;
t0
(1.53)
with c D G.ª; t0 /x 0 .
Exercise 1.4.3. For the support function

¡.l j X Œª/ D hl; W .ª; t0 /li1=2 C l 0 c; W .ª; t0 / D W .ª; t0 /0 > 0;

prove that

X Œª D fx W h.x  c/; W 1 .ª; s/.x  c/i  g; (1.54)

which is a nondegenerate ellipsoid.

3
Instead of the traditional letter p paired with q, we use q  , since p is used to denote for the
dimension of u.
1.4 The Reachability Problem: Time-Optimal Control 23

Calculation of the reach set is more difficult for the bilinear system (1.17), which
with v.s/ D 0 becomes

xP D A.t; u/x: (1.55)

The difficulty is due to the fact that even if u is constrained to a convex compact set
with x 0 6D 0 the reach set X λ may be nonconvex. This is seen in the next example.
Exercise 1.4.4. Show that the differential inclusion for x 2 R2 ,

xP 1 2 Œ1; 1x2 ; xP 2 D 0; t 2 Œ0; 1;

with X 0 D fx W x10 D x1 .0/ D 0; x20 D x2 .0/ 2 Œ’; ’; ’ > 0g; has a nonconvex
reach set X Œª D X .ª; 0; X 0 /; ª > 0:
Remark 1.4.2. A more general problem than finding the reach sets X Œt  at time t is
S find reach sets within a given time interval t 2 T . The calculation of such sets,
to
fX Œt  j t 2 T g, requires more complex operations and is treated in Sect. 2.6 of
Chap. 2.
Exercise 1.4.5. Find a two-dimensional S autonomous system x.t P / D Ax.t / and a
convex, compact initial set X 0 so that fX Œt  j t 2 Œ0; 1g is not convex.
The next problem is basic. It initiated research in optimal control.

1.4.3 Time-Optimal Control

Problem 1.4.3. Given starting position ft0 ; x 0 g and final point x F , find control
u.s/ 2 P .s/; s 2 Œt0 ; ª that steers the system from x 0 to x F in minimum time,
i.e., ª  t0 is minimized.
We shall solve this problem as follows. Consider the reach set X Œª D X .ª; t0 ; x 0 /
with ª > t0 . Assuming x F 6D x 0 ; let us look for the first instant of time ª0 when
x F 2 X Œª. Namely, denoting the (Euclidean) distance as –.ª/ D d.x F ; X Œª/, and
assuming –.t0 / > 0, we will find the smallest root ª0 of the equation –.ª/ D 0: Then
ª0  t0 will be the minimum time.
Taking B .0/ D fx W hx; xi  1g and expressing x F 2 X Œª C –B .0/ in terms of
its support function,

hl; x F i  ¡.l j X Œª C ©B .0// D ¡.l j X Œª/ C ©hl; li1=2 ; 8fl W hl; li  1g;

we find that the distance d.x F ; X Œª/ D –.ª/ is the smallest © satisfying this
inequality,

–.ª/ D maxfhl; x F i  ¡.l j X Œª/ j hl; li  1g: (1.56)

24 1 Linear Control Systems

Hence

ª0 D minfª j –.ª/ D 0g:

Recall that the support function ¡.l j X Œª/ was calculated earlier (see
Lemma 1.4.2).
Theorem 1.4.1. The minimal time for transferring system (1.44) from starting point
xŒt0  to final point xŒª0  D x F is ª0 t0 , where ª0 is the minimal root of the equation
–.ª/ D 0:
Let l 0 be the optimizer of problem (1.56) with ª D ª0 .
Theorem 1.4.2. The time-optimal control u0 .t /; which steers the system from x 0 to
point x F satisfies the maximum condition of type (1.50) with l; ul ; t; ª substituted
by l 0 ; u0 .t /; t0 ; ª0 of Problem 1.4.3.
With l 0 6D 0 this maximum condition is known as the maximum principle,
discussed in more detail in the next section.
Remark 1.4.3. The optimal time ª0  t0 may be discontinuous in the boundary
points x 0 ; x F .
Exercise 1.4.6. Construct an example of Problem 1.4.3 for which ª0  t0 is
discontinuous in both x 0 ; x F .
We shall further deal with various versions of the time-optimal control problem
in detail, when solving specific cases.
In the next section we first introduce the maximum principle for linear systems as
the necessary condition of optimality, then indicate when it is a sufficient condition.
We shall mostly emphasize the nondegenerate case which is the main situation in
applications, the one for which the maximum principle was indeed introduced and
for which the optimal control may be found precisely from this principle. But we
shall also indicate degenerate abnormal or singular situations, which may occur and
for which the maximum principle is noninformative.

1.5 Optimal Open-Loop Control for Linear-Convex

Systems: The Maximum Principle

The necessary conditions for optimality of control problems were formulated by

Pontryagin and his associates in the form of the “Maximum Principle,” [226]. These
conditions apply to a broad class nonlinear systems under nonsmooth constraints
on controls. In the linear case they may be extended to necessary and sufficient
conditions of optimality as indicated in the following text. (See also [30, 81, 120,
195].)
1.5 Optimal Open-Loop Control for Linear-Convex Systems: The Maximum. . . 25

Here we deal with control problems for linear-convex systems, namely those of
type (1.2), with convex constraints on the control. In this section the problems are
solved in the class of open-loop controls UO , with no disturbance term, v.t /  0.
Problem 1.5.1. Consider system (1.2), with T D Œt0 ; ª. Given starting position
ft0 ; x 0 g, find

J .t0 ; x 0 / D minf®.ª; xŒª/ j x.t0 / D x 0 g (1.57)

u

under constraints (1.2), (1.18). Here ®.t; x/ is continuous in ft; xg and convex
in x, bounded from below, with bounded level sets.
 
The level sets of ®.t; x/ are bounded iff 0 2 int Dom® .t; / ; 8t 2 T: Recall
that

®.t; x/ D maxfhx; li  ® .t; l/ j l 2 Rn g; (1.58)

l

and ® .t; l/, the (Fenchel) conjugate of ®.t; x/ in the second variable, is defined by

® .t; l/ D maxfhx; li  ®.t; x/ j x 2 Rn g:

x

The attainability of the maximum is ensured by the properties of ®.t; x/.

1.5.1 The Necessary Conditions of Optimality

Substituting ª for t and xŒª for x in (1.58) and using the integral representa-
tion (1.3) for xλ, we come to

J .t0 ; x 0 / D min max ˆ.u./; l j t0 ; x 0 /; (1.59)

u l

in which u.t / 2 P .t /; l 2 Rn and

Z ª
ˆ.u./; l j t0 ; x 0 / D hl; G.ª; t0 /x 0 i C hl; G.ª; s/B.s/u.s/ids  ® .ª; l/:
t0

In (1.59), the minimization is with respect to u.t / 2 P .t /. As before, u./ D

u.t /; t 2 T , denotes the function u.t / regarded as an element of an appropriate
function space, which here is L2 .T / and P ./ D fp./g is the class of all measurable
functions p.t / that satisfy p.t / 2 P .t / a.e. in T .
26 1 Linear Control Systems

The functional ˆ.u./; l j t0 ; x 0 / is concave in l (since ® .ª; l/ is convex in l)

and linear in u./. Since u./ belongs to a weakly compact set P ./ in L2 .T /, one
may apply the minmax theorem of Ky Fan [70] and interchange the order of min
and max in (1.59):

J .t0 ; x 0 / D min max ˆ.u./; l j t0 ; x 0 / D max min ˆ.u./; l j t0 ; x 0 / D max ˆ0 .l j t0 ; x 0 /;

u l l u l
(1.60)
in which
Z ª
ˆ0 .l j t0 ; x / D h§Œt0 ; x i 
0 0
¡.§Œt  j B.t /P .t //dt  ® .ª; l/: (1.61)
t0

Here §Œt  D §.t; ª; l/ is the solution to the adjoint equation

P / D A0 .t /§.t /; §.ª/ D l;

§.t (1.62)

for system (1.2). The second term in the right-hand side of (1.61) comes from the
relations
Z ª 
min hl; G.ª; s/B.s/u.s/ids ju./ 2 P ./
u t0
Z ª
D minfhl; G.ª; s/B.s/ui j u 2 P .s/gds
t0 u
Z ª
D maxfhl; G.ª; t /B.t /ui j u 2 P .s//ds
t0 u
Z ª
D ¡.§Œt  j B.t /P .t //dt: (1.63)
t0

In the above lines the order of operations of min and integration may be
interchanged (see [238, p. 675]).
Let l 0 be a maximizer of ˆ0 .l j t0 ; x 0 / and let u0 ./ be an optimal control, i.e., a
minimizer of ®.ª; xŒª/ D maxl ˆ.u./; l j t0 ; x 0 /:
Recall the definition of saddle point for ˆ.u./; l j t0 ; x 0 / D ˆ.u; l/:
Definition 1.5.1. A pair fus ; l s g is said to be a saddle point for ˆ.u; l/ if

ˆ.us ; l/  ˆ.us ; l s /  ˆ.u; l s /; 8u; l:

Lemma 1.5.1. The pair .u0 ./; l 0 / is a saddle point of ˆ.u./; l j t0 ; x 0 / D ˆ.u; l/
and ˆ.u0 ./; l 0 j t0 ; x 0 / D ˆ.u0 ; l 0 / D J .t0 ; x 0 /:
1.5 Optimal Open-Loop Control for Linear-Convex Systems: The Maximum. . . 27

Proof. From (1.60) we have

ˆ.u0 ; l/  max ˆ.u0 ; l/ D min max ˆ.u; l/ D J .t0 ; x 0 / D

l u l

D max min ˆ.u; l/ D min ˆ.u; l 0 /  ˆ.u; l 0 /;

l u u

substituting u D u0 and l D l 0 we obtain J .t0 ; x 0 / D ˆ.u0 ; l 0 /, from which the

lemma follows. u
t
Thus

ˆ.u0 ; l j t0 ; x 0 /  ˆ.u0 ; l 0 j t0 ; x 0 / D J .t0 ; x 0 /  ˆ.u; l 0 j t0 ; x 0 /; 8u; l:

(1.64)
Concentrating on the necessary conditions of optimality for the control we first
introduce the next assumption.
Let 0 D minf®.ª; x/ j x 2 Rn g:
Assumption 1.5.1. The “regularity condition” 0 < J .t0 ; x 0 / holds.
Lemma 1.5.2. Under Assumption 1.5.1 the maximizer l 0 D l 0 .t0 ; x 0 / 6D 0.
Proof. Indeed, suppose l 0 D 0. From (1.60), J .t0 ; x 0 / D ® .ª; 0/, so that

J .t0 ; x 0 / D ® .ª; 0/ D  maxf®.ª; x/ j x 2 Rn g D minf®.ª; x/ j x 2 Rn g D 0 :

contrary to the assumption. t

u
This assumption means that x abs D arg minf®.ª; x/ j x 2 Rn g, the absolute
minimizer of function ®.ª; x/, does not lie in X Œª D X .ª; t0 ; x 0 /—the reachability
set of system (1.2).
The second inequality in (1.64) is

ˆ.u0 ./; l 0 j t0 ; x 0 /  ˆ.u./; l 0 j t0 ; x 0 /; 8u.t / 2 P .t /: (1.65)

Let §0 Œt  D §.t; ª; l 0 /. Assumption 1.5.1, which we now assume, yields §0 Œt  6 0

so that the solution of the adjoint equation is nontrivial.
Then, rewriting the inequality of (1.65) in detail, using (1.60)–(1.62) and
omitting similar terms in the left and right-hand sides of this inequality, we arrive at
the relation
Z ª Z ª
¡.§ Œt  j B.t /P .t //dt 
0
§0 Œt 0 B.t /u.t /dt; 8u.t / 2 P .t /:
t0 t0
28 1 Linear Control Systems

Here equality is attained at u0 ./ if and only if for almost all t 2 T the following
pointwise maximum condition holds:

 §0 Œt 0 B.t /u0 .t / D maxf§0 Œt 0 B.t /u j u 2 P .t /g D ¡.§0 Œt  j B.t /P .t //:

(1.66)
The last relation may also be rewritten in the form of a minimum condition:

§0 Œt 0 B.t /u0 .t / D minf§0 Œt 0 B.t /u j u 2 P .t /g: (1.67)

Definition 1.5.2. An open-loop control u.t / 2 P .t /; t 2 T; is said to satisfy the

maximum principle if there exists a nontrivial solution .§0 Œt  6D 0/ of the adjoint
equation (1.62), such that the following conditions hold:
(i) u.t / satisfies the maximum condition (1.66), where
(ii) vector §0 Œª D l 0 is a maximizer of ˆ0 .l/ (1.61).
We have thus proved the next result.
Theorem 1.5.1. Under Assumption 1.5.1 every minimizing control u0 .t / of
Problem 1.5.1 satisfies the maximum principle.
Let xu0 D x 0 Œª denote the end point at time ª of the trajectory x 0 Œt  D
x .t; t0 ; x 0 /, emanating from position ft0 ; x 0 g under an optimizing control u0 .t / of
0

the previous theorem. Then, under Assumption 1.5.1, we have

maxfhl; xu0 i  ® .ª; l/g D ®.ª; xu0 / D hl 0 ; xu0 i  ® .ª; l 0 /;

l

which leads to the next property.

Lemma 1.5.3. An optimal control of Theorem 1.5.1 satisfies the equality

hl 0 ; xu0 i D ®.ª; xu0 / C ® .ª; l 0 /: (1.68)

This is the transversality condition which allows nonsmooth functions ®.

Corollary 1.5.1. Relation (1.68) is equivalent to condition (ii) in Definition 1.5.2
of the maximum principle.
The proof follows from the definition of the conjugate function.
Lemma 1.5.4. The pair fl 0 ; xu0 g of Lemma 1.5.3 satisfies the following relations:

l 0 2 @®.ª; xu0 /; and xu0 2 @® .ª; l 0 /; (1.69)

where @®.ª; xu0 /; @® .ª; l 0 / are the subdifferentials in the second variable of
®.ª; xu0 / and ® .ª; l 0 / at xu0 and l 0 :
1.5 Optimal Open-Loop Control for Linear-Convex Systems: The Maximum. . . 29

If ®.ª; x/ is differentiable in x, we have

@®.ª; xu0 /
l 0 D ®x .ª; xu0 / D :
@x
Exercise 1.5.1. Prove Lemma 1.5.4. (See Lemma 1.5.7 below.)
Relations (1.69) are simpler under some additional conditions.
Lemma 1.5.5 ([238], Sect. 11.C). Suppose function ®.x/ is finite, coercive (this
means lim inff®.x/=kxkg ! 1 as kxk ! 1/, and also convex, of class C 2
(twice continuously differentiable) on Rn and its Hessian matrix r 2 ®.x/ is positive
definite for all x. Then its conjugate ® .l/ satisfies the same properties, namely, it
is also a finite, coercive, convex function of class C 2 on Rn with its Hessian matrix
r 2 ® .l/ positive definite for every l.
Corollary 1.5.2. Suppose function ®.ª; x/ satisfies conditions of Lemma 1.5.5 in
the second variable. Then relations (1.69) of Lemma 1.5.4 have the form

@®.ª; xu0 / @® .ª; l 0 /

l 0 D ®x .ª; xu0 / D ; xu0 D ®l .ª; l 0 / D :
@x @l
The last corollary indicates when the Fenchel transformation used in this chapter
to calculate the conjugate ® coincides with the classical Legendre transformation.

1.5.2 Degenerate Cases

The first degenerate case. As we observe, the maximum principle requires

§0 Œª 6D 0: Suppose, however, that §0 Œª D l 0 D 0: Then the maximum condition,
which looks like

0  B.t /u0 D maxf0  B.t /uju 2 P .t /g;

is degenerate. It is trivially fulfilled, but it is not the maximum principle. The optimal
control then has to be found from considerations other than the maximum principle.
As we shall now observe, this case occurs when Assumption 1.5.1 is not fulfilled.
Example 5.1. For the one-dimensional system

P / D u.t /; x.0/ > 0; ju.t /j  ; t 2 Œ0; ª;

x.t

consider the problem of minimizing ®.ª; xŒª/ D 12 x 2 Œª:

30 1 Linear Control Systems

Fig. 1.2 Example 5.1

Here
1
max min ˆ.u; l/ D maxfl 0 x.0/  ªjlj  hl; li2 g D max ˆ0 .l/:
l u l 2 l

Assume ª  x.0/ > 0. Then ˆ0 .l/ < 0; 8l 6D 0; and maxl ˆ0 .l/ D ˆ0 .l 0 / D 0:

Hence

l 0 D 0; and max min ˆ.u; l/ D ˆ 0 D J .0; x.0// D 0:

l u

The pair l 0 D 0 with any u D u0 satisfies the maximum condition but not the
maximum principle. Among these there exists an optimal control u0 which must
satisfy condition
Z ª
u.t /dt C x.0/ D 0
0

This is a singular control which has to be found from conditions other than the
maximum principle (Fig. 1.2).
We shall now see that there is a second degenerate case.
The second degenerate case
This is when §0 Œt  6D 0, but B 0 .t /§0 Œt   0:
Example 5.2. Consider system

xP1 D u; xP2 D u; juj  ; t 2 Œ0; ª;

so that x 2 R2 ; u 2 R D R1 : Here A D 0; B 0 .t /  b 0 D .1; 1/ and the system is

not controllable.
1.5 Optimal Open-Loop Control for Linear-Convex Systems: The Maximum. . . 31

Take ®.ª; xŒª/ D 12 hxŒª; xŒªi and x1 .0/ > 0; x2 .0/ < 0: We have
Z ª 1
ˆ 0 D max min ˆ.u; l/ D max minf.l1 x1 .0/ C l2 x2 .0// C .l1 C l2 /u.t /dt  hl; lig D
l u l u 0 2

1
D maxf.l1 x1 .0/ C l2 x2 .0//  ªjl1 C l2 j  hl; lig:
l 2

Calculating the maximum in l one may observe that

with l1 C l2 > 0 we have ˆ 0 D ˆ0 .l 0 j 0; x.0// D 12 ..x1 .0/  ª/2 C .jx2 .0/j C
ª/2 /;
with l1 C l2 < 0 we have ˆ 0 D 12 ..x1 .0/ C ª/2 C .jx2 .0/j C ª/2 /;
with l1 C l2 D 0 we have ˆ 0 D 12 .x12 .0/ C x22 .0//:
With x1 .0/ C x2 .0/ D 0; the maximum is attained at l10 D x1 .0/; l20 D x2 .0/
and ˆ 0 D 12 .x12 .0/ C x22 .0//:
The optimizing pair is fl 0 ; u0 g and the maximum condition

l 0 b  u0 D maxfl 0 b  u j juj  g D maxf0  ug D 0  u0

u u

is degenerate. However, §0 Œt  D l 0 6D 0 and condition (ii) of Definition 1.5.2,

as well as (1.69) are satisfied. Here the maximum principle is satisfied, with any
ju.t /j  ; which demonstrates that it is a necessary but not a sufficient condition
for optimality. In this example the maximum principle is noninformative. The
optimal control u0  0 is a singular control which means it has to be found from
conditions other than the maximum principle (see Remark 1.5.3 below).
A similar situation arises with x1 .0/ D 0; x2 .0/ < 0: Then with ª  jx2 .0/j=2

1
ˆ0 D hx.ª/; x.ª/i; x.ª/ D x.0/ C bc.u.//; c.u.//
2
Z ª
D u.t /dt; x1 .ª/ D x2 .ª/ D x D x2 .0/=2 > 0:
0

The optimal control u0 must satisfy c.u0 .// D x ; and with ª  jx2 .0/j=2 it is
singular, while u0 .t / 6D 0: There are many such controls (see Fig. 1.3).
We have thus indicated two types of degenerate problems. A protection from the
first type was ensured by Assumption 1.5.1. A protection from the second type is
ensured by the following assumption.
Let b .i/ .t / be the i -th column of B.t / and ui the i -th coordinate of u, i D
1; : : : ; p.
32 1 Linear Control Systems

Fig. 1.3 Example 5.2

Assumption 1.5.2. The system (1.44) is strongly completely controllable. This

means that the condition of complete controllability of Corollary 1.3.1 is applicable
to each column b .i/ .t / of B.t /.
This assumption implies that functions §.s; ª; l/0 b .i/ .s/ 6D 0; 8l 6D 0; 8i D
1; : : : ; n; for almost all t 2 T:

1.5.3 Sufficiency Conditions for Optimality

Some preliminary results

Lemma 1.5.6. Suppose 0 2 intP .t /  Rp . Then, under Assumption 1.5.2, the
reachability set X Œª D X .ª; t0 ; x 0 /  Rn has an interior point x 0 :
Proof. Since P .t / is continuous, there exists a number © > 0; such that the ball

B© .0/ D fx W hx; xi  ©2 g P .t /; 8t 2 Œt0 ; ª:

Then ¡.l j P .t //  ©klk; 8t and

Z ª Z ª
¡.l j X .ª; t0 ; 0// D ¡.l jG.ª; s/B.s/P .s//ds  © kl 0 G.ª; s/B.s/kds:
t0 t0

Under the strong controllability condition (see Lemma 1.3.1), we have

Z t2 Z t2
0
kl G.ª; s/B.s/kds D .l 0 D.s/D 0 .s/l/1=2 ds > 0; 8klk 6D 0;
t1 t1
1.5 Optimal Open-Loop Control for Linear-Convex Systems: The Maximum. . . 33

whatever be Œt1 ; t2  Œt0 ; ª; t1 < t2 ; and since

Z t2 
min .l 0 D.s/D 0 .s/l/1=2 ds j klk D 1 > 0;
l t1

this proves the lemma. t

u
Exercise 1.5.2. Prove that Lemma 1.5.6 is true when intP .t / 6D ;:
For any l 6D 0 we also have
Z ª Z ª
¡.l jG.ª; s/B.s/P .s//ds D maxfl 0 G.ª; s/B.s/u.s/ j u.s/ 2 P .s/gds D
t0 t0

Z ª
D l 0 G.ª; s/B.s/u0l .s/ds > 0; (1.70)
t0

where u0l .s/ 6D 0; a.e. Thus we came to the next proposition.

Corollary 1.5.3. Under the assumptions of Lemma 1.5.6 the maximum condi-
tion (1.70), taken for any l 6D 0; is achieved only by function u0l .s/ 6D 0; a.e.
Remark 1.5.1. Note that in Example 5.2, in the absence of controllability, we have
x 2 R2 ; but the reach set X λ is one-dimensional and it has no interior points, but
of course has a nonempty relative interior (see Fig. 1.3).

The Sufficiency Conditions

We shall now indicate, under Assumption 1.5.1, some conditions when the
maximum principle turns out to be sufficient for optimality of the solution to
Problem 1.5.1.
Theorem 1.5.2. Suppose Assumption 1.5.1 is satisfied and the following conditions
for a function u .t / are true:
(a) u .t / satisfies the maximum principle of Definition 1.5.1 for l 0 6D 0—the
maximizer of ˆ0 .l/I
(b) u .t / satisfies the condition

max ˆ.u ; l j t0 ; x 0 / D max ˆ.u ; l/ D ˆ.u ; l 0 / D ˆ.u ./; l 0 j t0 ; x 0 /:

l l

Then u .t / is an optimal control of Problem 1.5.1 and J .t0 ; x 0 / D ˆ0 .l 0 / is the

optimal cost.
34 1 Linear Control Systems

Proof. Denote x 0 Œª; x  Œª to be the vectors generated from xŒt0  D x 0 under an
optimal control u0 .t / and under control u .t /, respectively, with xλ being the one
generated from the same point under any admissible control u.t /:
Then, on the one hand, since u .t / satisfies condition (a) (the maximum principle
for l 0 ), we have
Z ª

ˆ.u ./; l / D h§ Œt0 ; x i C
0 0 0
§0 Œt 0 B.t /u .t /dt  ® .ª; l 0 / D
t0

Z ª
D h§ Œt0 ; x i C
0 0
minf§0 Œt 0 B.t /u j u 2 P .t /gdt  ® .ª; l 0 / D ˆ.u0 ./; l 0 / D
t0

D hl 0 ; x 0 Œªi  ® .ª; l 0 / D ®.ª; x 0 Œª/;

and on the other, due to condition (b),

Z ª
ˆ.u ./; l 0 / D max ˆ.u ; l/ D maxfl 0 G.ª; t0 /x 0 C l 0 G.ª; t /B.t /u .t /dt  ® .ª; l 0 /g
l l t0

D maxfhl; x  Œªi  ® .ª; l/ j l 2 Rn g D ®.ª; x  Œª/;

l

so that u .t / is the optimal control. t

u
Remark 1.5.2. Condition (b) of Theorem 1.5.2 may be substituted by the following
equivalent condition.
(b’) u .t / generates a trajectory x  Œt  such that l 0 2 @®.ª; x  Œª/:
This statement is proved below, in Lemma 1.5.9.
However, the last theorem does not ensure that the maximum principle is
nondegenerate and may serve to determine the control.
Remark 1.5.3. Indeed, to single out the optimal control u0  0 of Example 5.2 from
all the others (recall that there the redundant maximum condition is satisfied by all
the possible controls), one has to apply something else than the maximum principle.
This could be condition (b) of the sufficiency Theorem 1.5.2, which would serve as
a verification test.
Thus, taking control u .t /  k 6D 0; x1 .0/ D x2 .0/; and calculating

1
maxfhl; x.0/i C k.l1 C l2 /ª  hl; lig D ®.x  .ª//;
l 2

we find

®.x  .ª// D .x1 .0/ C k/2 C .x2 .0/ C k/2 > ®.x 0 .ª// D x12 .0/ C x22 .0/;

which indicates that the controls u .t /  k with k 6D 0 are not optimal, though they
satisfy the maximum condition.
1.5 Optimal Open-Loop Control for Linear-Convex Systems: The Maximum. . . 35

Theorem 1.5.3. Suppose control u .t / satisfies conditions of Theorem 1.5.2 and

the assumptions of Lemma 1.5.6 are satisfied. Then u .t / 6D 0 a.e. and it may be
found from the maximum principle which is nondegenerate.
Here is an example where the optimal control in not unique.
Example 5.3. For the two-dimensional system

P / D u.t /; x.0/ D .1; 2/0 ; jui .t /j  ; i D 1; 2; t 2 Œ0; ª;

x.t

find the optimal control that minimizes ®.xŒª/ D maxi jxi .ª/j; i D 1; 2:
Recall that here ® .l/ D I .l j B  .0//; where B  .0/ D fl W jl1 j C jl2 j  1g and
I .l j B / is the indicator function for set B (as defined in convex analysis), so that
I .l j B / D 1 if l 2 B and I .l j B / D C1 otherwise.
We have

ˆ 0 D ˆ.u0 ; l 0 / D
 Z ª 
D max hl; x.0/i C minfhl; u.s/i j ju.s/j  gds  I .l j B .0// D max ˆ0 .l/ D
0 l

D maxfhl; x.0/i  .jl1 j C jl2 j/ª j jl1 j C jl2 j  1g D

l

D maxfhl; x.0/i j jl1 j C jl2 j  1g  ª D max jxi .0/j  ª:

i

Taking further ª D 1;  D 1; we have l 0 D .0; 1/ 6D 0: (Note that

here Assumptions 1.5.1, 1.5.2 are both satisfied, so that the maximum principle
is nondegenerate.)
The minimum condition gives

minfhl 0 ; u.t /i j ju.t /j  1g D .jl10 j C jl20 j/; u1 .t / 2 Œ1; 1; u2 .t /  1;

where l 0 is the maximizer of ˆ0 .l/: The maximum principle is thus satisfied with
u2 .t /  1 and any control u1 .t /:
Let us now check condition (b) of Theorem 1.5.2, calculating
Z ª
maxfhl; x.0/i C hl; u .s/ids j jl1 j C jl2 j  1g D max ˆ.u ; l/:
l 0 l

This gives, for u2 .t /  1; and any u1 2 Œ1; 0; maxl ˆ.u ; l/ D
maxfjx1 .1/j; jx2 .1/jg D 1; which means all such controls are optimal.
At the same time, for u2 .t /  1; and any u1 2 Œk; 1; k 2 .0; 1 we have
maxl ˆ.u ; l/ D maxfjx1 .1/j; jx2 .1/jg  1 C k > ˆ.u ; l 0 / so such controls are
not optimal (Fig. 1.4).
36 1 Linear Control Systems

Fig. 1.4 Example 5.3

The next example illustrates the case when the optimal control is unique.
Exercise 1.5.3. For the two-dimensional system

P / D u.t /; x.0/ D .1; 0/0 ; jui .t /j  1; i D 1; 2; t  0;

x.t

consider the problem of minimizing ®.ª; xŒª/ D 12 kxŒªk2 : For ª < 1, show that
the unique maximizer of ˆ0 .l j t0 ; x 0 / is l0 D .1; 0/0 . Hence u.t /; t 2 Œ0; ª,
satisfies the maximum principle iff u1 .t /  1. The optimal control in addition
satisfies u02 .t /  0. Prove that the optimal control u01 .t /  1; u02 .t /  0 is unique.
It is interesting to observe that the notion of saddle point allows us to give the
following interpretation of sufficiency.
Theorem 1.5.4. Suppose fus ./; l s g is a saddle point of ˆ.u./; l j t0 ; x 0 /. Then
us ./ is an optimal control of Problem 1.5.1 which satisfies the maximum condition
of Definition 1.5.2 with §s Œª D l s :
Note that Theorem 1.5.4 covers both degenerate cases. If Assumption 1.5.1 is
satisfied, we moreover have l s 6D 0:
Exercise 1.5.4. Prove Theorem 1.5.4.

1.5.4 A Geometrical Interpretation

A geometrical interpretation of the maximum principle and saddle point condi-

tion (1.64) is given in Lemma 1.5.7 and Exercise 1.5.4, and illustrated in Fig. 1.5.
1.5 Optimal Open-Loop Control for Linear-Convex Systems: The Maximum. . . 37

Fig. 1.5 xu0 is the point of

support for X λ in direction
l 0 and the set
f®.x/  ®.xu0 /g in direction
l0

Lemma 1.5.7. Let xu0 D x.ª; t 0 ; x 0 / be the state at time ª corresponding to a

control u0 . Then l 0 satisfies the first inequality in (1.64),

ˆ.u0 ; l j t0 ; x 0 /  ˆ.u0 ; l 0 j t0 ; x 0 /; 8l;

if and only if

®.ª; x/  ®.ª; xu0 / C hl 0 ; x  xu0 i; 8x 2 Rn : (1.71)

If l 0 ¤ 0, then xu0 is the point of support for the set fx W ®.ª; x/  ®.ª; xu0 /g in
direction l 0 , i.e. l 0 2 @®.ª; xu0 /:
Proof. Evidently

hl 0 ; xu0 i  ® .ª; l 0 / D ˆ.u0 ; l 0 / D max ˆ.u0 ; l/ D maxfhl; xu0 i  ® .ª; l/g D ®.ª; xu0 /;
l l

is equivalent to

® .ª; l 0 / D hl 0 ; xu0 i  ®.ª; xu0 /;

which, since ® .ª; l 0 / D maxx fhl 0 ; xi  ®.ª; x/g, is equivalent to (1.71).

Further on, if ®.ª; x/  ®.ª; xu0 /, by (1.71) one must have hl 0 ; x  xu0 i  0,
which proves the last assertion. u
t
Lemma 1.5.8. Suppose l 0 ¤ 0. Then u0 satisfies the second inequality in (1.64),

ˆ.u0 ; l 0 j t0 ; x 0 /  ˆ.u; l 0 j t0 ; x 0 /; 8u;

if and only if xu0 is the point of support for the reach set X λ in direction l 0 , i.e.,

hl 0 ; xu0 i  hl 0 ; xi; 8x 2 X λ:

38 1 Linear Control Systems

Exercise 1.5.5. Prove Lemma 1.5.8.

The next lemma can be used to verify whether a proposed control is indeed optimal.
Lemma 1.5.9. Let x 0 .ª/ be the state at time ª corresponding to control u0 and
l 0 2 @®.ª; xu0 /. Then u0 is optimal if it satisfies the maximum principle of
Definition 1.5.2.
Proof. By (1.71), for any x.ª/ 2 Rn ; we have

hl 0 ; x.ª/  x 0 .ª/i  0 ) ®.ª; x.ª//  ®.ª; x 0 .ª//I

and by Lemma 1.5.8 (the maximum principle in the form of a minimum rule),

x.ª/ 2 X Œª ) hl 0 ; x.ª/  x 0 .ª/i  0I

whence

x.ª/ 2 X Œª ) ®.ª; x.ª//  ®.ª; x 0 .ª//;

so u0 is optimal. t
u
Remark 1.5.4. Be careful, since, as indicated in Examples 5.1 and 5.2, the control
u0 of Lemma 1.5.9 may turn out to be singular and the maximum principle for u0
may be degenerate.

1.5.5 Strictly Convex Constraints on Control

Recall that a compact set P is said to be strictly convex, if its boundary @P does not
contain any line segment. That is, if p 0 ; p 00 2 @P , then œp 0 C.1œ/p 00 62 @P ; 8œ 2
.0; 1/:
Strictly convex sets may be presented as level sets of strictly convex functions.
A function ®.x/ is strictly convex in the convex domain D if for points p 0 ; p 00 2 D
one has

®.œp 0 C .1  œ/p 00 / < œ®.p 0 / C .1  œ/®.p 00 /; 8œ 2 .0; 1/:

Note that the inequality here is strict.4

Lemma 1.5.10. Suppose the function ®.ª; x/ is strictly convex in x. Then the
maximizer l 0 of ˆ0 .l j t0 ; x 0 / is unique.

4
Recall that here, as mentioned in Sect. 1.2, set P .t / is assumed to have an interior point for all t:
1.5 Optimal Open-Loop Control for Linear-Convex Systems: The Maximum. . . 39

Exercise 1.5.6. Prove Lemma 1.5.2.

An example of a strictly convex function is the strictly positive quadratic function
®.x/ D hp  a; P .p  a/i with P D P 0 > 0.
Denote a degenerate ellipsoid as

E .p  ; P / D fp W hp  p  ; P 1 .p  p  /i  1g; P D P 0 > 0:

Here p  is the center and P is the shape matrix.

Another example is as follows. Take the function ®.x/ D d 2 .x; E .0; P //; P D
0
P > 0; and a ray fx D ’h; h 62 E .0; P / ’ > 0g; with scalar ’. Then function
f .’/ D ®.’h/ defined for ’ > 1 will be strictly convex. (Prove this.)
The constraint on u may also be strictly convex.
Lemma 1.5.11. Suppose d 6D 0 and P is strictly convex. Then the maximizer u0 of
the problem

maxfhd; ui j u 2 P g D hd; u0 i

is unique.
Lemma 1.5.12. Suppose the continuous function d.t / 6D 0 almost everywhere in
T and function P .t / is Hausdorff-continuous with values in strictly convex compact
sets. Then the optimizer u0 .t / to

hd.t /; u0 .t /i D maxfhd.t /; ui j u 2 P .t /g

is unique and integrable.

Exercise 1.5.7. Prove Lemmas 1.5.11, 1.5.12.
An example of a strictly convex set P is the nondegenerate ellipsoid, P D
E .p  ; P /:
Another example is the intersection of m nondegenerate ellipsoids: \m
iD1 E .0; Pi /;
Pi > 0; Pi D Pi0 : (Prove this.)
Lemmas 1.5.10–1.5.12 imply the following assertion.
Theorem 1.5.5. Suppose Assumptions 1.5.1 and 1.5.2 hold, the sets P .t /; t 2 T;
are strictly convex and the function ®.ª; x/ is strictly convex in x. Then the optimal
control u0 .t / of Problem 1.5.1 is unique.
Before completing this section we emphasize that finding the optimal control
under hard bounds requires one to solve optimization problems contained in the
maximum principle of type (1.50) or (1.66). These problems involve calculating a
support function in the following form.
Problem 1.5.2. Given vector d and convex compact set P in Rp , find

¡.d j P / D maxfhd; ui j u 2 P g D hd; u0 i: (1.72)

40 1 Linear Control Systems

This optimization problem is easily solved if P is defined by a standard norm in the

space lq W P D fu W kuk  g: For example,
Pk
• if q D 1, kuk D iD1 jui j and u
0
D .0; : : : ;  sgndj ; : : : ; 0/0 ; j D
arg maxfjdi j; i D 1; : : : ; kg;
 1=2
Pk
• if q D 2, kuk D iD1 jui j
2
and u0 D  d.hd; d i/1=2 ;
• if q D 1, kuk D maxfjui j j i D 1; : : : ; kg and u0i D  sgndi ; i D 1; : : : ; k:
Exercise 1.5.8. Solve Problem 1.5.2 for
• (i) P D fu W hu  a; P .u  a/i1=2  ; P D P 0 > 0g:
 1=q
Pk
• (ii) P D fu W .u
iD1 i  ai / q
 g, with q 2 .2; 1/:

for given vector a.

Summarizing this section we observe that the solution to the primal Problem 1.5.1
of optimal control was reduced to the dual optimization problem of maximizing the
dual cost functional ˆ0 .l j t0 ; x 0 /. The dual problem leads us to a crucial element
of the solution—the vector §.ª/ D l 0 that uniquely defines the trajectory §Œt  D
§.t; ª; l 0 / of the adjoint equation (1.61) which, in its turn, determines the optimal
control itself through the maximum principle.
In general, the dual problem for optimization under constraints is simpler than
the primal problem. This justifies its usefulness. In most problems in the forthcom-
ing sections the primal problem is to minimize a cost over a class of functions (the
controls), while the dual problem is one of finite-dimensional optimization with
either no constraints at all, or with simpler ones.
In this section the properties of system linearity and convexity of the constraints
and of the “cost function” (the performance index) allowed us to indicate assump-
tions which allow the maximum principle to be not only a necessary condition of
optimality (as in the general case), but also a sufficient condition.
A natural move is to apply the techniques of convex analysis and its duality
methods used above to other types of optimal control problems in “linear-convex”
systems. This is done in the next section.

1.6 Duality Methods of Convex Analysis in Problems

of Optimal Control

In the investigation of optimization problems the notion of convexity turned out to

be extremely important. This resulted in the development of convex analysis, with
pioneering work by Fenchel, [73,74] Moreau, [217], and Rockafellar, [237,238], see
also [48, 99, 227]. One of the key elements here is in introducing dual optimization
problems solving which sometimes simplifies the optimization procedure.
1.6 Duality Methods of Convex Analysis in Problems of Optimal Control 41

1.6.1 The Primal and Dual Optimization Problems

In this section we elaborate further on the reduction of linear-convex problems of

optimal control to respective dual problems of optimization.
Problem 1.6.1. Consider system (1.2) with T D Œt; ª. Given starting position
ft; xg, find
Z ª 
J .t; x/ D min .®1 .s; H.s/xŒs/ C ®2 .s; u.s///ds C ®.ª; xŒª/ ; (1.73)
u t

along the trajectories xŒs D x.s; t; x/ of system (1.2).

The m  n matrix H.s/ is continuous; ®1 .t; z/; ®.ª; x/; ®2 .t; u/ are continuous
in t and convex in z 2 Rm , x 2 Rn , and u 2 Rp , respectively, and assumed to have
bounded level sets.
Note that there are no bounds on u; x. In convex analysis ®1 .t; z/; ®.t; x/; ®2 .t; u/
are called normal integrands, [238].
However, boundedness of the level sets is equivalent to Assumption 1.6.1 which
we further presume to hold without additional reference.
Assumption 1.6.1. The following inclusions are true

0 2 intDom®1 .t; /; 0 2 intDom® .ª; /; 0 2 intDom®2 .t; /:

Here and in the sequel all conjugates ® ; ®i and all sets Dom./ are taken
with respect to the second variable unless otherwise indicated. Next, we reduce
Problem 1.6.1 to a dual problem of optimization. We first rewrite functions
®1 .t; z/; ®.t; x/; ®2 .t; u/ in terms of their conjugates:

®1 .s; H.s/xŒs/ D maxfhœ.s/; H.s/xŒsi  ®1 .s; œ.s// j œ.s/ 2 Rm g; (1.74)

œ

®.ª; xŒª/ D maxfhl; xŒªi  ® .ª; l/ j l 2 Rn g; (1.75)

l

®2 .s; u/ D maxfhp.s/; ui  ®2 .s; p.s// j p.s/ 2 Rp g: (1.76)

p

Using formula (1.3), substitute for xŒs; xŒª first in (1.74) and (1.75), and
then substitute these results into (1.73). This leads to the following problem of
minimaximization:

J .t; x/ D min max ˆ.u./; l; œ./ j t; x/; (1.77)

u l;œ
42 1 Linear Control Systems

in which the variables range over u.s/ 2 Rp ; œ.s/ 2 Rm ; s 2 Œt; ª, and l 2 Rn ,
and
Z ª
ˆ.u./; l; œ./ j t; x/ D hl; G.ª; t /xi C hl; G.ª; Ÿ/B.Ÿ/u.Ÿ/id Ÿ  ® .ª; l/C
t

Z ª  Z s 
C œ.s/; H.s/G.s; t /xCH.s/ G.s; Ÿ/B.Ÿ/u.Ÿ/d Ÿ ®1 .s; œ.s//C®2 .s; u.s// ds:
t t
(1.78)
We now introduce a new adjoint system for Problem 1.6.1,

P
§.s/ D A0 .s/§.s/  H 0 .s/œ.s/; §Œª D l: (1.79)

Its trajectory §Œt  D §.t; ª; l/ is described by an integral formula analogous

to (1.3),
Z ª
0
§Œt  D G .ª; t /l C G 0 .s; t /H 0 .s/œ.s/ds; (1.80)
t

with G.s; t / defined by (1.4), (1.5).

Interchanging the order of integration in the second integral transforms (1.78)
into

ˆ.u./; l; œ./ j t; x/
Z ª 
D h§Œt ; xi C §ŒŸ0 B.Ÿ/u.Ÿ/ C ®2 .Ÿ; u.Ÿ//  ®1 .Ÿ; œ.Ÿ// d Ÿ  ® .ª; l/:
t

The functional ˆ.u./; l; œ./ j t; x/ is convex in u and concave in fl; œ./g.

Since we assumed 0 2 intDom®u .s; /; we observe that

®2 .s; p/ D maxfhp; ui  ®2 .s; u/g D  minfhp; ui C ®2 .s; u/g;

u u

which yields

min ®2 .s; u/ D ®2 .s; 0/:

u

The last equality indicates that the minimizing set

U 0 .s/ D fu W u 2 arg min ®2 .s; u/g

is bounded for each s 2 T D Œt; ª. Together with the continuity of ®2 .s; u/
in s this yields the Hausdorff-continuity of U 0 .s/, which means that in the
minimaximization (1.77) the functions u.s/ may be selected in a weakly compact
1.6 Duality Methods of Convex Analysis in Problems of Optimal Control 43

subset set of L2 ŒT . The last fact allows us to apply Ky Fan’s minmax theorem
to (1.77), which yields

J .t; x/ D min max ˆ.u./; l; œ./ j t; x/ D max min ˆ.u./; l; œ./ j t; x/

u l;œ l;œ u

D max ˆ0 .l; œ./ j t; x/; (1.81)

l;œ

in which
Z ª  
ˆ0 .l; œ./ j t; x/ D h§Œt ; xi ®2 .s; B 0 .s/§Œs/C®1 .s; œ.s// ds® .ª; l/:
t
(1.82)
Here we have used the relations
Z ª Z ª
0
min .§Œs B.s/u.s/C®2 .s; u.s///ds D min.§Œs0 B.s/u.s/C®2 .s; u.s///ds D
u./ t t u

Z ª
D .®2 .s; B 0 .s/§Œs/ds: (1.83)
t

In summary we have two optimization problems: the Primal and the Dual.
Primal Problem: Find the optimal control u0 .s/; s 2 Œt; ª that minimizes the
primal functional
Z ª
….t; x; u/ D .®1 .s; H.s/xŒs/ C ®2 .s; u.s///ds C ®.ª; xŒª/;
t

along trajectories of the primal system

xP D A.s/x C B.s/u C v.s/; s 2 Œt; ª; (1.84)

with given starting position ft; xg; x.t / D x, so that

J .t; x/ D minf….t; x; u/ j u 2 Rn ; (1.84)g:

u

Dual Problem: Find the optimal pair fl; œ.s/; s 2 Œt; ªg that maximizes the
dual functional
Z ª  
ˆ0 .l; œ./ j t; x/ D h§Œt ; xi ®2 .s; B 0 .s/§Œs/C®1 .s; œ.s// ds® .ª; l/;
t

along the trajectories of the dual (adjoint) system

P
§.s/ D A0 .s/§.s/  H 0 .s/œ.s/;

with boundary value §.ª/ D l, (§ is a column vector).

44 1 Linear Control Systems

The attainability of the maximum in fl; œ./g in (1.81), (1.82) and of the
minimum in u./ in (1.83) is ensured by Assumption 1.6.1. This pointwise min-
imaximization in u.s/; œ.s/; s 2 Œt; ª with a further maximization in l yields
functions u0 .s/; œ0 .s/ and also a vector l 0 . The indicated optimizers need not
be unique. However, due to the properties of systems (1.2), (1.79) and functions
®1 .t; x/; ®.t; x/; ®2 .t; u/, there always exists a pair of realizations u0 .s/; œ0 .s/ that
are integrable and, in fact, may even turn out to be continuous.
Exercise 1.6.1. Prove the existence of integrable realizations u0 .s/; œ0 .s/; s 2
Œt; ª, for the optimizers of (1.81)–(1.83).
Exercise 1.6.2. Prove that whatever be l 0 6D 0; one has §.t / 6 0; t 2 Œt0 ; ª;
whatever be œ.t /:
The relations above indicate the following necessary conditions of optimality of
the control realization u0 .s/; s 2 Œt; ª.
Theorem 1.6.1. Suppose u0 .t / is the optimal control for Problem 1.6.1 and
fl 0 ; œ0 ./g are the optimizers of the dual functional ˆ0 .l; œ./ j t; x/ that generate
the solution §0 Œs D §.s; ª/ under l D l 0 ; œ./ D œ0 ./. Then u0 .t / satisfies the
following pointwise minimum condition

§0 Œs0 B.s/u0 .s/C®2 .s; u0 .s// D minf§0 Œs0 B.s/uC®2 .s; u/ j u 2 Rp g; s 2 Œt; ª;
u
(1.85)
which is equivalent to the pointwise maximum condition

 §0 Œs0 B.s/u0 .s/  ®2 .s; u0 .s// D maxf§0 Œs0 B.s/u  ®2 .s; u/ j u 2 Rp g

u

D ®2 .s; B 0 .s/§0 Œs/; s 2 Œt; ª; (1.86)

producing with l 0 6D 0 the maximum principle.

In order to ensure conditions l 0 ; œ0 6D 0 we will need some additional
assumptions. Let
Z ª  
 .t; x/ D ˆ0 .0; 0 j t; x/ D 
0
®2 .s; 0/ C ®1 .s; 0/ ds  ® .ª; 0/ D J0 .t; x/:
t

Assumption 1.6.2. The inequality 0 .t; x/ D J0 .t; x/ < J .t; x/ is true.

Lemma 1.6.1. Under Assumption 1.6.2 the maximizer fl 0 ; œ0 ./g 6D 0.
Denote

1 .t; x/ D maxfˆ0 .l; 0 j t; x/ j l 2 Rn g;

2 .t; x/ D maxfˆ0 .0; œ./ j t; x/ j œ./ 2 L2 .T /g:
1.6 Duality Methods of Convex Analysis in Problems of Optimal Control 45

Assumption 1.6.3. The following inequalities hold:

.i / 1 .t; x/ < J .t; x/; .i i / 2 .t; x/ < J .t; x/:

Lemma 1.6.2. (a) Suppose Assumption 1.6.3(i) holds. Then l 0 6D 0:

(b) Suppose Assumption 1.6.3(ii) holds. Then œ0 ./ 6D 0:
Exercise 1.6.3. Prove Lemma 1.6.2.
Assumption 1.6.4. (i) Functions ®1 .t; z/; ®.t; x/ are strictly convex in z; x and
continuous in t .
(ii) Function ®2 .t; u/ is strictly convex in u and continuous in t .
Lemma 1.6.3. Suppose Assumption 1.6.4(i) is true. Then the optimizers fl 0 ; œ0 ./g
is unique.
Exercise 1.6.4. Prove Lemma 1.6.3.
Lemma 1.6.4. Under the boundedness of level sets for ®1 .t; H.t /x/; ®.t; x/;
®2 .t; u/ and with a given nonzero optimizer fl 0 ; œ0 ./g one has the inclusion

 B 0 .s/§0 Œs 2 int Dom®2 .s; /; s 2 T: (1.87)

Proof. Indeed, if inclusion (1.87) does not hold, it would contradict the assumption
that fl 0 ; œ0 ./g 6D 0: This could be checked by direct substitution. t
u
Note that inclusion (1.87) is also the necessary and sufficient condition for the
level sets of the function

§0 Œs0 B.s/u C ®.s; u/

to be bounded and hence for the attainability of the minimum in u in (1.85) (which
is also the maximum in (1.86)) for s 2 T .
Lemma 1.6.5. Under the assumptions of Lemma 1.6.4 the minimum in (1.85) is
attained and the minimizer u0 .s/; s 2 T; may be selected as a piecewise continuous
realization.
This brings us to the sufficient conditions for optimality of the control realization
u0 .s/; s 2 Œt; ª.
Theorem 1.6.2. Suppose Assumptions 1.6.2, 1.6.3(i) are true, the requirements of
Lemma 1.6.4 hold and
(i) a control u .s/; s 2 T , satisfies the maximum principle (1.86), with fl 0 6D 0;
œ0 .t / 6D 0g as the maximizer for the Dual Problem,
46 1 Linear Control Systems

(ii) the equality

max ˆ.u ./; l; œ./ j t; x/ D ˆ.u ./; l 0 ; œ0 ./ j t; x/

l;œ

is true.
Then u .s/; s 2 T , is the optimal control for the Primal Problem.
Proof. The proof is similar to that of the previous section (see Theorem 1.5.2). t
u
An additional requirement for functions ®1 .t; z/; ®.t; x/; ®2 .t; u/ to be strictly
convex in z; x and u results in the uniqueness of the optimal control.
Theorem 1.6.3. Suppose under the conditions of Theorem 1.6.2 Assumption 1.6.4
is also true. Then solutions u0 .s/; and fl 0 ; œ0 .s/g to both Primal and Dual Problems
are unique.
Exercise 1.6.5. Suppose that functions ®1 .t; z/; ®.t; x/; ®2 .t; u/ are quadratic
forms in z and u:

1 1 1
®1 .t; z/ D hz; M.t /zi; ®.t; x/ D hx  m; L.x  m/i; ®2 .t; u/ D hu; N.t /ui;
2 2 2

with z D H x; M.t / D M 0 .t / > 0; N.t / D N 0 .t / > 0; L D L0 > 0, so that

Z ª
1
J .t; x/ D min .hH.s/xŒs; M.s/H.s/xŒsi C hu.s/; N.s/u.s/i/dsC
u t 2

1
C hxŒª  m; L.xŒª/  mi j u./ 2 L2 ŒT  : (1.88)
2

(i) Solve the linear-quadratic problem (1.88) by duality theory of this subsection.
(ii) Solve the same problem through methods of classical variational calculus.

1.6.2 General Remark: Feedforward Controls

Problem 1.5.1 of open-loop optimal control was solved in Sect. 1.5 for a fixed
starting position ft0 D t; x 0 D xg with solution given as an optimal cost J .t0 ; x 0 /
under control u0 .Ÿ/ D u0 .Ÿ j t0 ; x 0 /; Ÿ 2 Œt0 ; ª; in the class u0 ./ 2 UOO :
Now, solving this problem of feedforward control for any interval Œt; ª and any
starting position ft; xg; such solution may be used to create a model-predictive
control uf .t; x/ D u0 .t j t; x/ which will be a function of ft; xg: However, such
moves require a more general approach to variational problems of control that would
indicate solutions of the same problem in the class UC C of closed-loop control
strategies. Such are the Dynamic Programming techniques discussed in Chap. 2.
Chapter 2
The Dynamic Programming Approach

Abstract This chapter describes general schemes of the Dynamic Programming

approach. It introduces the notion of value function and its role in these schemes.
They are dealt with under either classical conditions or directional differentiability
of related functions, leaving more complicated cases to later chapters. Here the
emphasis is on indicating solutions to forward and backward reachability problems
for “linear-convex” systems and the design of closed-loop control strategies for
optimal target and time-optimal feedback problems.

Keywords Dynamic programming • Value function • HJB equation • Reach-

ability • Linear-convex systems • Colliding tubes • Terminal control • Time-
optimal control

Dynamic Programming techniques provide a powerful conceptual and computa-

tional approach to a broad class of closed-loop optimal control problems. They
are an application of the Hamiltonian formalism developed in calculus of varia-
tions and analytical dynamics [29, 33, 87, 269] to problems of feedback control
using Hamilton–Jacobi–Bellman (HJB) equations also known as the Dynamic
programming equations [16, 22, 24]. We start with the case when these partial
Differential equations can be solved under either classical conditions that presume
differentiability of value functions, or those where the value function is convex in
the state variables and hence allows their directional differentiability.1 The emphasis
here is on linear-convex systems. For these systems it is not necessary to integrate
the HJB equations since, as indicated here, the value function may be calculated
through methods of convex analysis. These methods also apply to the calculation of
support functions for the forward and backward reachability sets. The latter case is
also useful for calculating feedback controls.
Note that Sects. 2.1, 2.3, 2.7, 2.8 of this chapter are applicable not only to linear
systems but to nonlinear systems as well.

1
The general nondifferentiable case for the value function is discussed later in Sect. 5.1.

© Springer International Publishing Switzerland 2014 47

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__2
48 2 The Dynamic Programming Approach

2.1 The Dynamic Programming Equation

Consider system (1.1) expressed as

xP D f .t; x; u/; u 2 U.t /; t 2 Œ£; ª D T£ ; x 2 Rn ; u 2 Rp (2.1)

with starting position f£; xg; x.£/ D x: Here we have either U.t / D Rp or
U.t / D P .t /, where P .t / is compact-valued and Hausdorff-continuous.
The control will be sought for as either open-loop u.t /; u./ 2 UO , or closed-
loop u.t; x/; u.; / 2 UC :
We wish to find the optimal control u that minimizes the Mayer–Bolza functional
of the general form

Zª
J .£; x j u.// D L.t; xŒt ; u.t //dt C ®.ª; xŒª/: (2.2)
£

In (2.2) ª is a fixed terminal time, and xŒt  D x.t; £; x/ is the trajectory of (2.1)
starting from initial position f£; xg. It is customary to call L.t; x; u/ the running
cost and ®.ª; x/ the terminal cost. To begin, we suppose that the functions L; ® are
differentiable and additional conditions are satisfied that justify the mathematical
operations considered in this section. We will explicitly specify these conditions
later.
Definition 2.1.1. The value function for the problem of minimizing (2.2) is
defined as
0 ª 1
Z ˇ
ˇ
V .£; x£ / D inf @ L.t; xŒt ; u.t //dt C ®.ª; xŒª/ ˇˇ u./ 2 U./; xŒ£ D x£ A ;
u./
£
(2.3)
where U./ D fu./ W u.t / 2 U.t /; t 2 T£ g:
Thus V .£; x£ / is the infimum of the cost (2.2) incurred by any feasible control u./ 2
U./ and starting in state xŒ£ D x£ . For any u.t / 2 U.t / and resulting trajectory
xŒt  D x.t; £; x£ / we must have
0 1
Z¢ Zª
V .£; x£ /  L.t; xŒt ; u.t //dt C @ L.t; xŒt ; u.t //dt C ®.ª; xŒª/A ;
£ ¢
2.1 The Dynamic Programming Equation 49

for £ < ¢ < ª. Minimizing the second term with respect to u gives

V .£; x£ /
Z¢
 L.t; xŒt ; u.t //dt
£
8 ª 9
<Z ˇ =
ˇ
C inf L.t; xŒt ; u.t //dt C ®.ª; xŒª/ ˇˇ u./ 2 U./; xŒ¢ D x.¢; £; x£ /
: ;
¢

Z¢
D L.t; xŒt ; u.t //dt C V .¢; x.¢//: (2.4)
£

On the other hand, since V .£; x£ / is the infimum, for each © > 0 there exists uQ ./
such that

V .£; x£ / C ©  J .£; x j uQ .//

Z¢
 Q ; uQ .t //dt
L.t; xŒt
£
0 ª 1
Z ˇ
ˇ
C inf @ L.t; xŒt ; u.t //dt C ®.ª; x.ª// ˇˇ u./ 2 U./; xŒ¢
Q Q £; x£ /A
D x.¢;
¢

Z¢
D Q ; uQ .t //dt C V .¢; xŒ¢/;
L.t; xŒt Q (2.5)
£

Q  D x.t;
with xŒt Q £; x£ / being the trajectory resulting from control uQ .t /. From (2.4)
and (2.5)

Z¢
V .£; x£ /  Q ; uQ .t //dt C V .¢; xŒ¢/
L.t; xŒt Q  V .£; x£ / C ©:
£

Letting © ! 0 leads to the Dynamic Programming Equation

0 ¢ 1
Z ˇ
ˇ
V .£; x£ / D inf @ L.t; xŒt ; u.t //dt C V .¢; xŒ¢/ ˇˇ u./ 2 U./A : (2.6)
£
50 2 The Dynamic Programming Approach

Denote by V .t; x j ª; ®.ª; // the value function for the problem to minimize (2.2)
with terminal cost ®.ª; / (and the same running cost). With this notation, the value
function (2.3) is V .t; x j ª; ®.ª; // and the terminal cost serves as the boundary
condition

V .ª; x j ª; ®.ª; // D ®.ª; /: (2.7)

Lastly, recognizing the right-hand side of (2.6) as the value function for the
problem with terminal cost V .¢; / allows us to express (2.6) as the Principle of
Optimality:

V .£; x j ª; V .ª; // D V .£; x j ¢; V .¢;  j ª; V .ª; ///; £  ¢  ª: (2.8)

Observe that (2.8) is the semigroup property for the mapping V .£; / 7! V .ª; /:
Substituting t for £, t C ¢ for ¢ and s for t , rewrite (2.6) as
0 tC¢ 1
Z ˇ
ˇ
V .t; x/ D inf @ L.s; xŒs; u.s//ds C V .t C ¢; xŒt C ¢/ ˇˇ u./ 2 U./A :
t
(2.9)

Here ¢  0 and xŒs D x.s; t; x/ is the trajectory of (2.1) starting from ft; xg. For
¢  0, (2.9) can be written as
8 tC¢ 9
<Z ˇ =
ˇ
inf L.s; xŒs; u.s//ds C V .t C ¢; xŒ£ C ¢/  V .t; x/ ˇˇ u./ 2 U./ D 0;
: ;
t

and so
0 1
Z
tC¢ ˇ
1 V .t C ¢; xŒt C ¢/  V .t; x/ ˇˇ
lim inf @ L.s; xŒs; u.s//ds C A
ˇ u./ 2 U./ D 0:
¢!C0 ¢ ¢
t
(2.10)

Suppose now that V .t; x/ is continuously differentiable at .t; x/. Then reversing the
order of the operations lim and inf and passing to the limit with ¢ ! C0, we get
 ˇ 
d V .t; xŒt / ˇˇ
inf L.t; xŒt ; u/ C ˇ u 2 U.t / D 0: (2.11)
u dt

Expanding the total derivative d V .t; xŒt /=dt in terms of partial derivatives and
noting that @V .t; xŒt /=@t does not depend on u, (2.11) may be rewritten as
 ˇ 
@V .t; x/ @V .t; x/ ˇ
C inf ; f .t; x; u/ C L.t; x; u/ ˇˇ u 2 U.t / D 0; (2.12)
@t @x
2.1 The Dynamic Programming Equation 51

with boundary condition (2.7). This partial differential equation is known as the
(backward) Hamilton–Jacobi–Bellman equation or simply the HJB equation. It is
“backward” because the boundary condition (2.7) is at the terminal time; the
“forward” HJB equation has boundary condition at the initial time.
Lemma 2.1.1. If the value function V .t; x/ given by (2.3) is differentiable, it
satisfies the HJB equation (2.12) with boundary condition (2.7).
Exercise 2.1.1. Justify the reversal of lim and inf in (2.10).
Theorem 2.1.1. (i) Suppose the value function V .t; x/ for problem (2.3) is differ-
entiable at point ft; xg. Then it satisfies the inequality
 
@V .t; x/ @V .t; x/
C infw ; f .t; x; w/ C L.t; x; w/ j w 2 UŒt   0:
@t @x
(2.13)

(ii) If in addition to (i) there exists an optimal control u0 .s/ 2 U.s/, such that
u0 .s/ ! w0 2 U.t / as s ! t C 0, then

@V @V .t; x/
C ; f .t; x; w0 / C L.t; x; w0 / D 0: (2.14)
@t @x

Theorem 2.1.2 (The First Verification Theorem). Suppose a differentiable func-

tion ¨.t; x/ satisfies the HJB equation (2.12) in domain D D T£  Rn together
with boundary condition (2.7). Then in this domain ¨.t; x/  V .t; x/. Moreover, if
control u0 ./, with trajectory xŒs D x 0 .s; t; x/; s  t; x 0 Œt  D x, satisfies

@¨.s; x 0 Œs/
L.s; x 0 Œs; u0 .s// C ; f .s; x 0 Œs; u0 .s// D
@x
  
@¨.s; x 0 Œs; u/
D min L.s; x Œs; u/ C
0
; f .s; x Œs; u/ j u 2 U.s/ ;
0
(2.15)
@x

then u0 ./ is optimal and ¨.t; x/ D V .t; x/.

Proof. Let u./ 2 U./ be any control and xŒs D x.s; t; x/ the resulting trajectory
starting at xŒt  D x. From (2.12),
Z ª   
@¨.s; x.s// @¨.s; x.s//
¨.ª; xŒª/  ¨.t; x/ D C f .s; x.s/; u.s//; ds 
t @s @x
Z ª
 L.s; x.s/; u.s//ds:
t
52 2 The Dynamic Programming Approach

Since ¨.ª; x.ª// D ®.ª; xŒª/, moving the integral to the left side of the last
relation gives
Z ª
¨.t; x/  L.s; x.s/; u.s//ds C ®.ª; xŒª/ D J .t; x j u.//:
t

Hence ¨.t; x/  V .t; x/:

Repeating this calculation for u D u0 ./, but using (2.15) instead of (2.12), yields
¨.t; x/ D J .t; x j u0 .//, which implies that ¨.t; x/ D V .t; x/ and u0 ./ is optimal.
t
u
Definition 2.1.2. A control in the form u.t; x/ is called a closed-loop or feedback
control. It is sometimes known as positional control. A control u D u.t / that is a
function only of time is an open-loop control.
Corollary 2.1.1. The optimal feedback (closed-loop) control for problem (2.3) is
given by
 ˇ 
@V ˇ
u0 .t; x/ 2 arg min ; f .t; x; u/ C L.t; x; u/ ˇˇ u 2 U.t / : (2.16)
@x

Remark 2.1.1. Formally, given a closed-loop control u0 .t; x/, one may find the
corresponding open-loop control u0 D u0 Œs by first obtaining the solution xŒs D
x.s; t; x/ of the differential equation

xP D f .s; x; u0 .s; x//; xŒt  D x; (2.17)

and then substituting to get u0 Œs D u0 .s; xŒs/: However, one must ensure
that (2.17) has a solution in some reasonable sense, particularly, when the
feedback control u0 .s; x/ is nonlinear (even discontinuous) in x, but the original
system is linear in x.
Consider now the problem of finding u that minimizes the inverse Mayer–Bolza
functional

Zt
J0 .t; x j u.// D ®.t0 ; x.t0 // C L.s; x.s/; u.s//ds; (2.18)
t0

with given final position ft; xg; xŒt  D x; and initial time t0  t: The value function
for this problem is

V0 .t; x/ D inffJ0 .t; x j u.// j u./ 2 U./; x.t / D xg: (2.19)

u./
2.1 The Dynamic Programming Equation 53

Mimicking the previous argument now leads to the Dynamic Programming Equation
Z t
V0 .t; x/ D inffV0 .t ¢; xŒt ¢/C L.s; xŒs; u.s//ds j u./ 2 U./; x.t / D xg;
t¢

from which, supposing V0 .t; x/ is differentiable, one obtains the forward HJB
equation
 ˇ 
@V0 @V0 .t; x/ ˇ
.t; x/ C sup ˇ
; f .t; x; u/  L.t; x; u/ ˇ u 2 U.t / D 0; (2.20)
@t @x

with boundary condition

V0 .t0 ; x/ D ®.t0 ; x/: (2.21)

The Principle of Optimality in forward time now has the form

V0 .t; x j t0 ; V0 .t0 ; // D V0 .t; x j £; V0 .£;  j t0 ; V0 .t0 ; ///; t0  £  t: (2.22)

Exercise 2.1.2. Derive the Dynamic Programming Equation in forward time and
then the forward HJB equation (2.20).
Theorem 2.1.3 (The Second Verification Theorem). Suppose a differentiable
function ¨0 .t; x/ satisfies the HJB equation (2.20) in domain D D Tt0 Rn ; together
with boundary condition (2.21). Then in this domain ¨0 .t; x/  V0 .t; x/. Moreover,
if control u0 ./, with trajectory x 0 Œs D x 0 .s; t; x/; x 0 Œt  D x; s  t , satisfies

@¨0
L.s; x 0 Œs; u0 .s// C .s; x 0 Œs/; f .s; x 0 Œs; u0 .s// D
@x
  ˇ 
@¨0 ˇ
D max L.s; x 0 Œs; u/ C .s; x 0 Œs; u/; f .s; x 0 Œs; u/ ˇ u 2 U.s/ ;
@x ˇ
(2.23)
then u0 ./ is optimal and ¨0 .t; x/ D V0 .t; x/.
Exercise 2.1.3. Prove Theorem 2.1.3.
Corollary 2.1.2. The optimal feedback (closed-loop) control for problem (2.19) is
 ˇ 
@V0 ˇ
u0 .t; x/ 2 Arg max ; f .t; x; u/  L.t; x; u/ ˇˇ u 2 U.t / : (2.24)
@x
54 2 The Dynamic Programming Approach

Remark 2.1.2. The optimal closed-loop control u0 .t; x/ found here is feasible only
when accompanied by an existence theorem for the equation

xP D f .t; x; u0 .t; x//:

We shall now use the Dynamic Programming Verification Theorem to minimize a

quadratic cost for a linear system.

2.2 The Linear-Quadratic Problem

Minimizing a quadratic integral cost over the trajectories of a linear control system
is the basic problem in all courses on linear control. It has a solution in explicit
form, fairly easily obtained and smooth enough to satisfy all requirements for a
classical solution to the HJB equation of the previous section. This problem has
a vast literature, but it is not the subject of this book, and is presented here as a
necessary transitionary passage to main topics.
Consider the linear system (1.2) or (1.44), namely

xP D A.t /x C B.t /u; t0  t  t1 ; x.t0 / D x0 ; (2.25)

with control u.t / 2 Rp and the quadratic integral cost functional

Zt1  
J.t0 ; x0 ; u.// D hx; M.t /xi C hu; N.t /ui dt C hx.t1 /; T x.t1 /i; (2.26)
t0

in which M.t / D M 0 .t /  0, N.t / D N 0 .t / > 0, are continuous in t 2 Œt0 ; t1  and

T D T 0 > 0.
Problem 2.2.1 (Linear-Quadratic Control Problem). Find a control u./ that
minimizes the cost (2.26).
Introduce the value function

V .t; x/ D minfJ.t; x; u.// j x.t / D xg: (2.27)

u./

Its backward HJB equation is (see (2.12))

 
@V @V
C min ; A.t /x C B.t /u C hx; M.t /xi C hu; N.t /ui D 0; (2.28)
@t u @x

with boundary condition

V .t1 ; x/ D hx; T xi: (2.29)

2.2 The Linear-Quadratic Problem 55

The internal minimization problem in (2.28) is to find

 ˇ 
@V ˇ
u.t; x/ D arg min ; A.t /x C B.t /u C hu; N.t /ui ˇˇ u 2 Rp :
@x

Solving this minimization by differentiating with respect to u and equating the

derivative to zero gives

1 @V
u.t; x/ D  N 1 .t /B 0 .t / : (2.30)
2 @x
Substituting (2.30) into (2.28), we come to
 
@V @V 1 @V 1 0 @V
C ; Ax  ; BN B C hx; M xi D 0: (2.31)
@t @x 4 @x @x

We follow Theorem 2.1.2 and look for V .t; x/ as a quadratic form V .t; x/ D
hx; P .t /xi, with P .t / D P 0 .t /. Then

@V @V
D hx; PP .t /xi; D 2P .t /x:
@t @x
Substituting these into (2.31) gives

hx; PP .t /xi C 2hP .t /x; A.t /xi  hP .t /x; BN 1 B 0 P .t /xi C hx; M xi D 0;

and so we arrive at the (backward) matrix differential equation of the Riccati type

PP C PA C A0 P  P 0 BN 1 B 0 P C M D 0; (2.32)

with terminal matrix boundary condition

P .t1 / D T: (2.33)

Consider the special case of Eq. (2.32) with M.t /  0. Differentiating the
identity I D P .t /P 1 .t / gives PP P 1 C P PP 1 D 0 or

PP 1 D P 1 PP P 1 :

Multiplying both sides of Eq. (2.32) by P 1 , and using the last relation, we have

PP 1 D AP 1 C P 1 A0  BN 1 B 0 ; P 1 .t1 / D T 1 : (2.34)
56 2 The Dynamic Programming Approach

The solution of this linear matrix equation is given by the integral formula
Z t1
P 1 .t / D G.t; t1 /T 1 G0 .t; t1 / C G.t; s/B.s/N 1 .s/B 0 .s/G0 .t; s/ds;
t
(2.35)
in which G.t; s/ is defined in (1.11), (1.12).
Theorem 2.2.1. The value function for Problem 2.2.1 has the quadratic form
V .t; x/ D hx; P .t /xi, in which P .t / satisfies the matrix Riccati equation (2.32),
with boundary condition (2.33). The optimal control, given by the linear feedback
function (2.30), is

u.t; x/ D N 1 .t /B 0 .t /P .t /x:

If, furthermore, M.t /  0, then P 1 .t / is given by formula (2.35).

The next problem is similar to Problem 2.2.1 and involves the quadratic cost
functional

Zt1  
J0 .t1 ; x1 ; u.// D hx.t0 /; Lx.t0 /i C hx; M.t /xi C hu; N.t /ui dt; (2.36)
t0

with final condition x.t1 / D x .1/ : Here L D L0 > 0:

Problem 2.2.2 (Inverse Linear-Quadratic Control Problem). Find a control u./
that minimizes the cost (2.36).
The value function for Problem 2.2.2 is

V0 .t; x/ D minfJ0 .t; x; u.// j x.t / D xg: (2.37)

u./

The corresponding forward HJB equation is

 
@V0 @V0
C max ; A.t /x C B.t /u  hx; M.t /xi  hu; N.t /ui D 0; (2.38)
@t u @x

with boundary condition

V0 .t0 ; x/ D hx; Lxi: (2.39)

The internal maximization in (2.38) gives

1 1 @V0
u.t; x/ D N .t /B 0 .t / ; (2.40)
2 @x
2.2 The Linear-Quadratic Problem 57

so that V0 .t; x/ satisfies the relation

 
@V0 @V0 1 @V0 @V0
C ; A.t /x C ; B.t /.N.t //1 B 0 .t /  hx; M.t /xi D 0;
@t @x 4 @x @x
(2.41)
with V0 .t0 ; x/ D hx; Lxi: As before, assuming V0 .t; x/ D hx; P0 .t /xi and
repeating the type of substitutions given above, we arrive at the (forward) Riccati
equation

PP0 C P0 A C A0 P0 C P0 BN 1 B 0 P0  M D 0; P0 .t0 / D L: (2.42)

The solution to Problem 2.2.2 may be used to determine the reach set X Œt  of
system (2.25) under the integral constraint

J0 .t; x.t /; u.//  :

In fact,

X Œt  D fx W V0 .t; x/  g D fx W hx; P0 .t /xi  g: (2.43)

Exercise 2.2.1. Prove (2.43).

Exercise 2.2.2. Solve Problem 2.2.1 for the cost

J .t0 ; x0 ; u.// D

Zt1  
D hxx  .t /; M.t /.xx  .t //iChu; N.t /ui dt Chx.t1 /a; T .x.t1 /a/i; x.t0 / D x 0 ;
t0
(2.44)

in which T D T 0 > 0, M.t / D M 0 .t /  0, N.t / D N 0 .t / > 0 are continuous in

t 2 Œt0 ; t1 , and function x  ./ and vector a are given.
Exercise 2.2.3. Solve Problem 2.2.2 for the cost

Zt1
J0 .t1 ; x1 ; u.// D maxfhx.t0 /; Lx.t0 /i; hu; N.t /uidt g; x.t1 / D x .1/ : (2.45)
t0

We shall now use the Dynamic Programming approach to the calculation of reach
sets under hard bounds on the controls.
58 2 The Dynamic Programming Approach

2.3 Reachability Through the HJB Equation: Hard Bounds

For problems of reachability and closed-loop control we derive the main Dynamic
Programming equation which is a first-order partial differential equation with
appropriate boundary conditions.

2.3.1 Forward Reachability

We extend Definition 1.4.1 of reach sets to the nonlinear system (2.1), under the
hard bound u 2 P .t /.
Definition 2.3.1. The reach set X Œª D X .ª; t0 ; x 0 / of system (2.1) at given time
ª, from position ft0 ; x 0 g, is the set of all points x, for each of which there exists a
trajectory xŒs D x.s; t0 ; x 0 /, generated by a control subject to the given constraint,
that transfers the system from position ft0 ; x 0 g to position fª; xg; x D xŒª:

X .ª; t0 ; x 0 / D fx W 9u./ 2 P ./ such that x.ª; t0 ; x 0 / D xg:

The reach set X Œª D X .ª; t0 ; X 0 / from set-valued position ft0 ; X 0 g at given time
ª is the union
[
X .ª; t0 ; X 0 / D fX .ª; t0 ; x 0 / j x 0 2 X 0 g:

The set-valued function X Œt  D X .t; t0 ; X 0 /; t0  t  ª, is the reach tube from

ft0 ; X 0 g.
This definition of reach sets does not involve an optimization problem. However,
we shall relate it to a problem of optimization that can be solved by Hamiltonian
methods in their Dynamic Programming version. We characterize the (forward)
reach set by the value function

V0 .£; x/ D min fd 2 .x.t0 /; X 0 / j x.£/ D xg D min fd 2 .x.t0 /; X 0 / j x 2 X .£; t0 ; x.t0 //g

u./;x.t0 / x.t0 /
(2.46)
over all measurable functions u.t / that satisfy (1.18). See Fig. 2.1. Above,

d 2 .x; X / D minfhx  z; x  zi j z 2 X g

is the square of the distance d.x; X / from point x to set X . Observe that (2.46) is a
special case of (2.19) with zero running cost in (2.18).
2.3 Reachability Through the HJB Equation: Hard Bounds 59

Fig. 2.1 The value function V0 .£; x/ in (2.46) with x.t0 / … X 0

Lemma 2.3.1. The following relation holds:

X .£; t0 ; X 0 / D fx W V0 .£; x/  0g: (2.47)

As seen from Fig. 2.1, (2.47) follows from the definition of the reach set
X .£; t0 ; X 0 /, which thus turns out to be the level set (at level zero) of the function
V0 .£; /.
For the value function V0 .t; x/ we again use the notation

V0 .t; x/ D V0 .t; x j t0 ; V0 .t0 ; //;

to emphasize its dependence on the boundary condition V0 .t0 ; x/ D d 2 .x; X 0 /. The

next assertion merely restates (2.22).
Theorem 2.3.1. The value function V0 .t; x/ satisfies the Principle of Optimality in
forward time and has the semigroup form

V0 .t; x j t0 ; V0 .t0 ; // D V0 .t; x j £; V0 .£;  j t0 ; V0 .t0 ; ///; t0  £  t: (2.48)

This theorem together with Lemma 2.3.1 also implies the following assertion.
Lemma 2.3.2. The reachability set X Œt  D X .t; t0 ; X 0 / satisfies the next relation

X Œt  D X .t; £; X .£; t0 ; X 0 //: (2.49)

60 2 The Dynamic Programming Approach

The solution of the reachability problem, namely the one of calculating the reach set,
now depends on the properties of either the “classical” or the generalized “viscosity”
solutions of the forward HJB equation
 
@V0 .t; x/ @V0 .t; x/
C max ; f .t; x; u/ j u 2 P .t / D 0; (2.50)
@t u @x

with boundary condition

V0 .t0 ; x/ D d 2 .x; X 0 /: (2.51)

Equation (2.50) follows from (2.48) or it may be seen as a special case of (2.20).
This equation is solvable in the classical sense (that is, (2.50) holds everywhere),
if the partials of V .t; x/ in t; x exist and are continuous. Otherwise (2.50) is a
symbolic relation for the generalized HJB equation which has to be described in
terms of subdifferentials, Dini derivatives or their equivalents. However, the typical
situation is that V is not differentiable. The treatment of Eq. (2.50) then involves
the notion of generalized “viscosity” or “minmax” solution for this equation (see
[16, 50, 247]). The last notion will be later treated in a separate Sect. 5.1 which
deals with nondifferentiable value functions and generalized solutions. In the case
of linear systems with convex constraints the value functions are convex in the
state space variables and hence directionally differentiable these solutions belong
the generalized, which property is checked directly.
The calculation of reach sets and tubes can thus be reduced to the calculation
of the value function V0 .t; x/ or its level sets X Œt  D fx W V0 .t; x/  0g and their
evolution in time t . On the other hand, we note that X Œt  may also be treated as
the cross-section (“cut”) X Œt  D X .t; t0 ; X 0 / of the solution tube X .; t0 ; X 0 / to the
differential inclusion

xP 2 F .t; x/; t  t0 ; x0 2 X 0; (2.52)

in which F .t; x/ D [ff .t; x; u/ j u 2 P .t /g.

A set-valued function X Œt  is a solution to Eq. (2.52) on the interval T D Œt0 ; £
if it satisfies this equation for almost all t 2 T . Under the previous assumptions on
f and P there is a unique solution X ΠD X .; t0 ; X 0 / issuing from X 0 for any time
interval Œt0 ; £. The solution X Œ may also be based on using the “funnel equation”
described in the next lines (see also Sect. 8.1 and [158]).
Assumption 2.3.1. F .t; x/ is convex compact-valued .F W Œt0 ; t1   Rn !
convRn /, continuous in t; x in the Hausdorff metric and Lipschitz-continuous in x:

h.F .t; x/; F .t; y//  Lhx  y; x  yi1=2 ; L < 1: (2.53)

2.3 Reachability Through the HJB Equation: Hard Bounds 61

Here h is the Hausdorff distance

h.Q ; M / D maxfhC .Q ; M /; hC .M ; Q /g;

and the Hausdorff semidistance between sets Q ; M is

hC .Q ; M / D max minfhx  z; x  zi1=2 j x 2 Q ; z 2 M g:

x z

The Hausdorff distance is a metric in the space of compact sets.

The “ordinary distance” d.Q ; M / between sets Q ; M ; is

d.Q ; M / D min minfhx  z; x  zi1=2 j x 2 Q ; z 2 M g:

x z

Theorem 2.3.2. Under Assumption 2.3.1 the forward reach set X Œt  is compact and
the set-valued tube X Œ is the unique solution to the following “funnel” equation for
the differential inclusion (2.52):

lim ¢ 1 h.X Œt C ¢; [f.x C ¢F .t; x// j x 2 X Œt g/ D 0; (2.54)

¢!C0

with boundary condition X Œt0  D X 0 .

Theorem 2.3.2 was introduced in [75, 224], see also [26, 158]. Note that the set-
valued function X Œt  of the last theorem is continuous in the Hausdorff metric. Thus
it is possible to single out two alternative approaches to the treatment of reachability:
namely, through the solution of the HJB equation (2.50) (the calculation of the value
function V0 .t; x/ and its level sets) or through the calculation of solution tubes
X Œt  to the differential inclusion (2.52) by dealing, for example, with the funnel
equation (2.54). In either approach, the calculation of reach sets is solved in forward
time. A reciprocal approach considers reach sets in backward (reverse) time.

2.3.2 Backward Reachability or the Solvability Problem

Whereas the reach set is the set of all states that can be reached from a given initial
set, the backward reach set comprises all those states, taken at instant £, from which
it is possible to reach a given “target” set M taken to be a convex compact in Rn .
Definition 2.3.2. Given a closed target set M Rn , the backward reach
(solvability) set W Œ£ D W .£; t1 ; M / at given time £, from set-valued position
ft1 ; M g; is the set of states x 2 Rn for each of which there exists a control
u.t / 2 P .t / that steers system (2.1) from state x.£/ D x to x.t1 / 2 M . The set-
valued function W Œt  D W .t; t1 ; M /; £  t  t1 ; is the solvability (or backward
reach) tube from ft1 ; M g.
Backward reach sets W Σ are also known as weakly invariant sets relative to
position ft1 ; M g:
62 2 The Dynamic Programming Approach

Definition 2.3.3. A set W Σ is said to be weakly invariant relative to system (2.1)
[(1.2), (1.7)] and position ft1 ; M g, if it consists of all those points x D x.£/
(positions f£; xg), from which set M is reachable at time t1 by some of the possible
controls.
Hence, from each position f£; xg 2 W Œ£ there exists at least one trajectory that
stays in the backward reach tube and reaches M at t D t1 : The backward reach set
W .£; t1 ; M / is the largest or, in other words, the inclusion maximal weakly invariant
set relative to M .
The calculation of backward reach sets may be achieved through the value
function

V .£; x/ D minfd 2 .x.t1 /; M / j x.£/ D xg;

u

V .£; x/ D V .£; x j t1 ; V .t1 ; //; V .t1 ; x/ D d 2 .x; M /; (2.55)

in which the minimization is over all u./ 2 P ./. Similarly to Lemma 2.3.1, the
backward reach set is a level set.
Lemma 2.3.3. The following relation is true:

W .£; t1 ; M / D fx W V .£; x/  0g; (2.56)

The next result is the analog of Theorem 2.3.1.

Theorem 2.3.3. The set-valued mapping W .t; t1 ; M / satisfies the semigroup
property

W .t; t1 ; M / D W .t; £; W .£; t1 ; M //; t  £  t1 : (2.57)

(in backward time) and the value function V .t; xjt1 ; V .t1 ; // satisfies the Principle
of Optimality in the semigroup form

V .t; x j t1 ; V .t1 ; // D V .t; x j £; V .£;  j t1 ; V .t1 ; ///; t  £  t1 : (2.58)

Associated with the Principle of Optimality (2.58) is the backward HJB equation
 
@V .t; x/ @V
C min ; f .t; x; u/ D 0; (2.59)
@t u @x

with boundary condition

d 2 .x; M / D V .t1 ; x/: (2.60)

2.3 Reachability Through the HJB Equation: Hard Bounds 63

Associated with the semigroup (2.57) is the “backward” funnel equation for the
differential inclusion (2.52):

lim ¢ 1 h.W Œt  ¢; [f.x  ¢F .t; x//jx 2 W Œt g/ D 0; (2.61)

¢!C0

with boundary condition W Œt1  D M .

Lemma 2.3.4. The set-valued function W Œt  D W .t; t1 ; M /; W Œt1  D M ; is the
unique solution to Eq. (2.61).
Remark 2.3.1. The above assertions emphasize two ways of describing reachability
sets and tubes, namely through set-valued functions like W Œt  or through “ordi-
nary” functions like V .t; x/. These are connected through (2.56).
Finally we formulate an important property of reach sets for our systems in the
absence unknown disturbances .v D 0/:2 This is when the forward and backward
reach sets X .£; t0 ; X 0 /; W .£; t1 ; M / have the next connection.
Theorem 2.3.4. Suppose X .t1 ; t0 ; X 0 / D X Œt1  D M : Then W Œt0  D
W .t0 ; t1 ; M / D X 0 :
Each of these sets X .£; t0 ; X 0 /; W .£; t1 ; M / does not depend on whether it is
calculated in the class of open-loop controls UO or closed-loop controls UC .
Exercise 2.3.1. Prove Theorem 2.3.4.
Strongly Invariant Sets. In contrast with the weakly invariant backward reach sets
W Σ from which the target M is reachable at time t1 with some control we now
introduce the notion of strongly invariant sets.
Definition 2.3.4. A set Ws Σ is strongly invariant relative to system (2.10) and
position ft1 ; M g; if from each of its points x D x.£/ set M is reachable at time t1
by all possible controls.
A set Ws Œ£ that consists of all such points x D x.£/ is said to be the inclusion-
maximal strongly invariant set relative to position ft1 ; M g.
The strongly invariant set relative to M is given by the value function

Vs .£; x/ D maxfd 2 .x.t1 /; M / j x.£/ D xg; (2.62)

u

in which the maximization is over all u./ 2 P ./; u./ 2 UO ./.

Consider the backward HJB equation
 
@Vs .t; x/ @Vs
C max ; f .t; x; u/ D 0; (2.63)
@t u @x

2
Problems of reachability under unknown but bounded disturbances are beyond the scope of this
book, along the approaches of which they are treated in papers [133, 176, 183].
64 2 The Dynamic Programming Approach

with boundary condition

Vs .t1 ; x/ D d 2 .x; M /: (2.64)

Theorem 2.3.5. If Eq. (2.64) has a unique (viscosity or classical) solution Vs .t; x/,
the strongly invariant set Ws Œ£ D Ws .£; t1 ; M / is the level set

Ws Œ£ D fx W Vs .£; x/  0g:

The next result is obvious.

Lemma 2.3.5. The set Ws Σ 6D ; iff there exists a point x such that the reach set
X .t1 ; £; x/ M .
Exercise 2.3.2. Does the mapping Ws .£; t1 ; M / satisfy a semigroup property
similar to (2.57)?
Solutions of the HJB equations above require rather subtle analytical or numerical
techniques. Some solutions may however be obtained through methods of convex
analysis. In particular, this is the case for linear-convex systems, which are when the
system is linear in x; u and the constraints on u; x 0 are convex. These will be studied
next.

2.4 Reachability for Linear-Convex Systems

In this section we calculate the value functions through methods of convex analysis
and also find the support functions for the forward and backward reachability sets.

2.4.1 Forward Reachability: Calculating the Value Functions

Consider system (2.25), namely,

xP D A.t /x C B.t /u; (2.65)

with continuous matrix coefficients A.t /; B.t / and hard bound u.t / 2 P .t /, with
P .t / convex, compact, and Hausdorff-continuous.

The Value Function (Forward)

Problem 2.4.1. Given starting set-valued position ft0 ; X 0 g, terminal time ª and
condition x.ª/ D x, calculate the value function V0 .ª; x/ of (2.46) at any position
fª; xg:
2.4 Reachability for Linear-Convex Systems 65

After solving this problem we may use (2.47) to find the forward reach set X Œt  D
X .t; t0 ; X 0 / as the level set X Œt  D fx W V0 .ª; x/  0g.
The HJB equation (2.50) now has the form
 ˇ 
@V0 .t; x/ @V0 .t; x/ ˇ
C max ; A.t /x C B.t /u ˇˇ u 2 P .t / D 0;
@t @x

or, since

maxfhl; ui j u 2 P .t /g D ¡.l j P /

is the support function of set P .t /, we may write

  ˇ 
@V0 .t; x/ @V0 .t; x/ @V0 .t; x/ ˇ
C ; A.t /x C ¡ ˇ B.t /P .t / D 0: (2.66)
@t @x @x ˇ

The boundary condition for (2.66) is

V0 .t0 ; x/ D d 2 .x; X 0 / D minfhx  q; x  qi j q 2 X 0 g: (2.67)

q

Instead of solving the HJB equation, V0 .£; x/ may be calculated through duality
techniques of convex analysis along the scheme given, for example, in [174,
Sect. 1.5].
Indeed, observing that the Fenchel conjugate for function ®.£; x/ D d 2 .x; X 0 /
in the second variable is ® .£; l/ D ¡.l j X 0 / C 14 hl; li, we may write, making use
of the minmax theorem of [72] to interchange min and max below,

V0 .£; x/ D minfd 2 .x.t0 /; X 0 / j x.£/ D xg

u

1
D min maxfhl; x.t0 /i  hl; li  ¡.l j X 0 / j x.£/ D xg
u l 4
D max min ‰.£; x; l; u.//; (2.68)
l u

where
Z £
1
‰.£; x; l; u.// D hs.£; t0 ; l/; xi  hs.t; t0 ; l/; B.t /u.t /idt  hl; li  ¡.l j X 0 /:
t0 4

Here sŒt  D s.t; t0 ; l/ is the solution of the adjoint equation

sP D A0 .t /s; s.t0 / D l; (2.69)

and s; l are column vectors.

66 2 The Dynamic Programming Approach

The function ‰.£; x; l; u.// is concave in l and ‰.£; x; l; u.// ! 1 as

hl; li ! 1. It is also convex in u with u.t / bounded by P .t /. This allows us again,
in view of ([72]), to interchange min and max in (2.68). Thus

V0 .£; x/ D max ˆ0 Œ£; x; l; (2.70)

l

with

ˆ0 Œ£; x; l D minf‰.£; x; l; u.// j u.t / 2 P .t /; t0  t  £g D

Z £
1
D hs.£; t0 ; l/; xi  ¡.B 0 .t /s.t; t0 ; l/ j P .t //dt  hl; li  ¡.l j X 0 /:
t0 4

We now show by a direct substitution of V0 .t; x/ into (2.66) that it satisfies this
equation with boundary condition (2.67). Indeed, let l 0 D l 0 .t; x/ be the maximizer
of function ˆ0 Œt; x; l in l (see (2.70)). The structure of ˆ0 Œt; x; l indicates that l 0
is unique. (Prove this assertion.)
Next we calculate the partial derivatives @V0 .t; x/=@t ; @V0 .t; x/=@x. According
to the rules of differentiating the maximum of a function like in (2.70), (see [61]),
we have

@V .t; x/ @ˆ0 Œt; x; l 0 

D D hPs .t; t0 ; l 0 /; xi  ¡.s.t; t0 ; l 0 / j B.t /P .t //;
@t @t

@V .t; x/ @ˆ0 Œt; x; l 0 

D D s.t; t0 ; l 0 /:
@x @x
Substituting these partials into (2.66) and keeping (2.69) in view, one observes
that V0 .t; x/ is indeed a solution to Eq. (2.66).
The boundary condition (2.67), namely,

1
V .t0 ; x/ D maxfhl; xi  hl; li  ¡.l j X 0 / j l 2 Rn g D
4

D maxfhl; xi  ® .t0 ; l/ j l 2 Rn g D d 2 .x; X 0 /;

is also fulfilled. We summarize these results in the next assertion.

Theorem 2.4.1. For the linear system (1.44) the value function V0 .£; x/ of the
forward HJB equation (2.66) has the following properties:
(i) V0 .£; x/ is given by (2.70).
(ii) The maximizer l 0 .£; x/ D arg maxl ˆ0 Œ£; x; l is unique and continuous in £; x.
(iii) V0 .£; x/ is a proper convex function in x and is therefore directionally
differentiable along any direction f1; lg.
(iv) V0 .t; x/ satisfies the HJB equation (2.66) for all t; x and also the boundary
condition (2.67).
2.4 Reachability for Linear-Convex Systems 67

Proof. The second assertion follows from the strict convexity of ˆ0 .£; x; l/ in l due
to the quadratic term and from the continuity of ˆ.£; x; l/ in £; x. Property (iii) may
be checked directly from the explicit formulas for the partials of V0 .t; x/, while (iv)
is verified through direct substitution (Check whether V0 .t; x/ is unique). t
u
Remark 2.4.1. Note that for a time-invariant system one has

V0 .£; x/ D V0 .£; x j t0 ; X 0 / D V0 .£  t0 ; x j 0; X 0 /:

2.4.2 The Conjugate of the Value Function V0

Having calculated V0 .t; x/ according to (2.70), let us now find its conjugate in the
second variable, denoted by V0 .t; l/. We have

V0 .t; l/ D maxfhl; xi  V0 .t; x/ j x 2 Rn g D

x

maxfhl; xi  maxfˆ0 Œt; x; œ j œ 2 Rn g j x 2 Rn g

x œ

D max minfhl; xi  ˆ0 Œt; x; œ j x; œ 2 Rn g

x œ

Z t
D min maxfhl; xi  hs.t; t0 ; œ/; xi C ¡.B 0 .Ÿ/s.Ÿ; t0 ; œ/ j P .Ÿ//d Ÿ
œ x t0

1
C hœ; œi C ¡.œ j X 0 / j x; œ 2 Rn g:
4

The minmax is here attained at s.t; t0 ; œ/ D l (for values of œ where this

equality is not true the internal maximum in x is C1). Hence, keeping in mind
that s.t; t0 ; œ/ D G 0 .t0 ; t /œ D l; we come to the formula
Z t 1
V0 .t; l/ D ¡.l j G.t; Ÿ/B.Ÿ/P .Ÿ//d Ÿ C ¡.l j G.t; t0 /X 0 / C hl; G.t; t0 /G 0 .t; t0 /li;
t0 4
(2.71)

Theorem 2.4.2. (i) The conjugate V0 .t; l/ to the value function V0 .t; x/ is given
by (2.71).
(ii) The forward reach set X Œt  D X .t; t0 ; X 0 / is given by

X Œt  D X .t; t0 ; X 0 / D fx W hl; xi  V0 .t; l/  0 j 8l 2 Rn g (2.72)

68 2 The Dynamic Programming Approach

Proof. Assertion (ii) follows from

V0 .t; x/ D maxfhl; xi  V0 .t; l/ j l 2 Rn g;

l

since V0 is conjugate to V0 . t

u

2.4.3 The Value Function (Backward)

A scheme similar to the previous one works for the solution V .t; x/ of the backward
equation (2.59) written for linear system (2.65) as
  ˇ 
@V .t; x/ @V .t; x/ @V .t; x/ ˇ
C ; A.t /x  ¡  ˇ B.t /P .t / D 0: (2.73)
@t @x @x ˇ

The boundary condition for (2.55) or (2.73) is

V .t1 ; x/ D d 2 .x; M / D minfhx  q; x  qi j q 2 M g: (2.74)

q

Omitting the calculation, which is similar to the above, we come to

V .£; x/ D max ˆŒ£; x; l; (2.75)

l

with
Z t1
1
ˆŒ£; x; l D hs.£; t1 ; l/; xi  ¡.s.t; t1 ; l/ j B.t /P .t //dt  hl; li  ¡.l j M /:
£ 4

Theorem 2.4.3. The value function V .£; x/ for the backward HJB equa-
tion (2.73), (2.74) has the following properties:
(i) V .£; x/ is given by (2.75).
(ii) The maximizer l 0 .£; x/ D arg maxl ˆŒ£; x; l is unique and continuous in £; x.
(iii) V .£; x/ is a proper convex function in x and is directionally differentiable for
any £; x.
(iv) V .t; x/ satisfies the HJB equation (2.73) for all t; x and also the boundary
condition (2.74).
Exercise 2.4.1. Find the conjugate function V  .t; l/:
Formulas of type (2.70), (2.75) may be also calculated with criteria d.x.t0 /; X 0 /;
d.x.t1 /; M / instead of d 2 .x.t0 /; X 0 /; d 2 .x.t1 /; M / in the definitions of
V0 .t; x/; V .£; x/. In this case the term 14 hl; li has to be omitted and the maxima
taken over a unit sphere hl; li  1. These calculations lead to the next result.
2.4 Reachability for Linear-Convex Systems 69

Theorem 2.4.4. Let Vf .t; x/; Vb .t; x/ be the solutions of HJB equations (2.66),
(2.73) with boundary conditions

Vf .t0 ; x/ D d.x; X 0 /; Vb .t1 ; x/ D d.x; M /; (2.76)

respectively. Then the following relations hold:

Vf .£; x/ D minfd.x.t0 /; X 0 / j x.£/ D xg D maxfˆf Œ£; x; l j hl; li  1g D

u l

D maxfˆf Œ£; x; l  I .ljB .0// j l 2 Rn g; (2.77)

l

with I .ljB .0// being the indicator function for the unit ball B .0/ and
Z £
ˆf Œ£; x; l D hs.£; t0 ; l/; xi  ¡.s.t; t0 ; l/ j B.t /P .t //dt  ¡.l j X 0 /:
t0

Similarly,

Vb .£; x/ D minfd.x.t1 /; M / j x.£/ D xg D maxfˆb Œ£; x; l j hl; li  1g D

u l

D maxfˆb Œ£; x; l  I .ljB .0// j l 2 Rn g; (2.78)

l

with
Z t1
ˆb Œ£; x; l D hs.£; t1 ; l/; xi  ¡.s.t; t1 ; l/ j B.t /P .t //dt  ¡.l j M /:
£

The conjugate value functions are

Z £
Vf .£; l/ D ¡.ljG.£; Ÿ/B.Ÿ/P .Ÿ//d Ÿ C ¡.l j G.£; t0 /X 0 / C I .ljG.£; t0 /B .0//;
t0
Z (2.79)
t1
Vb .£; l/ D ¡.l j G.£; Ÿ/B.Ÿ/P .Ÿ//d ŸC¡.l j G.£; t1 /M /CI .ljG.£; t1 /B .0//:
£
(2.80)
Here, as in the above, G.£; Ÿ/ is defined by (1.11), (1.12). The forward and backward
reach sets may now be described as follows.
Theorem 2.4.5. The support functions for the forward and backward reach sets are
Z £
¡.l j X Œ£/ D ¡.l j X .£; t0 ; X 0 // D ¡.l j G.£; Ÿ/B.Ÿ/P .Ÿ//d Ÿ C ¡.l j G.£; t0 /X 0 /;
t0
Z (2.81)
t1
¡.l j W Œ£/ D ¡.l j W .£; t1 ; M // D ¡.l j G.£; Ÿ/B.Ÿ/P .Ÿ//d Ÿ C ¡.l j G.£; t1 /M /:
£
(2.82)
70 2 The Dynamic Programming Approach

Exercise 2.4.2. Prove Theorems 2.4.4, 2.4.5.

Figure 2.2 illustrates the four-dimensional value function of type V0 .t; x/,
(see (2.46)), for system

xP 1 D x3 ; xP 3 D .k1 C k2 /x1 C k2 x3 ;

xP 2 D x4 ; xP 4 D k1 x1  .k1 C k2 /x3 C u;

k1 ; k2 > 0; under bounded control juj   through its two-dimensional projections

on subspaces Lfx1 ; x2 g and Lfx3 ; x4 g. Shaded are the projections of related level
sets.
Having described the forward and backward reachable tubes, we shall now
analyze the notion of colliding tubes, then use it to calculate all the points of the
reachable sets.

2.5 Colliding Tubes: Calculating All Reachable Points

In this section we introduce the Principle of Colliding Tubes investigating the

interaction between the forward reach tube emanating from a starting point with
the backward reach tube emanating from a terminal point. Using this principle,
one may calculate optimal trajectories from starting point to terminal point without
calculating the controls.

Colliding Tubes

Consider two set-valued positions of system (2.25): the starting position ft0 ; X 0 g
and the terminal position ft1 ; M g with t0  t1 . For each of these one may specify a
reach tube—in forward and backward time, respectively:

X Œt  D X .t; t0 ; X 0 /; and W Œt  D W .t; t1 ; M /; t 2 Œt0 ; t1 :

Lemma 2.5.1. Suppose at any time t D t  the intersection X Œt   \ W Œt   6D ;.

Then

X Œt  \ W Œt  6D ;; 8t 2 Œt0 ; t1 :

The proof of this statement follows from the definitions of forward and backward
reach sets. This yields the next conclusion.
2.5 Colliding Tubes: Calculating All Reachable Points 71

Fig. 2.2 Two-dimensional projections of the four-dimensional value function with their level sets
72 2 The Dynamic Programming Approach

Corollary 2.5.1. The tube

Z Œt  D X Œt  \ W Œt ; t 2 Œt0 ; t1 ;

consists of all trajectories of system (2.1) that connect the set X 0 at t D t0 to the set
M at t D t1 : Here

Z Œt  D fx W V0 .t; x/  0; V .t; x/  0g D X Œt  \ W Œt ;

for all t 2 Œt0 ; t1 ; V0 .t; x/ is the value function for the forward reach set (2.46) and
V .t; x/ is the value function for the backward reach set (2.55).
In particular Z Œt0  D X 0 \ W Œt0 ; Z Œt1  D X Œt1  \ M : Note that the knowledge of
tube Z ./ does not require any calculation of the corresponding controls.
The problem of calculating all reachable points may be separated into two
classes, namely the one of calculating the boundary points of the reach set X Œt1  D
X .t1 ; t0 ; x 0 / and that of calculating its interior points.
The first class of problems is easily solved in the following way. Suppose x 1 is
a point on the boundary of X Œt1 . The corresponding support vector (or vectors) l 
are given by

l  2 fl W ¡.l j X Œt1 / D hl; x 1 ig D L (2.83)

Taking any l  2 L and applying the maximum principle in its simplest form (see
Sect. 1.4, (1.46)), we observe that the desired control u .t / satisfies the relation

h§ .t /; B.t /u .t /i D ¡.§ .t / j B.t /P .t // D maxfh§ .t /; B.t /ui j u 2 P .t /g

(2.84)
for almost all t 2 Œt0 ; t1 . Here § .t / is the solution of the adjoint equation (1.47)
(1.61), with § .t1 / D l  .3
Lemma 2.5.2. The control u .t / that steers the trajectory x.t / from point x 0 D
x.t0 / to given point x 1 D x.t1 / on the boundary of reach set X Œt1  satisfies the
maximum principle (2.84), in which § Œt  is the solution to Eq. (1.47) ((1.62)) with
boundary condition § .t1 / D l  , and vector l  given by (2.83).
Note that u .t / will be defined for almost all t 2 Œt0 ; ª provided that each
coordinate function hi .t / of the row h.t / D .§ .t //0 B.t / is nonzero almost
everywhere. This property is ensured by strong controllability of the system under
consideration (see Sect. 1.3.3).

3
With additional information on P .t / (see Remark 1.5.3), in degenerate cases the control u .t /
may be written down in more detail.
2.5 Colliding Tubes: Calculating All Reachable Points 73

Suppose now that x 1 is an interior point of X Œt1  (x 1 2 intX Œt1  D

intX .t1 ; t0 ; x 0 // and we are to find the control that steers x.t / from x.t0 / D x 0
to x.t1 / D x 1 . We may then proceed as follows.
First find the forward reach sets X Œ£ D X .£; t0 ; x 0 / from point x 0 and the
backward reach sets W Œ£ D W .£; t1 ; x 1 / from point x 1 . This may be done, for
example, by applying formulas (2.81) and (2.82). The latter yield two relations:
Z £
¡.l j X Œ£/ D hl; G.£; t0 /x 0 i C ¡.l j G.£; s/B.s/P .s//ds; (2.85)
t0

and
Z t1
¡.l j W Œ£/ D hl; G.£; t1 /x i C 1
¡.l j G.£; s/B.s/P .s//ds; (2.86)
£

where tube X Œ£ develops from X Œt0  D x 0 with £ increasing and tube W Œ£
develops from W Œt1  D x 1 , with £ decreasing.
We thus have two intersecting tubes (X Œ£ \ W Œ£ 6D ;; 8£ 2 Œt0 ; ª), heading
towards each other in opposite directions. If x 0 lies on the boundary of W Œt0 , then
we arrive at the situation described in the previous case, Lemma 2.5.2, with x 0 and
x 1 interchanged and problem solved in backward time. Hence we further assume
x 0 2 intW Œt0 .
Then we have x 1 2 intX Œt1  and x 0 2 intW Œt0  and there exists an instant £ 2
Œt0 ; t1  when the boundaries of the two tubes collide. Namely, developing W Œt  from
t1 backwards towards t0 , we first have W Œt   X Œt  till we reach instant t D £ ,
when W Œt  bumps into the boundary of X Œt , touching it from inside at point x  , so
that

W Œ£  X Œ£ ; and 9l  W hl  ; x  i D ¡.l  jX Œ£ / D ¡.l  jW Œ£ /: (2.87)

Let us find this instant £ . With W Œ£  X Œ£; we have

k.£; l; x 0 ; x 1 / D ¡.ljX Œ£/  ¡.ljW Œ£/ > 0;

with

k.£; l; x 0 ; x 1 /
Z £
D hl; G.£; t0 /x 0  G.£; t1 /x 1 i C ¡.l j G.£; s/B.s/P .s/ds
t0
Z t1
 ¡.l j G.£; s/B.s/P .s//ds;
£
74 2 The Dynamic Programming Approach

and

”.£; x 0 ; x 1 / D minfk.£; l; x 0 ; x 1 / j hl; li D 1g > 0: (2.88)

l

With x 0 ; x 1 fixed, instant £ will then be the largest root of equation ”.£; x 0 ; x 1 / D
0—the first time when W Œ£ touches X Œ£ from inside.
Function ”.£; x 0 ; x 1 / is defined for £ 2 Œt0 ; t1 , with ”.t0 ; x 0 ; x 1 / <
0; ”.t1 ; x 0 ; x 1 / > 0, and it is continuous in £. Therefore, point £ , where
”.£ ; x 0 ; x 1 / D 0 exists. The maximizer l  of (2.88) is the support vector to X Œ£ 
at touching point, which is precisely the vector x  D x.£ /, that ensures (2.87).
Lemma 2.5.3. The first instant £ of collision for W Œ£ X Œ£ with boundary of
tube X Œ£ from inside is the largest root of equation ”.£ ; x 0 ; x 1 / D 0. The point x 
of such collision satisfies the maximum rule

maxfhl  ; xijx 2 X Σ g D hl  ; x  i D maxfhl  ; xijx 2 W Σ g:

The calculation of the control u .t / which steers x.t / from x.t0 / D x 0 to x.t1 / D
1
x is now reduced to the class of points considered in Lemma 2.5.2, namely the
points that lie on the boundaries of the respective reach sets.
Here we first specify the control u .t / for interval t 2 Œt0 ; £ , working with
point x  , which lies on the boundary of set X Σ . Then u .t / satisfies the maximum
principle

hl  ; G.£ ; t /B.t /u .t /i D maxfhl  ; G.£ ; t /B.t /uiju 2 P .t /g

D ¡.l  j G.£ ; t /B.t /P .t //; t  £ : (2.89)

Applying similar reasoning to the interval Œ£ ; t1 ; with “starting point” x 1 D x.t1 /
and x  .£ / being on the boundary of reach set W Œ£  with same support vector
l  , we may calculate u .t / for t 2 Σ ; t1 . This leads again to relation (maximum
principle) (2.89), but now taken for t > £ .
Theorem 2.5.1. The control u .t / that steers the trajectory x.t / from x.t0 / D x 0
to x.t1 / D x 1 satisfies the maximum principle relation (2.89), where l  is the
minimizer of problem (2.88), with £ D £ being the largest root of equation
”.£; x 0 ; x 1 / D 0:
Here Fig. 2.3 illustrates the idea of colliding tubes.4 Namely, given in green is
the backward reach tube from the terminal point x m at target set M (marked in red).
Given in blue is the forward reach tube emanating from the starting point x 0 located
in set X 0 : The intersection of these two tubes is the union of all system trajectories
that connect the starting point with the terminal point.

4
This example is animated in the toolbox [132].
2.6 The Closed-Loop Target Control 75

Fig. 2.3 Colliding tubes for control “from point to point”

The original equations for this picture are (x 2 R2 ):

       
0 1 0 3 2
xP D Ax C Bu; A D ; BD ; x0 D ; xm D B D :
2 0 1 1 0

with 2  u  2; t 2 Œ0; 5: The time of switching from one tube to the other is
£ D 0:886:
We shall now indicate how to construct feedback strategies for target control by
using backward reach sets.

2.6 The Closed-Loop Target Control

Here we introduce a scheme for finding the closed-loop control for minimizing at
given time the distance from a given target set. This would require to know the
backward reach set. Once this set is known, there would be no need of integrating
the HJB equation which produces it as a level set. The problem will be therefore
shifted to computation of these level sets without solving the HJB equation. Such
computation could be done through ellipsoidal methods, as explained in the next
chapter.
Consider system (1.2) with target set M . Let W Œ£ D W .£; ª; M / be its
backward reach set (the solvability set) from position fª; M g.
76 2 The Dynamic Programming Approach

Problem 2.6.1. In the class of set-valued feedback strategies UC D fU.t; x/g find
feedback strategy U  .t; x/ that ensures the inclusion xλ 2 M , whatever be the
starting position f£; xg; x 2 W Œ£.
From the definition of W Σ it follows that the problem of reaching target set
W Œª D M from position f£; xg is solvable if and only if x 2 W Œ£.
We now construct a solution strategy U  .t; x/ for Problem 2.6.1. Let

VC .t; x/ D d.G.ª; t /x; G.ª; t /W Œt /

and let d VC .t; x/=dt ju be the total derivative of VC .t; x/ at position ft; xg, along
the trajectory of Eq. (1.2) under control u.
The solution strategy is defined as
8  ˇ 
< arg min d V .t; x/=dt j ˇˇ u 2 P .t / ; if V .t; x/ > 0;
u ˇ
U  .t; x/ D
C C
(2.90)
:
P .t /; if VC .t; x/ D 0:

Observe that the set-valued function U  .t; x/ is upper semicontinuous in ft; xg

with respect to inclusion. This ensures the existence and extendability of solution to
the differential inclusion

xP 2 A.t /x C B.t /U  .t; x/; (2.91)

from x.£/ D x 2 W Œ£: The next fact turns out to be true.

Lemma 2.6.1. With U  .t; x/ selected according to (2.90) we have

d VC .t; x/ ˇˇ
ˇ  0;
dt u

for all u 2 U  .t; x/.

It now remains to verify that u may indeed be selected so as to ensure the derivative
d VC .t; x  Œt /=dt ju is non-positive within t 2 Œ£; ª. Calculate

@VC .t; x/ @VC .t; x/
d VC .t; x/=dt ju D C ; A.t /x C B.t /u (2.92)
@t @x

in which

VC .t; x/ D maxf0; maxfhG 0 .ª; t /l; xi  ¡.G 0 .ª; t /l j W Œt /jhl; li  1gg:

Note that we need to ensure d VC .t; x/=dt ju  0 only where VC .t; x/ > 0: Now
suppose VC .t; x/ > 0:
2.6 The Closed-Loop Target Control 77

Then let

l 0 .t; x/ D arg maxfhl; G.ª; t /xi  ¡.l j G.ª; t /W Œt / j hl; li  1g: (2.93)

Under the constraint hl; li  1 the vector l 0 D l 0 .t; x/ is unique. Applying the rules
for differentiating a “maximum function,” [61], we further have

@VC .t; x/ @¡.l 0 j G.ª; t /W Œt /

D hl 0 ; G.ª; t /A.t /xi  ;
@t @t

@VC .t; x/
D G 0 .ª; t /l 0 :
@x
Since
Z ª
xP D A.t /x C B.t /u; W Œt  D G.t; ª/M  G.t; s/B.s/P .s/ds;
t

then from (2.82), (2.80) we have

Z ª
¡.l j G.ª; t /W Œt / D ¡.l j M / C
0
¡.l 0 j G.ª; s/B.s/P .s//ds;
t

so that

@¡.l 0 j G.ª; t /W Œt /
D ¡.l 0 j G.ª; t /B.t /P .t //:
@t
Substituting the results into Eq. (2.92) we finally have

d VC .t; x/=dt ju D ¡.l 0 j G.ª; t /B.t /P .t // C hl 0 ; G.ª; t /B.t /ui:

Taking

U  .t; x/ D arg maxfhl 0 ; G.ª; t /B.t /ui j u 2 P .t /g; (2.94)

we get

d VC .t; x/=dt ju D 0; u 2 U  .t; x/:

To prove that U  .t; x/ is a solution to Problem 2.6.1, first integrate d VC .t; x/=dt
from £ to ª along the trajectories x  Œt  of (2.91) emanating from x  .£/ D x. Then,
with U  .t; x/ selected according to (2.90), one gets
Z ª
d VC .t; x  Œt / D VC .ª; x  Œª/  VC .£; x/  0;
£
78 2 The Dynamic Programming Approach

so VC .ª; x  Œª/  VC .£; x/, which implies that once VC .£; x/  0 (which means
x  Œ£ D x 2 W Œ£), one will have VC .ª; x  Œª/  0 (which means x  Œª 2
W λ D M ).
Note that under VC .t; x.t //  0, with control selected due to (2.90), the
trajectory x.t / never leaves the tube W Œt . So if, say VC .t; x.t // D 0, but at some
point t 0 D t C ¢ we have

VC .t; x.t 0 // D d.G.ª; t 0 /x.t 0 /; G.ª; t 0 /W Œt 0 / > 0;

then, due to the directional differentiability of function VC .t; x.t //, we would have
at some point t 00 2 .t; t 0  that the derivative d VC .t 00 /=dt > 0; which is not possible
due to (2.90).
Theorem 2.6.1. The closed-loop strategy U  .t; x/ which solves Problem 2.6.1 is
defined by (2.90), (2.94), in which l 0 D l 0 .t; x/ is the optimizer of (2.93) and W Œt 
is the backward reach tube from position fª; M g.
Remark 2.6.1. A strategy U  .t; x/ given by (2.94), (2.93), is said to be produced
by the extremal aiming rule (see [121]).
Exercise 2.6.1. Find the closed-loop strategy that steers point x 2 W Σ to a given
point x  2 M (provided x; x  are such that the problem is solvable).

2.7 Reachability Within an Interval

The problem of reachability may be formulated as a request to know what states

may be reached within a time interval rather than at fixed time.
Thus we consider a generalization of Problem 2.6.1.
Problem 2.7.1. Find the set WŒ£; ‚ D W.£; ‚; M / of all points for which there
exists a control u .t /; which ensures the inclusion xŒª D x.ª; £; x/ 2 M , for some
ª 2 ‚ D Œ’; “; “  ’  £.
This is the backward reach set relative to f‚; M g: Unlike Problem 2.6.1 where
the terminal time is fixed at ª, in Problem 2.7.1 the terminal time can be any instant
within the interval ‚ D Œ’; “:
The set WŒ£; ‚ is weakly invariant relative to f‚; M g, namely, such that for
each point x 2 WŒ£; ‚ there exists at least one trajectory that reaches set M at
some time ª 2 ‚:
Remark 2.7.1. Note that the maximal backward reach sets WŒ£; ‚ are the same
whether calculated in the class of open-loop or closed-loop controls.
Problem 2.7.2. In the class of set-valued feedback strategies UC D fU.t; x/g find
feedback strategy U  .t; x/ which ensures the inclusion xŒª 2 M for some ª 2 ‚,
whatever be the starting position f£; xg; x 2 WŒ£; ‚.
2.7 Reachability Within an Interval 79

Problem 2.7.1 may be solved by calculating the set WŒ£; ‚. Clearly,
[
WŒ£; ‚ D fW .£; ª; M / j ª 2 ‚g: (2.95)

Let

V.£; x/ D minfV .£; x j ª; V .ª; // j ª 2 ‚g; (2.96)

ª

with V .£; x j ª; V .ª; // taken from (2.55). That is,

V.£; x/ D

D min minfd 2 .xŒª; M / j xŒ£ D x; ª 2 ‚g D min minfd 2 .xŒª; M / j xŒ£ D x; ª 2 ‚g:

u ª ª u
(2.97)

Lemma 2.7.1.

WŒ£; ‚ D fx W V.£; x/  0g: (2.98)

Returning to formula (2.55), introduce the notation

V .£; ª; x/ D minfd 2 .xŒª; M / j xŒ£ D xg:

u

Hence

V.£; x/ D minfV .£; ª; x/ j ª 2 ‚g D V .£; ª0 ; x/;

ª

where ª0 D ª0 .£; x/: The solution to Problem 2.7.2 is now reduced to Problem 2.6.1
with ª D ª0 :
Exercise 2.7.1. Solve Problem 2.7.2 in detail, by applying the scheme of Sect. 2.6
to WŒ£; ‚:
The notion of strong invariance is applicable also to reachability within an
interval.
Definition 2.7.1. A set Ws Œ£; ‚ is strongly invariant relative to set M within
interval ‚, if for each of its points x the entire reach set X .ª; £; x/ D X Œª M
for some ª 2 ‚:
If Ws Œ£; ‚ contains all such points, then it is said to be the maximal strongly
invariant set relative to M , within interval ‚.
80 2 The Dynamic Programming Approach

Define value function

Vs .£; x/ D minfVs .£; ª; x/ j ª 2 ‚g; (2.99)

ª

Vs .£; ª; x/ D maxfd.x.ª/; M / j u./ 2 P ./; x.£/ D xg:

u

Lemma 2.7.2. Ws Œ£; ‚ is the level set

Ws Œ£; ‚ D fx W Vs .£; x/  0g: (2.100)

Denoting Ws .£; ª; M / D fx W Vs .£; ª; x/  0g, we have

[
Ws Œ£; ‚ D fWs .£; ª; M / j ª 2 ‚g:

Following (2.63), we also have (for ª fixed)

 
@Vs .t; ª; x/ @Vs .t; ª; x/
C max ; A.t /x C B.t /u D 0; (2.101)
@t u @x

with boundary condition

d.x; M / D Vs .ª; ª; x/: (2.102)

Let us now find the support function ¡.l j Ws .£; ª; M //: Following the scheme of
calculating (2.78), (2.80), but with minu substituted for maxu , we get

Vs .£; ª; x/ D maxfˆs Œ£; ª; x; l j hl; li  1g; (2.103)

l

with
Z ª
ˆs Œ£; ª; x; l D hs.£; ª; l/; xi C ¡.s.t; ª; l/ j B.t /P .t //dt  ¡.l j M /:
£

Hence

Ws .£; ª; M / D fx W ˆs Œ£; ª; x; l  0g

for all hl; li  1 and therefore, since the function ˆs Œ£; ª; x; l is homogeneous in
l, also for all l 2 Rn :
After a substitution similar to (2.71) we observe that 8x 2 Ws .£; ª; M /,
Z ª
hl; xi  ¡.l j G.t; ª/M /  ¡.l j G.t; Ÿ/B.Ÿ/P .Ÿ//d Ÿ D k.t; ª; l/: (2.104)
t

This yields the next result.

2.7 Reachability Within an Interval 81

Lemma 2.7.3. If k.t; ª; l/ is convex in l, one has

¡.l j Ws .£; ª; M // D k.t; ª; l/;

while in the absence of convexity in l one has

¡.l j Ws .£; ª; M // D k  .t; ª; l/ (2.105)

where the second conjugate is taken in the variable l.

This lemma follows from a well-known result in convex analysis on lower
envelopes of convex functions (see [237, 238]). Namely, the second Fenchel
conjugate k  .t; ª; l/ is the lower convex envelope of k.t; ª; l/ in the variable l.
This means epigraph epi kl in the variable l is the convex hull of the set epi kl .
Recall that the epigraph of a function k.l/ is the set of points lying “above” the
graph of k:

epi k WD ffl; ’g 2 Rn  R W ’  k.l/g:

Theorem 2.7.1. The value function Vs .£; x/, (2.99), that determines Ws Œ£; ‚
through (2.100) is given by (2.101)–(2.103).
The next item is reachability within an interval (see Remark 1.4.2).
Definition 2.7.2. The reach set XŒ‚ D X.‚; t0 ; X 0 / of system (1.44) under
constraint u.s/ 2 P .s/; s  t0 ; within given time-interval ‚ D Œ’; “; “ 
’  t0 , from set-valued position ft0 ; X 0 g, is the set of all points x, for each of
which there exists a trajectory xŒs D x.s; t0 ; x 0 /, for some x 0 2 X 0 ; generated
by some control subjected to the given constraint, that transfers the system from
position ft0 ; x 0 g to position fª; xg; x D xŒª, for some ª 2 ‚:

XŒ‚ D X.‚; t0 ; X 0 / D fx W 9 x 0 2 X 0 ; u./ 2 P ./; ª 2 ‚; x D x.ªI t0 ; x 0 /g:

Problem 2.7.3. Find the reach set XŒ‚:

Denote

V0 .‚; x/ D minfV0 .ª; x j t0 ; V0 .t0 ; // j ª 2 ‚g; (2.106)

where V0 .ª; x j t0 ; V0 .t0 ; // is taken from (2.48).

That is,

V0 .‚; x/ D minfminfd 2 .xŒ£; X 0 / j x D xŒª; ª 2 ‚gg D (2.107)

u ª

D minfminfd 2 .xŒ£; X 0 / j x D xŒªg j ª 2 ‚g:

ª u
82 2 The Dynamic Programming Approach

Lemma 2.7.4. The reach set XŒ‚ within a given time interval ‚ is the level set

XŒ‚ D fx W V0 .‚; x/  0g: (2.108)

The exact calculation of sets XŒ‚; WŒ£; ‚ may be achieved through formu-
las (2.106), (2.96) applied to functions V0 .£; x/; V .£; x/ given in (2.68), (2.77).

Reachability from a Tube

Consider a tube f‚0 ; X 0 ./g D ft; x W t 2 ‚0 D Œt 0 ; t 00 ; x 2 X 0 .t /g where compact

set-valued function X 0 .t / is Hausdorff-continuous on ‚0 :
Definition 2.7.3. The reachability set XŒ‚; ‚0  D X .‚; ‚0 ; X 0 .//; ‚ \ ‚0 ¤ ¥,
from tube f‚0 ; X 0 ./g within given time-interval ‚ is the union
[
XŒ‚; ‚0  D fx.ª; t0 ; x 0 / j ª 2 ‚; t0 2 ‚0 ; x 0 2 X 0 .t0 /g:

Problem 2.7.4. Find the reach set XŒ‚; ‚0 :

Taking

V0 .‚; ‚0 ; x/ D minfminfd 2 .xŒt0 ; X 0 .t0 // j xŒª D x; ª 2 ‚; t0 2 ‚0 gg D

u ª;t0
(2.109)
D minfminfd 2 .xŒt0 ; X 0 Œt0 / j xŒª D xg j ª 2 ‚; t0 2 ‚0 gg
ª;t0 u

we come to the next relation.

Lemma 2.7.5. The reach set from given tube ‚0 within given interval ‚ is the level
set

XŒ‚; ‚0  D fx W V0 .‚; ‚0 ; x/  0g: (2.110)

Sets WŒ£; ‚; XŒ‚ of course need not be convex.

The Total Reachability Sets

Definition 2.7.4. The total backward reachability set from position fª; M g is the
union
[
W Œª D W .ª; M / D fW .£; ª; M / j £ 2 .1; ªg:
2.8 Dynamic Programming: Time-Optimal Control 83

This is the set of all points from which set M may be reached at time ª from
some finite £ 2 .1; ª. For an autonomous system the set W Œª does not depend
on ª.
Denote V .ª; x/ D inf£ fV .£; xjª; V .ª; // j £ 2 .1; ªg; V .ª; x/ D
d 2 .x; M /.
Lemma 2.7.6. The total backward reachability set W λ is the level set

W Œª D fx W V .ª; x/  0g:

Definition 2.7.5. The total forward reachability set XC Œt0  from position ft0 ; X 0 g
is the union
[
XC Œt0  D XC .t0 ; X 0 / D fX .ª; t0 ; X 0 / j ª 2 Œt0 ; 1/g:

This is the set of all points that can be reached at some time ª  t0 . For an
autonomous system the set XC Œt0  does not depend on t0 .
Denote VC .t0 ; x/ D inffV0 .ª; x jt0 ; V0 .t0 ; // j ª 2 Œt0 ; 1/g; where V0 .t0 ; x/ D
d 2 .x; X 0 /:
Lemma 2.7.7. The total reachability set XC Œt0  is the level set

XC Œt0  D fx W VC .t0 ; x/  0g:

Exercise 2.7.2. For a linear autonomous system (1.44) with u.t / 2 P D

const; 0 2 intP and X 0 6D f0g given, describe conditions when XC D XC Œ0 D
Rn and when XC ¤ Rn .
Exercise 2.7.3. Suppose in Definition 2.7.2 we have ‚ D Œt0 ; ¢. Find a direct
HJB-type relation for V0 .¢; x/ D V0 .‚; x/ without reducing it to (2.106).

2.8 Dynamic Programming: Time-Optimal Control

We shall now use the notion of reachability to solve the problem of closed-loop
time-optimal control.
Problem 2.8.1. Given system (2.25), hard bound u 2 P , starting position ft; xg and
target set M ; x … M , find

£0 .t; x/ D minf£ W x.£/ 2 M g: (2.111)

Here target set M is taken similar to the above, being convex and compact.
In particular, one may have M D fmg as an isolated point.
84 2 The Dynamic Programming Approach

The solution scheme is as follows: for the given starting position ft; xg; we
construct the reachability tube X Œ£ D X .£; t; x/; £  t; then find the first instant
of time £ .t; x/ when X Œ£ touches M . The value £ .t; x/  t will be the minimal
time for reaching set M from position ft; xg, namely, £ .t; x/ D £0 .t; x/.
To calculate X Σ, recall problem (2.46), (2.47). Then

X Œ£ D fz W V0 .£; z/  0g;

where function

V0 .£; z/ D V0 .£; zjt; x/ D minfd 2 .x.t /; x/ j x.£/ D zg

u

satisfies the HJB equation (2.50) with boundary condition

V0 .t; z/ D d 2 .z; x/:

Let us now look how to find the first instant of time, when X Œt  touches M .
From Sect. 2.3 it follows that for inclusion m 2 X Σ to be true it is necessary and
sufficient that V0 .£; m/ D 0. Now, having noted that V0 .t; m/ > 0, solve equation
V0 .£; m/ D 0 and find the smallest root £0 > t of this equation. The number £0
exists if point m is reachable in finite time.
Suppose set M D fmg; then £0 D £0 .t; x/ is already the solution to
Problem 2.8.1. If not, consider equation

minfV0 .£; m/ j m 2 M g D f .£/ D 0; (2.112)

Here f .£/ D f .£ j t; x/ > 0 and f .£/ D 0 for some £ > 0, provided set M is
reachable in finite time.5
Taking the smallest root £0 .t; x/ of equation

f .£ j t; x/ D 0; (2.113)

we come to the solution of Problem 2.8.1.

Theorem 2.8.1. The solution of Problem 2.8.1 is the smallest root £0 .t; x/ of
Eq. (2.113).
In order to better understand Eq. (2.113) we look at a simple example.

5
A closed set Q is said to be reachable in finite time if the intersection Q \ XC .t; x/ 6D ; for
some £ > t. Here XC .t; x/ is the total (forward) reachability set from position ft; xg:
2.8 Dynamic Programming: Time-Optimal Control 85

Example 8.1

Consider the one-dimensional system

xP D u C f .t /; juj  ;

with target set M D m D f0g:

The value function V0 .t; z/ for this case, calculated according to (2.77), is as
follows:

V0 .t; z/ D minfd.x.0/; x/ j x.t / D zg D

u

Z t Z t
D maxfl.z  x/   jljds  lf .s/ds j jlj  1g:
l 0 0

This gives:
 Rt Rt
z  x  t  0 f .s/ds if z > x C 0 f .s/ds;
V0 .t; z/ D Rt Rt
z C x  t C 0 f .s/ds if z < x C 0 f .s/ds:

Fig. 2.4 Discontinuity of optimal time £0 in x

86 2 The Dynamic Programming Approach

Now, take for x > 0

8
< 0 if 0  t  x=;
f .t / D . C –/ if 1 C x= > t > x=; – > 0;
:
r if t  1 C x=; r > 0:

Then one may notice that the minimal time for reaching m D 0 is £0 .t; x/ D
x=. However, with starting position ft; x  g instead of ft; xg, where x  D x C • >
x; • > 0; the minimal time will be £0 .t; x  / D £0 .t; x/ C 1 C .– C •/=.r C /.
Hence, function £0 .t; x/ is discontinuous in x. This example yields the next remark.
Remark 2.8.1. In general the minimal time £0 .t; x/ is not robust relative to the
starting point x.
Exercise 2.8.1. Indicate additional conditions for Problem 2.8.1 that ensure conti-
nuity of function £0 .t; x/ in ft; xg (Fig. 2.4).
Chapter 3
Ellipsoidal Techniques: Reachability
and Control Synthesis

Abstract This chapter describes the ellipsoidal techniques for control problems
introduced in earlier chapters. We derive formulas for reachability sets using the
properties of ellipsoids and relations from convex analysis. The formulas are derived
through inductive procedures. They allow calculation of both external and internal
ellipsoidal approximations of forward and backward reachability sets with any
desired level of accuracy. The approximations are illustrated on examples explained
in detail, then followed by ellipsoid-based formulas for problems of reachability and
control synthesis.

Keywords Ellipsoidal approximations • Parametrization • Parallel calculation •

Ellipsoidal maximum principle • Reachability • Control synthesis

In this chapter we begin to apply ellipsoidal techniques to problems of control. We

consider systems that operate in the absence of uncertainty. We propose methods
to represent trajectory tubes for controlled systems through parameterized varieties
of ellipsoidal-valued tubes which achieve external and internal approximation of
set-valued solution functions with any desired accuracy. This is done by using
intersections and unions of a finite number of ellipsoidal tubes. Then, by appropri-
ately tending the number of approximating ellipsoids to infinity, one may approach
the exact solutions. We describe the construction of forward and backward set-
valued trajectories of reach sets (with these being taken either at given instants of
time or calculated for preassigned time-intervals). The routes for constructing the
external and internal ellipsoidal approximations employ algorithms that are based
on recurrent procedures. They do not require to be calculated “afresh” for any new
instant of time. The structure of the suggested solutions also allows their natural
parallelization. The results of Sects. 3.1–3.11 of this chapter are based on those
given in [177, 181, 182]. On the other hand, for the problem of internal ellipsoidal
approximations there exists a “second approach” which introduces other formulas
than in Sects. 3.7–3.10. A concise review of this approach is given in Sect. 3.12.
The “second approach” may also be recommended as a solution tool for problems
of reachability and control synthesis. This approach was discussed in [45, 174].

© Springer International Publishing Switzerland 2014 87

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__3
88 3 Ellipsoidal Techniques: Reachability and Control Synthesis

We thus start to introduce an array of methods for treating trajectory tubes rather
than isolated trajectories. Of special importance are methods that permit systems of
high dimensions in limited time. Such types of examples are presented in Chap. 4,
being solved with the aid of the ellipsoidal toolbox, [132].

3.1 Linear Systems Under Ellipsoidal Constraints

Consider the linear time-variant system model (1.1) described as

P / D A.t /x.t / C B.t /u C f .t /:

x.t (3.1)

The values u of the controls are restricted for almost all t by hard bounds

u 2 Q .t /; (3.2)

where Q .t / is a set-valued function Q W T ! convRq , Hausdorff-continuous in t ,

f .t / is a given integrable function.
Considering all the possible functions of time u.t /; t 2 T ,—the open-loop
controls—restricted by constraint (3.2), we came to a linear differential inclusion

P / 2 A.t /x.t / C B.t /Q .t / C f .t /

x.t (3.3)

under condition

x .0/ D x.t0 / 2 X 0 ;

where X 0 2 convRn .
In this chapter we further restrict the constraints on u; x 0 to be ellipsoidal-valued,
namely:

< u  q.t /; Q1 .t /.u  q.t // > 1; (3.4)

and

< x .0/  x 0 ; .X 0 /1 .x .0/  x 0 / > 1; (3.5)

where continuous functions q.t / and vector x 0 are given together with continuous,
positive, symmetric matrix-valued function Q0 .t / D Q.t / > 0 and matrix .X 0 /0 D
X 0 > 0. Here the respective ellipsoids are nondegenerate.
In terms of inclusions we have

u 2 E .q.t /; Q.t //; (3.6)

x.t0 / D x .0/ 2 E .x 0 ; X 0 /; (3.7)

3.1 Linear Systems Under Ellipsoidal Constraints 89

or, in terms of support functions, the inequalities

1
< l; u >< l; q.t / > C < l; Q.t /l > 2 ; (3.8)
1
< l; x .0/ >< l; x 0 > C < l; X 0 l > 2 ; (3.9)

8l 2 Rn :

Remark 3.1.1. Note that constraints in the form (3.8), (3.9) also allow degenerate
matrices Q.t /  0; X 0  0. In this case ellipsoids E .q.t /; Q.t // or E .x 0 ; X 0 / turn
out to be elliptical cylinders. This case will be discussed separately in Chap. 4.
Let us now start with the reachability issue described in Chap. 1, Sect. 1.4. Note
that we further take q.t / 6D 0 and presume f .t /  0, since keeping it here does not
add much to the procedure. The case f .t / 6D 0 will be important in Chap. 10, while
considering systems under uncertainty.
The reach set X Œ£ may be treated as the cut X Œ£ D X .£; t0 ; E .x 0 ; X 0 // of the
solution tube X ./ D fX Œt  W t  t0 g to the differential inclusion .t  t0 /;

xP 2 A.t /x C E .B.t /q.t /; B.t /Q.t /B 0 .t //; x .0/ 2 E .x 0 ; X 0 /: (3.10)

The reach set X .t; t0 ; E .x 0 ; X 0 // may also be presented through a set-valued

(“Aumann”) integral (see [9]).
Lemma 3.1.1. The following relation is true
Z t
X .t; t0 ; E .x 0 ; X 0 // D x ? .t / C G.t; t0 /E .0; X 0 / C G.t; s/E .0; B.s/Q.s/B 0 .s//ds;
t0
(3.11)
where
Z t
x .t / D G.t; t0 /x C
? 0
G.t; s/B.s/q.s/ds: (3.12)
t0

A standard direct calculation indicates

Lemma 3.1.2. The support function

¡.ljX .t; t0 ; E .x 0 ; X 0 // D (3.13)

< l; x ? .t / > C < l; G.t; t0 /X 0 G 0 .t; t0 /l >1=2 C

Z t
C < l; G.t; s/B.s/Q.s/B 0 .s/G 0 .t; s/l >1=2 ds:
t0

A direct consequence of the last representation leads to the next fact.

90 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Lemma 3.1.3. The reach set X Œt  D X .t; t0 ; E .x 0 ; X 0 // is a convex compact set in

Rn that evolves continuously in t .
The continuity in time of a convex compact set X Œt  is understood here as the
continuity in time of the support function ¡.ljX Œt / uniformly in l W< l; l > 1.
For an ellipsoidal-valued function E .q.t /; Q.t // continuity means that the center
q.t / and “shape matrix” Q.t / are continuous.
An obvious consequence of Corollary 1.3.1 is the next property.
Lemma 3.1.4. System (3.1) with f .t / D 0; is completely controllable iff the
quadratic form
Z £
WQ .£; ¢/ D < l; G.£; s/B.s/Q.s/B 0 .s/G 0 .£; s/l > ds; l 2 Rn ;
¢

is positive definite for any £ > ¢, provided Q.s/ D Q0 .s/ > 0:

The boundary @X Σ of set X Σ may be here defined as @X Σ D X Σ n i nt X Σ.
Under the controllability condition of the above lemma set X Σ has a non-void
interior i nt X Œ£ 6D ; for £ > t0 .
We further assume the following property.
Assumption 3.1.1. The symmetric matrix WQ .£; ¢/ is positive definite whenever
£ > ¢.
Points on the boundary of the reach set X Œt  satisfy some important properties.
Namely, consider point x  2 @X Σ. Then there exists a related support vector l 
such that

< l  ; x  >D ¡.l  jX Œ£/ D maxf< l  ; x > jx 2 X Œ£g: (3.14)

Denote u D u .t / to be the control which transfers system (3.10) from state

x.t0 / D x .0/ to x.£/ D x  . Then the Maximum Principle of Sect. 2.5 appears as
follows [120, 195, 226].
Theorem 3.1.1. Suppose the state x.£/ D x  is given and x  2 @X Œ£. Then
the control u D u .t / and the initial state x.t0 / D x .0/ ; which yield the unique
trajectory x  .t / that reaches point x  D x  .£/ from point x .0/ D x.t0 /; and ensures
relations (3.14), satisfy the following “maximum principle” for the control:

< l  ; G.£; t /B.t /u .t / >D (3.15)

D maxf< l  ; G.£; t /B.t /u > ju 2 E .q.t /; Q.t //g D

D< l  ; G.£; t /B.t /q.t / > C < l  ; G.£; t /B.t /Q.t /B 0 .t /G 0 .£; t /l  >1=2
3.2 Ellipsoidal Approximation of Reach Sets 91

for all t 2 Œt0 ; £, and the “maximum” (transversality) condition for the initial state:

< l  ; G.£; t0 /x .0/ >D maxf< l  ; x > jx 2 G.£; t0 /E .x 0 ; X 0 /g D (3.16)

D< l  ; G.£; t0 /x 0 > C < l  ; G.£; t0 /X 0 G 0 .£; t0 /l  >1=2 :

Here l  is the support vector for set X Œt  at point x  that satisfies relation (3.14).
The product l 0 G.£; t / D sŒt  D s.t; £; l/ may be presented as the backward solution
to the “adjoint system”

sP D sA.t /; sΣ D l 0 ;

where s is a row-vector.
Remark 3.1.2. The ellipsoidal nature of the constraints on u.t /; x .0/ yield unique-
ness of the optimal control and trajectory. (Prove this fact.)

3.2 Ellipsoidal Approximation of Reach Sets

Observe that although the initial set E .x 0 ; X 0 / and the control set E .q.t /; Q.t //
are ellipsoids, the reach set X Œt  D X .t; t0 ; E .x 0 ; X 0 // will not generally be an
ellipsoid. (Since already the sum of two ellipsoids, E1 C E2 , is generally not an
ellipsoid.)
As indicated in [174], the reach set X Œt  may be approximated both externally
and internally by ellipsoids E and EC , with E X Œt  EC .
Definition 3.2.1. An external approximation EC of a reach set X Œt  is tight if there
exists a vector l 2 Rn such that

¡.˙ljEC / D ¡.˙ljX Œt /:

The last definition is relevant for reach sets which are compact convex bodies,
symmetrical around the center x ? .t / (see formula (3.14) of the above). However it
does not produce a unique ellipsoid. A more precise notion is given through the next
definitions, where tightness is defined within a certain subclass EC of ellipsoids that
does not coincide with the variety of all possible ellipsoids.
Definition 3.2.2. An external approximation EC is tight in the class EC , if for any
ellipsoid E 2 EC , the inclusions X Œt  E EC imply E D EC .
This paper is concerned with external approximations, where class EC D fEC g
is described through the following definition.
92 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Definition 3.2.3. The class EC D fEC g consists of ellipsoids that are of the form
EC Œt  D E .x ? ; XC Œt /, where x ? .t / satisfies the equation1

xP ? D A.t /x ? C B.t /q.t /; x ? .t0 / D x 0 ; t  t0 ; (3.17)

and

XC Œt  D XC .t jpŒ/ D (3.18)
Z t  Z t
D p.s/ds C p0 .t / p 1 .s/G.t; s/B.s/Q.s/B 0 .s/G 0 .t; s/ds
t0 t0

Cp01 .t /G.t; t0 /X 0 G 0 .t; t0 / :

Here X 0 ; Q.s/; s 2 Œt0 ; t  are any positive definite matrices with function Q.s/
continuous, q.t / is any continuous function, p.s/ > 0; s 2 Œt0 ; t ; is any integrable
function and p0 .t / > 0.

In particular, this means that if an ellipsoid E .x ? ; XC /
X Œt  is tight in EC , then

there exists no other ellipsoid of type E .x ; kXC /; k  1 that satisfies the inclusions
?

X Œt   E .x ? ; kXC /  E .x ? ; XC / (ellipsoid E .x ? ; XC / touches set X Œt ).

Definition 3.2.4. We further say that external ellipsoids are tight if they are tight
in EC .
Functions p.t / are parameterizing parameters which generate the class EC of
approximating ellipsoids and we actually further deal only with ellipsoids EC 2 EC .
For problems of this book Definition 3.2.1 is motivated by 3.2.4, so we usually apply
Definition 3.2.1, only within the class EC . Then, however, with direction l given,
the related tight ellipsoid will be unique.
The class EC is rich enough to arrange effective approximation schemes, though
it does not include all possible ellipsoids. In fact, all the schemes of books [44, 174]
on external ellipsoids actually do not go beyond the class EC . A justification for
using class EC is due to the next proposition.
Theorem 3.2.1. The following inclusions are true:

X Œt  E .x ? .t /; XC .t jpŒ//; 8p0 ; p.s/ > 0; t0  s  t: (3.19)

Moreover,

X Œt  D \fE .x ? .t /; XC .t j pŒ/jp0 ; p.s/ > 0; s 2 Œt0 ; t g: (3.20)

1
In the following formulas symbol pŒ stands for the pair fp0 ; p.s/; s 2 Œt0 ; t g.
3.2 Ellipsoidal Approximation of Reach Sets 93

Relations (3.19), (3.20) are true for all p0  0 and p.s/ ranging over all
continuous positive functions. These results follow from ellipsoidal calculus (see
[174], Sects. 2.1, 2.7). What also holds is the following characterization of tight
external ellipsoids.
Theorem 3.2.2. For a given time £ the tight external ellipsoids EC Œ£ D
E .x ? .£/; XC Œ£/, with EC Œ£ 2 EC , are those for which the functions p and
p0 are selected as

p.s/ D< l; G.£; s/B.s/Q.s/B 0 .s/G 0 .£; s/l >1=2 ; t0  s  £; (3.21)

p0 .£/ D< l; G.£; t0 /X 0 G 0 .£; t0 /l >1=2 ; (3.22)

with vector l given. Each ellipsoid EC Σ touches X Σ at points

fx  W< l; x  >D ¡.ljX Œ£/g;

so that

¡.ljX Œ£/ D< l; x ? .£/ > C < l; XC Œ£l >1=2 D ¡.ljE .x ? .£/; XC Œ£//:

Remark 3.2.1. Note that the result above requires the evaluation of the integrals
in (3.18) for each time £ and vector l. If the computation burden for each evaluation
of (3.18) is C , and we estimate the reach tube via (3.19) for N values of time £ and
L values of l, the total computation burden would be C  N  L. A solution to the
next problem would reduce this burden to C  L.
Problem 3.2.1. Given a unit-vector function l  .t /; < l  ; l  >D 1, continuously

differentiable in t , find an external ellipsoid EC Œt 
X Œt  that would ensure for all
t  t0 ; the equality

¡.l  .t /jX Œt / D ¡.l  .t /jEC Œt / D< l  .t /; x  .t / >; (3.23)

so that the supporting hyperplane for X Œt  generated by l  .t /, namely, the plane

hx  x  .t /; l  .t /i D 0 that touches X Œt  at point x  .t /, would also be a supporting

hyperplane for EC Œt  and touch it at the same point.
This problem is solvable in the class EC . In order to solve it, we first make use
of Theorem 3.2.1. However, the function p.s/ and vector p0 (3.21), (3.22), used for
the parametrization in (3.18), will now depend on an additional variable t , so that
p.s/ D pt .s/ will be a function of two variables, s; t , and p0 D p0 .t /—since the
requirement is that relation (3.23) should now hold for all t  t0 .
94 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Theorem 3.2.3. With l.t / D l  .t / given, the solution to Problem 3.2.1 is an


ellipsoid EC Œt  D E .x ? .t /; XC Œt /, where

XC Œt  D (3.24)
Z t  Z t
pt .s/ds C p0 .t / .pt .s//1 G.t; s/B.s/Q.s/B 0 .s/G 0 .t; s/ds
t0 t0


Cp01 .t /G.t; t0 /X 0 G 0 .t; t0 / ;

and

pt .s/ D< l  .t /; G.t; s/B.s/Q.s/B 0 .s/G 0 .t; s/l  .t / >1=2 ; (3.25)

p0 .t / D< l  .t0 /; G.t; t0 /X 0 G 0 .t; t0 /l  .t0 / >1=2 :

The proof is obtained by direct substitution, namely, by substituting (3.25)

into (3.24) and further comparing the result with (3.13), (3.23). Rela-
tions (3.24), (3.25) need to be solved “afresh” for each t . It may be more convenient
for computational purposes to have them given in the form of recurrent relations
generated through differential equations.
Remark 3.2.2. In all the ellipsoidal approximations considered in this book the
center of the approximating ellipsoid is always the same, being given by x ? .t /
of (3.17).
The discussions in the next section will therefore actually concern only the relations

for the matrices XC Œt ; XC Œt  of these ellipsoids .x ? .t /  0/.

3.3 Recurrent Relations: External Approximations

We start with a particular case.

Assumption 3.3.1. The function l  .t / is of the following form l  .t / D G 0 .t0 ; t /l,
0
with l 2 Rn given. For the time-invariant case l  .t / D e A .tt0 / l.
Such curves l.t / will be further referred to as “good ones.” Under Assumption 3.3.1
the vector l  .t / may be expressed as the solution to equation

lP D A0 .t /l  ; l  .t0 / D l;

which is the adjoint to the homogeneous part of Eq. (1.1).

3.3 Recurrent Relations: External Approximations 95

Then pt .s/; p0 .t /; XC


Œt  of (3.24), (3.25) transform into

pt .s/ D (3.26)

D< l; G.t0 ; s/B.s/Q.s/B 0 .s/G 0 .t0 ; s/l >1=2 D p  .s/I p0 .t / D< l; X 0 l >1=2 D p0 ;

and

XC Œt  D G.t; t0 /XC .t /G 0 .t; t0 /; (3.27)
Z t 
XC Œt  D p  .s/ds C p0 ‰.t /; (3.28)
t0

where

‰.t / D
Z t
D < l; G.t0 ; s/B.s/Q.s/B 0 .s/G 0 .t0 ; s/l >1=2 G.t0 ; s/B.s/Q.s/B 0 .s/G 0 .t0 ; s/dsC
t0

C < l; X 0 l >1=2 X 0 :

In this particular case pt .s/ does not depend on t (pt0 .s/ D pt00 .s/ for t 0 6D t 00 ) and
the lower index t may be dropped.
Direct differentiation of XC Œt  yields

XP C Œt  D (3.29)

D   .t /XC Œt  C  1 .t /G.t0 ; t /B.t /Q.t /B 0 .t /G 0 .t0 ; t /; XC Œt0  D X 0 ;

where
Z t 1
  .t / D p  .t / p  .s/ds C p0 :
t0

Calculating
Z t  Z t 2
< l; XC Œt l >D p  .s/ds C p0 < l; ‰.t /l >D p  .s/ds C p0 ;
t0 t0

one may observe that

  .t / D< l; G.t0 ; t /B.t /Q.t /B 0 .t /G 0 .t0 ; t /l >1=2 < l; XC Œt l >1=2 : (3.30)

96 3 Ellipsoidal Techniques: Reachability and Control Synthesis


In order to pass to the matrix function XC Œt  we note that

XP C

Œt  D A.t /G.t; t0 /XC Œt G 0 .t; t0 /CG.t; t0 /XC Œt G 0 .t; t0 /A0 .t /CG.t; t0 /XP C Œt G 0 .t; t0 /:

After a substitution from (3.29) this gives

XP C
 
D A.t /XC  0
C XC A .t /C (3.31)

C  .t /XC

C  1 .t /B.t /Q.t /B 0 .t /; XC

.t0 / D X 0 :


We shall denote the ellipsoid constructed with matrix XC .t / described by
? 
Eqs. (3.31), (3.30), as E.x .t /; XC .t //. We now summarize the last results.
Theorem 3.3.1. Under Assumption 3.3.1 the solution to Problem 3.2.1 is given by
  
the ellipsoid EC Œt  D E.x ? .t /; XC Œt /, where x ? .t / satisfies Eq. (3.17) and XC Œt 
is a solution to Eqs. (3.31), (3.30).
  
Since set XC Œt  depends on vector l 2 Rn , we further denote XC Œt  D XC Œt l ,
using it in the following text whenever it will be necessary.
Theorem 3.3.2. For any t  t0 the reach set X Œt  may be described as

X Œt  D \fE.x ? ; XC Œt /j l W< l; l >D 1g: (3.32)

This is a direct consequence of Theorem 3.2.2 and of the selection of good curves
for representing the solution. Differentiating   .t /, according to (3.30), we arrive
at the next result.
Corollary 3.3.1. If B.t /  B  const and function   .t / is differentiable, then it
satisfies the differential equation

P D f .t /    2 ; .t0 / D 1; (3.33)

where
P /B 0 .t //G 0 .t0 ; t /l >
< l; G.t0 ; t /.A.t /B.t /Q.t /B 0 .t /  B.t /Q.t /B 0 .t /A0 .t / C B.t /Q.t
f .t / D 0 0
:
2 < l; G.t0 ; t /B.t /Q.t /B .t /G .t0 ; t /l >

Corollary 3.3.2. With A; B D const; Q.t / D Q D const , and l being an

eigenvector of A0 .A0 l D œl/, with real eigenvalue œ, the function f .t / will be

f .t / D œ:

Thus, if l  .t / satisfies Assumption 3.3.1, the complexity of computing a tight,

external ellipsoidal approximation to the reach set for all t , is the same as computing
the solution to the differential equation (3.31).
3.3 Recurrent Relations: External Approximations 97


In the general case, differentiating relation (3.24) for XC Œt , we have

XP C

Œt  D
 Z t Z t
  
D pt .t /C .@pt .s/=@t /dsCpP0 pt1 .s/G.t; s/B.s/Q.s/B 0 .s/G 0 .t; s/ds
t0 t0


Cp01 .t /G.t; t0 /X 0 G 0 .t; t0 /

Z t 
C pt .s/ds C p0 .t / .pt .t //1 B.t /Q.t /B 0 .t /C
 
t0

Z t
[email protected] .s//1 G.t; s/B.s/Q.s/B 0 .s/G 0 .t; s//=@t /dsC
t0


Cd.p01 .t /G.t; t0 /X 0 G 0 .t; t0 //=dt ;

which, using the notations


XC Œt  D
Z t  Z t
D pt .s/ds C p0 .t / .pt .s//1
t0 t0


0 0
G.t; s/B.s/Q.s/B .s/G .t; s/ds C p01 .t /G.t; t0 /X 0 G 0 .t; t0 / ;

Z t 1
  .t / D pt .t / pt .s/ds C p0 .t / ; (3.34)
t0

gives

XP C

D (3.35)

  0
D A.t /XC C XC A .t / C   .t /XC

C  1 .t /B.t /Q.t /B 0 .t /

CG.t; t0 /¥.t; l.t /; B./Q./B 0 .//G 0 .t; t0 /;


XC .t0 / D X 0 :
98 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Here

¥.t; l.t /; B.t /Q./B 0 .t // D (3.36)

Z t  Z t 
D .@pt .s/=@t /ds C pP0 .t / .pt .s//1 P .t0 ; s/ds C .p0 .t //1 P0 C
t0 t0

Z t  Z t 
   1  1
C pt .s/ds Cp0 .t / [email protected] .s// =@t /P .t0 ; s/ds C.d.p0 .t // =dt /P0
t0 t0

and

P .t0 ; s/ D G.t0 ; s/B.s/Q.s/B 0 .s/G 0 .t0 ; s/; P0 D X 0 :

It is important to note that under Assumption 3.3.1 (l  .t / D G 0 .t0 ; t /l) the terms
@pt .s/=@t D 0; pP0 .t / D 0, so that ¥.t; l.t /; B./Q./B 0 .// D 0.

Recall that an external ellipsoid constructed in the general case, with XC .t /
 ? 
taken from Eq. (3.35), (3.34), is denoted as E .x .t /; XC .t //, in contrast with an

ellipsoid constructed along good curves, with XC .t / taken from Eqs. (3.31), (3.30)

and denoted earlier as EC Œt :
 
Theorem 3.3.3. The solution EC Œt  D E .x ? .t /; XC .t // to Problem 3.2.1 is given
? 
by vector function x .t / of (3.17) and matrix function XC .t / that satisfies Eq. (3.35),
  
with  .t / defined in (3.34) and pt .s/; p0 .t / in (3.25). Under Assumption 3.3.1
Eq. (3.35) appears with term ¥  0.
Throughout the previous discussion we have observed that under Assump-

tion 3.3.1 the tight external ellipsoidal approximation E .x ? .t /; XC .t // is governed
by the simple ordinary differential equations (3.31). Moreover, in this case the
points x l .t / of support for the hyperplanes generated by vector l.t / run along a
system trajectory of (3.1) which is generated by a control that satisfies the Maximum
Principle (3.15) and the relation (3.16) of Theorem 3.1.1.
In connection with this fact the following question arises:
Problem 3.3.1. Given is a curve l.t / such that the supporting hyperplane generated
by vectors l.t / touches the reach set X Œt  at the point of support x l .t /. What should
be the curve l.t /, so that x l .t / would be a system trajectory for (3.1)?
Let us investigate this question when x 0 D 0; q.t /  0: Suppose a curve l.t / is
such that (xtl0 D x l .t0 /)
Z t
x .t / D
l
G.t; t0 /xtl0 C G.t; £/B.£/ut .£/d £ (3.37)
t0

is a vector generated by the pair fxtl0 ; ut ./g. In order that x l .t / be a support vector
to the supporting hyperplane generated by l.t / it is necessary and sufficient that this
3.3 Recurrent Relations: External Approximations 99

pair satisfy at each time t the maximum principle (3.15) and the condition (3.16)
under constraints

xtl0 2 E .0; X 0 /; ut .£/ 2 E .0; Q.£//;

which means

Q.£/B 0 .£/G 0 .t; £/l.t /

ut .£/ D ; (3.38)
hl.t /; G.t; £/B.£/Q.£/B 0 .£/G 0 .t; £/l.t /i1=2

with t0  £  t; and

X 0 G 0 .t; t0 /l.t /
xtl0 D : (3.39)
< l.t /; G.t; t0 /X 0 G 0 .t; t0 /l.t / >1=2

Note that since x l .t / varies in t , the variables xtl0 D x l .t0 /; ut .£/ should in general
also be taken as being dependent on t .
Now the question is: what should be the function l.t / so that x l .t / of (3.37)–
(3.39) is a trajectory of system (3.1)?
Differentiating (3.37), we come to

xP l .t / D A.t / G.t; t0 /xtl0 C (3.40)

Z t 
C G.t; £/B.£/ut .£/d £ C B.t /ut .t / C Y .t /;
t0

where
Z t
Y .t / D G.t; t0 /.@xtl0 =@t / C G.t; £/B.£/.@ut .£/=@t /d £: (3.41)
t0

In order that x l .t / would be a system trajectory for (3.1) it is necessary and

sufficient that Y .t /  0. Here the control at time t is ut .t / and the initial position
is xtl0 .
Let us look for the class of functions l.t / that yield Y .t / D Yl .t /  0. (Since
we are about to vary the problem parameters depending on choice of l./, we further
denote Y .t / D Yl .t / and pt .£/ D pt .£; l/; p0 .t / D p0 .t; l/, see below, at (3.43).)
For any t we have

Yl .t / D G.t; t0 /[email protected] 0 G 0 .t; t0 /l.t /.p0 .t; l.t ///1 /=@t /C (3.42)
Z t
C G.t; £/B.£/[email protected].£/B 0 .£/G 0 .t; £/l.t /.pt .£; l.t ///1 /=@t /d £;
t0
100 3 Ellipsoidal Techniques: Reachability and Control Synthesis

where, as in (3.25),

pt .s; l.t // D< l.t /; G.t; s/B.s/Q.s/B 0 .s/G 0 .t; s/l.t / >1=2 ; (3.43)

p0 .t; l.t // D< l.t /; G.t; t0 /X 0 G 0 .t; t0 /l.t / >1=2 :

Continuing the calculation of Yl .t /, after some transformations we come to

Yl .t / D ˆ0 .t; t0 ; l.t //.p0 .t; l.t ///3 (3.44)

Z t
C ˆ.t; £; l.t //.pt .£; l.t ///3 d £;
t0

where
D E
ˆ0 .t; t0 ; l/ D P0 .t; t0 /.A0 .t /l C d l=dt / l; G.t; t0 /X 0 G 0 .t; t0 /l 
 
P0 .t; t0 /l .l 0 A.t / C d l 0 =dt /P0 .t; t0 /l ;

and
D E
ˆ.t; £; l/ D P .t; £/.l 0 A.t / C d l 0 =dt /0 l; P .t; £/l 
 
P .t; £/l .l 0 A.t / C d l 0 =dt /P .t; £/l :

Here, in accordance with previous notation,

P0 .t; t0 / D G.t; t0 /X 0 G 0 .t; t0 /; P .t; £/ D G.t; £/B.£/Q.£/B 0 .£/G 0 .t; £/:

Let us now suppose that l.t / satisfies the equation

d l=dt C A0 .t /l D k.t /l; (3.45)

where k.t / is a continuous scalar function. Then, by direct substitution, we observe

that Yl .t / D 0.
On the other hand, suppose Yl .t /  0. Let us prove that then l.t / satisfies
Eq. (3.45). By contradiction, suppose a function l0 .t / yields Yl0 .t /  0, but does
not satisfy (3.45) for any k.t / (including k.t /  0). Then we should have

.l00 .t /A.t / C d l00 .t /=dt /Yl0 .t /  0;

3.3 Recurrent Relations: External Approximations 101

so that the expression

.l00 .t /A.t / C d l00 .t /=dt /

 Z t 
3 3
 ˆ0 .t; t0 ; l0 .t //.p0 .t; l0 .t /// C ˆ.t; £; l0 .t //.pt .£; l0 .t /// d £ D
t0


D .l00 .t /A.t / C d l00 .t /=dt /P0 .t; t0 /.A0 .t /l0 .t / C d l0 .t /=dt / < l0 .t /; P0 .t; t0 /l0 .t / > 


.l00 .t /P0 .t; t0 /.A0 .t /l0 .t / C d l0 .t /=dt //2 p0 .t; l0 .t //3 C

Z t
C .l00 .t /A.t / C d l00 .t /=dt /P .t; £/.A0 .t /l0 .t / C d l0 .t /=dt / < l0 .t /; P .t; £/l0 .t / > 
t0


..l00 .t /A.t / C d l00 .t /=dt /P .t; £/l/2 .pt .£; l0 .t ///3 d £;

should be equal to zero. But due to the Hölder inequality (applied in finite-
dimensional version for the first term and in infinite-dimensional version for the
integral term) the above expression is equal to zero if and only if l0 .t / satisfies
relation (3.45) for some k.t /: .A0 .t /l C d l.t /=dt / is collinear with l.t /.2 This
contradicts the assumption Yl .t /  0 and proves the following issues.
Theorem 3.3.4. Given a curve l.t /, the function xl .t / of (3.37)–(3.39), formed of
support vectors to the hyperplanes generated by l.t /, is a system trajectory for (3.1)
if and only if l.t / is a solution to the differential equation (3.45).
Through similar calculations the following assertion may be proved
Theorem 3.3.5. In order that G.t; t0 /¥.t; l.t /; B./Q./B 0 .//G 0 .t; t0 /  0; it is
necessary and sufficient that l.t / satisfies Eq. (3.45).
Exercise 4.1. Work out the proofs of Theorems 3.3.4, 3.3.5 in detail.
We have thus come to the condition that l.t / should satisfy (3.45). This equation
may be interpreted as follows. Consider the transformation
Z t
lk .t / D exp. k.s/ds/l.t /: (3.46)
t0

2
Here we also take into account that .l 0 .t /A.t / C d l.t /=dt /Yl .t / is bounded with p0 .t / > 0 and
pt .£/ > 0 almost everywhere due to the controllability assumption.
102 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Then
Z t Z t
d lk .t /=dt D exp. k.s/ds/d l.t /=dt  k.t / exp. k.s/ds/l.t / D
t0 t0

Z t Z t
D .A0 .t / C k.t /I / exp. k.s/ds/l.t /  k.t / exp. k.s/ds/l.t / D A0 .t /lk .t /
t0 t0

which means that lk .t / is again a solution to the adjoint equation d l=dt D A0 .t /l,
but in a new scale. However, the nature of transformation (3.46) is such that lk .t / D
”.t /l.t /; ”.t / > 0.
Lemma 3.3.1. The solutions lk .t / to Eq. (3.45) generate the same support hyper-
planes to the reach set X Œt  and the same points of support x lk .t /, whatever be
the functions k.t / (provided all these solutions have the same initial condition
lk .t0 / D l0 ).
In particular, with k.t /  0, we come to functions l.t / of Assumption 3.3.1 and
with Al D kl (k is an eigenvalue of constant matrix A) to equation d l=dt D 0 and
to condition l.t / D l D const:
In general the function l.t / of Assumption 3.3.1 is not normalized so that <
l.t /; l.t / >6D 1. In order to generate a function lu .t / with vectors lu .t / of unit length,
take the substitution lu .t / D l.t / < l.t /; l.t / >1=2 , where d l=dt C A0 .t /l D 0.
Direct calculations indicate the following statement.
Lemma 3.3.2. Suppose function lu .t / satisfies Eq. (3.45) with

k.t / D< l.t /; A0 .t /l.t / >< l.t /; l.t / >1 ;

where l.t / satisfies Assumption 3.3.1. Then < lu .t /; lu .t / > 1.

Remark 3.3.1. Lemma 3.3.1 indicates that the necessary and sufficient conditions
for the solution of Problem 3.3.1 do not lead us beyond the class of functions given
by Assumption 3.3.1.
Relations (3.35), (3.17), above describe the evolution of the basic parameters
 
x ? .t /; XC Œt  of the ellipsoids EC Œt . This allows to proceed with the next topic.

3.4 The Evolution of Approximating Ellipsoids

It is known that the dynamics of the reach set X Œt ; X Œt0  D E 0 D E .x 0 ; X 0 / may

be described by an evolution equation for the “integral funnel” of the differential
inclusion

xP 2 A.t /x C B.t /Q .t /; (3.47)

3.4 The Evolution of Approximating Ellipsoids 103

where Q .t / D E .q.t /; Q.t //, so that the set-valued function X Œt  satisfies for all t
the relation

lim ©1 h.X Œt C ©; .I C ©A.t //X Œt  C ©E .B.t /q.t /; B.t /Q.t /B 0 .t /// D 0;
©!0
(3.48)
with initial condition X Œt0  D E 0 [158].
Here, as before, h.Q ; M / stands for the Hausdorff distance between sets Q ; M .
Recall that

h.Q ; M / D maxfhC .Q ; M /; h .Q ; M /g; hC .Q ; M / D h .M ; Q /;

while the Hausdorff semidistance is defined as

h .M ; Q / D max minf< x  z; x  z >1=2 jx 2 Q ; z 2 M g:

x z

Equation (3.48) has a unique solution.

The idea of constructing external ellipsoidal approximations for X Œt  is also
reflected in the following evolution equation, where we will be interested only in
ellipsoidal-valued solutions:

lim ©1 h .E Œt C ©; .I C ©A.t //E Œt  C ©E .B.t /q.t /; B.t /Q.t /B 0 .t /// D 0
©!0
(3.49)
with initial condition E Œt0  D E .x 0 ; X 0 / D E 0 .
Definition 3.4.1. A set-valued function EC Œt  is said to be the solution to
Eq. (3.49) if
(i) EC Œt  satisfies (3.49) for all t ,
(ii) EC Œt  is ellipsoidal-valued and EC Œt  2 EC .
A solution to Eq. (3.49) is said to be minimal in EC if together with
conditions (i), (ii) it satisfies condition
(iii) EC Œt  is a minimal in EC solution to (3.49) with respect to inclusion.
Condition .i i i / means that there is no other ellipsoid E Œt  6D EC Œt ; E Œt  2 EC
that satisfies both Eq. (3.49) and the inclusions EC Œt 
E Œt 
X Œt . This also
means that among the minimal solutions EC Œt  to (3.49) there are non-dominated
(tight) ellipsoidal tubes EC Œt  that contain X Œt . For a given initial set E 0 D E Œt0 
the solutions to (3.49) as well as its minimal solutions may not be unique.
Remark 3.4.1. Note that Eq. (3.49) is written in terms of Hausdorff semidistance
h rather than Hausdorff distance h as in (3.48).
Denote an ellipsoidal-valued tube EŒt  D E .x ? .t /; X Œt /, that starts at
ft0 ; E 0 g; E 0 D E .x 0 ; X 0 / and is generated by given functions x ? .t /; X Œt  (matrix
X Œt  D X 0 Œt  > 0) as EŒt  D E.t jt0 ; E 0 / D E .x ? .t /; X Œt /.
The evolution Eq. (3.49) defines a generalized dynamic system in the following
sense.
104 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Lemma 3.4.1. Each solution E.t jt0 ; E 0 / D EŒt  to Eq. (3.49), in the sense of
Definition 3.4.1, defines a map with the semigroup property

E.t jt0 ; E 0 / D E.t jE.£; jt0 ; E 0 //; t0  £  t: (3.50)

This follows from Definition 3.4.1. Equation (3.49) implies that a solution EŒt 
to this equation should satisfy the inclusion

EŒt C – C o.–/B .0/

.I C –A.t //EŒt  C –E .B.t /q.t /; B.t /Q.t /B 0 .t //;

where –1 o.–/ ! 0 with – ! 0 and B .0/ D fx W hx; xi  1g is the unit ball in Rn .
Further denote qB .t / D B.t /q.t /; QB .t / D B.t /Q.t /B 0 .t /.
Given a continuously differentiable function l.t /; t  t0 ; and assuming that
EŒt  D E .x ? .t /; X Œt / is defined up to time t , let us select EŒt C – as an external
ellipsoidal approximation of the sum

Z.t C –/ D .I C –A.t //E .x ? .t /; X Œt / C –E .qB .t /; QB .t //;

requiring that it touches this sum Z.t C –/ (which, by the way, is not an ellipsoid),
according to relation

¡.l.t C –/jEŒt C –/ D ¡.l.t C –/jZ.t C –//; (3.51)

that is at those points z.t C–/ where Z.t C–/ is touched by its supporting hyperplane
generated by vector l.t C –/. Namely,

fz.t; –/ W< l.t C –/; z.t; –/ >D< l.t C –/; x ? .t C –/ > C < l.t C –/; X Œt C –l.t C –/ >1=2 D

D< l.t C –/; x ? .t C –/ > C < l.t C –/; .I C –A.t //X Œt .I C –A.t //0 l.t C –/ >1=2 C

C– < l.t C –/; QB .t /l.t C –/ >1=2 Co1 .–/g:

In the latter case, according to the definitions of tightness, the ellipsoid EŒt C –
is among the tight external approximations for Z.t C –/ (relative to terms of order
higher than –).3
The requirement above is ensured if EŒt C – externally approximates the sum
Z tC–
Z .t C –/ D G.t C –; t /E .x .t /; X Œt / C
?
G.t C –; s/E .qB .s/; QB .s//ds
t

3
Here and in the sequel the terms of type oi .–/ are assumed to be such that oi .–/–1 ! 0 with
– ! 0.
3.4 The Evolution of Approximating Ellipsoids 105

with the same criterion (3.51) as in the previous case. Select

X Œt C – D .1 C p 1 .t C –//G.t C –; t /X Œt G 0 .t C –; t /C (3.52)

–2 .1 C p.t C –//G.t C –; t /QB .t /G 0 .t C –; t /;

where

< l.t C –/; G.t C –; t /X Œt G 0 .t C –; t /l.t C –/ >1=2

p.t C –/ D : (3.53)
– < l.t C –/; G.t C –; t /QB .t /G 0 .t C –; t /l.t C –/ >1=2

Indeed, with p.t C–/ chosen as in (3.53), a direct substitution of p.t C–/ into (3.52)
gives

< l.t C–/; X Œt C–l.t C–/ >1=2 D< l.t C–/; G.t C–; t /X Œt G 0 .t C–; t /l.t C–/ >1=2 C
(3.54)
C– < l.t C –/; G.t C –; t /QB .t /G 0 .t C –; t /l.t C –/ >1=2 :

Estimating the Hausdorff distance

Z tC– 
0
R.t; –/ D h G.t C –; s/E .0; QB .s//ds; – G.t C –; t /QB .t /G .t C –; t / ;
2
t

by direct calculation, we have the next result.

Lemma 3.4.2. The following estimate holds,

R.t; –/  K–2 ; K > 0: (3.55)

In view of (3.53), (3.55), and the fact that G.t C –; t / D I C –A.t / C o2 .–/, we arrive
at the desired result below.
Lemma 3.4.3. With p.t C –/ selected as in (3.53), relations (3.52)–(3.55) reflect
equalities

¡.l.t C –/jE .x ? .t C –/; X Œt C –// D (3.56)

D ¡.l.t C –/jZ Œt C –/ C o3 .–/ D ¡.l.t C –/jZ.t C –// C o4 .–/:

Lemma 3.4.3 implies

X Œt C – D .1 C p 1 .t C –//.I C –A.t //X Œt .I C –A.t //0 C (3.57)

C–2 .1 C p.t C –//.I C –A.t //0 QB .t /.I C –A.t // C o5 .–/:

106 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Substituting p.t C –/ of (3.53) into (3.57), denoting –p.t C –/ D  1 .t C –/, and

expanding X Œt C –; G.t C –; s/; .t C –/ in –, one arrives at

X Œt C – D X Œt  C –A.t /X Œt  C –X Œt A0 .t / C .t /X Œt  C  1 .t /QB .t / C o6 .–/:

(3.58)
This further gives, by rewriting the last expression, dividing it by –, and passing to
the limit with – ! 0,

XP D A.t /X C XA0 .t / C .t /X C  1 .t /QB .t /; (3.59)

where

< l.t /; QB .t /l.t / >1=2

.t / D : (3.60)
< l.t /; X Œt l.t / >1=2

Starting the construction of EŒt  with EŒt0  D E 0 , we have thus constructed

the ellipsoid EŒt  as a solution to (3.49) with its evolution selected by the
requirement (3.51), (3.54), (3.56), which resulted in Eqs. (3.59), (3.60), X Œt0  D X 0 .
Equation (3.59) obviously coincides with (3.35) if ¥ D 0. However, as indicated
in Sects. 3.3, 3.4, the ellipsoid EŒt  constructed from Eq. (3.35), ¥ D 0; which then
turns into (3.31), touches the reach set X Œt  at points generated by support vector
l.t / D l  .t / if and only if the latter is of the type l  .t / D G 0 .t0 ; t /l (for any
preassigned l 2 Rn ). Then it even satisfies for all t  t0 the requirement (iii)
of Definition 3.4.1. On the other hand, as we have just shown, an ellipsoidal tube
EŒt  D E .x ? .t /; X Œt / evolves due to funnel equation (3.49) under Definition 3.4.1,
only if it satisfies Eqs. (3.59), (3.60). Therefore, an ellipsoid EŒt  which satisfies
Definition 3.4.1 may touch the exact reach tube only if it follows a good curve l.t /.
The above observations lead to the next result.
Theorem 3.4.1. An ellipsoid EŒt  D E .x ? .t /; X Œt / is a minimal solution
to the ellipsoidal funnel equation (3.49) only if it is constructed following
Eqs. (3.31), (3.59), (3.60), with X Œt0  D X 0 , where curve l.t / D G 0 .t0 ; t /l for
some l 2 Rn (it is a good curve).
Note that the opposite of the last Theorem is also true.

Theorem 3.4.2. In order that E .x ? Œt ; XC Œt / would be described by Eqs. (3.31),
(3.59), (3.60) it is necessary and sufficient that l.t / satisfy Eq. (3.45) with some k.t /.

The ellipsoid E .x ? Œt ; XC Œt / will be tight in EC .
The solutions given in this theorem are thus tight in the sense that there exists no

other ellipsoid in EC which could be squeezed in between E .x ? .t /; XC Œt / and
X Œt .
From Theorems 3.4.1, 3.4.2 and Lemma 3.4.1 it follows that the mapping

EŒt  D E.t jt0 ; X 0 / constructed as in Theorem 3.4.1, with X Œt  D XC Œt , satisfies
the semigroup property (3.50). This may be also checked by direct calculation,
using (3.26)–(3.28).
3.5 The Ellipsoidal Maximum Principle and the Reachability Tube 107

Remark 3.4.2. The ellipsoid EŒt  of 3.4.1 solves Problem 3.2.1 for any (non-
normalized) curve l.t /. In order to have a solution for a normalized curve with
< l.t /; l.t / >D 1; one has to follow Lemmas 3.3.1, 3.3.2.
 
In the general case an ellipsoidal tube EC Œt  D E .x ? ; XC Œt / may be such that
it touches X Œt  along some curve according to the requirement of Problem 3.2.1.

But this curve may not be a good one. Then equations for XC Œt  will differ

from (3.31), (3.59), but will coincide with (3.35), where ¥ 6D 0. And EC Œt  need
not satisfy the semigroup property. An example of such case is when the touching
curve is “drawn” on the surface of exact reach tube by an external minimum-volume
ellipsoid. This touching curve is not a good one. The failure to distinguish good
curves from any curves led to confusion and controversial statements in some
publications in the 1980s on minimum-volume external ellipsoidal approximations.
But what if Eqs. (3.31), (3.59) are still used to approximate the exact reach set
from above, disregarding the effect of being tight? Then the ellipsoid EC Œt  under
consideration will be a conservative upper estimate for the exact reach set so that
X Œt  EC Œt , with no promise of being tight.

3.5 The Ellipsoidal Maximum Principle

and the Reachability Tube

We may now summarize the earlier results emphasizing that they were directed
at the calculation of reachability tubes rather than fixed-time reach sets. Theo-
rems 3.4.1, 3.4.2, lead to the next proposition.
Theorem 3.5.1. The solution of Problem 3.2.1 allows the following conclusions:
 
(i) the ellipsoid EC Œt  D E  .x ? .t /; XC .t // constructed from Eqs. (3.24), (3.25)
 
for any given l .t / D l.t /; XC .t0 / D X 0 ; is a solution to Problem 3.2.1 and

XC .t / satisfies the “general” equation (3.35) with parameters   .t / defined
in (3.34).

(ii) The ellipsoid EŒt  D E.x ? .t /; XC .t // constructed from Eqs. (3.31), (3.30)
(or (3.59), (3.60), which are the same) is an upper bound for the solution
EC Œt  D E  .x ? .t /; XC .t // of (3.24), (3.25), so that for any given function
l  .t / we have

¡.l.t /jE.x ? .t /; XC .t ///  ¡.l  .t /jE  .x ? .t /; XC

.t ///; (3.61)

with equality reached for all t  t0 provided l  .t / is chosen according to

Assumption 3.3.1. In the last case the external ellipsoids are tight.

(iii) In order that E .x ? .t /; XC .t // be described by Eqs. (3.31), (3.30), it is
necessary and sufficient that l.t / satisfy Eq. (3.45) with some function k.t /.
108 3 Ellipsoidal Techniques: Reachability and Control Synthesis


Remark 3.5.1. If in the previous theorem l  .t / is chosen with k.t /  0, and XC .t /

is its related matrix with  .t / being its related parameterizing function, then we

further drop the asterisks, assuming l  .t / D l.t /; XC .t / D XC .t /;   .t / D .t /
and are using Eqs. (3.31), (3.30) (or (3.59), (3.60)). For vector x ? .t / of the center of

ellipsoid E .x ? .t /; XC .t // the asterisk remains since it does not depend on l:
The solutions to the problem of the last theorem are tight in the sense of
Definitions 3.2.1, 3.2.4. It is also useful to note the following.
Lemma 3.5.1. Each ellipsoidal tube of type E.t jt0 ; E 0 / D EŒt , generated as a
solution to Eqs. (3.31), (3.30), defines a map with the semigroup property

E.t jt0 ; E 0 / D E.t jE.£; jt0 ; E 0 //; t0  £  t: (3.62)

We now indicate how to calculate the reach tube with the results above.
An application of Theorem 3.1.1 under Assumption 3.3.1 yields the following
conclusion.

Theorem 3.5.2. Suppose Assumption 3.3.1 is fulfilled. Then the points x l .t / of
support for vector l D l  .t /, namely, those for which the equalities

< l  .t /; x l .t / >D ¡.l  .t /jX Œt / D ¡.l  .t /jE.x ? .t /; XC Œt // (3.63)

are true for all t  t0 , are reached from initial state x l .t0 / D x l0 using a system
trajectory with control u D u .t / which satisfies the maximum principle (3.15) and
condition (3.16) which now have the form l  .t / D G 0 .t0 ; t /l

< l; G.t0 ; s/B.s/u .s/ >D maxf< l; G.t0 ; s/B.s/u > ju 2 E .q.t /; Q.t //g;
(3.64)
and

< l; x l0 >D maxf< l; x > jx 2 E .x 0 ; X 0 /g; (3.65)

for any t  t0 . For all s  t0 the control u .s/; s 2 Œt0 ; t  may be taken to be the
same, whatever be the value t .
From the maximum principle (Theorem 3.1.1) one may directly calculate the
optimal control u .s/ which is

Q.s/B 0 .s/G 0 .t0 ; s/l

u .s/ D ; t0  s  t; (3.66)
< l; G.t0 ; s/B.s/Q.s/B 0 .s/G 0 .t0 ; s/l >1=2

and the optimal trajectory

Z t
G.t; s/B.s/Q.s/B 0 .s/G 0 .t0 ; s/l
x .t / D x .t / C
l ?
ds; t  t0 ;
t0 < l; G.t0 ; s/B.s/Q.s/B 0 .s/G 0 .t0 ; s/l >1=2
(3.67)
3.5 The Ellipsoidal Maximum Principle and the Reachability Tube 109

where
X 0l
x l .t0 / D x l0 D C x 0 D x ? .t0 /: (3.68)
< l; X 0 l >1=2
These relations are not very convenient for calculation, being expressed in a non-
recurrent form.
However, one should note that due to (3.38), (3.39), the trajectory x l .t / of
Theorem 3.5.2 also satisfies the following “ellipsoidal” maximum principle.
Theorem 3.5.3. The next condition holds

< l  .t /; x l .t / >D maxf< l  .t /; x > jx 2 E.x ? .t /; XC Œt /g; (3.69)

and is attained at

x l .t / D x ? .t / C XC Œt l  .t / < l  .t /; XC Œt l  .t / >1=2 ; (3.70)

where l  .t / D G 0 .t0 ; t /l,

XC Œt0  D X 0 ; x l .t0 / D x 0 C X 0 l < l; X 0 l >1=2 ; (3.71)

and XC Œt  may be calculated through Eq. (3.31) with explicitly known parame-
ter (3.30).
Thus the “trick” is to substitute at each point of the boundary of the reach set
X Œt  the “original” maximum principle (3.15) of Theorem 3.1.1 by the “ellipsoidal”
maximum principle (3.69).

The maximum principles

This results in the next proposition.

110 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Theorem 3.5.4. The trajectory x l .t / that runs along the boundary @X Œt  of the
reach set X Œt  and touches X Œt  at the points of support for the vector l  .t / D
G 0 .t; t0 /l is given by equalities (3.71), (3.70), where XC Œt  is the solution to
Eqs. (3.31), (3.30) (or, what is the same, to Eqs. (3.59), (3.60)).
Due to Lemma 3.3.1 the last theorem may be complemented by
Corollary 3.5.1. Given vector l 0 D l  .t /, the related ellipsoid E.x ? .t /; XC Œt /
and vector x l .t / 2 @X Œt ; generated by function l  .t / of Eq. (3.45), do not depend
on the choice of function k.t / in this equation.
Denoting x l .t / D xŒt; l, we thus come to a two-parametric surface xŒt; l that
defines the boundary @X Œ of the reachability tube X Œ D [f@X Œt ; t  t0 g. With
t D t 0 fixed and l 2 S D fl W< l; l >D 1g varying, the vector xŒt 0 ; l runs along the
boundary @X Œt 0 . On the other hand, with l D l 0 fixed and with t varying, the vector
xŒt; l 0  moves along one of the system trajectories x l .t / that touch the reachability
set X Œt  of system (3.1) with control u .t / of (3.66) and x l0 of (3.68). Then

[fxŒt; ljl 2 Sg D @X Œt ; [fxŒt; ljl 2 S; t  t0 g D @X :

Remark 3.5.2. Relation (3.70) is given in a recurrent form and throughout the
calculation of curves x l Œt  and surface xŒt; l one need not compute the respective
controls u .t /.
Remark 3.5.3. This remark concerns the question of optimality of external ellip-
soids. Suppose for each time t a globally volume-optimal ellipsoid E .x ? .t /; Xv .t //
is constructed. It will obviously touch the reach set X Œt  at each instant t and thus
produce a certain curve xv .t / on the surface @X Œt  of the reach tube X Œ. However,
an example given in [174, Sect. 2.7, Example 2.7.1] indicates that the volume-
optimal ellipsoid may not exist in the class EC . The conclusion which follows is
that in general the volume-optimal curve xv .t / is not a “good one” in the sense
of Assumption 3.3.1. Therefore, one should not expect that matrix Xv .t / of the
volume-optimal ellipsoid to be described by Eq. (3.31) ((3.59)), as claimed by some
authors.
On the other hand, the tight ellipsoids of Sect. 3.2 do satisfy Definition 3.2.1.
This implies

d Œl D ¡.ljX Œt / C ¡.ljX Œt / D ¡.ljE.x ? .t /; XCl .t /// C ¡.ljE.x ? .t /; XCl .t // D

D 2 < l; XCl .t /l >1=2

where d Œl is the length of the exact projection of set X Œt  on the direction l. Here
matrix XCl .t / is calculated due to (3.31), (3.30), for the given vector l 2 Rn .
3.6 Example 3.6 111

3.6 Example 3.6

For an illustration of the results consider system

xP 1 D x2 ; xP 2 D u; (3.72)

x1 .0/ D x10 ; x2 .0/ D x20 I juj  ;  > 0:

Here we have:
Z t
x1 .t / D x10 C x20 t C .t  £/u.£/d £;
0

Z t
x2 .t / D x20 C u.£/d £:
0

The support function

¡.ljX Œt / D maxf< l; x.t / > j juj  g

may be calculated directly and is given by formula

Z t
¡.ljX Œt / D l1 x10 C .l1 t C l2 /x20 C  jl1 .t  £/ C l2 jd £:
0

The boundary of the reach set X Œt  may be calculated from the formula ([174])

minf¡.ljX Œt /  l1 x1  l2 x2 j < l; l >D 1g D 0: (3.73)

l

Then the direct calculation of the minimum gives a parametric representation for
the boundary @X Œt  by introducing parameter ¢ D l20 = l10 , where l10 ; l20 are the
minimizers in (3.73). This gives two curves

x1 .t / D x10 C x20 t C .t 2 =2  ¢ 2 /; (3.74)

x2 .t / D x20 C 2¢ C t;

and
x1 .t / D x10 C x20 t  .t 2 =2  ¢ 2 /; (3.75)

x2 .t / D x20  2¢  t:

for ¢ 2 Œt; 0:4

4
Note that for l20 = l10 > 0 or l20 = l10 < t we have ¢ 62 Œt; 0. For such vectors the point of support
x l Œt  will be at either of the vertices of set X Œt .
112 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Fig. 3.1 The reach set at

given time

These curves (parameterized by ¢) form the upper and lower boundaries of X Œt 

with parameter t fixed (see Fig. 3.1). With t increasing this allows to draw the
reachability tube X Œt  as a set-valued function of time t , see Fig. 3.2.
0
The formula l.t / D e A t l of Assumption 3.3.1 here transforms into .l1 .t / D
l1 I l2 .t / D l2  t l1 /. This allows to write down the relations for the points of support
to the hyperplanes generated by vector l.t /. A substitution into (3.74), (3.75), gives
for l20 = l10 2 Œ0; t 

x1l .t / D x10 C x20 t C .t 2 =2  .t l10  l20 /= l10 /2 /; (3.76)

x2l .t / D x20 C 2.l20  t l10 /= l10 C t;

and

x1l .t / D x10 C x20 t  .t 2 =2  ..t l10  l20 /= l10 /2 /; (3.77)

x2l .t / D x20  2.l20  t l10 /= l10  t:

With vector l 2 R2 fixed, this gives a parametric family of curves x l .t / that cover
the surface of the reach tube X Πand are the points of support for the hyperplanes
generated by vectors l 2 R2 through formula l.t / D .l1 ; t l1 C l2 /. These curves
are shown in thick lines in Fig. 3.2.
3.6 Example 3.6 113

Fig. 3.2 The reach tube

Let us now choose two pairs of vectors l 2 R2 , for example

l  D .10; 4/; l  D .10; 4/I l  D .4; 0/; l  D .4; 0/:

Each of these pairs generates a tight ellipsoid E  Œt  and E  Œt  due to Eq. (3.71),
 
where the elements xi;j of XC satisfy the equations

      
xP 11 D x12 C x21 C .t /x11 ; xP 12 D x22 C x12 ;
    
xP 21 D x22 C .t /x21 ; xP 22 D .t /x22 C ..t //1 2 ;


with XC .0/ D 0 and

.t / D f1 .t /=f2 .t /;

with
 2  
f1 .t / D jl2  t l1 jI f2 .t / D .x11 l1 C 2x12 l1 .l2  t l1 / C x22 .l2  t l1 /2 /1=2 :

Ellipsoid E  Œt  is expressed similarly.

Here E  Œt  touches the reach set X Œt  at points xl .t / generated by vector l 
through formulas (3.75), (3.77) and E  Œt  touches it at points xl .t / generated by
vector l  through formulas (3.74), (3.76) (see cross-section in Fig. 3.3 for t D 0:5).
114 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Fig. 3.3 Two tight ellipsoids

For a given time t the intersection

E  Œt  \ E  Œt   X Œt 

gives an approximation of X Œt  better than the four supporting hyperplanes that

could be placed at the same points of support as for the ellipsoids (see Fig. 3.3).
At the same time, by taking more ellipsoids for other values of l we achieve,
as a consequence of Theorem 3.2.1, a more accurate approximation of set X Œt 
(see Fig. 3.4 for seven ellipsoids) with exact representation achievable if number
of appropriately selected ellipsoids tends to infinity.
Having introduced a family of external ellipsoidal approximations for reach sets
and specified its basic properties, we now indicate that similar properties are also
true for internal approximations which are often required whenever one has to deal
with guaranteed performance. This is a more difficult problem, though, as compared
with external approximations.

3.7 Reachability Sets: Internal Approximations

Existing approaches to the calculation of internal ellipsoidal approximations for the

sum of a pair of ellipsoids are given in [174, Sect. 2.3, pp. 121–127]. However,
the important question for dealing with dynamics is how to effectively compute
3.7 Reachability Sets: Internal Approximations 115

Fig. 3.4 Many tight ellipsoids

an internal approximation of the sum of any finite number of ellipsoids and going
further of an integral of ellipsoidal-valued functions. It is also important, as in the
external case, to single out families of tight internal approximations of reach tubes
or their neighborhoods through such ellipsoidal-valued functions that would touch
their boundary from inside at any point on their surface. Both of the questions are
answered in the forthcoming sections.
Thus, given as before is system (3.1) or differential inclusion (3.3), with
A.t /  0. The problem is to approximate the respective reach sets and tubes
internally, through appropriate tight ellipsoidal sets and ellipsoidal-valued tubes.
The text of this section relies on the next constructions introduced earlier in
[177, 182]. Though described there for nondegenerate ellipsoids, they are true for
degenerate ellipsoids as well.
Consider the internal approximation of sum
Z t
E .x 0 ; X0 / C E .B.s/q.s/; B.s/Q.s/B 0 .s//ds;
t0

of an n-dimensional ellipsoid E .x 0 ; X0 / and a set-valued integral of an ellipsoidal-

valued function E .B.s/q.s/; B.s/Q.s/B 0 .s// D E .qB .s/; QB .s// with continuous
n  p-matrix function B.s/ > 0. (Recall notations qB .t / D B.t /q.t /; QB .t / D
B.t /Q.t /B 0 .t /.)
Theorem 3.7.1. (i) The following inclusion is true
Z t

E .x ; X .t // x.t / C E .0; X0 / C
0
E .0; QB .£//d £;
t0
Z t
x.t / D x C 0
qB .s/ds;
t0
116 3 Ellipsoidal Techniques: Reachability and Control Synthesis

whatever be the matrix

X  .t / D (3.78)
 Z t 0  Z t 
1=2 1=2 1=2 1=2
D S0 X 0 C S.£/QB .£/d £ S0 X 0 C S.£/QB .£/d £ :
t0 t0

Here X0 D .X0 /0 > 0 is an n  n matrix, Q.£/ D Q0 .£/; £ 2 Œt0 ; t  are

any positive definite, p  p matrices with continuous function Q.£/, matrices
S0 S00 D I; S 0 .£/S.£/  I are orthogonal n  n matrices and S.£/ are
continuous in time.
(ii) For a given vector l 2 Rn relation

< l; x.t / > C¡.ljE .0; X  .t /// 

Z t
 ¡.ljE .0; X0 // C ¡.ljE .0; QB .£//d £; l 2 Rn ; (3.79)
t0

turns into an equality iff matrices S0 ; S.£/ may be chosen such that for some
scalar function œ.£/ > 0 the equality

1=2 1=2
S.£/QB .£/l D œ.£/S0 X0 l (3.80)

would be fulfilled for all £ 2 Œt0 ; t .

Remark 3.7.1. Note that with m < n matrices QB .t / D B.t /Q.t /B 0 .t / turn out
1=2
to be degenerate, of rank  m: Then the square root QB is understood to be an
n  n matrix which may be calculated through the “square root” version of the
1=2 1=2
singular value matrix decomposition of QB which gives QB QB D QB (see
[245, pp. 80–83]).
We now pass to the definition of internally tight ellipsoids.
Definition 3.7.1. An internal approximation E of a reach set X Œt  is tight if there
exists a vector l 2 Rn such that

¡.˙ljE / D ¡.˙ljX Œt /:

This definition may be appropriate for the reach sets of the present paper, but the
respective ellipsoids may not be unique. A more general definition follows.
Definition 3.7.2. An internal approximation E is tight in the class E , if for any
ellipsoid E 2 E , X Œt 
E
E implies E D E .
This section is concerned with internal approximations, where class E D fE g
is described within the following definition.
3.7 Reachability Sets: Internal Approximations 117

Definition 3.7.3. The class E D fE g consists of ellipsoids that are of the form
E Œt  D E .x ? .t /; X  .t //, where x ? .t / satisfies the equation

xP ? D B.t /q.t /; x ? .t0 / D x 0 ; t  t0 ;

X  .t / is of the form (3.78), q.t / 2 Rp is any Lebesgue-measurable function.

In particular, this means that if an ellipsoid E .x; X / X Œt  is tight in E , then
there exists no other ellipsoid of type E .x; kX /; k > 1; that satisfies the inclusions
X Œt 
E .x; kX /
E .x; X / (ellipsoid E .x; X / touches set X Œt ).
Definition 3.7.4. The internal ellipsoids are said to be tight if they are tight in E .
We actually further deal only with ellipsoids E 2 E . For the problems of this
book Definition 3.7.1 follows from 3.7.4.
The class E is rich enough to arrange effective approximation schemes, though
it does not include all possible ellipsoids. A justification for using this class is due
to the propositions of Theorem 3.7.3 which also gives conditions for the internal
ellipsoids E .0; X  .£// to be tight in the previous sense.
Let us now return to equation

xP D A.t /x C B.t /u; t0  t  t1 ; (3.81)

of Sect. 3.1.
Then the problem consists in finding the internal ellipsoid E .x ? .t /; X  .t // for
the reach set
Z t
X Œt  D G.t; t0 /E .x 0 ; X0 / C G.t; £/B.£/E .q.£/; Q.£//d £:
t0

Since for a matrix-valued map we have B E .q; Q/ D E .qB ; QB /, the formula of

Theorem 3.7.1 given there for X  .t / will now transform into
 Z t 
X  .t / D G.t; t0 / X0 S00 .t0 / C G.t0 ; £/QB .£/S 0 .£/d £ 
1=2 1=2

t0

 Z t 
S.£/QB .£/G .t0 ; £/ G 0 .t; t0 /;
0
1=2 1=2
 S0 .t0 /X0 C (3.82)
t0

and
Z t
x ? .t / D G.t; t0 /x 0 C G.t; £/qB .£/d £: (3.83)
t0

Theorem 3.7.1 now transforms into the following statement.

118 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Theorem 3.7.2. The internal ellipsoids for the reach set X Œt  satisfy the inclusion

E .x ? .t /; X  .t // E .G.t; t0 /x 0 ; G.t; t0 /X0 G 0 .t; t0 //C (3.84)

Z t
C E .G.t; £/qB .£/; G.t; £/QB .£/G 0 .t; £//d £ D X Œt ;
t0

where X  .t /; x ? .t / are given by (3.82), (3.83), with S0 ; S.£/ being any orthogonal
matrices and S.£/ continuous in time.
The tightness conditions now transfer into the next proposition.
Theorem 3.7.3. For a given instant t the internal ellipsoid E .x ? .t /; X  .t // will
be tight and will touch X Œt  at the point of support x l of the tangent hyperplane
generated by given vector l  , namely,

¡.l  jX Œt / D (3.85)
Z t
D< l  ; x ? .t / > C < l  ; X0 l  >1=2 C < l  ; G.t; £/QB .£/G 0 .t; £/l  >1=2 d £ D
t0

¡.l  jE .x ? .t /; X  .t /// D< l  ; x ? .t / > C < l  ; X  .t /l  >1=2 D< l  ; x l >;

iff S0 ; S.£/ satisfy the relation

S.£/QB .£/G 0 .t; £/l  D œ.£/S0 X0 G 0 .t; t0 /l  ; t0  £  t;

1=2 1=2
(3.86)

for some function œ.£/ > 0:

Direct calculation indicates the following.
Lemma 3.7.1. The function œ.£/ of Theorem 3.7.3 is given by

œ.£/ D< l  ; G.t; £/QB .£/G 0 .t; £/l  >1=2 < l  ; G.t; t0 /X0 G 0 .t; t0 /l  >.1=2/ ; t0  £  t:
(3.87)

The previous Theorems 3.7.2, 3.7.3 were formulated for a fixed instant of time
t and a fixed support vector l  . It is important to realize what would happen if l 
varies in time.

3.8 Example 3.8

Consider system

xP 1 D x2 ; xP 2 D u; (3.88)

x1 .0/ D x10 ; x2 .0/ D x20 I juj  ;  > 0:

3.8 Example 3.8 119

Here
Z t
x1 .t / D x10 C x20 t C .t  £/u.£/d £;
0

Z t
x2 .t / D x20 C u.£/d £:
0

Assume X0 D B– .0/ D fx W< x; x > –2 g. The support function

¡.ljX Œt / D maxf< l; x.t / > j juj  ; x 0 2 X0 g

of the reach set X Œt  D X .t; 0; X0 / may be calculated directly and is given by

Z t
¡.ljX Œt / D –.l12 C .l1 t C l2 /2 /1=2 C jl1 .t  £/ C l2 jd £:
0

The boundary @X Œt  of the reach set X Œt  is the set of vectors such that

minf¡.ljX Œt /  l1 x1  l2 x2 jhl; li D 1g D 0: (3.89)

l

This leads to the next parametric presentation of @X Œt  through two bounding curves
(one with sign plus in ˙ and the other with minus)

x1 .t / D x10 C x20 t ˙ .t 2 =2  ¢ 2 /; (3.90)

x2 .t / D x20 ˙ 2¢ ˙ t;

taken for values of parameter ¢ 2 .t; 0/.The values ¢ 62 .t; 0/correspond to two
points—the two vertices of X Œt .
Here ¢ D l20 = l10 where l10 ; l20 are the minimizers in (3.89).
Solving the problem for all t > 0, set l 0 D l.t /. Then

x1 .t / D (3.91)

D –.l1 .t /.t 2 C 1/ C l2 .t /t /=.l12 .t / C .l1 .t /t C l2 .t //2 /1=2 ˙ .t 2 =2  l22 .t /= l12 .t //;

x2 .t / D

D –.l1 .t /t C l2 .t //=.l1 .t /2 C .l1 .t /t C l2 .t //2 /1=2 ˙ 2l2 .t /= l1 .t / ˙ t;

where l 2 R2 ; t  0.
Here, for each t , vector l 0 D l.t / is the support vector to X Œt  at point x.t / 2
@X Œt . Moreover, with t fixed, and x D x .l/ .t / running along @X Œt  (which is a
120 3 Ellipsoidal Techniques: Reachability and Control Synthesis

closed curve in Rn for a given t ), its support vector l 0 D l.t / will sweep out all
directions in Rn . Therefore, considering any function l.t / of t with t  t0 , we may
be sure that there exists a corresponding trajectory x .l/ .t / 2 @X Œt  for t  t0 .
Proceeding further, we select l.t / satisfying Assumption 3.3.1, namely, as l.t / D
0
e A t l  . This transforms into l1 .t / D l1 ; l2 .t / D l2  t l1 and (3.91) simplifies to

x1 .t / D (3.92)

D –.l1 C t l2 /=.l12 C l22 /1=2 ˙ .t 2 =2  .t l1  l22 .t //2 = l12 .t //;

x2 .t / D –l2 =.l12 C l22 /1=2 ˙ 2.l2  t l1 /= l1 ˙ t:

These relations depend only on the two-dimensional vector l  . They produce a

parametric family of curves fx1 .t /; x2 .t /g that cover all the surface of the reach tube
X Œt  so that vectors x.t /; x.t / D fx1 .t /; x2 .t /g are the points of support for the
hyperplanes generated by vectors l.t / D .l1 ; t l1 C l2 /0 . The reach tube that starts
at X 0 6D f0g with these curves on its surface is shown in Fig. 3.5.
We now construct the tight internal ellipsoidal approximations for X Œt  that touch
the boundary @X Œt  from inside at points of support taken for a given vector l D l  .
The support function ¡.l  j X Œt / for the exact reach set may be rewritten as
Z t
   1=2
¡.l j X Œt / D –hl ; I l i C hl  ; QB .£/l  i1=2 d £; (3.93)
0

0
1
0.2
0.5
0.4
0
0.6
−0.5
0.8
−1
1

−1

−2

−3

Fig. 3.5 Reach tube

3.8 Example 3.8 121

where B 0 D .0; 1/ and

 
£2 ; £
QB .£/ D
£; 1

and QB .£/ D QB .£/.1 C £2 /1=2 .

1=2

According to (3.102) and in view of Assumption 3.3.1, we have (taking S0 D I ),

X  .£/ D
 Z t  Z t 
QB .£/S 0 .£/d £
1=2 1=2
D –I C –I C S.£/QB .£/d £ ; (3.94)
0 0

where matrix S.£/ must satisfy the conditions

S 0 .£/S.£/ D I; S.£/QB .£/l  D –œ.£/l  ; £  0

1=2
(3.95)

for some œ.£/ > 0. Calculations give

–2 œ2 .£/ D hl  ; QB .£/l  ihl  ; l  i1 D .l1 £ C l2 /2 .l12 C l22 /1 : (3.96)

Denote
   
£2 l1 C £l2 cos ¥p .£/
p.£/ D QB .£/l  D .1 C £2 /1=2 D rp .£/
1=2
;
£l1 C l2 sin ¥p .£/

where

rp .£/ D jl1 £ C l2 j; ¥p .£/ D ˙ arccos.˙£=.1 C £2 /1=2 /

and also
 
   1=2 cos ¥l .£/
l D hl ; l i ; ¥l D ˙ arccos.l1 =.l12 C l22 /1=2 /:
sin ¥l .£/

Selecting further the orthogonal matrix-valued function S.£/ as

 
cos ’.£/;  sin ’.£/
S.£/ D ;
sin ’.£/; cos ’.£/

we may rewrite the second relation (3.95) as

   
cos.¥p .£/ C ’.£// cos ¥l .£/
rp .£/ D –œ.£/.l12 C l22 /1=2 ; (3.97)
sin.¥p .£/ C ’.£// sin ¥l .£/
122 3 Ellipsoidal Techniques: Reachability and Control Synthesis

where £ 2 Œt0 ; t . Here ’.£/ has to be selected from equality

¥p .£/ C ’.£/ D ¥l .£/; £ 2 Œt0 ; t ; (3.98)

and œ.£/ is given by (3.96).

Equations (3.97), (3.98) need no recalculation for new values of t .
Thus we have found an orthogonal matrix function S.£/
 
cos.¥l .£/  ¥p .£//;  sin.¥l .£/  ¥p .£//
S.£/ D ; (3.99)
sin.¥l .£/  ¥p .£//; cos.¥l .£/  ¥p .£//

that depends on l  , that is continuous in £ and satisfies (3.97).

Matrix X .t / may now be calculated from equations
0
XP  D .XP  .t //0 X .t / C X .t /XP  .t /; X .0/ D –2 I; (3.100)

where

XP  D S.t /QB .t /; X .0/ D –I:

1=2

The internal ellipsoids for the reach set X Œt  D X .t; 0; X 0 / are shown in Figs. 3.6,
3.7, and 3.8 for X 0 D E .0; –I /; with epsilon increasing from – D 0 (Fig. 3.6) to
– D 0:175 (Fig. 3.7), and – D 1 (Fig. 3.8). The tube in Fig. 3.5 corresponds to the
epsilon of Fig. 3.7. One may also observe that the exact reach sets X .t; 0; f0g/ taken
for – D 0 are located within the sets X .t; 0; X 0 / D X Œt , calculated for – 6D 0 (see
Figs. 3.7 and 3.8).

3.9 Reachability Tubes: Recurrent Relations—Internal

Approximations

We start with a question similar to Problem 3.2.1, but now formulated for internal
approximations.
Problem 3.9.1. Given a vector function l  .t /, continuously differentiable in t , find
an internal ellipsoid E .x .t /; X .t // X Œt  that would ensure for all t  t0 ; the
equality

¡.l  .t /jX Œt / D ¡.l  .t /jE .x .t /; X .t /// D hl  .t /; x l .t /i; (3.101)

so that the supporting hyperplane for X Œt  generated by l  .t /, namely, the plane

hx  x l .t /; l  .t /i D 0 that touches X Œt  at point x l .t /, would also be a supporting
hyperplane for E .x .t /; X .t // and touch it at the same point.
3.9 Reachability Tubes: Recurrent Relations—Internal Approximations 123

Fig. 3.6 Internal ellipsoids 1.5

for X 0 D E .0; 0/

0.5

x2

−0.5

−1

−1.5
−0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6
x1

This problem is solvable in the class E . In order to solve this problem, we

shall refer to Theorems 3.7.2, 3.7.3. However, functions S.£/; œ.£/ used for the
parametrization in (3.82), (3.86) should now be functions of two variables, namely,
of £, t , since the requirement is that relation (3.101) should now hold for all
t  t0 (and therefore S0 should also depend on t ). We may therefore still apply
Theorems 3.7.2, 3.7.3 but now with S0 ; S.£/; œ.£/ substituted by S0t ; St .£/; œt .£/.
Theorem 3.9.1. With l D l  .t / given, the solution to Problem 3.9.1 is an ellipsoid
E .x .t /; X .t //; where x .t / D x ? .t / and

X .t / D (3.102)
 Z t 
D G.t; t0 / X0 S0t0 .t0 / C G.t0 ; £/QB .£/St0 .£/d £ 
1=2 1=2

t0

 Z t 
St .£/QB .£/G .t0 ; £/ G 0 .t; t0 /;
0
1=2 1=2
 S0t .t0 /X0 C
t0
124 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Fig. 3.7 Internal ellipsoids 1.5

for X 0 D E .0; 0/;
E .0; 0:175/

0.5

x2

−0.5

−1

−1.5
−0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8
x1

with S0t ; St .£/ satisfying relations

St .£/PB .£/G 0 .t; £/l  .t / D œt .£/S0t X0 G 0 .t; t0 /l  .t /;

1=2 1=2
(3.103)

and S0t0 S0t D I; St0 .£/St .£/  I for all t  t0 ; £ 2 Œt0 ; t , where

œt .£/ D (3.104)

D< l  .t /; G.t; £/QB .£/G 0 .t; £/l  .t / >1=2 < l  .t /; G.t; t0 /X0 G 0 .t; t0 /l  .t / >.1=2/ :

The proof follows by direct substitution. The last relations are given in a “static”
form and Theorem 3.9.1 indicates that the calculation of parameters S0t ; St .£/; œt .£/
has to be done “afresh” for every new instant of time t . We shall now investigate
whether the calculations can be made in a recurrent form, without having to perform
the additional recalculation.
In all the ellipsoidal approximations considered in this book the center of the
approximating ellipsoid is always the same, being given by x ? .t / of (3.83). The
discussions shall therefore concern only the relations for X .t /.
3.9 Reachability Tubes: Recurrent Relations—Internal Approximations 125

Fig. 3.8 Internal ellipsoids 2.5

for X 0 D E .0; 0/; E .0; 1/
2

1.5

x2 0.5

−0.5

−1

−1.5

−2

−2.5
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
x1

Remark 3.9.1. Without conditions (3.103), (3.104) equality (3.101) turns into an
inequality

¡.l  .t /jX Œt /  ¡.l  .t /jE .x .t /; X .t /// D hl  .t /; x l .t /i; (3.105)

for any orthogonal matrix-valued function S.t /:

Remark 3.9.2. The results of Sects. 3.7 and 3.9 are also true for degenerate ellip-
soids E .x 0 ; X0 /; E .q.t /; Q.t //. This will further allow to treat systems with
box-valued constraints in the form of parallelotopes as well as to treat zonotope-
valued constraints (see Chap. 5, Sects. 5.4, 5.5).
We will now pass to the treatment of internal approximations of reachability
tubes. We start with a particular function l  .t /, namely, the one that satisfies the
Assumption 3.3.1, where the requirement is that l  .t / is of the form l  .t / D
G 0 .t0 ; t /l.
Substituting l  .t / in (3.103), (3.104), we observe that the relations for calculating
St .£/; œt .£/ transform into

St .£/QB .£/G 0 .t0 ; £/l D œt .£/S0t X0 l;

1=2 1=2
(3.106)

S0t0 S0t D I; St0 .£/St .£/  I;

126 3 Ellipsoidal Techniques: Reachability and Control Synthesis

and

œt .£/ D< l; G.t0 ; £/QB .£/G 0 .t0 ; £/l >1=2 < l; X0 l >1=2 : (3.107)

Here the known vectors and functions used for calculating S0t ; St .£/; œt .£/ do
not depend on t . But then the matrix S0t and functions St .£/; œt .£/ do not depend
on t either, no matter what is the interval Œt0 ; t . Therefore, the lower indices t in
S0t ; St ; œt may be dropped and the matrix X .t / D X.l/ will depend only on l.
Then we have

X.l/ .t / D G.t; t0 /K .t /G 0 .t; t0 /;

where

K .t / D K 0 .t /K.t /; KP D KP 0 K C K 0 K;
P

and
Z t
S.£/QB .£/G 0 .t0 ; £/d £:
1=2 1=2
K.t / D S0 X0 C (3.108)
t0

Differentiating X.l/ .t / in view of the last notations, we come to

XP .l/ D A.t /X.l/ C X.l/ A0 .t / C G.t; t0 /.KP 0 .t /K.t / C K.t /KP 0 .t //G 0 .t; t0 /;
(3.109)
where

KP D H.t /G 0 .t0 ; t /; H.t / D S.t /QB .t /; K.t0 / D S0 X0 :

1=2 1=2
(3.110)

The differentiation of (3.83) also gives

xP ? D A.t /x ? C B.t /q.t /; x ? .t0 / D x 0 : (3.111)

This leads to the following theorem.

Theorem 3.9.2. Under Assumption 3.3.1 the solution to Problem 3.9.1 is given by
ellipsoid E .x ? .t /; X.l/ .t // where X.l/ .t /; x ? .t / are given by Eqs. (3.109), (3.111),
and functions S.t /, œ.t / involved in the calculation of K.t / satisfy together with
S0 the relations (3.106), (3.107), where the lower indices t in S0t ; St ; œt are to be
dropped.
We may also denote X .t / D K.t /G 0 .t; t0 /, so that X .l/ .t / D .X .t //0 X .t /:
Lemma 3.9.1. Function X. t / may be expressed through equation

XP  D X A0 .t / C H.t /; X .t0 / D S0 X0 :

1=2
(3.112)

This yields the next result.

3.9 Reachability Tubes: Recurrent Relations—Internal Approximations 127

Lemma 3.9.2. The ellipsoid E .x ? .t /; X.l/ .t // of Theorem 3.9.2, given by

Eqs. (3.109)–(3.112) depends on the selection of orthogonal matrix function S.t /
with orthogonal S0 ; and for any such S0 ; S.t / the inclusion

E .x ? .t /; X.l/ .t // X Œt ; t  t0 ;

is true with equality (3.105) attained for X .t / D X.l/ .t / and x .t / D x ? .t / under

conditions (3.106), (3.107).
Let us now suppose that l.t / of Problem 3.9.1 is the vector function that generates
any continuous curve of related support vectors on the surface of X Œt . Namely,
Assumption 3.3.1 is not fulfilled. Then one has to use formula (3.102), having in
mind that S0t ; St .£/ depend on t . After a differentiation of (3.102) in t , one may
observe that (3.110) transforms into

XP  D A.t /X C X A0 .t / C H 0 .t /X .t / C X .t /H.t / C ‰.t; /; (3.113)

X .t0 / D X0 ;

where
0
‰.t; / D ‰  .t; / C ‰  .t; /

and
 Z t 
 1=2 0 1=2 0
‰ .t; / D G.t; t0 / X0 .@S0t .t0 /=@t / C G.t0 ; £/QB .£/.@St .£/=@t /d £ 
t0

 Z t 
St .£/QB .£/G 0 .t0 ; £/d £/ G 0 .t; t0 /:
1=2 1=2
S0t .t0 /X0 C
t0

Lemma 3.9.3. Under Assumption 3.3.1 the functional ‰.t; /  0.

Similarly to Sects. 3.3 and 3.4, we come to the proposition.
Theorem 3.9.3. Let l.t / generate a curve x l .t / of related support points located on
the surface of set X Œt , forming a system trajectory for (3.81) due to some control
u.t /. Then Assumption 3.3.1 is satisfied and the functional ‰.t; /  0 iff l.t / is a
“good” curve.
We would finally like to emphasize that the suggested approach appears to be
appropriate for parallel computations (see Chap. 4, Sect. 4.5).
128 3 Ellipsoidal Techniques: Reachability and Control Synthesis

3.10 Backward Reachability: Ellipsoidal Approximations

The backward reachability or “solvability” sets W Œ£ D W Œ£; t1 ; M  were

introduced earlier in Sect. 2.3. As indicated in this section, their calculation is
crucial for constructing strategies of feedback target control. We shall now indicate
the ellipsoidal techniques that allow effective calculation of feedback control
strategies for such problems. The suggested methods apply to systems of fairly high
dimensions.
Ellipsoidal approximations of solvability sets are derived through procedures
similar to forward reach sets. Here are the respective results. A detailed derivation
of the coming relations may serve to be one of the good exercises on the topics of
this book.
Keeping notation QB .£/ D B.£/Q.£/B 0 .£/, we have the next assertion.
Theorem 3.10.1. Given is system (3.1) with ellipsoidal bounds u.t / 2
E .q.t /; Q.t // and target set E .m; M /. Then the following results are true.
.l/
(i) With E .w? .£/; WC .£// and E .w? .£/; W.l/ .£// being the external and internal
ellipsoidal approximations of set W Σ the next inclusions are true

E .w? .£/; W.l/ .£// W Œ£ E .w? .£/; WC.l/ .£//: (3.114)

(ii) Vector w? .t / satisfies equation

wP ? D A.t /w? C B.t /q.t /; w? .t1 / D m; (3.115)

.l/
while matrix WC .t / satisfies equation

WP C D A.t /WC C WC A0 .t /  .t /WC   1 .t /QB .t /;

.l/ .l/ .l/ .l/
(3.116)

where

.t / D< l.t /; QB .t /l.t / >1=2 < l.t /; WC .t /l.t / >1=2 ; WC .t1 / D M;
.l/ .l/

(3.117)
with l.t / D G 0 .t1 ; t /l; l 2 Rn :
(iii) For each vector l 2 Rn the next equality is true

.l/
¡.ljE .w? .t /; WC .t /// D ¡.ljW Œt /; (3.118)

.l/
(ellipsoid E .w.t /; WC .t // is tight).
(iv) For any vector l an internal ellipsoid E .w? .£/; W.l/ .£// W Œ£ is defined
through matrix
3.10 Backward Reachability: Ellipsoidal Approximations 129

W.l/ .t / D
 Z t1 
D G.t; t1 / M 1=2
Sm0  G.t1 ; t /QB .£/ 1=2 0
S .£/d £
t

 Z t1 
Sm M 1=2  S.£/QB .£/1=2 G 0 .t1 ; £/d £ G 0 .t; t1 /;
t

which satisfies equation

WP .l/ D (3.119)

D A.t /W.l/ C W.l/ A0 .t / C G.t; t1 /HP .l/ .t /G 0 .t; t1 /; W.l/ .t1 / D M;

with

H .l/ .t / D H 0 .t /H.t /; HP .l/ D HP 0 H C H 0 HP ;

Z t1
H.t / D Sm M 1=2  S.£/QB .£/1=2 G 0 .t1 ; £/d £; (3.120)
t

and

HP D S.t /QB .t /G 0 .t1 ; t /; H.t1 / D Sm M 1=2 ;

1=2

where Sm is any orthogonal matrix and S.t / is any orthogonal matrix-valued

function.
.l/
(v) Suppose orthogonal matrices Sm D Sm ; S.t / D S .l/ .t / satisfy conditions

.Sm.l/ /0 Sm.l/ D I; .S .l/ .t //0 S .l/ .t /  I;

S .l/ .t /QB .t /1=2 G 0 .t1 ; t /l < l; G.t1 ; t /QB .t /1=2 G 0 .t1 ; t /l >1=2 D

D Sm.l/ M 1=2 l < l; M l >1=2 ;

for all t 2 Σ; t1  and therefore depend on l.

Then for each vector l 2 Rn the equality condition for support functions is
true:

¡.ljE .w? .t /; W.l/ .t /// D ¡.ljW Œt / (3.121)

(the ellipsoid E .w? .t /; W.l/ .t // is tight).

130 3 Ellipsoidal Techniques: Reachability and Control Synthesis

(vi) The next relations are true

.l/
[ fE .w? .t /; W.l/ .£//j < l; l > 1g D W Œ£ D \fE .w? .t /; WC .£//j < l; l > 1g:
(3.122)

Further, passing to notations

H.t / D W1=2 .t /S.t /QB ; W.l/ .t / D W .t /

1=2

we also come to equation

WP  D A.t /W C W A0 .t /  H0 .t /W  W H.t /; W .t1 / D M; (3.123)

which may be used as well as (3.119).

The given theorem is formulated for tight ellipsoids placed along good curves of
type l.t / D G 0 .t1 ; t /l. Equality (3.122) follows from conditions (3.121).
Exercise 3.10.1. Derive the formulas of Theorem 3.10.1 for A.t / 6D 0:
In Fig. 3.9 presented is a picture related to calculating the reachability tube for
system5

Fig. 3.9 Reachability tube with one external and one internal approximation

5
Thus example was worked out by P. Gagarinov.
3.11 The Problem of Control Synthesis: Solution Through Internal Ellipsoids 131

0 1 0 1
0 1 0 0 0 0 0 0 1
B 1 0 0 0 0 0 0 0C B 0 C
B C B C
B 0 0 0 0C B 0 C
B 0 1 0 0 C B C
B C B C
B 1 0 9 0 0 0 0 0C B 0 C
A.t / D B C; B.t / D B C;
B 0 0 0 0 0 1 0 0C B 0:3 C
B C B C
B 0 0 1 0 3 0 0 0C B 0 C
B C B C
@ 0 0 0 0 0 0 0 1A @ 0 A
0 0 0 0 1 0 2 0 0

X0 D I 2 R88 ; x00 D .0; : : : ; 0/ 2 DR8 ; P .t / D 1; p.t / D cos.2t /I t 2D Œ0; 6;

where given is the cross-section (“cut”) of its dynamic projection at time t D t  D 6

on the subspace L .t  / orthogonal to vector .1; 0; 0; 0; 1; 0; 0; 0/0 , with tight internal
and external ellipsoids touching the exact reach set along “good” direction at
l 0 .t  / D .1; 0; 0; 0; 1; 0; 0; 0/:
Remark 3.10.1. In the last two sections we have studied the behavior of tight
internal ellipsoidal approximations of reach sets and reach tubes. It shows
that Eqs. (3.110)–(3.112) with appropriately chosen parameter S.t / (an orthogonal
matrix-valued function restricted by an equality) generate a family of internal
ellipsoids that touch the reach tube or its neighborhood from inside along a special
family of “good” curves that cover the whole tube. Such “good” curves are the
same as for the external approximations. The suggested techniques allow an on-
line calculation of the internal ellipsoids without additional computational burden
present in other approaches. The calculation of similar ellipsoids along any other
given smooth curve on the boundary of the reach tube requires additional burden as
compared with the “good” ones. The internal approximations of this paper rely on
relations different from those indicated in [35, 45, 174] and are relevant for solving
various classes of control and design problems requiring guaranteed performance.

3.11 The Problem of Control Synthesis: Solution Through

Internal Ellipsoids

We now pass to the construction of synthesizing target control strategies through

ellipsoidal methods. Without loss of generality, following the transformations of
Sect. 1.1, we may take A.t /  0. This is done by applying transformation xN D
G.t1 ; t /x and getting for xN the reduced equation, but keeping for it the original
notation x. Such a move will also allow to avoid some cumbersome calculations and
to present a clearer demonstration of the main relations, since now the “good” curves
will turn out to be straight lines and we will have W.l/ .t /  H .l/ .t /; G.t1 ; t /  I .
Problem 3.11.1. Given is the system

xP D B.t /u; u 2 E .qB .t /; QB .t // D Q .t / (3.124)

132 3 Ellipsoidal Techniques: Reachability and Control Synthesis

and target set M D E .m; M /. One is to find a synthesizing strategy u D U  .t; x/,
that transfers system (3.124) from any position f£; x.£/g; x.£/ 2 W Œ£, to the set
M at time t1 , so that x.t1 / 2 M . The specified strategy u D U  .t; x/ must ensure
the existence of solutions to the differential inclusion

xP 2 B.t /U  .t; x/:

The solution to the given problem is well known and indicated in Sect. 2.6. It may
be found, once we know the solvability tube W Œt ; t 2 Œ£; t1 .
Namely, denoting

VC .t; x/ D d.x; W Œt /

and taking the total derivative d VC .t; x/=dt ju at position ft; xg, due to Eq. (3.124),
under control u, one gets the desired strategy as
8 
< u W d V .t; x/=dt j  0j u 2 Q .t / ; if V .t; x/ > 0 .x 62 W Œt /;
Ue .t; x/ D
C u C
:
Q .t /; if VC .t; x/ D 0 .x 2 W Œt /:
(3.125)
It will be further indicated that also

arg min fd VC .t; x/=dt ju j u 2 Q .t /g ; if VC .t; x/ > 0 .x 62 W Œt /;
Ue .t; x/ D
Q .t /; if VC .t; x/ D 0 .x 2 W Œt /:
(3.126)
This is the so-called extremal aiming rule (due to Krasovski, see [121, 123]).
We shall now demonstrate that the solution to Problem 3.11.1 may be obtained
by operating only with ellipsoidal functions.
Consider position f£; x.£/g, and a parameterized family of ellipsoidal tubes
E .w? .t /; W.l/ .t //; t 2 Σ; t1 , constructed according to Theorem 3.10.1. For a given
position f£; xg; x D x.£/ the solution to Problem 3.11.1 exists if x 2 W Œ£.
With x 2 int W Œ£ one may choose any control u 2 Q .£/, so the selection of
control solutions will evolve around the boundary of W Σ and beyond. We shall
start by supposing that given is position f£; x  g with x D x  2 @W Œ£ on the
boundary of W Œ£. We may then introduce an internal ellipsoid E .w? .£/; W .£//
that touches its boundary at this x  : To do this we first note that with A.t /  0
the “good” curves that cover the tube W Œt ; t 2 Œ£; t1  are such that l.t /  l  ; is
constant. Hence we may further take l D l  to be the support vector of x  2 @W Σ

so that the ellipsoid that touches would be with W .£/ D W.l / .£//, namely

hl  ; x.t /i D ¡.l  j W.l / Œ£/: (3.127)

And such property will hold within the whole interval t 2 Σ; t1 :
 
¡.l  j E .w? .t /; W .l / .t // D hl  ; w? .t /i C hl  ; W.l / .t /l  i1=2 ;

along the selected “good curve” l.t /  l  :

3.11 The Problem of Control Synthesis: Solution Through Internal Ellipsoids 133


The selection of vector l D l  and the related ellipsoid E .w? .t /; W.l / .£// is a
separate element of the solution process which is discussed further. But first recall
the following. Suppose for W .t / D W?0 .t /W? .t / we have
Z t1
1=2
W? .t / D Sm M 1=2  S.Ÿ/QB .Ÿ/d Ÿ; (3.128)
t

Then for any orthogonal-valued matrix Sm and matrix function S.t /; t 2 Σ; t1  the

ellipsoid E .w? .t /; W.l / .t // W Œt  will be an internal approximation of W Œt :
However, if for a given vector l  l  we select Sm , S.t / coordinated according to
relations

S.t /QB .t /l D œ.£/Sm M 1=2 l; œ.t / D< l; QB .t /l >1=2 < l; M l >1=2 :

1=2

(3.129)
1=2
Then, due to these relations, vectors Sm M 1=2 l and S.t /QB .t /l; 8t 2 Σ; t1  will
be collinear. Hence, taking such Sm D Sm ; S.t / D S  .t / together with l D l  ; and

taking W .t / D W.l / .t /; we will have
Z t1
 
hl ; W.l / .t /l  i1=2 
D hl ; M l i 1=2
C hl  ; QB .s/l  i1=2 ds; (3.130)
t

and since

WP  .t / D WP ?0 .t /W? .t / C W?0 .t /WP ? .t /

we further get by direct calculation,


hl  ; WP .l / .t /l  i1=2 D hl  ; QB .t /l  i1=2 : (3.131)

Also note that if items l.t / and Sm ; S.t / are not coordinated, as in (3.129),
then (3.130), (3.131) will turn into inequalities with D substituted by .
Also observe that for any fl  W hl  ; l  i D 1g, coordinated with related Sm ; S  .t /
according to (3.129), we also have
 
hl  ; W.l / .t //l  i  hl; W.l / .t //li (3.132)

where l need not be coordinated with Sm , S  .t /:


We further work with the selected tight tube E.l / Œt  substituted in place of the
solvability tube W Œt  of (3.129), while following exactly the respective “aiming”
scheme. To use the ellipsoidal tube is more convenient than the exact solvability
tube, since in the ellipsoidal case all the auxiliary operations may be presented in
explicit form.
To indicate the ways of finding the control solutions, we begin with the
nondegenerate case specified as follows:
134 3 Ellipsoidal Techniques: Reachability and Control Synthesis

 
Assumption 3.11.1. Given set E.l / Œ£ D E .w? .£/; W.l / .£// that touches W Œ£

internally, there exists © > 0 such that W.l / .£/  ©I; 8t 2 Œ£; t1 :
Checking whether x D x.£/ 2 W Œ£.
We first find if the distance d Œ£; x D d.x.£/; W Œ£/ > 0. Then

VC .£; x/ D d Œ£; x D maxfhl; xi  ¡.l j W Œ£/ j hl; li D 1g

D hl 0 ; xi  ¡.l 0 j W Œ£/ D © > 0; (3.133)

and

hl 0 ; xŒ£i D © C hl 0 ; x 0 Œ£i; hl 0 ; x 0 Œ£i D ¡.l 0 j W Œ£/

so that x 0 is the point in W Œ£ closest to x D x.£/ and x 0 2 @W Œ£: With l 0

calculated, the solution U  .t; x/ to Problem 3.11.1 would be given by (3.125).
Now we will do the calculations of (3.133) through ellipsoids.
Synthesizing control strategies through ellipsoids.
 
We first have to find an internal ellipsoid E.l / Σ D E .w? .t /; W.l / .t // that
touches W Σ from inside at closest point x Σ. And this will happen once l 0 D l  ;
0

with tube E.l / Œt  taken due to coordinated l  ; Sm ; S  .t /; t 2 Œ£; t1 :
To ensure l 0 D l  there are various options. Thus, for example
(i) we may take the distances del l Σ D d.x; E.l/ Σ/ over all internal ellipsoids
E Œ£/ D E .w? .£/; W.l/ .£// whatever be l; Sm ; S./ and find its minimum over
.l/

unit vectors l and orthogonal matrices Sm , S.t /, arriving at

VC .£; x/ D d Œ£; x D minfdel l Œ£ j hl; li D 1; Sm Sm0 D I; S.t /S 0 .t / D I;

t 2 Σ; t1 g D del0 l Σ:

or (ii) we may select among internal ellipsoids E.l/ Σ that internally touch the
boundary @W Σ the one that does it for l D l  D l 0 . This may not need to calculate

the whole reach set.6 Then vector l  with related W.l / Σ are those that satisfy

VC .£; x/ D minfd.x; x l / j hl; li D 1g D d.x; x  /; x l D x  ; l D l  :

The corresponding equations for finding an ellipsoid E.l/ Œt  D E .w? .t /; W.l/ .t //

which touches W Œt  at point x l 2 @W Œ£ with support vector l are
 (
w? .t1 / D m; W? .t1 / D Sm M 1=2 ;
(3.134)
WP ? .t / D S.t /QB .t /:
1=2
wP ? D qB .t /I

6
If W Œt  is already calculated, the procedure is simple.
3.11 The Problem of Control Synthesis: Solution Through Internal Ellipsoids 135

Here
Z t1
W .t / D W?0 .t /W? .t /;
1=2
W? .t / D W? .t1 /  S.Ÿ/QB .Ÿ/d Ÿ;
t

QB .t / D B.t /Q.t /B 0 .t / and vectors Sm M 1=2 l; S.Ÿ/QB .Ÿ/l should be collinear.

1=2

Now suppose Assumption 3.11.1 is true and the tight tube E.l / Œt ; l  D l 0 is
?
already selected. Then the ellipsoidal control strategy Ue .t; x/ is defined, starting
from t D £; in the following way. Taking
 
Ve .t; x/ D d.x; E .w? .t /; W.l / .t /// D minfhx  z; x  zi j z 2 E
.l /
Œt g D
z

 
D maxfhl; x  w? .t /i  ¡.l j E .0; W.l / .t // j hl; li D 1g D hl 0 ; xi  hl 0 ; W.l / .t /l 0 i1=2 ;
l
(3.135)

we observe that the maximizer l 0 of this problem is unique and due to (3.132) we
have
 
hl 0 ; xi D hl 0 ; w? .t /i C hl 0 ; W.l / .t /l 0 i1=2 D ¡.l 0 j E .w? .t /; W.l / .t /// 

 ¡.l 0 j W .t //

Then, following the reasoning of Sect. 2.6, we get for d Œx.t /; E .w? .t /;

W.l / .t //
> 0 the strategy
8  ˇ 
< arg min d V .t; x/=dt j ˇˇ u 2 E .q.t /; Q.t // ; if x 62 E .w? .t /; W .l  / .t //;
u ˇ
Ue .t; x/ 
e
D
: /
E .q.t /; Q.t //; if x 2 E .w .t /; W .t //:
? .l

(3.136)
Since l  D l 0 this gives
(
q.t /  Q.t /B 0 .t /l 0 < l 0 ; QB .t /l 0 >1=2 ; if x 2
0
6 E .w? .t /; W.l / .t //;
Ue .t; x/ D 0
E .q.t /; Q.t //; if x 2 E .w? .t /; W.l / .t //:
(3.137)
We further show that the ellipsoidal strategy Ue .t; x/ of (3.137) does solve the
problem of control synthesis if the starting position f£; xg is such that x D x.£/ 2
0
@E .w? .£/; W.l / .£// for related l 0 and also x.£/ 2 @W Œ£: Indeed, suppose x.£/ 2
E.l / Œ£, and xŒt  D x.t; £; x.£// that satisfies Eq. (3.124) with u 2 Ue .t; x/; £ 
0

t  t1 ; is the respective trajectory that emanates from f£; x.£/g. We will demonstrate
that any such solution xŒt  of (3.124) will guarantee the inclusion xŒt1  2 E .m; M /.
To do that we will need, as in Sect. 2.6, to know the total derivative d Ve .t; x/=dt
for Ve .t; x/ > 0.
136 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Recalling
0
Ve .t; xŒt / D d Œt  D d0 .t; xŒt / D< l 0 ; xŒt  > ¡.l 0 j E.l / Œt /;

for d Œt  > 0, and due to the uniqueness of the maximizer l  D l 0 6D 0, we have

d d 0
d Œt  D .< l 0 ; xŒt  > ¡.l 0 jE.l / Œt //;
dt dt
so that

d @ 0
d Œt  D< l 0 ; xŒt
P  >  ¡.l 0 jE .w? .t /; W .l / .t // D (3.138)
dt @t

d 0
D< l 0 ; B.t /.u  q.t // >  < l 0 ; W.l / .t /l 0 >1=2 :
dt

Then from (3.131), with u D u0e 2 Ue .t; x/ we have:

d 0
< l 0 ; B.t /.u0e q.t // >  < l 0 ; W.l / .t /l 0 >1=2  hl 0 ; B.t /.u0e q.t //iChl 0 ; QB .t /l 0 i:
dt
(3.139)
Here l 0 D l  is coordinated with Sm ; S  .t /; so (3.131) gives an inequality.
From here it follows that with u D u0e 2 Ue .t; x/ of (3.137), and vector l 0 D
0
l .t; x/ of (3.135) we ensure the inequality
ˇ
d ˇ
d Œt ˇˇ  0:
dt u

Summarizing, we finally have

Ue .t; x/ D arg minf< l 0 ; B.t /u > ju 2 E .q.t /; Q.t //g: (3.140)

Integrating d d Œt =dt D d d0 .t; x.t //=dt from £ to t1 , along any trajectory

x.t; £; x.£// generated by u D u0e 2 Ue .t; x/; we get

Ve .t1 ; x.t1 // D d0 .t1 ; x.t1 // D d.x.t1 /; E .m; M //  d0 .£; x.£// D Ve .£; x.£// D 0
0
which means x.t1 / 2 E .m; M /, provided x.£/ 2 E.l / Œ£ W Œ£, whatever be the
solution to

xP 2 B.t /Ue .t; x/; t 2 Σ; t1 ;

generated from position f£; xg; x 2 W Œ£.

3.11 The Problem of Control Synthesis: Solution Through Internal Ellipsoids 137

We thus came to
Theorem 3.11.1. The solution to Problem 3.11.1 (with A.t /  0) is given by the
“ellipsoidal strategy” Ue .t; x/ of (3.140).
Exercise 3.11.1. Prove an analogy of Theorem 3.11.1 for system (3.1) with
A.t / 6D 0:
We now indicate some techniques for finding l 0 .

Calculating l 0

The scalar product hx; zi taken above, in solving Problem 3.11.1, may be substituted
for one of more general kind, namely by hx; T zi; T D T 0 > 0; hx; T xi D kxkT ;
so that in the above we took T D I: However we will also need to consider T 6D I;
since the calculation of vector l 0 is especially simple if one takes T 1 D W.l / .t /.
0

Therefore, before moving towards the proof that control (3.136) does indeed
solve Problem 3.11.1, we first look at how to calculate l 0 . Namely, we have to find
the distance d Œt  > 0 in the more general form as

d Œt  D VT .t; x/ D dT Œt; x D dT .x; E .w? .t /; W.l / .t /// D

D maxf< T l; x  w? .t / > hT l; W.l / .t /T li1=2 / j < l; T l > 1g; (3.141)

where VT .t; x/ D Ve .t; x/ if T D I: This problem is solved through simple

optimization techniques. Here is its solution.
An auxiliary optimization problem. Consider a nondegenerate ellipsoid E D
E .y  ; Y /; Y D Y 0 > 0: We need to find

V .t; x/ D minfhx  y; T .x  y/i1=2 j y 2 E g:

y

Clearly, with V .t; x/ > 0,

minfkx  ykT j y 2 E g D minfmaxfhx  y; liT j klkT  1g j y 2 E g D

y y l

D maxfhx; liT C minfhy; liT / j y 2 E g j klkT  1 g D

l y

D max fhx; liT  ¡.T ljE / j klkT  1g D hx; l 0 iT  ¡.T l 0 j E /:

l

Here l 0 is the maximizer of function hx; liT  ¡.T ljE /.

138 3 Ellipsoidal Techniques: Reachability and Control Synthesis

In view of

¡.T ljE / D hy  ; T li C hT l; Y T li1=2 D hy  ; liT C kT lkY ;

we find

V .t; x/ D max fhx  y  ; liT  kT lkY j klkT  1g : (3.142)

l

Lemma 3.11.1. For dT Œt  > 0 the distance between given point x 2 Rn and
ellipsoid E D E.y  ; Y /  Rn ; Y D Y 0 > 0 is the solution to optimization
problem (3.142), with T D T 0 > 0:
Finding the solution to (3.142) may be pursued with various matrices T .
We emphasize two cases–when T D I and T D Y 1 :
Case (a) : T D I
Referring to Lemma 3.11.1, we use the standard Euclid metric, taking, y  D
w ; Y D T . Then relations (3.137)–(3.140) remain without formal change.
?

Now V .t; x/ of (3.142) transforms into

n o
V .t; x/ D max hx  w? ; li  klkW.l  / j klk  1 D (3.143)
l

 
D d.x  w? ; E .0; W.l / // D minfkz  pk j hp; .W.l / /1 pi  1; z D x  w? g:
p


For kzkT > 1; T D .W.l / /1 ; this may be reduced to finding minp fkz 

pk2 j hp; .W.l / /1 pi D 1g > 0 through conventional Lagrangian techniques.
The related Lagrangian is

L .p; œ/ D hz  p; z  pi C œ.hp; .W.l / /1 pi  1/

with
 
Lp0 .p; œ/ D 2..z  p/ C œ.W.l / /1 p/ D 0; so that p D .I  œ.W.l / /1 /1 z:

Multiplier œ satisfies equation h.œ/ D 0; where

  / 1  
h.œ/ D h.I  œ.W.l / /1 /1 z; .W .l .I  œ.W.l / /1 /1 zi  1 D hp; .W.l / /1 pi  1:

Here h.0/ > 0 and with œ ! 1 we have lim h.œ/ D 1; so there exists a root
œ0 > 0 of equation h.œ/ D 0. This root œ0 is unique, since h0 .œ/ < 0: We have thus
found
 
d.x  w? ; E .0; W.l / // D kz  p 0 k; p 0 D .I  œ0 .W.l / /1 /1 z

and the element of E .w? ; W.l / / closest to x is x  D w? C p 0 :
3.11 The Problem of Control Synthesis: Solution Through Internal Ellipsoids 139

Summarizing we come to the conclusion

Lemma 3.11.2. (i) The vector l 0 which solves (3.143) (or (3.142)) under T D I
is
  
0; if hx  w? ; .W.l / /1 .x  w? /i  1 .x 2 E .w ; W.l / //;
l0 D
.x  x  /kx  x  k1 ; otherwise;
(3.144)

Here x  2 E .w? ; W.l / / is the closest to x in the Euclid metric and

x  D w? C p 0 ; p 0 D .I  œ0 .W.l / /1 /1 .x  w? /:
/
(ii) The related controls are then specified by (3.137) with w? D w? .t /; W.l D

W.l / .t /; p 0 D p 0 .t /:
Exercise 3.11.2. Work out the formulas for the “ellipsoidal strategy” U  .t; x/ of
Problem 3.11.1 for A.t / 6D 0:
Case (b) : T D Y 1
This is the simpler case. Indeed, substituting Y 1 for T , we transform (3.142)
into

V .t; x/ D max fhx  y  ; liT  klkT j klkT  1g : (3.145)

l

Due to the inequality of Cauchy–Bunyakovski–Schwartz we now have

hx  y  ; liT  klkT  kx  y  kT klkT  klkT ;

with equality attained at l D œ.x  y  /; œ  0: Since hl; T li D 1; this gives

œ D kx  y  k1
T , so that under T D Y
1
the maximizer of (3.142) is

0; if kx  y  kT  1;
lT0 D (3.146)
.x  y  /kx  y  k1
T ; otherwise:

Hence, with V .t; x/ > 0 .x … E .y  ; Y //, we have

V .t; x/ D maxfkx  y  kT klkT  klkT j hl; T li  1ig D kx  y  kT  1:

l
(3.147)
Otherwise, with x 2 E ; we have V .t; x/ D 0:
The element y 0 D y  C q 0 2 E .y  ; Y / closest to x is found as follows:
find

minfkx  ykY 1 j y 2 E .y  ; Y /g D minfkx  .y  C q/kY 1 j q 2 E .0; Y /g

y q

D kx  .y  C q 0 /kY 1 :
140 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Then y 0  y  D q 0 , so that kq 0 kY 1 D ky 0  y  kY 1 D 1 (y 0 lies on the boundary

of E .y  ; Y /).
Lemma 3.11.3. (i) Under T D Y 1 the distance in metric kx  ykT between
x … E and ellipsoid E D E .y  ; Y / is

V .t; x/ D d.x; E /T D maxf0; kx  y 0 kT  1g:

(ii) The optimizer lT0 in problem (3.147) under metric kx  ykT ; T D Y 1 is given
by (3.146).
(iii) The element of E closest to x under metric kx  ykT is

y 0  y  D q 0 ; kq 0 kT D 1:

Exercise 3.11.3. In the previous lemma show that vectors lT0 of (ii) and p 0 of (iii)
are collinear.
We now return to the original Problem and apply the previous Lemma 3.11.3 to
find the synthesizing control Ue .t; x/: In terms of Problem 3.11.1 we have: x D

x.t /; y  D w? .t /; Y D W.l / .t /:
Now, in order to rewrite these results in terms of Lemma 3.11.3, we need to

substitute its l 0 by T l 0 D lT0 ; T D .W.l / /1 , with metric kx  ykT D hx  y;
.l  / 1
.W / .x  y/i : We thus get
1=2


0; if kx.t /  w? .t /kT  1;
lT0 D (3.148)
.x.t /  w? .t //kx.t /  w? .t /k1
T ; otherwise:

The related control uT e .t; x/, for x.t / … E ; is then found due to (3.140), as

uT e .t; x/ D q.t /  Q.t /B 0 .t /lT0 hlT0 ; QB .t /lT0 i1=2 :

After following (3.140) and recalling QB .t / D B.t /Q.t /B 0 .t /, this yields the
solution strategy
8 
< q.t / if x 2 E .w? .t /; W.l / .t //;
UTe .t; x/ D q.t /  Q.t /B 0 .t /.x.t /  w? .t //kQ1=2 .t /B 0 .t /.x.t /  w? .t //k1
T ;
: .l  /
if x 62 E .w .t /; W .t //:
?

(3.149)
Exercise 3.11.4. By direct substitution of u D UTe .t; x/ into (3.124) prove the
following:
(i) the existence of solution to (3.124) under such substitution,
(ii) that u D UTe .t; x/ does solve Problem 3.11.1.
3.12 Internal Approximations: The Second Scheme 141

Remark 3.11.1. A solution in the absence of Assumption 3.11.1 has to deal with
degenerate internal ellipsoids. The calculation of related controls then has to
undergo a procedure of regularization. This issue will be treated at the end of
Chap. 4.
The given scheme thus describes the applicability of ellipsoidal techniques
to the calculation of trajectory tubes for controlled systems with further design
of synthesizing controls within a fairly simple scheme as compared to existing
methods. This scheme is realized in the ellipsoidal toolbox used further to solve
the presented examples (see [45, 132]).

3.12 Internal Approximations: The Second Scheme

Apart from the techniques for internal approximation of sets and tubes given in
Sects. 3.7–3.10, there exists another set of formulas which may be used for the same
purpose. The restrictive element in this “second approach” is that here the ellipsoidal
approximations are derived through operations with only a pair of ellipsoids rather
than any number of these as in the above. The treatment of internals for sums of
ellipsoids may therefore be achieved only through an inductive sequence of pairwise
operations rather than through a single operation. Nevertheless this scheme turns out
to be useful and is also being recommended. The scheme was discussed in [174] (see
also [45]).
We shall calculate the reachability set X Œt  D X .t; t0 ; X0 / for system (3.1) with
v.t /  0: Using the funnel equation (2.54), we have

.I C ¢A.t //X Œt  C ¢ E .qB .t /; QB .t // X Œt C ¢ C o1 .¢/B .0/; (3.150)

where B .0/ D fx W hx; xi  1g,

¢ 1 o1 .¢/ ! 0 if ¢ ! 0:

With X Œt  being an ellipsoid of type

X Œt  D E .x ? .t /; X  .t //; X  .t / > 0;

we may apply the following formula. For two given ellipsoids E .a.1/ ; X1 /;
E .a.2/ ; X2 /; X1 D X10 > 0; X2 D X20 > 0; it indicates the matrix

X  .H / D H 1 Œ.HX1 H 0 /1=2 C .HX2 H 0 /1=2 2 H 01 ; (3.151)

where H 2 † and

† D fH 2 Rnn W H 0 D H; jH j 6D 0g:

(Matrices H are therefore symmetrical and nondegenerate.)

142 3 Ellipsoidal Techniques: Reachability and Control Synthesis

Matrix X  ensures the relations

E .a.1/ C a.2/ ; X  .H // E .a.1/ ; X1 / C E .a.2/ ; X2 /; 8H 2 †; (3.152)

and
[
E1;2 D fE .a.1/ C a.2/ ; X  .H // j H 2 †g D E .a.1/ ; X1 / C E .a.2/ ; X2 /:
(3.153)
If one of the two ellipsoids is degenerate, say X2  0; then the union E1;2 at the
left-hand side of (3.153) should be substituted by its closure E1;2 .
Applying the last relations to (3.150), with X Œt  D E .x ? .t /; X  .t // being an
ellipsoid, we take

X1 D .I C ¢A.t //X  Œt .I C ¢A.t //0 ; X2 D ¢ 2 B.t /Q.t /B 0 .t /:

Then in view of (3.152) we have

X Œt C ¢ C o1 .¢/B .0/
E .x ? .t C ¢/; X  .t C ¢//;

where

X  .t C ¢/ D H 1 .t /ŒH.t /X  .t /H 0 .t / C
C ¢.A.t /X  .t / C X  .t /A0 .t // C ¢ 2 A.t /X  .t /A0 .t / C
1 1
C ¢.H.t /.I C ¢A.t //X  .t /.I C ¢A0 .t //H 0 .t // 2 .H.t /QB .t /H 0 .t // 2 C
1 1
C ¢.H.t /QB .t /H 0 .t // 2 .H.t /.I C ¢A.t //X  .t /.I C ¢A0 .t //H 0 .t // 2 C
C ¢ 2 H.t /QB.t /H 0 .t /H 01 .t /;

and

x ? .t C ¢/ D .I C A.t /¢/x ? .t / C ¢B.t /q.t /:

From here, after subtracting X  .t / from both sides, then dividing them by ¢ and
applying a limit transition with ¢ ! 0, we come to ordinary differential equations

XP .t / D A.t /X  C X  A0 .t / C H 1 ..H.t /X  .t /H 0 .t // 2 .H.t /QB .t /H 0 .t // 2 C

1 1

1 1
C .H.t /QB .t /H 0 .t // 2 .H.t /X  .t /H.t // 2 /.H 1 /0 ; (3.154)

xP? D A.t /x ? C B.t /q.t /; (3.155)

3.12 Internal Approximations: The Second Scheme 143

with initial conditions

x ? .t0 / D x 0 ; X  .t0 / D X 0 :

What follows from here is the inclusion

X Œt 
E .x ? .t /; X  .t //; (3.156)

where x ? .t /; X  .t / satisfy (3.154), (3.155) and H.t / is a continuous function of t

with values in †—the variety of symmetric nondegenerate matrices.
This leads us to the next conclusion.
Theorem 3.12.1. The internal approximation of the reachability set X Œt  D
X .t; t0 ; E .x 0 ; X 0 // for system (3.1) is given by the inclusion (3.156), where
x ? .t /; X  .t / satisfy Eqs. (3.155) and (3.154). Moreover, the following representa-
tion is true

X Œt  D [fE .x ? .t /; X  .t //jH./ 2 †g; (3.157)

where the union is taken over all measurable matrix-valued functions with values
in †.
Remark 3.12.1. The closure of the union of sets at the right-hand side of equal-
ity (3.157) appears due to the fact that matrix QB D B.t /Q.t /B 0 .t / may turn out
to be degenerate. Then the exact reachability set X Œt  may have kinks while being
approximated by internal ellipsoids all of which are nondegenerate. This effect is
absent in the “first approach” of Sects. 3.7–3.10 which allows degenerate elements
among the internal approximating ellipsoids.
One should note that internal approximations of reachability sets by nondegen-
erate ellipsoids require that the “reach set” itself would be nondegenerate.
Definition 3.12.1. A reachability set X Œt  is said to be nondegenerate if there exists
an n-dimensional ball B© .c/ D fx W< x  c; x  c > ©g of radius © > 0, such that
B© .c/ X Œt :
In other words, set X Œt  a nonempty interior. The last property may arise either from
the properties of the system itself and its constraints or from the degeneracy of the
starting set X Œt0 , or from both factors. To check degeneracy due to the system itself
it suffices to deal with reach sets of type X Œt  D X .t; t0 ; f0g/; emanating from point
x.t0 / D 0:
Consider system

xP 2 A.t /x C B.t /E .q.t /; Q.t //; (3.158)

x.t0 / D 0;

where B.t / is continuous and q.t / 2 Rp ; Q.t / 2 Rpp ; p < n are also continuous.
144 3 Ellipsoidal Techniques: Reachability and Control Synthesis

The parameters of this system allow to generate the set-valued integral

Zt

X Œt  D G.t; s/B.s/E .0; Q.s//ds; (3.159)
t0

where matrix G.t; s/ is defined in Sect. 1.1.

Assumption 3.12.1. There exists a continuous scalar function “.t / > 0; t > t0 ;
such that the support function

¡.ljX  Œt /  “.t / < l; l >1=2 ;

for all l 2 Rn ; t  t0 .
With Q.t / > 0; t > t0 ; this assumption implies that the reachability domain
X Œt  of system (3.158) has a nonempty interior (intX Œt  ¤ ;) and is therefore
nondegenerate. Assumption 3.12.1 is actually equivalent to the requirement that
system (3.158) with unbounded control u.t / would be completely controllable
[109, 195], on every finite time interval Œt0 ; t .
Exercise 3.12.1. Prove that under Assumption 3.12.1 and with Q.t / > 0; t > 0;
x.t0 / D 0 the reachability set X Œt  for system (3.158) is nondegenerate. (Use
Exercise 1.4.2.)
In a similar way the “second approach” may be applied to the approximation of
backward reachability sets W Σ. A reasoning similar to the above in this section
gives the result.
Theorem 3.12.2. The internal approximation of the backward reachability set
W Œt  is given by the inclusion

E .w? .t /; W Œt / W Œt ;

where fw? .t /; W Œt g satisfy Eq. (3.114) and

WP  .t / D A.t /W .t / C W .t /A0 .t / (3.160)

1 1
H 1 .t /Œ.H.t /W .t /H 0 .t // 2 .H.t /QB .t /H 0 .t // 2
1 1
C .H.t /QB .t /H 0 .t // 2 .H.t /W .t /H.t // 2 H 01 .t / ;

with boundary conditions x.t1 / D m; W Œt1  D M:

Moreover,the following representation is true

W Œt  D [fE .w? .t /; W .t //jH./ 2 †g; (3.161)

where the union is taken over all measurable matrix-valued functions with values
in †.
3.12 Internal Approximations: The Second Scheme 145

Problem 3.11.1 of ellipsoidal control synthesis is then solved according to the

scheme of Sect. 3.10.
Exercise 3.12.2. (i) Prove formulas (3.160) and (3.161).
(ii) Design an ellipsoidal synthesis for Problem 3.11.1 using Eq. (3.161) and prove
that it solves the problem.7

7
Such a proof is indicated in [174, Sect. 3.6, p. 212].
Chapter 4
Solution Examples on Ellipsoidal Methods:
Computation in High Dimensions

Abstract In this chapter we describe solution examples for controlled systems that
illustrate the contents of Chaps. 1–3. These include the multiple integrator, planar
Newtonian motions and calming down a chain of springs. Special sections are
devoted to relevant computational formulas and high-dimensional systems. Also
discussed are possible degeneracy effects in computation and the means of their
avoidance.

Keywords Multiple integrator • Newtonian motion • Oscillating system

• Graphic illustration • Regularization • High dimensions

In this chapter we first present examples of control problems discussed above. All
these problems are treated in finite time. The solutions are reached through the
theory given in Chaps. 1, 2 and the computation is done using methods of Chap. 3.1
The second part deals with specifics of calculation for systems of high
dimensions.
We begin with the examples.

4.1 The Multiple Integrator

The calculation of reach sets for a two-dimensional system with one integrator was
treated in detail in Chap. 3, Sect. 3.9. We continue now by dealing with the multiple
integrator with single input given by system (i D 1; : : : ; n  1I n  2)

xP i D xiC1 ;
(4.1)
xP n D b.t /u

1
The illustrations for the presented solutions were provided by N.Bairamov, A.Mesiats,
D.Odinokov, and V.Stepanovich.

© Springer International Publishing Switzerland 2014 147

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__4
148 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

with b.t / taken continuous. The fundamental transition matrix for this system is
0 1
1 t s : : : .t  s/n1 =.n  1/Š
B0 1 : : : .t  s/n2 =.n  2/Š C
B C
G.t; s/ D B : : :: :: C
@ :: :: : : A
0 0 0 1

Here, with initial conditions x10 D 0; : : : ; xn0 D 0; we have

Z t Z Ÿn1 Z t 
x1 .t /D ::: b.Ÿn /u.Ÿn /d Ÿn : : : d Ÿ1 D .t s/n1 ..n  1/Š/1 b.s/u.s/ ds
0 0 0

which is an iterated integration of multiplicity n.

Exercise 4.1.1. Is system (4.1) controllable?
Example 4.1. For system (4.1) assume x D .x1 ; : : : ; xn /0 ; x 2 Rn ; and n  n -
0
matrix X 0 D X 0 > 0. Find the forward reach set X Œt  D X .t; t0 ; X 0 / from starting
position ft0 ; X g; under constraint ju.s/j  ; with
0

X 0 D fx W hx  x 0 ; .X 0 /1 .x  x 0 /i  1g:

Note that system (4.1) is written in the form

xP D Ax C B.t /u; u 2 Q .t /; t  t0 ; (4.2)

where Q .t / D fu W juj  g and

0 1 0 1
010  00 0
B0 0 1  0 0C B C
B C B 0 C
B C B C
B0 0 0  0 0C B 0 C
ADB
B :: :: :: :: :: :: C
C;
B
B.t / D B :: C:
C
B: : : : : :C B : C
B C B C
@0 0 0  0 1A @ 0 A
000  0 0 b.t /

In order to calculate the reach sets for this system we shall follow Chap. 1 and
transform the coordinates so that the new system has A  0 and a new matrix
B. Applying transformation z.t / D G.t0 ; t /x.t /; t  t0 to (4.2), (see Sect. 1.1,
formula (1.6)), we get

zP D B.t /u; B.t / D G.t0 ; t /B.t /: (4.3)

4.1 The Multiple Integrator 149

We consider this system without loss of generality, under the same starting set X 0
and same constraint on control u as before. Now, with t  t0 , the forward reach
set is
Z t
Xz Œt  D X 0 C B.s/Q .s/ds; (4.4)
t0

while in the original coordinates

Z t
X Œt  D X .t; t0 ; X 0 / D G.t; t0 /X 0 C G.t; s/B.s/Q .s/ds: (4.5)
t0

Recall that here the transformation

0 1
1t s : : : .t0  t /n1 =.n  1/Š
B0 1 : : : .t0  t /n2 =.n  2/Š C
B C
z.t / D G.t0 ; t /x.t / D B : :: :: :: C x.t /
@ :: : : : A
0 0 ::: 1

and
0 1
.t0  t /n1 =.n  1/Š
B .t0  t /n2 =.n  2/Š C
B C
B.t / D B :: C b.t /:
@ : A
1

We now return in (4.3)–(4.4) from boldface letter B to ordinary B, getting xP D

B.t /u; where B.t / is time-dependent even when the original system (4.2) is time-
invariant.
We begin with some examples on calculating forward reach sets through
ellipsoidal approximations.
Given X 0 D E .x 0 ; X 0 /; Q .t / D E .q.t /; Q.t //; M D E .m; M /, we now
assume

Q .t / D E .qb .t /; QB .t //; QB .t / D B.t /Q.t /B 0 .t /; qb .t / D B.t /q.t /;

Xz0 D X 0 ; so that we deal with system

zP D B.t /u; z.t0 / 2 E .x 0 ; X 0 /; u 2 Q .t /; t  t0 : (4.6)

The required reach set is now

Z t
Xz Œt  D E .x 0 ; X 0 / C QB .s/ds
t0
150 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

with its internal and external approximating ellipsoids being E .xz .t /; Xz .t //;
E .xz .t /; XzC .t //: These are given by
Z t
xz .t / Dx C 0
qb .s/ds; (4.7)
t0

Z t
 0
  1=2
Xz .t / D Xz .t /Xz .t /; Xz .t / D .X 0 /1=2 C S.s/QB .s/ds; .S0 D I /;
t0
(4.8)

 Z  Z !
0 1=2
t
1=2 X0 t QB .s/
XzC .t / D hl; X li C hl; QB .s/li ds C ds ;
t0 hl; X 0 li1=2 t0 hl; QB .s/li1=2

(4.9)

where S.t / D S.t I l/ is an orthogonal matrix, so that S 0 .t /S.t / D I .

The reach set Xz Œt  is approximated as

E .xz .t /; Xz .t // Xz Œt  E .xz .t /; XzC .t //; (4.10)

where tightness ensures that the support functions

¡.lj E .xz .t /; Xz .t // D ¡.l j Xz Œt / D ¡.lj E .xz .t /; XzC .t //

along good curves l.t / 2 Rn : (See Sect. 3.3.) Here the good curves lz .t /  lz .t0 / D
l turn out to be constant. Therefore the previous relations turn into equalities for
any l under proper selection of parameterizing coefficients .t /; S.t /; in ODEs that
describe functions xz .t /, Xz .t /, XzC .t / for the approximating ellipsoids.
These ODEs are
( ( 8
 .t / D .X 0 /1=2 ; < XzC .t0 / D X 0 ;
xz .t0 / D x 0 ; Xz 0
P D /XzC C  1 .t /QB .t /
xP z D qb .t /I XP z
 D S.t /Q1=2 .t /I
B
X
: zC .t

(4.11)

with
 Z t 1
.t / D hl; QB .t /li 1=2
hl; X li C
0 1=2
hl; QB .s/li ds
1=2
t0

and with matrices S 0 .t /S.t / D I; selected such that vectors .X 0 /1=2 l; S.t /QB .t /l
1=2

are collinear.
4.1 The Multiple Integrator 151

Returning to the original coordinates fxg , we have

E .x  .t /; XC Œt / X Œt  E .x  .t /; X Œt /; (4.12)

where x  Œt  D G.t; t0 /xz Œt , and

XC Œt  D G.t; t0 /XzC Œt G 0 .t; t0 /; X Œt  D G.t; t0 /Xz Œt G 0 .t; t0 /:

Remark 4.1.1. In further relations below we omit the lower indices z for all the
related variables in (4.12).
For the backward reach sets the passage to new coordinates fzg is given by
z.t / D G.ª; t /x.t /; t  ª: Then, with given target set M D E .m; M /, we
again have Eq. (4.6) and in these new coordinates the backward reach set W Œt  D
W .t; ª; M /; M D E .m; M / also allows tight ellipsoidal approximations

E .w .t /; W .t // W Œt  E .w .t /; WC .t //:

The parameterized functions w .t /; WC .t /; W .t / may also be calculated

directly, through equations of Chap. 3 (see (3.97), (3.59), (3.98), (3.99)). The
corresponding ODEs are
( ( (
w .ª/ D m; W .ª/ D M 1=2 ; WC .ª/ D M;
wP  D qb .t /I WP  D S.t /Q .t /I
1=2
B
WP C D .t /WC   1 .t /QB .t /:
(4.13)

Here
Z !1
ª
.t / D hl; QB .t /li 1=2
hl; M li 1=2
C hl; QB .s/li 1=2
ds
t

with
Z ª
0
W D W W ; W D .W 0 /1=2 
1=2
S.t /QB .t /dt;
£

and S 0 .t /S.t / D I; while vectors M 1=2 l; S.t /QB .t /l are collinear.

1=2

4.1.1 Computational Results

Demonstrated here are the projections of reachability sets for system (4.2) of
dimension Dim = 4 and 8 on the plane fx1 ; x2 g at time ª: N stands for the number
152 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.1 N D 5; Di m D 4. Internal and external estimates

of approximating ellipsoids. The related parameters are x 0 D 0; £ D 0; ª D 2 for

dimension 4 and ª D 3 for dimension 8; q D 0; Q D 1;
0 1
1 0  0
B 0 0:001    0 C
B C
X0 D B : : : C:
@ :: :: : : ::: A
0 0    0:001

The directions l that ensure tight approximations are selected for time ª to be a
uniform partition on the unit circle in the plane fx1 ; x2 g (Figs. 4.1, 4.2, 4.3, 4.4, 4.5,
4.6, 4.7, and 4.8).

4.2 A Planar Motion Under Newton’s Law

This is the motion of a particle of mass m > 0 given by system

8
ˆ
ˆ xP 1 D x2 ;
<
mxP 2 D b1 u1 ;
(4.14)
ˆ xP 3 D x4 ;
:̂
mxP 4 D b2 u2

with b1 ; b2 > 0; t 2 Œt0 ; ª:

4.2 A Planar Motion Under Newton’s Law 153

Fig. 4.2 N D 5; Di m D 4. Internal estimates and their convex hull

Fig. 4.3 N D 20; Di m D 4. Internal and external estimates

154 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.4 N D 20; Di m D 4. Internal estimates and their convex hull

Fig. 4.5 N D 5; Di m D 8. Internal and external estimates

4.2 A Planar Motion Under Newton’s Law 155

Fig. 4.6 N D 5; Di m D 8. Internal estimates and their convex hull

Fig. 4.7 N D 20; Di m D 8. Internal and external estimates

156 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.8 N D 20; Di m D 8. Internal estimates and their convex hull

Taking further m D 1, we may transform this system to canonical first-order

form, which is
8
ˆ
ˆ xP 1 D x2 ;
<
xP 2 D b1 u1 ;
(4.15)
ˆ xP 3 D x4 ;
:̂
xP 4 D b2 u2 :

Our first task will be to find forward and backward reach sets

X Œª D X .ª; t0 ; X 0 /; X 0 D E .x 0 ; X 0 /; W Œt  D W .t; ª; M /; M D E .m; M /

under constraints
 
u1
u.t / D u D 2 Q .t / D E .q.t /; Q.t //; x.t0 / D x .0/ 2 X 0 :
u2

Denote
0 1 0 1 0 1
x1 0100 0 0
 
B x2 C B0 0 0 0C u1 B b1 0 C
xDB C B C B C
@ x3 A ; A D @ 0 0 0 1 A ; u D u2 ; B D @ 0 0 A :
x4 0000 0 b2
4.2 A Planar Motion Under Newton’s Law 157

Then system (4.14) has the form (4.2), here with B D const: The fundamental
matrix of its homogeneous part has the form G.t; £/ D exp A.t  £/; where Ak D
0; k  2: Hence
0 1
1t £ 0 0
B0 1 0 0 C
G.t; £/ D e A.t£/ D I C A.t  £/ D B
@0
C:
0 1t  £A
0 0 0 1

Using transformation z D G.t0 ; t /x; t  t0 ; taking B.t / D G.t0 ; t /B the

system (4.14) may be transformed, as in the previous section (see (4.6)), but without
changing original notations, into

xP D B.t /u.t /; u.t / 2 Q .t / D E .qB .t /; QB .t //; t  t0 ; (4.16)

with
0 1
.t0  t /b1 0
B b1 0 C
B.t / D G.t0 ; t /B D B
@
C ; qB .t / D B.t /q;
0 .t0  t /b2 A
0 b2
B.t /Q.t /B0 .t / D QB .t /:

Returning now from boldface B to B, the forward reach set will be

Z t
Xz Œt  D Xz .t; t0 ; X / D X C
0 0
B.s/Q .s/ds:
t0

With X 0 D E .x 0 ; X 0 / and support function

¡.l j Xz Œt / D (4.17)
Z t
hl; x 0 i C hl; X 0 li1=2 C .hl; qB .t /i C hl; QB .t /li1=2 /dt:
t0

Similarly, for the backward reach set we use transformation z D G.ª; t /x; t  ª;
and take B.t / D G.ª; t /B: Then system (4.14), without changing notations, may
be again transformed into (4.17), but now with t  ª: Here we also return from
boldface B to B:
For target set M D E .m; M / the backward reach set will be
Z ª
Wz Œt  D Wz .t; ª; M / D M  B.s/Q .s/ds
t
158 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

with support function

¡.l j Wz Œt / D (4.18)
Z ª
hl; mi C hl; M li 1=2
 .hl; qB .t /i  hl; QB .t /li1=2 /dt:
t

The ellipsoidal approximations of Xz Œt ; Wz Œt  are given by relations

(4.7)–(4.13).

4.2.1 Computational Results

Forward Reach Sets

Presented are calculations of the forward reach set X Œ1 D X .1; 0; E .x 0 ; X 0 // for
the four-dimensional system

xP D B.t /u; b1 D 2; b2 D 1; t 2 Œ0; 1 (4.19)

and
0 1 0 1
0 3000
   
B0C B C
x0 D B C ; X0 D B 0 1 0 0 C ; q D 1 ; Q D 1 0 :
@0A @0 0 1 0A 1 04
0 0003

Indicated are two-dimensional projections of external and internal ellipsoidal

approximations for X Œt  on the system of coordinates .x1 ; : : : ; x4 /0 . Also shown
are the convex hulls of the intersection of external estimates (as a red line, Figs. 4.9
and 4.11) and the union of internal estimates (as a blue line in Fig. 4.10 and dashed
line in Fig. 4.11). Here internal ellipsoids are in green.
Indicated here are two two-dimensional projections of the reach tube for the
system of the above (external and internal, Figs. 4.12 and coinciding 4.13).

Backward Reachability Sets and Control Synthesis: Aiming Methods

We are now passing to the calculation of control strategies. Our task in the simplified
coordinates (from z to x D G.t; ª/z) will be to transfer system (4.19) from given
position ft0 ; xg to the final one m 2 M within prescribed time ª  t0 . As we have
seen in Sects. 2.6 and 3.11, the task will be solvable iff x 2 W Œt0 , where W Œt0  D
W .t0 ; ª; M / is the backward reachability set from target set M .
4.2 A Planar Motion Under Newton’s Law 159

Fig. 4.9 N D 15; Di m D

4; fx1 ; x2 g. External convex
hull and internals

Fig. 4.10 N D 15; DIM D

4; fx1 ; x2 g. Internals with
their convex hull
160 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.11 N D 15; Di m D

4; fx1 ; x2 g. External and
internal convex hulls

Fig. 4.12 N D 15; Di m D

4; fx1 ; x3 g. Projection of
reach tube—coordinates of
motion
4.2 A Planar Motion Under Newton’s Law 161

Fig. 4.13 N D 15; Di m D

4; fx2 ; x4 g. Projection of
reach tube—velocities of
motion

We therefore have a problem in the next form: find feedback control strategy
u D U.t; x/ which ensures the following conditions

P / D B.t /u.t /; t 2 Œt0 ; ª;
x.t
(4.20)
x.t / 2 Rn ; x.ª/ 2 E .m; M /; u.t / 2 E .q.t /; Q.t //;

whatever be the starting position ft0 ; xg; x 2 W Œt0 .

To find the feedback solution strategies we will apply the aiming methods
described in Sects. 2.6 and 3.11. For calculating the solution we will substitute W Œt 
by its tight internal ellipsoidal approximation E Œt  D E .w? .t /; W Œt /. Then, as
indicated in Sect. 3.11, the synthesizing feedback control u D U.t; x/ has the next
form .q.t / 2 E .q.t /; Q.t ///
(
q.t /; if x 2 E Œt ,
U.t; x/ D 0 0 0
1
0 2
q.t /  Q.t /B .t /l hB .t /l ; Q.t /B .t /l i
0 0
; if x … E Œt .
(4.21)
Here the unit vector l 0 D .x.t /  s 0 /=kx.t /  s 0 k, where

s 0 D argminfkx  sk W s 2 E Œt ; x D x.t /g

so that l 0 is the gradient of the distance function d.x; E Œt / with t fixed and s 0 is
the nearest point of E Œt  from x 62 E Œt .
162 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Aiming method (a). As indicated in Sect. 3.11 (T=I) vector s 0 D .Q C

œE/1 Q.z  q/ C q for E Œt  D E .q.t /; Q.t // may be found from equation
h.œ/ D 0, where

h.œ/ D h.Q C œE/1 Q.z  q/; .Q C œE/1 .z  q/i  1:

and œ > 0 is its only positive root (prove that).

Here the bottleneck is to find a proper E Œt  among such internal ellipsoids which
would actually be the closest to x.
A second option—aiming method (b)—is to calculate the distance function in
the metric generated by E .0; Q/, namely,

s 0 D argminfkx  skQ1 W s 2 E Œt ; x D x.t /g:

Then s 0 will lie on the line that connects point x and w? , hence

l 0 D .x  w? /=kx  w? k:

The final formula will then be as follows

8
ˆ
< q.t /; if x 2 E Œt ,
U.t; x/ D q.t /  Q.t /B 0 .t /l 0 (4.22)
1 ; if x … E Œt .
:̂
hB 0 .t /l 0 ; Q.t /B 0 .t /l 0 i 2

Such are the two approaches to the calculation of l 0 .

In the second approach one may use several internal ellipsoids Ek Œt  since they
all have a common center w? .t / for each t . The solution formula is as follows
8 S
ˆ
ˆ q.t /; if x 2 Ek Œt ,
< k
U.t; x/ D Q.t /B 0 .t /l 0 S (4.23)
ˆ q.t /  1 ; if x … Ek Œt .
:̂
hB 0 .t /l 0 ; Q.t /B 0 .t /l 0 i 2 k

In the calculation of reachability sets for phase coordinates x of large dimensions

and controls of small dimensions the internal ellipsoids Ek Œt  may turn out to be
degenerate. A separate treatment of such situations is given below, in Sect. 4.4.3.
For the two-dimensional system

1
xP 1 D x2 ; xP 2 D u C 1; M D E .m; M /; m D .1; 1/0 ; M D I; juj  1;
2

indicated here are the projections of the backward reach set W Œ£ for £ D 0; ª D 3
(Figs. 4.14, 4.15, 4.16, 4.17, 4.18, 4.19, 4.20, 4.21, and 4.22).
4.2 A Planar Motion Under Newton’s Law 163

Fig. 4.14 N D 5; DIM D 2. Backward reach set—internal and external estimates

Fig. 4.15 N D 5; DIM D 2. Backward reach set—internal estimates and their convex hull
164 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.16 N D 30; DIM D 2. Backward reach set—internal and external estimates

Fig. 4.17 N D 30; DIM D 2. Backward reach set—internal estimates and their convex hull
4.2 A Planar Motion Under Newton’s Law 165

Fig. 4.18 N D 5; Di m D 2. Backward reach tube—internal and external estimates

Fig. 4.19 N D 15; DIM D 2. Backward reach tube—coinciding internal and external estimates
166 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.20 N D 5; Di m D 2.
Closed-loop chattering
control—aiming method (b)
in the ellipsoidal norm

Fig. 4.21 N D 15;

Di m D 2. Closed-loop
chattering control—aiming
in the Euclidean norm
4.2 A Planar Motion Under Newton’s Law 167

Fig. 4.22 N D 15;

DIM D 2. Continuous
closed-loop target control

Indicated further is the closed-loop controlled trajectory that reaches target M

with aiming in the Euclidean metric (a) and in the metric of the approximating
ellipsoid (b).

Aiming Method (c) for Continuous Control

In the previous figures the controls are of chattering type, ensuring sliding-type
regimes when the trajectory fluctuates along the boundary of the backward reach set
W Œt . However, one may introduce a solution with continuous control, when having
started from inside W Œ£ the trajectory does not reach the boundary of W Œt ; t > £.
Such control is constructed as follows (T D W1 .t /)
8
< q.t /; if hx  w? .t /; W1 .t /.x  w? .t //i  1;
Ue .t; x/ D otherwise
:
q.t /  QB 0 .t /.x  w? .t //kQ1=2 B 0 .t /.x  w? .t //k1 :
168 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

4.3 Damping Oscillations

4.3.1 Calming Down a Chain of Springs in Finite Time

Given is a vertical chain of a finite number n of suspended springs subjected to a

vertical control force applied to the lower end of the chain. The chain also includes
loads of given mass attached in between the springs at their lower ends and the
masses of the springs are taken to be negligible relative to the loads. The upper end
of the chain is rigidly fixed to a suspension (see Fig. 4.23).
The oscillations of the chain are then described by the following system of
second-order ODEs (i D 2; : : : ; j  1; j C 1; : : : ; n  1):
8
ˆ m1 wR 1 D k2 .w2  w1 /  k1 w1 ;
ˆ
ˆ
ˆ
ˆ
ˆ
ˆ
:::
ˆ
ˆ
< mi wR i D kiC1 .wiC1  wi /  ki .wi  wi1 /;
::: (4.24)
ˆ
ˆ
ˆ
ˆ mj wR j D kj C1 .wj C1  wj /  kj .wj  wj 1 / C uj .t /;
ˆ
ˆ
ˆ
ˆ:::
:̂
mn wR n D kn .wn  wn1 /

with t > t0 D 0. Here n is the number of springs as numbered from top to bottom.
The loads are numbered similarly, so that the i -th load is attached to the lower end
of the i -th spring; wi is the displacement of the i -th load from the equilibrium, mi is

Fig. 4.23 An oscillating

system
4.3 Damping Oscillations 169

the mass of the i -th load, ki is the stiffness coefficient of the i -th spring. The gravity
force enters (4.24) implicitly, through the lengths of the springs at the equilibrium.
The initial state of the chain at time t0 D 0 is given by the displacements w0i and
the velocities of the loads wP 0i , with

wi .t0 / D w0i ; w
P i .t0 / D wP 0i ; i D 1; : : : ; n:

The control u D uj .t / in this equation is indicated at the j-th load (which may
be j D n) with its values confined to the interval Q D Œumin ; umax . In particular,
this constraint may be symmetric: Q D Œ;  or one-sided: directed only down
(Q D Œ0; ) or only up (Q D Œ; 0).
We now rewrite system (4.24) in normal matrix form introducing the extended
vector x 2 R2n such that .x1 ; : : : ; xn /0 D w; .xnC1 ; : : : ; x2n /0 D w.
P Then
 
w0
P / D Ax.t / C b .j / uj .t /;
x.t x.t0 / D x 0 D (4.25)
P0
w
 
0 I 0
b .j / D .0; : : : ; 0; bnCj ; 0; : : : ; 0/; bnCj D m1
.j / .j /
AD ; j ;
M 1 K 0
0 1
m1
B :: C
M D@ : A;
mn
0 1
k1 C k2 k2
B k2 k2 C k3 k3 C
B C
B : : : C
K D B ::: :: :: :: ::: C :
B C
@ kn1 kn1 C kn kn A
kn kn

If the controls are applied at several loads j1 ; : : : ; jk , then vector b .j / in 4.25

has to be substituted by matrix B D .b .j1 / ; : : : ; b .jk / / and the control will be
k-dimensional: u0 D .uj1 .t /; : : : ; ujk .t //:
Exercise 4.3.1. (i) Prove that system (4.25) is controllable if j D 1 or j D n.
(ii) Is the system controllable for any j 6D 1 and j 6D n‹:
(iii) Find the necessary and sufficient conditions for controllability of sys-
tem (4.25), when mi  m; ki  k (m, k are the same for all i D 1; : : : ; n).
170 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Examples of Solutions Through Ellipsoidal Methods

The emphasis of this book is not only to describe exact solutions but also to indicate
effective numerical methods. Based on ellipsoidal approximations such methods
rely on specially derived ordinary differential equations, indicated above in Chap. 3.
We now present some examples of feedback control for a chain of springs. These
are solved by schemes explained in Sect. 3.11.
Example 4.2. Chain of four springs with four loads with control at each load
Indicated are projections of the backward reachability set for system (4.24) with
four springs and four loads of different masses fmi g D f3; 1; 1:5; 5g and different
coefficients of stiffness fki g D f2; 1; 4; 1g.
We assume target set M D E .0; M /; M D 0:1I and the control in (4.25) to be
four-dimensional: Bu D fb .5/ u1 ; : : : ; b .8/ u4 g (applied to each load),
0 1
1 1 0 0
B1 2 0 0C
u 2 E .0; Q/; QDB
@0
C:
0 2 0A
0 0 0 1

The phase space for our system is of dimension 8, with unit orths of from
e10 D .1; 0; : : : ; 0/ 2 R 8 to e80 D .0; : : : ; 0; 1/ 2 R 8 : Our aim is to calculate
the solvability set (backward reach set) W Œt  D W .t; ª; M / with ª D 3, seeking
for the projections of eight-dimensional W Œt  on two-dimensional planes fw5 ; w6 g
and fw7 ; w8 g of system velocities. To find these projections we need the related
external and internal ellipsoidal approximations that are calculated for good curves
l.t / along which they would be tight. The boundary conditions l D l.ª/ for such
curves are taken along a partition of unit circles on the planes e5 ; e6 and e7 ; e8 for
the velocities. The evolution of these projections in time will reflect the dynamics of
tube W Œt . The result of intersecting approximating ellipsoids is given in Figs. 4.24,
4.25, 4.26, and 4.27. 
Example 4.3. Target control trajectories for chain of four springs with four loads:
Control at each load
Indicated are the target controlled trajectories with four springs and four loads
of different masses fmi g D f5; 2; 4; 1g and different coefficients of rigidity fki g D
f2; 1; 1:5; 3g; taking M D E .0; 0:1I /: The aiming is done within scheme (b) of
Sect. 3.11, in the metric of internal ellipsoids. The results are illustrated in Figs. 4.28,
4.29, 4.30, and 4.31 with boundary of target set in blue.
Example 4.4. Oscillating springs with two loads. Considered is a system with two
loads and control applied to one of them.
4.3 Damping Oscillations 171

Fig. 4.24 Di m D 8. External approximations of backward reach set—coordinates x5 ; x6

Fig. 4.25 Di m D 8. External approximations of backward reach set—coordinates x7 ; x8

172 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.26 Di m D 8. Internal approximations of backward reach set—coordinates x5 ; x6

Fig. 4.27 Di m D 8. Internal approximations of backward reach set—coordinates x7 ; x8

4.3 Damping Oscillations 173

Fig. 4.28 Di m D 8: Projection of phase trajectory—position coordinates x1 ; x2

Fig. 4.29 Di m D 8: Projection of phase trajectory—position coordinates x3 ; x4

174 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.30 Di m D 8:
Projection of phase trajectory:
velocities x5 ; x6

Fig. 4.31 Di m D 8:
Projection of phase trajectory:
velocities x7 ; x8

Calculated are:
– projections of approximations of backward reach set at final time by arrays of
ellipsoids (their boundaries are marked blue for externals and green for internals;
an accurate approximation through many ellipsoids indicates that the result is
seen as almost the same).
– solutions to the problem of ellipsoidal control synthesis with pictures showing
starting set is in green and boundary of target set in red.
(i) Two loads. Control at upper load.
Here, in system (4.24), the number of loads is N D 2, m1 D 2; m2 D 2, k1 D 1:3,
k2 D 2, w 2 R2 , t0 D 0; t1 D 7. The control force u 2 Œ1; 1 is applied only to
the upper load. In system (4.25) x 2 R4 : The approximated backward reach tube,
as calculated in backward variables tN D 7  t; t 2 Œ0; 7; emanates at time tN D 0;
4.3 Damping Oscillations 175

Fig. 4.32 Dim 4: Projections of approximation of backward reach set with control at upper load
on two-dimensional planes in axes (position, velocity) for each load

.t D 7/; from target set M —a ball of radius 0:01 centered at 0, and terminates at
t D 0; T D tN D 7: Reach sets at shown in backward time at T D 7.t D 0/, see
Fig. 4.32.
The controlled trajectory evolves from starting set X 0 , centered at point x.0/ D
x D Œ0:5688; 1:0900; 0:8921; 0:24230 to target set M —a ball of radius 0:1
0

centered at m D 0. It is shown in Figs. 4.33 and 4.34. 

(ii) Two loads. Control at lower load.
The system in same as in (i), but control is applied only at lower load.
The approximation of backward reach set is calculated as in previous case (i),
within the same time interval, but with changed location of control. It is shown at
final time in Fig. 4.35.
The starting set is centered at x 0 D Œ1:4944; 1:9014; 0:2686; 1:2829T and
terminal set is the same as in (i), see Figs. 4.36 and 4.37.
Example 4.5. Oscillating springs with four loads. Considered is a system with four
loads and control applied to only one of them.
Calculated are:
– projections of backward reach set at final time (their boundaries are marked blue
for externals and green for internals; an accurate solution indicates that they are
seen as almost the same).
– solutions to the problem of ellipsoidal control synthesis with pictures showing
starting set is in green and boundary of target set in red.
Control Applied at First (Upper) Load

Here, in system (4.24), the number of loads is N D 4, m1 D 1:5; m2 D

2; m3 D 1:1; m4 D 4, k1 D 3; k2 D 1:4; k3 D 2:1; k4 D 1, w 2 R4 ,
t0 D 0; t1 D 10. The control force u 2 Œ1; 1 is applied only to the upper load. In
system (4.25) x 2 R8 : The backward reach tube, as calculated in backward variables
176 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.33 Dim 4: The target controlled trajectories for system of two loads with control at upper
load

tN D 10  t; t 2 Œ0; 10; emanates at time tN D 0; .t D 10/; from target set M —a

ball of radius 0:01 centered at 0, and terminates at t D 0; T D 10: The projections
of these reach sets at shown in backward time at T D 10.t D 0/, see Fig. 4.38.
The controlled trajectory evolves from starting set X 0 , centered at point x.0/ D
x D Œ1:3691I 0:4844I 0:8367I 0:0432I 1:6817I 0:5451I 1:9908I 0:22770 to tar-
0

get set M —a ball of radius 0:1 centered at m D 0. It is shown in Figs. 4.39 and 4.40.

Control Applied at Fourth (Lowest) Load

Here the problem parameters are the same as with control at first (upper) load except
that now it is applied to the fourth (lowest ) load.
Projections of the reach set are shown in Fig. 4.41. The controlled trajectory now
evolves from starting set X 0 , centered at point

x.0/ D x 0
D Œ0:6296I 1:9940I 2:6702I 3:6450I 0:1013I 0:1054I 0:2913I 0:63830

to the same target set M —a ball of radius 0:1 centered at f0g. It is depicted in
Figs. 4.42 and 4.43.
4.3 Damping Oscillations 177

Fig. 4.34 Dim 4: Projections of phase trajectories of target controlled system with control at upper
load

In order to calculate exact solutions to our control problems one needs to heavily
rely on calculating fundamental matrix G.t; s/: Their calculation may also occur in
other elements of the solution process. The calculation of such matrices for large
dimensions is a special direction in numerical methods (see [89, 90, 92]). In the next
lines we indicate an algorithmic method applicable to differential equations for the
chain of springs.
178 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.35 Projections of approximation of backward reach set with control at lower load on two-
dimensional planes in axes (position, velocity) for each load

Fig. 4.36 The target controlled trajectories for system of two loads with control at lower load

Computing the Fundamental Matrix

For stationary systems matrix function G.t; s/ D exp A.t  s/ which is a matrix
exponent. Consider calculation of such an exponent X.t / D e At , assuming
 
0 I
AD ;
M 1 A0 0
4.3 Damping Oscillations 179

Fig. 4.37 Dim 4: Projections of phase trajectories of target controlled system with control at lower
load

where matrix M is diagonal and matrix A0 is symmetric and tri-diagonal (nonzero

elements are allowed only on the main diagonal and those directly above and under).
Consider the powers of A. Having observed
 
M 1 A0 0
A D2
;
0 M 1 A0
180 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.38 Projections of approximation of backward reach set with control at first (upper) load on
two-dimensional planes in axes (position, velocity) for each load

we further obtain
 
.1/k .M 1 A0 /k 0
A2k D ;
0 .1/k .M 1 A0 /k
 
0 .1/k .M 1 A0 /k
2kC1
A D kC1 1 kC1 :
.1/ .M A0 / 0

1 1
Introducing new matrix L D .M  2 A0 M  2 /, we have

1 1
!
.1/k M  2 L2k M 2 0
A 2k
D 1 1 ;
0 .1/k M  2 L2k M 2

1 1
!
0 .1/k M  2 L2kC1 L1 M 2
A 2kC1
D 1 1
.1/kC1 M  2 L2kC1 LM 2 0
4.3 Damping Oscillations 181

Fig. 4.39 The target controlled trajectories for system of four loads with control at first (upper)
load

and further on
1
X 1
e At
D .At /k D
kŠ
kD0

1
!  1
!
M2 0 cos.Lt / sin.Lt /L1 M2 0
D 1 1 : (4.26)
0 M2  sin.Lt /L cos.Lt / 0 M2

The algorithm for calculating e At is as follows:

1 1
1. Calculate symmetric matrix L2 D M  2 A0 M  2 .
2. Calculate singular decomposition for matrix L2 D UK V T , where matrices U , V
are orthogonal (here U D V due to symmetry of L2 ) and K is diagonal. Since L2
is a tri-diagonal matrix, one may use special algorithms effective for this specific
class of matrices (with complexity of order N log3 N ).
1
3. Calculate matrix K 2 .
1 1
4. For each instant t calculate diagonal matrices sin.tK 2 /, cos.tK 2 /.
1 1
5. Calculate matrix cos.tL/ D U cos.tK 2 /V T , sin.tL/ D U sin.tK 2 /V T .
6. Calculate e At using formula (4.26).
182 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.40 Projections of phase trajectories of target controlled system with control at first (upper)
load

The presented algorithm relies on the special structure of matrix A and proves to
be far more effective than the general algorithms that do not rely on such structure.
A simulation experiment indicates an eight-times more effective performance of the
structure related algorithm as compared to the “ordinary” one.

4.4 Computation in High-Dimensional Systems. Degeneracy

and Regularization2

4.4.1 Computation: The Problem of Degeneracy

As we have observed, the basic problem treated here is how to calculate ellipsoidal
estimates for reachability tubes of a control system. Its solution is a crucial element
for calculating feedback target control U .t; x/ that steers the system trajectory from

2
This section follows paper [58].
4.4 Computation in High-Dimensional Systems. Degeneracy and Regularization 183

Fig. 4.41 Projections of approximations of backward reach set with control at fourth (lowest) load
on two-dimensional plane in axes (position, velocity) for each load

given position ft0 ; X 0 g to given target set M . Namely, as indicated in Sect. 2.6, to
apply the aiming rule, one needs to calculate the backward reachability (solvability)
tube W Œt  for the investigated system. For linear-convex systems of type (3.1) this
is done through ellipsoidal methods of Sect. 3.11 which proved successful for many
problems. This route is especially useful for systems of higher dimensions where
finding set W Œt  directly is computationally too cumbersome. Thus we use its
internal ellipsoidal approximations. Here we require that constraints in the problem
are ellipsoidal: X 0 D E .x 0 ; X 0 /, Q .t / D E .q.t /; Q.t //. Otherwise sets Q .t / have
to be approximated by their internal or external ellipsoids. Functions q.t /, Q.t / for
the constraints are assumed to be continuous.
With W Œt  given, the desired control strategy U D Ue .t; x/ is given
by (3.136), (3.137). Following Sects. 3.9, 3.11, and (3.123), we observe that the
internal ellipsoids are generated by the following ODEs:

wP ? .t / D A.t /w? .t / C B.t /q.t / (4.27)

184 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Fig. 4.42 The target controlled trajectories for system of four loads with control at fourth (lowest)
load

with initial condition w? .t0 / D m, and

WP  .t / D A.t /W .t / C W .t /A0 .t /

 ŒH0 .t /W .t / C W .t /H.t /; (4.28)

1 1
H.t / D W 2 .t /S.t /QB2 .t /;

with boundary condition W .t1 / D M: Here S.t / is a parameterizing orthogonal

1 1
matrix such that vectors S.t /QB2 l.t / and M 2 l.t / are collinear and column l.t / is a
“good” curve—the solution to equation

d l.t /
D A0 .t /l.t /; l.t0 / D l:
dt

Ellipsoidal approximations Ew Œt  D E .w? .t /; W .t // are “tight” in the sense that

they touch the exact reachability set in the direction of vector l.t /:

¡.l.t / j Ew Œt / D ¡.l.t / j X Œt /:

The computation of external estimates is less difficult than of the internals which
are therefore less investigated. However, the internals are crucial for designing
feedback controls, so these difficulties, which increase with dimension, have to be
coped with. We therefore concentrate on internals.
4.4 Computation in High-Dimensional Systems. Degeneracy and Regularization 185

Fig. 4.43 Projections of phase trajectories of target controlled system with control at fourth
(lowest) load

An attempt to apply established approximation formulas from [182] to oscillating

systems of high dimensions [58] with scalar control (nu D 1) revealed that the
configuration matrix W Œt  of ellipsoid Ew Œt  is ill-conditioned (namely, n  1 out
of n existing semi-axes of Ew Œt  are of length close to zero). This raises a serious
issue for computations for the following reasons:
(a) errors caused by numerical integration of approximating ODEs may cause
matrix W Œt  to have negative eigenvalues, which is unacceptable;
(b) the only information provided by a degenerate ellipsoid with a single positive
axis is the value of support function in the direction of that axis. However, this
value may be obtained by simpler calculations (not involving the solution of a
matrix ODE).
We indicate some means of coping with these difficulties. For case (a) this
will be to regularize the original procedures of Sect. 3.9. The idea is to calculate
a set of ellipsoidal approximations which are then being “mixed” throughout the
process. For case (b) the issue is that high-dimension systems require a polynomially
increasing computational burden. This is met by introducing a new efficient method
for calculating matrices S.t /.
186 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Finally in this section we discuss a parallelized software implementation of the

presented formulas. The given algorithms have proved to be effective for systems of
ODEs of dimension up to 500.

4.4.2 Regularizing the Ellipsoidal Estimates

Numerical experiments with internal ellipsoidal estimates of the reachability tube

for a high-order oscillating system of the (4.24) type (see [58]) with a scalar control
lead to the following conclusion. If the terminal set M has small diameter, then
the internal ellipsoidal estimates of the reach tube are close to degenerate. Namely,
as time t increases, only one eigenvalue of matrix W .t / grows. Other eigenvalues
remain close to those of matrix M .
We start this section by demonstrating the cause of such degeneracy. To do this,
we analyze the formula for the internal ellipsoidal estimate of the geometrical sum
of two ellipsoids. After that we indicate how one copes with this degeneracy by
weakening the requirements on the tightness of estimates. Finally we extend this
approach to calculating the internal estimates of the reachability tube.

Degeneracy of the Sum of Degenerate Ellipsoids

Recall from Sect. 3.7 that the formula for an internal ellipsoidal estimate of the
geometrical sum of m ellipsoids E .qi ; Qi /, i D 1; : : : ; m, tight in direction ` 2 Rn :

E .q1 ; Q1 / C    C E .qm ; Qm /
E .q; Q/; (4.29)

X
m
1 1 1
qD qi ; Q D QŒm0 QŒm; QŒm D Q12 C S2 Q22 C    C Sm Qm2 ;
iD1

where Si are orthogonal matrices satisfying condition:

1 1
Q12 ` is collinear with Sj Qj2 `; j D 2; : : : ; m. If any of the matrices Qi ; i D
1; : : : m; is degenerate, relation (4.29) makes sense for directions ` such that Qi ` ¤
0, for all i D 1; : : : ; m.
Theorem 4.4.1. Suppose rankQi D ri and Qi ` ¤ 0, i D 1; : : : ; m. Then the rank
of matrix Q D QŒm0 QŒm is limited by r1 C    C rm  .m  1/.
1 1
Proof. Denote column spaces of matricesPQ12 and Si Qi2 as L1 and Li , respectively,
i D 2; : : : ; m. Then the range imQ miD1 Li . By definition of matrices Si the
1
nonzero vector l .1/ D Q12 ` belongs to all subspaces Li . Therefore the total number
of linearly independent vectors in the image imQ cannot be greater than r1 C
.r2  1/ C    C .rm  1/. t
u
4.4 Computation in High-Dimensional Systems. Degeneracy and Regularization 187

Corollary 4.4.1. The set of rank 1 matrices is closed with respect to opera-
tion (4.29).
This means a collection Q of matrices of rank 1, made according to (4.29), is also
of rank 1.
Remark 4.4.1. The degeneracy of estimates is the result of their tightness (the
requirement that they touch the exact set). This is not a property of the particular
formula (4.29). Indeed, if rankQi D 1, then the summed ellipsoids may be
presented as Ei D E .qi ; Qi / D convfqi ˙ ai g, where vectors ai are the only
nonzero semi-axes of these ellipsoids. Then the exact sum E1 C    C Em will be a
polyhedron convfq1 C    C qm ˙ a1 ˙    ˙ am g. And for almost all directions `
the only tight approximation will be one of the diagonals of this polyhedron. In this
case formula (4.29) describes exactly those diagonals.
Remark 4.4.2. For some choices of orthogonal matrix S the dimension of the inter-
nal ellipsoidal estimate may be strictly less than the dimension of ellipsoids being
added. For example, if Q1 D Q2 D I , ` D l .1/ , S D diagf1; 1; 1; : : : ; 1g,
then (4.29) gives Q D diagf1; 0; 0; : : : ; 0g.

Regularizing the Estimate for the Sum of Degenerate Ellipsoids

Applying results of the above to systems of type (4.27), (4.28), we indicate how
several degenerate estimates may be combined (mixed) to get an estimate of higher
dimension. This approach is based on the formula for internal ellipsoidal estimates
of the convex hull for the union of ellipsoids [174].
Lemma 4.4.1. If ellipsoids Eiw D E .w; W.i/ /, i D 1; : : : ; m, are internal estimates
of a convex set X , then ellipsoid

X
m X
m
E’w D E .w; W ’ /; W’ D ’i W.i/ ; ’i  0; ’i D 1; (4.30)
iD1 iD1

will also be an internal estimate of X .

p
Proof. Since t is concave, we have
! 12
X
m
¡.` j Ew
’/ D hw; `i C ’i h`; W.i/ `i (4.31)
iD1

X
m
1
’i .hqi ; `i C h`; W.i/ `i 2 /  max ¡.` j Ew i /;
iD1;:::;m
iD1
188 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

hence

[
m
E’w conv Eiw X :
iD1

t
u
Theorem 4.4.2. Suppose w D 0 and the dimension of L (the linear hull of Eiw ) is r.
Then if ’i > 0, i D 1; : : : ; m, the following equality holds: the image of mapping
W’ is imW’ D L . In particular, the matrix W’ is of rank r.
Proof. This theorem means that

im.’1 W.1/ C    C ’m W.m/ / D imW.1/ C    C imW.m/ :

Due to the symmetricity of matrices W.i/ , the latter is equivalent in terms of matrix
kernels to

ker.’1 W.1/ C    C ’m W.m/ / D ker W.1/ \    \ ker W.m/ :

The inclusion of the right-hand side into the left is obvious. On the other hand, if

x 2 ker.’1 W.1/ C    C ’m W.m/ /;

then taking the inner product of equality ’1 W.1/ C    C ’m W.m/ D 0 by x, on both

sides, we get

’1 hx; W.1/ xi C    C ’m hx; W.m/ xi D 0:

Since matrices W.1/ are non-negative definite and ’i is positive, we observe that
hx; W.i/ xi D 0. Hence x 2 ker W.1/ . t
u
Note some properties of estimates E’w :
1. Suppose ellipsoidal approximation E1w is tight in direction `, i.e. ¡.` j E1w / D
¡.` j X /. Let us now estimate the difference between its support function and
that of ellipsoid E’w in the same direction. After some calculation we have:
1
¡.` j E’w /  hw; `i C ’1 h`; W.1/ `i 2
1 p
D hw; `i C h`; W.1/ `i 2 1  .1  ’1 / D

1 1
D ¡.` j X /  .1  ’1 /h`; W.1/ `i 2 C O..1  ’1 /2 /:
2
4.4 Computation in High-Dimensional Systems. Degeneracy and Regularization 189

Fig. 4.44 Internal ellipsoidal

approximations of sum of two
ellipsoids

Therefore, the closer ’1 is to 1, the closer are ellipsoids E’w to the tight internal
ellipsoid along direction `.
2. If all the ellipsoids Eiw , i D 1; : : : ; m, are tight approximations of X along the
same direction `, then in (4.31) an equality is true and ellipsoid E’w is also a tight
approximation of X along `.
Example 4.6. Figure 4.44 shows ellipsoidal estimates of the sum of two degenerate
ellipsoids. Original ellipsoids Ei are two line segments (thick dotted line), and their
sum is a parallelogram (thin dotted line). Due to Theorem 4.4.1 tight approximations
of the sum are also degenerate ellipsoids (line segments) shown by thick solid line.
Regularized approximations with ’1 D 10 1 1 9
; 2 ; 10 are presented with thin solid lines.
These are non degenerate due to Theorem 4.4.2 and touch the parallelogram (note
that they are tight in direction of normals to the sides of the parallelogram, see
property 2). Besides that, for ’ D 10 1
and 10 9
thanks to property 1 the support
functions for the estimates are close to the support function of the parallelogram
along corresponding directions.
Figure 4.45 shows internal ellipsoidal approximations of solvability set, as
calculated by (4.31), for an oscillating system xP 1 D x2 , xP 2 D x1 C u on
time interval Œ0;  with parameter ” D 20 1
(left) and 21 (right). Exact (tight)
approximations are degenerate ellipsoids (shown with thick lines).

4.4.3 Regularizing the Estimate for the Reachability Tube

We choose in (4.30) the following values of parameters ’:

’1 D 1  ¢ ” C ¢ ”“i ; ’i D ¢ ”“i ; i D 2; : : : ; m;
P
where “i  0 and m iD1 “i D 1; ”  0.
Here ¢ is sufficiently small, ensuring ’1 > 0. Then
!
X
m
¢ 1 .W’  W.1/ / D ” “i W.i/  W.1/ :
iD1
190 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

2 2

1.5 1.5

1 1

0.5 0.5

0 0

−0.5 −0.5

−1 −1

−1.5 −1.5

−2 −2
−2 −1 0 1 2 −2 −1 0 1 2

Fig. 4.45 Internal ellipsoidal estimates of the reachability set of a 2D system (left: ” D 1
20
, right:
” D 12 )

We use this result to mix m ellipsoidal estimates of the reachability tube as follows:

WP .i/ D A.t /W.i/ C W.i/ A0 .t /  ŒH0 .t /W.i/ C W.i/ H.t /C

0 1
X
m
C” @ “ij W.j /  W.i/ A ; W.i/ .t1 / D M;
j D1

1 1
with H.t / D .W.i/ .t // 2 S.t /QB2 .t /.
Recall that Si .t / are arbitrary orthogonal matrices such that vectors
1 1
Si .t /.B.t /Q.t /B 0 .t // 2 li .t / and .W.i/ .t // 2 li .t / are directionally collinear. The
column vector functions li .t / are solutions to related adjoint systems

d li .t /
D A0 .t /li .t /; li .t0 / D `i :
dt
Remark 4.4.3. Parameter ”  0 controls the level of “mixed” approximations: the
higher is ”, the greater is the impact of mixing. Parameters “ij  0 control the
configuration of the mixture (namely, which ellipsoids are being mixed).

Remark 4.4.4. The choice of identical coefficients “ij D “O j (in particular, “ij D m1 )
P
reduces the number of operations, since in this case the sum m j D1 “ij W .t / D
.j /
Pm O .j /
j D1 “j W .t / does not depend on j and is calculated only once for each time
step.
Theorem 4.4.3. Suppose matrix-valued solutions W.i/ .t / to Eq. (4.28) are extend-
able to interval Œt0 ; t1 , and are positive definite on this interval. Then the set-value
function

[
m [
m
W Œt  D conv Eiw Œt  D conv E .w .t /; W.i/ .t //
iD1 iD1
4.4 Computation in High-Dimensional Systems. Degeneracy and Regularization 191

satisfies for t 2 Œt0 ; t1  the funnel equation

lim ¢ 1 hC ..I  ¢A.t //W Œt  ¢; W Œt   ¢ E .0; QB .t // D 0 (4.32)

¢!0C

with boundary condition W Œt1  M D E .m; M /.

Without loss of generality consider q.t /  0; m D 0, and therefore,
w? .t /  0. Then, changing variables z.t / D G.t0 ; t /x.t /, as in Sect. 1.1, we come
to system (4.25) with A.t /  0.
The support function of the reach set W Œt  is
1
¡.` j W Œt / D maxfh`; W.i/ .t /`i 2 j i D 1; : : : ; mg: (4.33)

Let ¢ > 0 be sufficiently small, such that for • 2 Œ0; ¢ the maximum for given
1
direction ` is achieved at same i D i0 , i.e. ¡.` j W Œt / D h`; W.i0 / .t /`i 2 :
Assuming k`k D 1, the estimate for the support function of W Œt  ¢ is
1
¡.` j W Œt  ¢/ D h`; W.i0 / .t  ¢/`i 2 D

¢
h`; W.i0 / .t /`i 2 h`; WP .i0 / .t /`i 2 C o.¢/:
1 1 1
Ch`; W.i0 / .t /`i 2 
2

Omitting brackets .t / in the notations, we further estimate, due to (4.4.3),

1 1
h`; WP .i0 / .t /`i D h`; .W.i0 / / 2 SQB2 `i  h`; .W.i0 / / 2 SQB2 `iC
1 1

0 1
X
m
C” @ “i0 j h`; W.j / `i  h`; W.i0 / `iA  
j D1

1 1 1 1
2k.W.i0 / / 2 `k kS kkQB2 `k  2h`; W.i0 / `i 2 h`; QB `i 2 :

Here the multiplier of ” is non-positive, since for i D i0 the expression

h`; W .i/ .t /`i; .i D i0 / is at its maximum. Returning to the estimate of
¡.` j W Œt  ¢/, we finally get
1
¡.` j W Œt  ¢/  ¡.` j W Œt / C ¢h`; QB .t /`i 2 D o.¢/;

which is equivalent to the funnel equation (4.32) of the above (A.t /  0).
Corollary 4.4.2. Set-valued function W Œt  is an internal estimate of the reacha-
bility tube W Œt , and functions W.i/ Œt  are internal ellipsoidal estimates of W Œt .
This follows from the fact that the solvability set W Œt  is the maximum solution to
funnel equation (4.32) with respect to inclusion.
192 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

102

101
Eigenvalues of W1/2

100

10−1

10−2
100 101
Number of Approximations (m)

101
(Volume of E)1/20

100

10−1

10−2
100 101
Number of Approximations (m)

Fig. 4.46 The size of ellipsoidal estimates depending on the number of mixed estimates m (top:
eigenvalues, bottom: volume in the power n1 )

Exercise 4.4.1. Prove formula (4.33).

Example 4.7. Figure 4.45 shows internal ellipsoidal estimates of the reachability
set of an oscillating system xP 1 D x2 , xP 2 D x1 C u; within time interval Œ0;  with
” D 20 1
and 12 . Here exact approximations are degenerate ellipsoids (shown with
thick lines).
Exercise 4.4.2. Figure 4.46 demonstrates the dependence of the size of ellipsoidal
estimates for a higher-order oscillating system on the system dimension (number
of nodes N D 10, dimension n D 2N D 20). Shown on the top are 20 graphs of
4.4 Computation in High-Dimensional Systems. Degeneracy and Regularization 193

1
all eigenvalues of the matrix .W.1/ / 2 (these eigenvalues are the sizes of semi-axes
of the estimates). On the bottom is the volume of the estimates plotted in power
1
n
D 201
(the geometrical mean of the axes).
Analyzing similar plots for a number of values of N , one may conclude that for
robust computation of the reachability tube the number of approximations m should
be chosen close to the system dimension n.

4.4.4 Efficient Computation of Orthogonal Matrix S

Equation (4.28) for the ellipsoidal approximation includes the operation of finding
an orthogonal matrix S D S.v1 ; v2 / 2 Rnn , such that S v2 is collinear with v1 for
some nonzero vectors v1 ; v2 2 Rn .
Note that with n  2 the matrix S.v1 ; v2 / is not unique (for n D 2 there are at
least two such matrices, while for n  3 there are infinitely many).
Function S.v1 ; v2 / should be defined as sufficiently smooth in variables v1 ; v2 , so
that integration schemes for higher-order ODEs could be applied to (4.28).
Matrix S.v1 ; v2 / may be calculated, for example, by computing the singular value
decomposition of vectors v1 ; v2 and by multiplying the corresponding orthogonal
matrices [132]. The computational burden for this procedure is of order O.n3 /, and
continuous dependence of S on v1 ; v2 is not guaranteed.
The following theorem gives explicit formulas for calculating S.v1 ; v2 /, which
relax the burden to O.n2 / operations and also ensure S to be sufficiently smooth.
Theorem 4.4.4. Take some nonzero vectors v1 ; v2 2 Rn . Then matrix S 2 Rnn
calculated as

S D I C Q1 .S  I /Q1T ; (4.34)
  p
c s vi
SD ; ; c D hOv1 ; vO 2 i; s D 1  c 2 ; vO i D ; (4.35)
s c kvi k

Q1 D Œq1 ; q2  2 Rn2 ; (4.36)


s 1 .Ov2  c vO 1 /; s ¤ 0;
q1 D vO 1 ; q2 D
0; sD0

is orthogonal and satisfies the property of collinearity for S v2 and v1 .

Proof. Suppose v1 ; v2 are not collinear and pass to their normalized versions vO i D
vi kvi k1 . Denote Lv D L .v1 ; v2 / to be the plane generated by vectors v1 and v2 .
We shall describe an orthogonal transformation S which is a rotation in the plane
Lv that transforms vO 2 into vO 1 . We also impose an additional requirement on the
orthogonal complement Lv? to Lv which is that the induced operator Sv? will be
equal to identity.
194 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

Compose matrix V D ŒOv1 ; vO 2  and find its decomposition: V D QR, where

matrix Q 2 Rnn is orthogonal and matrix R 2 Rn2 is upper triangular. Matrices
Q and R may be written in block form as

Q D ŒQ1 ; Q2 ; Q1 2 Rn2 ; Q2 2 Rn.n2/ ;

  
R1 1c
RD ; R1 D :
O.n2/2 0s

Note that the columns of Q1 and Q2 form orthonormal bases in plane Lv and its
orthogonal complement Lv? , respectively. With an additional constraint R11 > 0,
R22 > 0 matrix Q1 is unique, while matrix Q2 may be arbitrary, with orthonormal
columns orthogonal to those of Q1 . The indicated relations may be regarded as the
Gram—Schmidt orthogonalization procedure for finding matrix Q1 .
Set


S O
S DQ Q0 :
O I

This matrix is orthogonal as the product of orthogonal matrices. It describes the

composition of three transformations: a rotation of plane Lv to plane L .e1 ; e2 /, a
rotation in that plane, and a return to original coordinates.
We now prove that S does not depend on Q2 . Indeed, S D Q1 SQ10 C Q2 Q20 . But
from the equality QQ0 D Q1 Q10 C Q2 Q20 D I it follows that Q2 Q20 D I  Q1 Q10 ,
so we get relation (4.34), which does not contain Q2 .
Matrix S is also orthogonal when v1 is collinear with v2 , i.e. s D 0. In this case
c D ˙1 where for c D 1 we have S D I and for c D 1 we get S D I  2Ov1 vO 01 ,
and check that S S 0 D I .
Now calculate S vO 2 . From (4.36) we get Q10 vO 2 D .c; s/0 for any value of s. We
further have
  
c1 s c
S vO 2 D vO 2 C Q1 D
s c  1 s
 
1 1c
vO 2 C ŒOv1 ; s .Ov2  c vO 1 / D vO 1 :
s

For any vector v ? vO 1 ; vO 2 we have Q10 v D 0, hence S v D v. t

u
Remark 4.4.5. The computational complexity in using (4.34)–(4.36) is of order
O.n2 /. Moreover, the multiplication by matrix S is now performed with O.n2 /
operations instead of the usual O.n3 /.
4.4 Computation in High-Dimensional Systems. Degeneracy and Regularization 195

Remark 4.4.6. It may be checked directly that for nonzero vectors v1 ; v2 function
S D S.v1 ; v2 / has continuous derivatives of any order everywhere except the cone
generated by collinear v1 ; v2 .

4.4.5 Parallel Computation

Here we discuss the application of parallel computations to the computation of

internal ellipsoidal estimates and their application to finding feedback controls.

Computing the Ellipsoidal Estimates

In order to solve the ODE (4.4.3) numerically, using  parallel processes, we

decompose the index set I D fi D 1; : : : ; mg into  disjoint subsets Ik , such
that I D I1 [ : : : [ I . Process k will calculate and store matrices W.i/ , i 2
Ik . (To balance the load between the processes the cardinalities of Ik should be
approximately identical, namely, of order m=.)
Direct integration of ODE (4.4.3) would lead to an excessive
Pm amount of data
.j /
exchange between processes for computing the term iD1 “ij W . To avoid
this, we combine (4.4.3) with (4.30). As aPresult each process P solves its own
ODE (4.4.3) with the above term replaced by i2Ik “ij W.j / , where j 2Ik “ij D 1.
Approximations from different processes are mixed at prescribed discrete times by
applying formula (4.30) with ’i D m1 . As a result the amount of data exchange is
reduced substantially.

Computing the Feedback Controls

For the feedback problem the exact “extremal aiming” control should be
calculated as
( 1
Q.t /B 0 .t /lhl; QB .t /li 2 if B 0 .t /l ¤ 0I
Ue .t; x/ D
E .q.t /; Q.t //; if B 0 .t /l D 0;

where vector l D l.t; x/ D @V  =@x, V  .t; x/ D d.G.t1 ; t /x; G.t1 ; t /W Œt /

indicates the shortest path from x to W Œt . Here set W Œt  is the convex hull of a
union of sets. So the calculation of vector l reduces to a computationally difficult
problem of the max-min type, namely,
˚ 
V  .t; x/ D max min hq; G.t1 ; t /xi  ¡.G 0 .t1 ; t /q j Eiw Œt / (4.37)
kqk1 iD1;:::;m

(especially so, since parameters of the sets are stored across multiple processes).
196 4 Solution Examples on Ellipsoidal Methods: Computation in High Dimensions

To cope with such a difficulty, we replace V  .t; x/ by

[
m [
m
O  Œt /; W
VO  .t; x/ D d.G.t1 ; t /x; G.t1 ; t /W O  Œt  D Eiw Œt  D E .w.t /; W.i/ .t //;
iD1 iD1

which is equivalent to interchanging the maximum and minimum in (4.37)

VO  .t; x/ D min max fhq; G.t1 ; t /xi  ¡.G 0 .t1 ; t /q j Eiw Œt /g: (4.38)
iD1;:::;m kqk1

This leads to a vector l  D l  .t; x/ D @V  =@x D li0 , where i0 2 1; : : : ; m is the

minimizers index and li0 the maximizer in (4.38) for the fixed i D i0 .
Thus, to synthesize controls, each process locally finds an ellipsoid nearest to
the on-line position. Then the process that is the nearest among such ellipsoids is
chosen to calculate the control by using its own ellipsoid. The specified control is
finally communicated to all the other processes.

Simulation Results

Numerical results were obtained in computing feedback controls for a chain of

oscillating springs (4.24), [57], with the following parameters (here N is the number
of links and n D 2N is the system dimension):
• N D 25 (n D 50) for a system with disturbance, without matching condition;
• N D 50 (n D 100) for a system with one-directional scalar control (u 2 Œ0; );
• N D 50 (n D 100) for an inhomogeneous oscillating system (the lower part is
twice as heavy as the upper);
• N D 100 (n D 200) for a system with scalar control;
• N D 250 (n D 500) for a system with control of dimension N .
We do not compare these results with a non-parallel version of our code, since
in the latter case the existing memory limitations usually prevent it from being run
with large values of n.
Chapter 5
The Comparison Principle: Nonlinearity
and Nonconvexity

Abstract This chapter introduces generalizations and applications of the presented

approach prescribed earlier to nonlinear systems, nonconvex reachability sets and
systems subjected to non-ellipsoidal constraints. The key element for these issues
lies in the Comparison Principle for HJB equations which indicates schemes of
approximating their complicated solutions by arrays of simpler procedures. Given
along these lines is a deductive derivation of ellipsoidal calculus in contrast with
previous inductive derivation.

Keywords Nonlinearity • Nonconvexity • Comparison principle • Unicycle

• Boxes • Zonotopes • Ellipsoids

As indicated above, the solutions to many problems of control synthesis for systems
described by ODEs reduce to an investigation of first-order PDEs—the HJB type—
and their modifications [21, 24, 178, 198, 226, 248]. Similar equations may also be
used for calculating forward and backward reachability sets for control systems with
or without disturbances (the HJB equations).1
It is also well known that solutions to equations of the HJB type are rather
difficult to calculate, and their respective algorithms are still being developed
[221, 244]. However, for many problems, as those on reachability, on design of
safety zones for motion planning or on verification of control algorithms, one may
often be satisfied with approximate solutions that require a smaller computational
burden and may be achieved through substituting original HJB equations by
variational inequalities [23] due to certain Comparison Principles [48,95,149,186].
This chapter indicates such comparison theorems which are also applicable to
nonlinear systems for both smooth and nonsmooth solutions and to description of
nonconvex sets. They also allow to derive ellipsoidal methods through deductive
procedures in contrast with inductive approaches of Chap. 3. At the same time,

1
Such considerations are also true for systems with uncertain (unknown but bounded) disturbances
(the HJBI equation) [123, 176, 222], and for dynamic game-type problems. They involve more
complicated equations of Hamilton–Jacobi–Bellman–Isaacs (HJBI) type, [18, 19, 102, 123]. [121,
150,183]. These issues are mostly, except Chaps. 9 and 10, beyond the scope of the present volume.

© Springer International Publishing Switzerland 2014 197

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__5
198 5 The Comparison Principle: Nonlinearity and Nonconvexity

when applied to linear-convex reachability problems, such estimates may lead to

effective external and internal approximations of reach sets that converge to exact.
This includes the treatment of controls with hard bounds generated not only by
ellipsoids, but also by box-valued sets and symmetric polyhedrons (zonotopes).

5.1 The Comparison Principle for the HJB Equation

5.1.1 Principal Propositions for Comparison Principle

In this section we deal with ordinary systems without disturbances. Consider first
the nonlinear equation

xP D f .t; x; u/; t 2 Œt0 ; ª D Tª (5.1)

which coincides with Eq. (1.1) whose properties are described in the first lines of
Sect. 1.1.
As indicated in Sect. 2.3, given initial set X 0 (for t D t0 ) and a target set M
(for t D ª/, it makes sense to construct forward X Œt  D X .t I t0 ; X 0 /; 8t  t0 /;
and backward W Œt  D W .t I ª; M /; t  ª; reachability tubes for system (5.1),
emanating from set-valued positions ft0 ; X 0 g and fª; M g, respectively [121, 178].
Introduce notations

H .t; x; p/ D maxf< p; f .t; x; u/ > ju 2 P .t /g:

It is well known (see Sect. 2.3 and also [145, 178, 198]) that the solution V .t; x/
of the respective “forward” HJB equation of type

Vt C H .t; x; Vx / D 0; V .t0 ; x/ D d.x.; X 0 /; (5.2)

allows to calculate X Œt  D X .t; t0 ; X 0 / as the level set:

X Œ£ D fx W V .£; x/  0g:

The last property is independent of whether V was obtained as a classical or a

generalized solution of Eq. (5.2).
Thus the exact description of set X Σ in general requires to solve the first-order
PDE (5.2). Such a problem may be rather difficult to solve as the reachability sets
for nonlinear systems may turn out to have a very peculiar form (see [225, 255]).
We shall therefore seek for the upper and lower estimates of functions V .t; x/,
and as a consequence, also the external and internal estimates of sets X Σ: This
move is motivated by a desire to describe the estimates through relations simpler
than those for the exact solution.
5.1 The Comparison Principle for the HJB Equation 199

Assumption 5.1.1. There exist functions H.t; x; p/ and functions wC .t; x/ 2

C1 ŒTª ; .t / 2 L1 , which satisfy the inequalities

H .t; x; p/  H.t; x; p/; 8ft; x; pg; (5.3)

wC C
t C H.t; x; wx /  .t /: (5.4)

Theorem 5.1.1. Let functions H.t; x; p/; wC .t; x/; .t / satisfy Assumption 5.1.1.
Then the following estimate is true

X Œt  XC Œt ; (5.5)

where
Z t
XC Œt  D fx W wC .t; x/  .s/ds C maxfwC .t0 ; x/jx 2 X 0 gg: (5.6)
t0

Let there exist a pair x 0 2 X 0 ; u .t / 2 P .t /; t  t0 , such that the respective

trajectory x  .t / 2 X Œt . Then

d wC .t; x/=dtjxDx  .t/ D wC  C   

t .t; x / C hwx .t; x /; f .t; x ; u /i 

 wC   C C   C
t .t; x / C H .t; x ; wx /  wt .t; x / C H.t; x ; wx /  .t /:

The last relations imply

d wC .t; x/=dtjxDx  .t/  .t /:

Integrating this inequality from t0 to t , we have

wC .t; x  .t // 
Z t Z t
 .s/ds C wC .t0 ; x  .t0 //  .s/ds C maxfwC .t0 ; x/jx 2 X 0 g;
t0 t0

which means x  .£/ 2 XC Œ£ and the theorem is proved.

Recall that when function V .t; x/ is nondifferentiable, Eq. (5.2) is written down
only as a formal symbolic notation and its solution should be considered in a
generalized “viscosity” sense [16, 50, 80] or equivalent “minimax” sense [247] or
“proximal” sense [48].
Similar theorems are true for backward reachability sets W Œt  (or, in other words
the “weakly invariant sets” relative target set M or the “solvability sets” in terms of
[174]). Namely, if W .t; ª; M / D W Œt  is a backward reachability set from target
set—a compact M Rn , then

W Œ£ D fx W V .b/ .£; x/  0g;

200 5 The Comparison Principle: Nonlinearity and Nonconvexity

where V .b/ .t; x/ is the solution (classical or generalized) of the “backward” HJB
equation

.b/
Vt  H .t; x; Vx.b/ / D 0; V .b/ .ª; x/ D d 2 .x; M /: (5.7)

which, in its turn, is generated by the problem

V .b/ .£; x/ D minfd 2 .xŒª; M / j xŒ£ D x; u.; / 2 UC g: (5.8)

Here UC is the class of closed-loop controls in which the last problem does have an
optimal closed-loop solution u0 .; / which generates an optimal trajectory x 0 λ D
x 0 .ªI £; x/:
Assumption 5.1.2. There exist functions H.t; x; p/, w.bC/ .t; x/ 2 C1 ; .t / 2 L1 ,
which satisfy the inequalities

H .t; x; p/  H.t; x; p/; 8ft; x; pg;

.bC/
wt  H.t; x; w.bC/
x /  .t /  0;

w.bC/ .ª; x/  V .b/ .ª; x/; 8x:

Denote

H .b/ .t; x; p/ D minfhp; f .t; x; u/i j u 2 P .t /g D H .t; x; p/:

Then, under the last assumption, we have

.bC/ .bC/ .bC/ .bC/

d w.bC/ .t; x/=dt .t /  wt  H .b/ .t; x; wx /  wt H.t; x; wx /.t /  0:

Integrating this inequality from £ to ª along an optimal trajectory x 0 Œt  which starts

at x and ends at x 0 Œª; generating the value V .b/ .£; x/, (as introduced in the previous
lines), we come to relations
Z ª
w.ª; x Œª/  w
0 .bC/
.£; x/ C .t /dt;
£

where w.bC/ .ª; x 0 Œª/  V .b/ .ª; x 0 Œª/ according to the last condition of
Assumption 5.1.2.
However, along the optimal trajectory x 0 Œt ; t 2 Œ£; ª; we have

V .b/ .£; x/ D V .b/ .t; x 0 Œt / D V .b/ .ª; x 0 Œª/: (5.9)

(see, for example, [48, Sects. 7.5–7.7]). This brings us to the next theorem.
5.1 The Comparison Principle for the HJB Equation 201

Theorem 5.1.2. Let functions H.t; x; p/; w.bC/ .t; x/; .t / satisfy Assumption
5.1.2. Then there exists a lower estimate for V .b/ .£; x/ W
Z ª
w.bC/ .£; x/ C .t /dt  V .b/ .£; x/;
£

and an external estimate for W Σ W

W Σ WC Σ;

where
Z ª
WC Σ D fx W w .bC/
.£; x/ C .t /dt  0g:
£

Exercise 5.1.1. (a) Prove a proposition similar to Theorem 5.1.1 for the backward
reach set W Σ:
(b) Prove a proposition similar to Theorem 5.1.2 for the forward reach set X Σ.
Let us now pass to the discussion of internal estimates for the reachability sets
and the related HJB equations. We shall consider the internal estimates for backward
reachability sets W Œt  D W .t; x; M /. As in the above, here we do not necessarily
require differentiability of the value function V .b/ .t; x/.
Let function V .b/ .t; x/ of Eq. (5.7) be continuous in all the variables, being a
generalized (viscosity) solution of this equation (in particular, it may also turn out
to be classical).
Consider the next assumption.
Assumption 5.1.3. There exist function h.t; x; p/ and function w .t; x/ 2 C1 ;
which satisfy the inequalities

H .t; x; p/  h.t; x; p/; 8ft; x; pg;

w 
t .t; x/  h.t; x; wx .t; x//  0;

w .ª; x/  V .b/ .ª; x/:

Under Assumption 5.1.3 we have

w   
t  H .t; x; wx /  wt  h.t; x; wx /  0:

Integrating the last inequality from £ to ª, along some optimal trajectory x 0 Œs D
x 0 .s; £; x/ which ends in x 0 Œª; we come to

w .ª; x 0 Œª/  w .£; x/

202 5 The Comparison Principle: Nonlinearity and Nonconvexity

and further, due to third condition, to

V .ª; x 0 Œª/  w .£; x/: (5.10)

However V .ª; x 0 Œª/ D V .£; x/, according to (5.9). Comparing this with (5.10),
we come to the next proposition.
Theorem 5.1.3. Under Assumption 5.1.3 the following estimate is true

V .b/ .£; x/  w .£; x/:

Denoting W  Œt  D fx W w .t; x/  0g and using the last inequality, we come to

the conclusion.
Corollary 5.1.1. Under Assumption 5.1.3 the next inclusion is true

W  Σ W Σ: (5.11)

We shall further observe, in the linear-convex case, how comparison theorems may
be used for obtaining ellipsoidal estimates of reachability sets.

5.1.2 A Deductive Approach to Ellipsoidal Calculus

Let us now return to linear systems of type

xP D A.t /x C B.t /u; (5.12)

under an ellipsoidal constraint on control u and a similar one on target set M ,

namely,

u 2 P .t / D E .pu .t /; P .t //; M D E .m; M /; t 2 Œt0 ; ª; (5.13)

where P 0 .t / D P .t / > 0; M 0 D M > 0. As usual, a nondegenerate ellipsoid is

given here by the inequality

E .pu ; P / D fu W hu  pu ; P 1 .u  pu /i  1g;

with its support function being

¡.l j E .pu ; P // D maxfhl; pijp 2 E .pu ; P /g D hl; pu i C hl; P li1=2 :

5.1 The Comparison Principle for the HJB Equation 203

Then

H .t; x; p/ D minfhp; f .t; x; u/i j u 2 P .t /g D H .b/ .t; x; p/

D hp; A.t /xi C minfhp; B.t /ui j u 2 E .pu .t /; P .t //g D

u

D hp; A.t /x C B.t /pu .t /i  hp; B.t /P .t /B 0 .t /pi1=2 ;

and with these relations the HJB equation (5.7), namely,

.b/
Vt C H .b/ .t; x; Vx.b/ / D 0; V .b/ .ª; x/ D d 2 .x; M /: (5.14)

acquires the form specific for a linear system.

In order to get the lower bound for V .b/ and the external bound for set W Œt , we
apply Theorem 5.1.2 and Corollary 5.1.1. We further use the next inequalities

hp; B.t /P .t /B 0 .t /pi1=2  ” 2 .t / C .4” 2 .t //1 hp; B.t /P .t /B 0 .t /pi;

where ” 2 .t / > 0 is arbitrary. Here an equality is attained with

” 2 .t / D .1=2/hp; B.t /P .t /B 0 .t /P .t /i1=2 :

Under Assumption 5.1.2, as applied to system (5.1), the function w.t; x/ is taken
to be quadratic:

w.t; x/ D hx  x  ; K.t /.x  x  /i  1; (5.15)

where K.t / D K 0 .t / > 0 is differentiable. Then we have

d w.t; x/=dt D wt C H .b/ .t; x; wx / D

D wt C hwx ; A.t /x C B.t /pu .t /i  hwx ; B.t /P .t /B 0 .t /wx i1=2 

 wt C hwx ; A.t /x C B.t /pu .t /i  ” 2 .t /  .4” 2 .t //1 hwx ; B.t /P .t /B 0 .t /wx i:

Substituting w.t; x/ D hx  x  .t /; K.t /.x  x  .t //i  1, we further obtain

˚
P /.x  x  .t //i  2hxP  .t /; K.t /.x  x  .t //iC
wt C H .b/ .t; x; wx /  hx  x  .t /; K.t

C2hK.t /.x  x  .t //; A.t /x C B.t /pu .t /i


 .” 2 .t //1 hK.x  x  .t //; B.t /P .t /B 0 .t /K.x  x  .t //i  ” 2 .t /: (5.16)
204 5 The Comparison Principle: Nonlinearity and Nonconvexity

In order that satisfy this scheme we shall demand that the expression in curly
brackets of (5.16) is equal to zero for all x  x  :

P /.x  x  .t //i  2hxP  .t /; K.t /.x  x  .t //iC

hx  x  .t /; K.t

C2hK.t /.x  x  .t /; A.t /.x  x  .t // C A.t /x  .t / C B.t /pu .t /i

.” 2 .t //1 hK.t /.x  x  .t //; B.t /P .t /B 0 .t /K.t /.x  x  .t //i D 0

Equalizing with zero the terms with multipliers of second order in x x  ; then those
of first order in the same variable, we observe that the last equality will be fulfilled
if and only if the following equations are true

KP C KA.t / C A0 .t /K  .” 2 .t //1 KB.t /P .t /B 0 .t /K D 0; (5.17)

xP  D A.t /x  .t / C B.t /pu .t /; (5.18)

with boundary conditions

K.ª/ D M 1 ; x  .ª/ D m: (5.19)

Under Eqs. (5.17), (5.18) relation (5.16) yields inequality

d w=dt  ” 2 .t /:

Integrating it from £ to ª, along an optimal trajectory x 0 Œt ; t 2 Œ£; ª, which starts

at xΣ D x and ends at xλ, generating V .b/ ; we come to condition
Z ª
w.£; x/  w.ª; x/ C ” 2 .t /dt; (5.20)
£

where w.t; x/ D .x  x  .t /; K.t /.x  x  .t //  1, and K.t /; x  .t / are defined

through Eqs. (5.17), (5.18) with

w.ª; x/ D hx  x  .ª/; M 1 .x  x  .ª//i:

Summarizing the above, we come to the proposition

Theorem 5.1.4. (i) Function
Z ª
w .t; xj”.// D w.t; x/ 
0
” 2 .s/ds
t
5.1 The Comparison Principle for the HJB Equation 205

is a lower bound of V .b/ .t; x/:

w0 .t; xj” 2 .//  V .b/ .t; x/: (5.21)

(ii) The inclusion

W Œt  WC Œt ; (5.22)

is true, where
 Z ª 
 
WC Œt  D x W hx  x .t /; K.t /.x  x .t //i  1 C 2
” .s/ds :
t

We further transform Eq. (5.17) to new variables, substituting K for KC according

to relations
 Z ª 
K D K 1 ; KP D K KP K ; KC .t / D 1 C ” 2 .s/ds K .t /
t

We then come to equations

KPC D A.t /KC C KC A0 .t /  .t /KC  ..t //1 B.t /P .t /B 0 .t /; (5.23)

and boundary condition KC .ª/ D M .

Here
 Z ª 1
.t / D ” .t / 1 C
2 2
” .s/ds :
t

The formulas derived here allow the next conclusion.

Theorem 5.1.5. The following inclusion (external estimate) is true

W Œt  WC Œt ;

where

WC Œt  D E .x  .t /; KC .t // D fx W hx  x  .t /; KC1 .t /.x  x  .t //i  1g;

whatever be the function .t / > 0.

Remark 5.1.1. The last relation, namely, the external ellipsoidal approximation of
the backward reach set W Œt  was derived through a deductive scheme, from the
HJBI equations. Recall that similar results were achieved earlier, in Chap. 3, through
an inductive scheme (see also [174, 180]), without applying the Hamiltonian
formalism. There the structure of differential equations indicated in Theorem 3.3.1
206 5 The Comparison Principle: Nonlinearity and Nonconvexity

for the configuration matrices of the approximating “inductive” ellipsoids is similar

to (5.23) for matrices KC of the “deductive” external ellipsoids derived in this
section.
We now pass to internal ellipsoidal approximations of sets W Σ: Starting with
Eq. (5.14), where

H .b/ .t; x; p/ D hp; A.t /x C B.t /pu .t /i  hp; B.t /P .t /B 0 .t /pi1=2 ;

we shall apply Theorem 5.1.3 and Corollary 5.1.1. In doing this we shall use the
inequality

hp; B.t /P .t /B 0 .t /pi1=2  hT .t /p; T .t /pi1=2 hp; S.t /.B.t /P .t /B 0 .t //1=2 pi;

where p 2 Rn is arbitrary and S.t / is any continuous matrix function, whose values
S.t / are orthogonal matrices, namely, S.t /S 0 .t / D I and T .t / D T 0 .t / is a matrix
function with T .t /p 6D 0:
Here an equality is attained under coordinated collinearity of vectors
S.t /.B.t /P .t /B 0 .t //1=2 p and T .t /p.
As in the “external” case, under Assumption 5.1.3, when applied to sys-
tem (5.12), function w .t; x/ is taken quadratic: w .t; x/ D hx  x  ; K .t /.x 
x  /i  1, with K .t / D .K /0 .t / > 0 and x  .t / differentiable. Taken T .t / D
.K /1 ; we then have

w
t CH
.b/
.t; x; w   0  1=2
x /Dwt Chwx ; A.t /xCB.t /pu .t /ihwx ; B.t /P .t /B .t /wx i 

 w 
t C hwx ; A.t /x C B.t /pu .t /i

h.K /1 .t /w 1  1=2

x ; .K / .t /wx i h.K /1 .t /w 0
x ; S.t /.B.t /P .t /B .t //
1=2 
wx i:

Note that in the domain

D.r/ D ft; x W hx  x  .t /; K .t /.x  x  .t //i < r 2 ; t 2 Œ£; ªg

the following condition is true (PB .t / D B.t /P .t /B 0 .t /)


w
t CH .b/
.t; x; w
x/  hx  x  .t /; KP .t /.x  x  .t //i

2hxP  .t /; K .t /.x  x  .t //i C 2hK .t /.x  x  .t //; A.t /x C B.t /pu .t /i

 2r 1 hx  x  .t /; S.t /PB .t /K .t /.x  x  .t //i :
1=2
(5.24)
5.1 The Comparison Principle for the HJB Equation 207

The last relation will reflect the scheme of Theorem 5.1.4 with r sufficiently
large. Then, in order to satisfy this scheme, it will be sufficient that the expression
in curly brackets will be equal to zero in D.r/: Demanding this equality, we have

hx  x  .t /; KP  .t /.x  x  .t //i  2hxP  .t /; K .t /.x  x  .t //iC

C2hK .t /.x  x  .t //; A.t /.x  x  .t // C A.t /x  .t / C B.t /pu .t /i

2r 1 hx  x  .t /; S.t /PB .t /K .t /.x  x  .t //i D 0

1=2

for all ft; x 2 D.r/g: Equalizing with zero the terms with multipliers of second
order in x  x  ; then those of first order in the same variable, we observe that the
last equality will be fulfilled if and only if the next equation

KP  C .K A.t / C A0 .t /K  r 1 .K PB S 0 .t / C S.t /PB .t /K / D 0;

1=2 1=2
(5.25)

is true together with Eq. (5.18) for x  .t / and with boundary conditions

K .ª/ D M 1 ; x  .ª/ D m: (5.26)

Under (5.25), (5.18), (5.26) relation (5.24) yields inequality

d w =dt  0;

integrating which from £ to ª, along an optimal trajectory x 0 Œt ; t 2 Œ£; ª; which

starts at xΣ D x; we come to condition

w .£; x/  w.ª; x 0 Œª/ D V .b/ .ª; x 0 Œª/ D V .b/ .t; x/; (5.27)

where w.t; x/ D hx  x  .t /; K .t /.x  x  .t //i  1:

Summarizing the above, we come to the proposition
Theorem 5.1.6. Function w .t; x/ is an upper bound of V .b/ .t; x/:

w .t; x/  V .b/ .t; x/:

The inclusion

W Œt 
W Œt ; (5.28)

is true, where

W Œt  D fx W hx  x  .t /; K .t /.x  x  .t //i  1g:

208 5 The Comparison Principle: Nonlinearity and Nonconvexity

Transforming Eq. (5.28) to new variables, with

K D K 1 ; KP  D K KP  K ;

we come to equations

KP  D A.t /K C K A0 .t /  r 1 .t /.K S.t /PB1=2 .t / C PB1=2 .t /S 0 .t /K /; (5.29)

and boundary condition K .ª/ D M . Then

W Œt  D fx W hx  x  .t /; .K .t //1 .x  x  .t //i  1g: (5.30)

Here r.t / is a tuning parameter which may be selected on-line, as a function of

jjK .t /jj. Then the structure of Eq. (5.29) will be of the type we had in (3.109).
Note that the centers x  .t / of the indicated internal ellipsoids will be the same
as for external estimates, being in both cases described by Eq. (5.18).
Remark 5.1.2. By an appropriate selection of parameterizers .t /; S.t / for
Eqs. (5.23), (5.29), similar to those of Sects. 3.3 and 3.8, one may obtain for each
“good” direction l 2 Rn of Assumption 3.3.1 an ellipsoid which is “tight” along
this direction. Namely, for each l one may indicate such parameters ./; S./ for
which equalities

¡.ljW Œt / D ¡.ljWC Œt /; ¡.ljW Œt / D ¡.ljW Œt /

would be true.

5.2 Calculation of Nonconvex Reachability Sets

Consider the reachability set XŒ‚ D X .‚; t0 ; X 0 / of Definition 2.3.1 within an

interval ‚ D Œ’; “: As indicated in Sect. 2.7
[
XŒ‚ D fX Œt  j t 2 ‚ g:

We shall now calculate XŒ‚ through ellipsoidal approximations restricting our-

selves to the “external” case. The “internal” solutions could be designed within a
similar scheme.
5.2 Calculation of Nonconvex Reachability Sets 209

Recall that due to Theorem 3.3.2 and Comparison in Principle as in Sect. 5.1.2,
we may write2 :

\
X Œt  D fE .x  .t /; XC
l
Œt / j hl; li D 1g
l

and equivalently X Œt  D fx W V 0 .t; x/  1g: where

V 0 .t; x/ D maxfVC0 .t; x; l/ j hl; li  1g;

l

VC0 .t; x; l/ D hx  x  .t /; .XC

l
Œt /1 .x  x  .t //i: (5.31)

Then we have
[
XŒ‚ D fX Œt  j t 2 ‚g D fx W V0 .‚; x/  1g; (5.32)

where

V0 .‚; x/ D minfVC0 .t; x/ j t 2 ‚g D min maxfVC0 .t; x; l/ j hl; li D 1; t 2 ‚g:

t t l
(5.33)
This leads to
Theorem 5.2.1. The reachability set XŒ‚ within the time interval ‚ D Œ’; “ may
be described through (5.31)–(5.33).
According to the theory of minimax problems (see [62]), we have

V0 .‚; x/ D min maxfVC0 .t; x; l/ j hl; li D 1; t 2 ‚g 

t l

 max minfVC0 .t; x; l/ j hl; li D 1; t 2 ‚g: (5.34)

l t

Then, in terms of sets, the relation for XŒ‚ may be expressed as

[\
XŒ‚ D fE .x  .t /; XC
l
Œt / j hl; li D 1; t 2 ‚g
t l

\[
fE .x  .t /; XC
l
Œt / j hl; li D 1; t 2 ‚g; (5.35)
l t

where x  Œt ; XC
l
Œt  are defined by Theorems 3.3.1, 3.3.2 (Eqs. (3.31), (3.32)).

2
In Sect. 5.1.2 we calculated the backward reach set W Œt  due to value function V .b/ .t; x/. Here,
in a similar way, we calculate the forward reach set X Œt  due to value function V 0 .t; x/.
210 5 The Comparison Principle: Nonlinearity and Nonconvexity

Recalling notation EC Œt; l D E .x  .t /; XC

l
Œt /, introduce
\ [
X Œt  D fEC Œt; l j hl; li D 1g; X Œ‚; l D fEC Œt; l j t 2 ‚g:
l t

Then, in terms of sets we have the following proposition

Theorem 5.2.2. The next relations are true
[ \
X Œ‚ D fX Œt  j t 2 ‚g fX Œ‚; l j hl; li D 1g:
t l

Remark 5.2.1. The conditions for equality of sets

[\ \[
fg D fg
t l l t

would be the set-valued analogy of the theorem on existence of a saddle point in l; t

(mint maxl VC .t; x; l/ D maxl mint VC .t; x; l/) for function VC .t; x; l/:
We further we observe
[\
fE .x  .t /; XC
l
Œt / j l./ 2 SSg ./; t 2 ‚g
t l./

\[
fE .x  .t /; XC
l
Œt / j l./ 2 SSg ./; t 2 ‚g; (5.36)
l./ t

where SSg ./ is the set of all good curves kl.t /k D 1; t 2 ‚; normalized according
to Sect. 3.3, to lie on a unit sphere.
Exercise 5.2.1. Indicate conditions when the operation of inclusion ( ) in (5.36)
may be substituted by an equality.
Exercise 5.2.2. Indicate internal ellipsoidal approximations for set XŒ‚:
Example 5.8.
Consider system

xP 1 D x2 C u C r cos ®t; xP2 D ¨2 t C r sin ®t; (5.37)

with t 2 Œ0; £; juj  ; x.t0 / 2 X0 .

We illustrate the calculation of XŒ‚; ‚ D Œ0; £; and its upper estimate XCl Œ‚;
assuming £ D 1; ¨ D 1;  D 1; X0 D xc C B .0/; xc0 D .1; 1/:3

3
The illustrations for this example were done by Zakroischikov.
5.2 Calculation of Nonconvex Reachability Sets 211

l
Fig. 5.1 The set XC Œ‚ (in red) and its covering by intersection of nonconvex sets XC Œ‚

Shown in Fig. 5.1 is the set XC Œ‚ (in red) and its covering by intersection of
nonconvex sets XCl Œ‚; calculated along good curves l.t /: An example of such
covering
\
XC Œ‚ j l  ; l   D XC Œ‚; l   XC Œ‚; l  

by only two such sets calculated from good curves generated by l  .£/ D
.1; 0/; l  .£/ D .0:54; 0:83/ is indicated in Fig. 5.2.
Figures 5.3 and 5.4 show set XŒ‚ and its external approximation by an ellipsoidal
tube.
212 5 The Comparison Principle: Nonlinearity and Nonconvexity

Fig. 5.2 Covering of XC Œ‚

l
by only two sets from XC Œ‚

Fig. 5.3 Set XŒ‚

5.3 Applications of Comparison Principle 213

Fig. 5.4 External approximation of set XŒ‚

5.3 Applications of Comparison Principle

5.3.1 Forward Reachability

Here we briefly indicate a result similar to Theorem 5.1.1 of Sect. 5.1 for a forward
reachability set. Take linear system (5.12), with constraints on x.t0 /; u.t / given by
nondegenerate ellipsoids:

x.t0 / 2 E .x 0 ; X 0 /; u.t / 2 E .pu .t /; P .t //; (5.38)

Then

H .t; x; p/ D hp; A.t /x C B.t /pu .t /i C hp; B.t /P .t /B 0 .t /pi1=2  (5.39)

 hp; A.t /x C pu .t /i C .t / C .4.t //1 .hp; B.t /P .t /B 0 .t /pi D H.t; p; x/

for any .t / > 0, with equality attained if

1
.t / D e .t; p/ D hp; B.t /P .t /B 0 .t /pi1=2 :
2
214 5 The Comparison Principle: Nonlinearity and Nonconvexity

Let us now look for w.t; x/ as a quadratic form, w.t; x/ D .hx  x  .t /; K.t /.x 
x  .t //i  1; where K.t / D K 0 .t / is differentiable and requires that w satisfies the
PDE

wt C hwx ; A.t /x C B.t /pu i C .4.t //1 hwx ; B.t /P .t /B 0 .t /wx i D 0 (5.40)

with boundary condition

w.t0 ; x/  hx  x 0 ; .X 0 /1 .x  x 0 /i: (5.41)

Then, after an integration from t0 to t , we come to the inequality

Z t Z t
w.t; x.t //  w.t0 ; x.t0 // C .s/ds  1 C .s/ds: (5.42)
t0 t0

Here K.t / may be obtained through a standard procedure of solving the resulting
Riccati equation

1
KP D KA.t /  A0 .t /K  .K; B.t /P .t /B 0 .t /K/; (5.43)
.t /

with equation

xP  D A.t /x  C B.t /pu .t /:

where K.t0 / D .X 0 /1 ; x  .t0 / D x 0 . Transformations

 Z t 
P P 1
K D K K K ; K D K ; KC .t / D 1 C .s/ds K .t /;
t0

convert into
1
KP C D A.t /KC KC A0 .t / C B.t /Q.t /B 0 .t / C .t /KC ; KC .t0 / D .X 0 /1 ;
.t /
(5.44)
Here
Z t
.t / D .t /=.1 C .s/ds/:
t0

These equations now coincide with those of [21, 23, 24].

The level sets for function w.t; x/ D w .t; x/ are ellipsoids E .x  .t /; KC .t //,
that depend on parameterizing functions , namely,

fx W w .t; x/  1g D E .x  .t /; KC .t // D EC Œt : (5.45)

5.3 Applications of Comparison Principle 215

Selecting k such functions i ./, we come to the estimate

X Œt  \fECi Œt  j 1  i  kg: (5.46)

An appropriate selection of i ./ ensures that ellipsoids ECi Œt  are tight in the sense
that there is no other external ellipsoid of type (5.44) that could be squeezed in
between ECi Œt  and X Œt .

5.3.2 Systems with Hamiltonians Independent of the State

For a system of type

xP D f .t; u/; u 2 P .t /; (5.47)

with P .t / compact and f continuous in t; x, find

V .£; x/ D minf®.x.t0 //jx.£/ D xg: (5.48)

u

Here are two simple examples of such systems:

xP i D hu; Li .t /ui; i D 1; : : : ; n; hu; P ui  1;

where P D P 0 > 0 and Li .t / are n  n matrices, and

xP i D sin.ui  ’i .t //; ui 2 Œi ; i ; i D 1; : : : ; n:

Function V .t; x/ could be sought for through equation

Vt C H.t; Vx / D 0; V .t0 ; x/ D ®.x/; (5.49)

with Hamiltonian

H.t; p/ D maxfhp; f .t; u/i ju 2 P .t /g;

and solution V .t; x/ taken, if necessary, as a viscosity solution. We shall indicate an

explicit formula for this function.
Assumption 5.3.1. (i) Function ®.x/ is proper, convex.
(ii) Its conjugate ® .l/ is strictly convex and 0 2 int .Dom®/, so that ® .l/ ! 1,
as hl; li ! 1, [237].
216 5 The Comparison Principle: Nonlinearity and Nonconvexity

Due to a theorem of Lyapunov on the range of a vector measure (see [3, 202])4 a
vector x.t0 / of type

Z£
x.t0 / D x C Q.£/; Q.£/ D  f .t; P .t //dt;
t0

ranges over x C Q.£/, where Q.£/ is a convex compact set.

Then5

V .£; x/ D minf®.q/ jq 2 .xCQ.£//g D min supfhl; qi® .l/ jl 2 Rn ; q 2 Q.£/g

q q l

D supfhl; xi  ¡.l jQ.£//  ® .l/g:

l

From here it follows that V .£; x/ is closed, convex in x, being the conjugate of a
closed convex function V  .£; x/ D ¡.l j  Q.£// C ® .l/.
Lemma 5.3.1. Under Assumption 5.3.1 on ®.x/ the following equality is true:

V .£; x/ D V  .£; x/: (5.50)

We further have

V  .£; l/ D supfhl; xi  V .£; x/g

x

D supfhl; xi  min maxfhp; x.t0 /i  ® .p/g

x u./ p

Z £
D supfhl; xi  min maxfhp; x  f .t; u.t //dt i  ® .p/gg
x u./ p t0

Z £
D max supfhl; xi  maxfhp; x  f .t; u.t //dt i  ® .p/gg
u./ x p t0

Z £
D max sup minfhl  p; xi C hp; f .t; u.t //idt C ® .p/g
u./ x p t0

Z £
D max min supfhl  p; xi C hp; f .t; u.t //idt C ® .p/g
u./ p x t0

4
Do not confuse famous Lyapunov (1911–1973) with celebrated A.M. Lyapunov (1856–1918),
founder of modern stability theory.
5
Here and below the conjugates of V are taken only in the second variables with £ fixed.
5.3 Applications of Comparison Principle 217

Z £
D maxf hl; f .t; u.t //idt C ® .l/g
u./ t0

Z £
D H .t; l/dt C ® .l/:
t0

Therefore, due to (5.50),

Z £
V .£; x/ D V  .£; x/ D maxfhl; xi  H .t; l/dt  ® .l/g:
l t0

Assumption 5.3.1 allowed to interchange the sup and min operations above by
applying the minmax theorem of Ky Fan [72]. Also used was the property
Z £ Z £
max hl; f .t; u.t //idt D maxff .t; u/ j u 2 P .t /gdt;
u./ t0 t0

indicated in [238, Chap. 14, Sect. F]. The above results in the next proposition.
Theorem 5.3.1. Under Assumption 5.3.1 the following relation is true:
Z £
V .£; x/ D maxfhp; xi  H .t; p/dt  ® .p/g: (5.51)
p t0

With H independent of both t; x the last relation is a formula of the Lax–Hopf type,
[17, 247]. The requirements of Assumption 5.3.1 are clearly satisfied by function
®.x/ D d 2 .x; X 0 / with X 0 convex and compact.
Exercise 5.3.1. For system

xP 1 D sin u; xP 2 D cos u; u 2 Œ; ; t 2 Œt0 ; £; (5.52)

find value function

V .£; x/ D minfjjx.t0 /  x 0 jj j x.£/ D xg; x 0 2 X 0 :

Here the Hamiltonian is

H .t; p/ D maxfp1 sin uCp2 cos u j u 2 Œ; g D maxf.p12 Cp22 /1=2 sin.uC’/g;
u ’

where ’ D arc cos .p1 .p12 C p22 /1=2 /: This gives H .t; p/ D .p12 C p22 /1=2 : We also
have

® .p/ D I .B .0// C hp; x 0 i;

where I .B .0// is the indicator function for a Euclidean unit ball.

218 5 The Comparison Principle: Nonlinearity and Nonconvexity

Following (5.52), we have

V .£; x/ D maxfhp; x  x 0 i  .£  t0 /.p12 C p22 /1=2 j p12 C p22 D 1g

p

Calculating the maximum, we come to the maximizers

p10 D.x1 x10 /..x1 x10 /2 C.x2 x20 /2 /1=2 ; p20 D.x2 x20 /..x1 x10 /2 C.x2 x20 /2 /1=2 :

This yields V .£; x/ D ..x1  x10 /2 C .x2  x20 /2 /1=2  £:

We may now check that V .£; x/ is a solution to the HJB equation (5.49) which is

Vt C maxfVx1 sin u C Vx2 cos ug D 0; u 2 Œ; :

u

Here Vt .£; x/ D 1; Vx .£; x/ D .p10 ; p20 /0 D .cos ®; sin ®/0 , so that H.t; p 0 / D 1:
Hence this equation is satisfied together with its boundary condition. We finally
calculate the reachability set for system (5.52) which is

X Œ£ D fx W V .£; x/  0g D fx W .x1  x10 /2 C .x2  x20 /2  £2 g:

5.3.3 A Bilinear System

Consider the two-dimensional nonlinear system

xP 1 D u1 x1  a12 x1 log x2 ; xP 2 D u2 x2  a21 x2 log x1 ; (5.53)

with positive coefficients aij ; i; j D 1; 2 and a hard bound

2 2 2 2
u 2 P D fp W p11 u1 C p22 u2  1g (5.54)

on the control u D .u1 ; u2 /0 with p D .p11 ; p22 /0 ; pi i > 0.

Let us look for the reach set X.t; t0 ; x 0 / for this equation, starting from point
x D .x10 ; x20 /, where xi0 > 0; i D 1; 2. Then the corresponding forward HJB
0

equation is

Vt C maxfVx1 .u1 x1  a12 x1 log x2 / C Vx2 .u2 x2  a21 x2 log x1 /ju 2 E g D 0;

u

where E is the ellipsoid defined by (5.54). This further gives

1
Vt C .p11 .Vx1 /2 x1 C p22 .Vx2 /2 x2 / 2  .a12 x1 log x2 Vx1 C a21 x2 log x1 Vx2 / D 0;
(5.55)
5.3 Applications of Comparison Principle 219

which has to be considered with boundary condition

V .t0 ; x/ D .log x  log x 0 ; log x  log x 0 /2 : (5.56)

Rather than solving the last equation, let us transform Eq. (5.53) to another
system of coordinates (system (5.53) may be linearized). Namely, taking zi D
log xi ; z0i D log xi0 , we have

zP1 D a12 z2 C u1 ; zP2 D a21 z1 C u2 ; z.0/ D z0 ; (5.57)

under the same constraint (5.54). The reach set Z .£; 0; z0 / from point z0 for this
system may be written down using relations similar to (3.32), (2.68), with changes
in the definition of value function V .t; x/ from d 2 .z.t0 /; z0 / to d.z.t0 /; z0 /.
This leads to the value function

V ? .£; z/ D maxfhl; zi  ˚.£; z0 ; l/ j hl; li  1g;

and further on, in view of Lemma 2.3.1, to a system of inequalities .l; z 2 R2 /

Z .£; 0; z0 / D fz W l1 z1 C l2 z2  ˚.£; z01 ; z02 ; l1 ; l2 /g; 8l 2 R2 ;

where

˚.£; z01 ; z02 ; l1 ; l2 /

Z £
D .l1 cosh ’.£  s/  l2 “1 sinh ’.£  s//2 p11
0
!1=2
1
C.l1 “ sinh ’.£  s/ C l2 cosh ’.£  s// p222
ds

C.l1 cosh ’£  l2 “1 sinh ’£/z01 C .l1 “ sinh ’£ C l2 cosh ’£/z02 ;

1 1
where ’ D .a12 a21 / 2 ; “ D .a12 =a21 / 2 : Returning to the initial coordinates x, we
observe that the boundary @X .£; 0; x 0 / of the exact reach set X .£; 0; x 0 / is given by
equation

maxf.l1 log x1 Cl2 log x2 /˚.£; log x10 ; log x20 ; l1 ; l2 /jl W hl; li  1g D 0 (5.58)
l

in the variables x1 ; x2 , where xi0 D exp z0i . The solution to (5.55), (5.56) then
happens to be V .£; x/ D V ? .£; log x1 ; log x2 /.
Rather than calculating the exact V .£; x/ and X .£; 0; x 0 /, it may be simpler to
approximate these. To explore this option, let us first approximate function V ? .£; z/
by quadratic functions and set Z .£; 0; z0 / by ellipsoids.
220 5 The Comparison Principle: Nonlinearity and Nonconvexity

To approximate the convex compact reach set Z.£; 0; z0 // by external ellipsoids

E .z? .t /; ZC .t // we may apply Eq. (5.44) where KC is substituted for ZC . We have
1
ZP C D AZC ZC A0 C Q.t / C .t /ZC ; ZC .t0 / D 0; (5.59)
.t /

and

zP? D Az? ; z? .t0 / D z0 ;

where
   
0 a12 10
AD ; PD :
a21 0 01

In order to ensure the property of tightness for ellipsoids E .z .t /; ZC .t //, and
therefore an equality in (5.46), we have to select the parameterizing functions .t /
as follows (see Chap. 3):
1 1
.t / D hl; F .t /QF 0 .t /li 2 hl; ZC .t /li 2 : (5.60)

Here
0 1
cosh ’t “ sinh ’t
F .t / D exp.At / D @ A:
1
“ sinh ’t cosh ’t

The reach tube ZŒt  D Z.t; 0; z0 / will now be exactly described by the formula (see
Chap. 3)
1
z .t; l/ D z? C ZC Œt F .t /lhl; F 0 .t /ZC Œt F .t /li 2 ; (5.61)

its surface being totally covered by the parameterized family of nonintersecting

curves z .t; l/ with parameter l 2 R2 and t  0. The equality

ZŒt  D \fE .z? .t /; ZC .t //j./g

will be true, where each ellipsoid

E .z .t /; ZC .t // D fz W ˙i;j
2
D1 hzi  zi Œt ; ZCij Œt .zj  zj Œt //i  1g:
? ?

Here ZCij are the coefficients of ZC . Returning to the original coordinates x and
taking xi? .t / D exp z?i .t /, we observe that ellipsoids E .z .t /; ZC .t // transform into
star-shaped nonconvex sets

S Œt  D fx W ˙i;j
2
D1 hlog xi  log xi Œt ; ZCij Œt .log xj  log xj Œt /i  1g;
? ?
5.3 Applications of Comparison Principle 221

Fig. 5.5 Cost at time t D 0

so that now

X Œt  D \fS Œt j./g: (5.62)

The surface of the tube X Œt  D X.t; 0; x 0 / will be totally covered by the family of
nonintersecting curves .i D 1; 2/
1
xi .t; l/ D exp.log.xi? Œt  C .ZC Œt F .t /l/i hl; F 0 .t /ZC Œt F .t /li 2 /;

where .h/i stands for the i -th coordinate of vector h.

It is clear from the text that solutions to this example all lie within the first
quadrant fx1 > 0; x2 > 0g. They are illustrated in Figs. 5.5, 5.6, 5.7, 5.8, 5.9,
and 5.10. However, these figures are actually produced for a more general system

jxP1 jDu1 jx1 ja12 jx1 j log jx2 j; jxP2 jDu2 jx2 ja21 jx2 j log jx1 j; jx1 j.0/>0; jx2 j.0/>0;

which includes Eq. (5.53), if considered within the domain fx1 > 0; x2 > 0g. The
pictures for the quadrants fx1 < 0; x2 > 0g; fx1 < 0; x2 < 0g; fx1 > 0; x2 < 0g are
symmetrical with the one for fx1 > 0; x2 > 0g, relative to either the coordinate axes
or the origin.
Here Figs. 5.5, 5.6, and 5.7 illustrate the exact value function V .£; x/ for
£ D 0; £ D 0:5; £ D 1, respectively. Figure 5.8 shows four reach tubes X Œt  each of
which originates in one of the four coordinate quadrants. Figures 5.9 and 5.10 show
the cuts of the value function (for each of the four reach tubes) surrounded by the
boundaries of sets S Œt —the preimages of ellipsoidal surfaces, for £ D 0:5; £ D 1.6

6
This example was worked out by O.L. Chucha.
222 5 The Comparison Principle: Nonlinearity and Nonconvexity

Fig. 5.6 Cost at time t D 0.5

Fig. 5.7 Cost at time t D 1

5.3.4 External Ellipsoids for the Unicycle: Reachability

It is well known that reachability sets for nonlinear systems are typically nonconvex
and may have a peculiar form. But in some cases it may suffice to have ellipsoidal
estimates of their convex hulls instead. This may be given by an intersection
of an array of ellipsoids. Explained here is an example of such a convexifying
approximation. Consider the nonlinear system
5.3 Applications of Comparison Principle 223

Fig. 5.8 Reach tube at time t=0

Fig. 5.9 Estimate at time t=0.5

Fig. 5.10 Estimate at time t=1

224 5 The Comparison Principle: Nonlinearity and Nonconvexity

xP 1 D x2 ;
xP 2 D r cos x5 ;
xP 3 D x4 ; (5.63)
xP 4 D r sin x5 ;
xP 5 D ’u;

which describes a controlled planar motion for the “unicycle.”

Here u is the control, r D const > 0, ’ D const > 0; x .0/ D x.t0 / 2 X 0 D
E x ; X0 :
0

The domain U ./ of admissible controls u is

U ./ D L1 .Œt0 ; t / \ fu W u.t / 2 U D Œ1; 1:; a:eg: (5.64)

As before, let X Œt  D X .t; t0 ; X 0 / stand for the reachability set of system (5.63):
 ˚ ˇ 
X t; t0 ; X 0 D x ˇ 9u./ 2 U ./ W x.t0 / D x .0/ 2 X 0 W x.t; t0 ; x .0/ / D x :

Problem 5.3.1. Find a family of external ellipsoidal estimates for the reachability
set X Œt  of system (5.63) such that for each t 2 Œt0 ; t1  it touches the convex hull
convX Œt  of X Œt  at a certain point x.t /.
To solve this problem we apply Sect. 5.1 of this chapter.

Using the Comparison Principle

Following Sects. 2.1 and 2.3, we introduce the value function:

˚  ˇ 
V .t; x/ D inf d xŒt0 ; X 0 ˇ xŒt  D x ;
u./

so that

X Œt  D fx j V .t; x/  0 g

Then the related HJB equation for V .t; x/ is as follows:

Vt C max fVx1 x2 C Vx3 x4 C rVx2 cos x5 C rVx4 sin x5 C ’uVx5 g D 0: (5.65)

juj1

Solving the internal problem, we have

Vt C H .t; x; Vx / D 0; H .t; x; p/ D p1 x2 C p3 x4 C rp2 cos x5 C rp4 sin x5 C ’jp5 j D 0:

(5.66)
5.3 Applications of Comparison Principle 225

To find the necessary approximation for convX Œt  we apply the comparison principle
(Theorem 5.1.1). Let wC .t; x/ 2 C1 , .t / 2 L1 , H .t; x; p/ satisfy conditions

H .t; x; p/  H .t; x; p/ ; 8 ft; x; pg ;


wC C
t .t; x/ C H t; x; wx .t; x/  .t /; 8t  t0 : (5.67)

Then the next estimate is true.

X Œt  XC Œt  ; (5.68)

where
8 9
ˆ
ˆ Z t >
>
< =
C C
XC Œt  D x w .t; x/  .£/d £ C max w .t0 ; x/ : (5.69)
ˆ x2X 0 >
>
:̂ t0
;

We now parameterize the family of functions wC .t; x/ by the pair fx./; p./g
which is the solution to the characteristic ODE system for our HJB PDE
equation (5.66) [51]. This ODE is written as

xP 1 D x2 ; pP1 D 0;
xP 2 D r cos x5 ; pP2 D p1 ;
xP 3 D x4 ; pP3 D 0;
xP 4 D r sin x5 ; pP4 D p2 ;
xP 5 D ’Nu; pP5 D r.p4 cos x5  p2 sin x5 /;

where uN D sgn.p5 / with p5 ¤ 0 and uN 2 Œ1; 1 with p5 D 0. We will look for such
ellipsoidal approximations XC Œt  of reachability set X Œt , that for each t would touch
convX Œt  at point x.t /. We further omit the argument “t ” in x.t N /; p.t
N /, denoting
them as x;N p:N
For arbitrary h > 0, œ1 , œ2 the next inequality is true

1 1 
p2 cos x5 C p4 sin x5  h hp; e2 e20 C e4 e40 piC
2

Ch1 hœ1 e2 C œ2 e4 ; pi C œ1 cos x5 C œ2 sin x5 C ”:

Here œ1 D pN2  r1 cos xN 5 ; œ2 D pN4  r1 sin xN 5 ; ” D 12 h C 12 h1 .œ21 C œ22 /,

Œe1 ; e2 ; e3 ; e4 ; e5  D I , with equality reached at .x;
N p/.
N Also true is inequality

N 2 C 2b.¥  ¥/
cos ¥  a.¥  ¥/ N C c;
226 5 The Comparison Principle: Nonlinearity and Nonconvexity

where
(  
N 2.¥N  / ; ¥N ¤ 
sin ¥=
aD ; b D a.¥N  /; N
c D cos ¥; ¥N 2 Œ0; 2
1
2
; ¥N D 

with equality at ¥ D ¥.N Upper estimates for functions  cos ¥, sin ¥,  sin ¥ may
be obtained similarly. Applying them, we come to

1 1 
p2 cos x5 C p4 sin x5  h hp; e2 e20 C e4 e40 pi C h1 hœ1 e2 C œ2 e4 ; piC
2

N .œ1 a1 C œ2 a2 /e5 e50 .x  x/i

Chx  x; N C 2hœ1 b1 C œ2 b2 /e5 ; x  xi
N C œ1 c1 C œ2 c2 C ”;

with equality at x D x,
N p D p. N
Using this relation, we find an upper estimate H.t; x; p/ for H .t; x; p/:

H .t; x; p/ D p1 x2 C p3 x4 C rp2 cos x5 C rp4 sin x5 C ’jp5 j 

 hp; A.x  x  /i C hp; Bpi C hx  x  ; C.x  x  /iC

C2hf; x  x  i C 2hC.x   x/;

N x  x  i C hg; pi C hAx  ; pi C  D H.t; x; p/;

N p D pN for any function x  D x  .t /. Here

with equality reached at x D x;

1 1 1
A D e1 e20 Ce3 e40 ; B D rh .e2 e20 Ce4 e40 /C ’e5 e50 ; C D r.œ1 a1 Cœ2 a2 /e5 e50
2 2jpN5 j

f D r.œ1 b1 C œ2 b2 /e5 ; g D rh1 .œ1 e2 C œ2 e4 /;

1
D ’jpN5 j C hx   x;
N C.x   x/i
N C 2hf; x   xi
N C r.œ1 c1 C œ2 c2 C ”/:
2

Now, for obtaining an ellipsoidal estimate we take wC .t; x/ as a quadratic form:


wC .t; x/ D hx  x  .t /; K.t / x  x  .t / i

and write down for functions wC .t; x/ and H .t; x; p/ the inequality (5.67):7

P /.x  x  .t //i  2hx  x  .t /; K.t /xP  .t /iC

hx  x  .t /; K.t

Chx  x  .t /; .K.t /A C A0 K.t / C 4B.t / C C.t //.x  x  .t //iC

C2h.K.t /g.t / C C.t /.x   x/

N C KAx  C f .t /; x  x  .t /i C .t /  .t /:

7
This example was worked out by V.V.Sinyakov.
5.3 Applications of Comparison Principle 227

In order that this inequality would be true for any .x  x  .t // it is sufficient to have
the next equations satisfied:

P / C K.t /A C A0 K.t / C 4KB.t /K C C.t / D 0;

K.t (5.70)

xP  .t / D .A C K 1 .t /C.t //x  .t / C K 1 .t / .f .t /  C.t /x.t

N // C g.t /: (5.71)

From initial condition

max wC .t0 ; x/ D 1
x2X 0

we now find such conditions for Eqs. (5.70), (5.71), namely:

1
K.t0 / D X 0 ; x  .t0 / D x 0 :

Substituting K for its inverse K as

K .t / D K 1 .t /; KP .t / D K .t /K.t
P /K .t /;

then multiplying both parts of (5.70) by matrix K 1 .t / we get

KP .t / D AK .t / C K .t /A0 C K .t /C.t /K .t / C 4B.t /; K .t0 / D X 0 ;

xP  .t / D .A C K .t /C.t //x  .t / C K .t / .f .t /  C.t /x.t

N // C g.t /; x  .t0 / D x 0 :

N
If under specific selection of fx./; N
p./g the solution to the last equations exists,
then we may introduce notation
0 1
Zt
KC .t / D @1 C .s/ds A K .t /;
t0

coming to the inclusion

˚ 
X Œt  XC Œt  D E .x  .t /; KC .t // D x W hx  x  .t /; KC1 .t /.x  x  .t //i  1 ;

N  (Figs. 5.11 and 5.12).

where XC Œt  touches X Œt  at point xŒt
228 5 The Comparison Principle: Nonlinearity and Nonconvexity

Fig. 5.11 The external ellipsoidal-valued estimate for the reachability tube (1 ellipsoid)

Fig. 5.12 The external ellipsoidal estimate for the reachability set at time t1 D 1 (3 ellipsoids)
5.4 Ellipsoidal Methods for Non-ellipsoidal Constraints 229

5.4 Ellipsoidal Methods for Non-ellipsoidal Constraints

In the previous texts the main types of hard bounds on system parameters were taken
as ellipsoidal-valued. However one of the conventional types of such bounds are also
polyhedrons which led to specific algorithms based, for example, on linear program-
ming and related constructions. A detailed theory of approximating reachability
sets and control solutions for linear-convex systems by parallelotopes, including
the rectangular case (boxes), was developed by Kostousova [115]. However, in this
section, we indicate that symmetrical polyhedrons may be well treated by ellipsoidal
approximations, with reasonable computational burden, especially if one involves
parallel calculation.

5.4.1 Degenerate Ellipsoids: Box-Valued Constraints

Let us now assume that system (5.12) is subjected to hard bounds of the “box” type,
namely,

u.t / 2 P .t /; x.t0 / 2 X 0 ;

where

P .t / D fu 2 Rm W jui  u0i .t /j  i .t /g; i .t /  0; (5.72)

X 0 D fx 2 Rn W jxj  xj0 j  j g; j  0; i D f1; : : : ; mg; j D f1; : : : ; ng;

and u0i .t /; xj0 are given.

Will it be possible to use ellipsoidal approximations for the respective reach sets
now, that P ; X 0 are not ellipsoids? To demonstrate that this is indeed possible, we
proceed as follows.
Let us define a box P with center p as P D B .p; P / where P D fp .1/ ; : : : ; p .m/ g
is an array of m vectors (“directions”) p .i/ such that

X
m
B .p; P / D fx W x D p C p .i/ ’i ; ’i 2 Œ1; 1g:
iD1

Then box P .t / of (5.72) may be presented as P .t / D B .u0 .t /; P .t //, where P D

fp .1/ ; : : : ; p .m/ g, with p .i/ D e.i/ i .t / and e.i/ is a unit orth oriented along the axis
0xi . Box B .u0 .t /; P .t // is a rectangular parallelepiped. A linear transformation T
of box B .p; P / will give

T B .p; P / D B .Tp; TP /:

Thus, in general, box B .T u0 .t /; TP .t // will not be rectangular. Let us now

approximate a box by a family of ellipsoids. Taking set B .0; P /, we may present it
230 5 The Comparison Principle: Nonlinearity and Nonconvexity

as the sum of m degenerate ellipsoids E .0; Qi i /, where

0
Qi i D e.i/ e.i/ qi i ; qi i D 2i :

Here Qi i is an m  m diagonal matrix with diagonal elements qkk D 0; k 6D

i; qi i D 2i (its only nonzero element is qi i D 2i ).
Then

X
m
B .0; P / D E .0; Qi i / E .0; Q.p//;
iD1

where p D fp1 ; : : : ; pm g and

X
m  X
m 
Q.p/ D pi pi1 Qi i ; pi > 0: (5.73)
iD1 iD1

Remark 5.4.1. These relations were usually applied to nondegenerate ellipsoids,

(see, [174], and Sect. 2.7 of this book). However, on applying Lemma 3.2.1 of [174],
one may observe that they are also true for the degenerate case.
Indeed, given vector l 2 Rm , take p D fp1 ; : : : ; pm g as

X
m
pi D jli ji if li 6D 0; pi D –m1 jjljj if li D 0; jjljj2 D li2 (5.74)
iD1

q
Here jli j D li2 , so

X
m X
m
¡.ljE .0; Q.p/// D hl; Q.p/li1=2  jli ji C–jjljj; jli ji D ¡.ljB .0; P //
iD1 iD1
(5.75)
and

¡.ljQ.p//  ¡.ljB .0; P //  –jjljj: (5.76)

Note that with © > 0 matrix Q.p/ is always nondegenerate. However, if we

allow – D 0, then, with p taken as in (5.74), relation (5.76) will turn into an
equality. Now let m be the number of nonzero coordinates of vector p in Q.p/.
5.4 Ellipsoidal Methods for Non-ellipsoidal Constraints 231

Then we observe that with m D m the matrix Q.p/ is nondegenerate. However,

with m < m; Q.p/ will turn out to be degenerate and set E .0; Q.p// will be an
elliptical cylinder.8
Theorem 5.4.1. (i) An external ellipsoidal approximation

B .0; P / E .0; Q.p//

is ensured by ellipsoid E .0; Q.p//, where Q.p/ is given by (5.73).

(ii) With p selected due to (5.74), the inequality (5.76) will be true and if one takes
– D 0 in (5.74), then (5.76) turns into an equality. However, with – D 0; m <
m the ellipsoid E .0; Q.p// becomes degenerate (an elliptical cylinder).
A similar approximation is true for box B .0; X / D X 0 .
Lemma 5.4.1. Under a linear transformation T we have

T B .0; Q/ D B .0; T Q/ E .0; TQ.p/T 0 / (5.77)

This follows directly from the above.

5.4.2 Integrals of Box-Valued Functions

Consider a set-valued integral

Z £
B .0; B.t /P .t //dt (5.78)
t0

and a partition ¢ŒN  similar to the one of Sect. 2.2. As before, the m  m diagonal
matrix P .t / and n  m matrix B.t / are continuous.
Then
Z £ X
N X
m
B .0; B.t /P .t //dt D lim E .0; B.tj /Qjj .tj /B 0 .tj //¢i
t0 iD1 j D1

with N ! 1 we have ¢ŒN  D maxf¢i g ! 0. Applying again the formula for the
external ellipsoidal approximation of the sum of ellipsoids, we have

8
In the next subsection related to zonotopes we indicate a regularization procedure that ensures
a numerical procedure that copes with such degeneracy. This is done by substituting cylinders for
far-stretched ellipsoids along degenerate coordinates.
232 5 The Comparison Principle: Nonlinearity and Nonconvexity

X
N X
m
E .0; B.tj /Qjj .tj /B 0 .tj //¢i E .0; XCN .pN Œ//;
iD1 j D1

X
N X
m  X
N X
m 
1
XCN .pN .// D pjj .ti / pjj .ti /Qjj .ti / ; pjj .ti / > 0:
iD1 j D1 iD1 j D1

Here pN ΠD fpjj .ti / jj D 1; : : : ; m; i D 1; : : : ; N g:

Taking pjj .ti / to be the values of continuous functions pjj .t /; j D 1; : : : ; m;
and passing to the limit in the previous relation, with N ! 1; ¢ŒN  ! 0, we
come to the next conclusion.
Lemma 5.4.2. The following inclusion is true
Z £
B .0; B.t /P .t //dt E .0; XC .£; pŒ// (5.79)
t0

n Z
X
£  X
n Z £ 
1
XC .pŒ/ D pjj .t /dt pjj .t /B.t /Qjj .t /B 0 .t /dt
j D1 t0 j D1 t0

for any continuous functions pjj .t / > 0.

Here pŒ D fpjj ./ jj D 1; : : : ; m; t 2 Œt0 ; £g: For a nonrectangular box
T .t /B .0; P .t // D B .0; T .t /P .t // and a nonzero box T0 B .0; X 0 / D B .0; T0 X 0 /
in a similar way we have
Theorem 5.4.2. The following inclusion is true
Z £
X Σ D B .0; T0 X / C 0
B .0; T .t /B.t /P .t //dt E .0; XC .£; pŒ// (5.80)
t0

where
X
n m Z
X
£ 
.0/
XC .£; pŒ/ D pkk C pjj .t /dt  (5.81)
kD1 j D1 t
0

X
n m Z
X £ 
T00 C 1
.t /T .t /B.t /Qjj .t /B 0 .t /T 0 .t /dt :
.0/1
 pkk 0
T0 Xkk pjj
j D1 j D1 t0

.0/
Here pŒ D fpkk ; pjj ./ j k D 1; : : : ; nI j D 1; : : : ; m; t 2 Œt0 ; £g:
5.4 Ellipsoidal Methods for Non-ellipsoidal Constraints 233

In order that an equality

Z £
¡.l j X Œ£/ D ¡.ljB .0; T0 X 0 // C ¡.ljB .0; T .t /B.t /P .t ///dt D ¡.ljE .0; XC .£; pŒ///
t0
(5.82)

would be possible for a given l 2 Rn , we would formally have to choose XC .£; pŒ/
taking

pjj .t / D hl; T .t /B.t /Qjj .t /B 0 .T /T 0 .t /li1=2 ; T00 li1=2 :

.0/
pkk D hl; T0 Xkk
0

(5.83)
But a nondegenerate matrix XC .£; pŒ/ would be possible only if pjj .t / ¤ 0 almost
.0/
everywhere and pkk ¤ 0. The equality is then checked by direct calculation.
Lemma 5.4.3. In order that for a given l 2 Rn there would be an equality (5.82), it
.0/
is necessary and sufficient that pjj .t /; pkk would be selected according to (5.83)
.0/
and both of the conditions pjj .t / ¤ 0 almost everywhere and pkk ¤ 0 would be
true.
Otherwise, either an equality (5.82) will still be ensured, but with a degenerate
E .0; XC .£; pŒ//, or, for any – given in advance, an inequality
Z £
¡.ljE .0; XC .£; pŒ//  ¡.ljB .0; T0 X 0 //  ¡.ljB .0; T .t /B.t /P .t ///dt  –jjljj
t0
(5.84)
may be ensured with a nondegenerate E .0; XC .£; pŒ//. This may be done by
selecting

.0/
pŒ D p – Œ D fpkk ; pjj ./ j k D 1; : : : ; nI j D 1; : : : ; m; t 2 Œt0 ; £; g

as

–jjljj
pjj .t / D hl; T .t /B.t /Qjj .t /B 0 .t /T 0 .t /li1=2 C
.–/
;
2m.£  t /
–jjljj
T00 li1=2 C
.0–/
pkk D hl; T0 Xkk
0
:
2n

It may be useful to know when pjj .t / ¤ 0 almost everywhere.

Lemma 5.4.4. In order that pjj .t / D hl; T .t /B.t /Qjj .t /B 0 .t /T 0 .t /li1=2 ¤ 0
almost everywhere, for all l 2 Rn , it is necessary and sufficient that functions
T .t /B.t /e.j / would be linearly independent. (The j -th column of T .t /B.t / would
consist of linearly independent functions.)
234 5 The Comparison Principle: Nonlinearity and Nonconvexity

This follows from the definition of linearly independent functions. Note that with
– D 0 we have
X
n m Z
X
£ 
.0/
pkk C pjj .t /dt D hl; XC .£; pŒ/li1=2 ; (5.85)
kD1 j D1 t
0

X
n X
n
pkk D hl; T0 X 0 T00 li1=2 T00 li1=2 :
.0/
hl; T0 Xkk
0

kD1 kD1

The parameters of the ellipsoid E .0; XC .£; pŒ// may be expressed through a
differential equation. Taking XC Œ£ D XC .£; pŒ/ and differentiating it in £, we get

X
m  X
n
.0/ 1
XP C Œ£ D pjj .£/ pkk 0
T0 Xkk T00
j D1 kD1

m Z
X
£ 
1
C pjj .t /T .t /B.t /Qjj .t /B 0 .t /T 0 .t /dt
j D1 t0

X
n m Z
X
£  X
m 
1
.£/T .£/B.£/Qjj .£/B 0 .£/T 0 .£/
.0/
C pkk C pjj .t /dt pjj
kD1 j D1 t0 j D1

Denoting

X
n m Z
X
£ 1
D j jlj jhl; XC Σli1=2 ;
.0/
j .£/ D pjj .£/ pkk C pjj .t /dt
kD1 j D1 t
0

X
n X
n
pkk /1 ; XC D hl; T0 X 0 T00 li1=2 T00 li1=2 T0 Xkk T00
.0/ .0/ .0/
k D pkk . 0
hl; T0 Xkk
0 0

kD1 kD1

and rearranging the coefficients similarly to Sect. 3.3, we come to

X
m  X
m
XP C Œ£D j .£/ XC Œ£C j1 .£/T .£/B.£/Qjj .£/B 0 .£/T 0 .£/; XC Œt0  D XC
0
:
j D1 j D1
(5.86)
Remark 5.4.2. If boxes B .p.t /; P .t //; B .x 0 ; X 0 / have nonzero centers p.t /, then
all the previous relations hold, with centers changed from 0 to x 0 .t /, where

xP 0 D B.t /p.t /; x.t0 / D x 0 ;

5.4 Ellipsoidal Methods for Non-ellipsoidal Constraints 235

so that E .0; XC .£; pŒ// turns into E .x 0 .£/; XC .£; pŒ//.

Theorem 5.4.3. (i) The matrix XC .£; pŒ/ of the external ellipsoid E .x 0 .£/;
XC .£; pŒ// that ensures inclusion (5.80) satisfies the differential equation and
initial condition (5.86).
(ii) The ellipsoid E .x 0 .£/; XC .£; pŒ// ensures the equality (5.82) (E .x 0 .£/;
XC .£; pŒ// touches set X Œ£ of (5.80) along direction l), if parameters pŒ
.0/
are selected as in (5.83) and j .t / are defined for all t 2 Œt0 ; £, with k ¤ 0.
0
Then E .x .£/; XC .£; pŒ// is nondegenerate for any l.
(iii) In order that E .x 0 .£/; XC .£; pŒ// would be nondegenerate for all l (with box
B .x 0 ; X 0 / D fx 0 g being a singleton), it suffices that functions T .t /B.t /e.j /
would be linearly independent on Œt0 ; t1 .
.–/ .0–/ .0/
(iv) In general, for any given –, selecting j .t /; k similarly to j .t /; k ,
.0/ .–/ .0–/
but with pjj .t /; pkk substituted for pjj .t /; pkk , one is able to ensure the
inequality (5.84).
We may now proceed with the approximation of reach sets for system (5.12).

5.4.3 Reachability Tubes for Box-Valued Constraints:

External Approximations

Consider system (5.12) under box-valued constraints (5.72). Its reach set will be
X  Œt  D G.t; t0 /X Œt ; where
Zt
X Œt  D B .x ; X / C
0 0
G.t0 ; s/B.s/B .u0 .s/; P .s//ds: (5.87)
t0

Let us first apply the results of the previous section to the approximation of X Œt .
Taking T .s/ D G.t0 ; s/; T0 D I , we have

X Œt  E .G.t0 ; t /x 0 .t /; XC Œt //;

where
X
m  X
m
XP C Œt  D j .t / XC Œt  C j1 .£/G.t0 ; t /B.t /Qjj .t /B 0 .t /G 0 .t0 ; t /;
j D1 j D1
(5.88)
with initial condition

X
n
.k /1 Xkk
.0/
XC Œt0  D 0
; (5.89)
kD1
236 5 The Comparison Principle: Nonlinearity and Nonconvexity

and with x 0 .t / evolving due to equation

xP 0 D A.t /x 0 C B.t /u0 .t /; x.t0 / D x 0 : (5.90)


Further on, denoting XC Œt  D G.t; t0 /XC Œt G 0 .t; t0 /, we obtain

XC Œt  E .x 0 .t /; G.t; t0 /XC Œt G 0 .t; t0 // D E .x 0 .t /; XC Œt /; (5.91)


0
where now XC D XC Œt0 
X
m  X
m
XP C
 
Œt  D A.t /XC  0
C XC A .t / C j .t / XC Œt  C j1 .t /B.t /Qjj .t /B 0 .t /:
j D1 j D1
(5.92)
Theorem 5.4.4. The inclusion (5.91) is true, whatever be the parameters
j .t / > 0; k > 0 of Eq. (5.92).
Let us now presume that Assumption 3.3.1 of Chap. 3 is fulfilled, so that vector
function l.t / along which we would like to ensure the tightness property is taken as
l.t / D G.t0 ; t /l; l 2 Rn . Then, following the schemes of Sect. 3.8 of Chap. 3, we
come to the next results.
Theorem 5.4.5. Under Assumption 3.3.1 of Chap. 3, in order that equality

¡.ljX Œt / D ¡.ljE .x 0 .t /; XC .t; pŒ//

would be true for a given “direction” l, the external ellipsoids E .x 0 .t /; XC .t; pŒ//
should be taken with

hl; G.t0 ; t /B.t /Qjj .t /B 0 .t /G 0 .t0 ; t /li1=2

j .t / D  ; (5.93)
hl; XC Œt li1=2
X
n  X
n 
0
XC D k jlk j .k jlk j/1 Xkk
0
; (5.94)
kD1 kD1

provided j .t / > 0 almost everywhere and jlk j > 0; k D f1; : : : ; mg.

Otherwise, the ellipsoids E .x 0 .t /; XC .t; pŒ// have to be taken for any given
– > 0 with parameters

hl; G.t0 ; t /B.t /Qjj .t /B 0 .t /G 0 .t0 ; t /li1=2

j– .t / D  ; (5.95)
hl; XC Œt li1=2 C –jjljj
X
n  X
m 
0–
XC D k jlk j C –jjljj .j jlj j/1 Xjj
0
; (5.96)
kD1 j D1
5.4 Ellipsoidal Methods for Non-ellipsoidal Constraints 237

0
instead of j .t /; XC . The inequality

¡.l jE .x 0 .t /; XC .t; pŒ///  ¡.l jX Œt /  –jjljj

will then be true.

The exact reach set X  Œt  is the sum of two sets:

X  Œt  D X0 Œt  C Xu Œt ;
where X0 Œt  D G.t; t0 /B .x 0 ; X 0 / is a (nonrectangular) box and

Zt

X u Œt  D G.t; s/B.s/B .u0 ; P .s//ds;
t0

is a convex compact set. Set X0 Œt  cannot be exactly approximated by nondegenerate

ellipsoids (see Fig. 5.13), while set Xu Œt  may be represented exactly by nonde-
generate ellipsoids under condition (iii) of Theorem 5.4.3. Let us reformulate this
condition.
Theorem 5.4.6. In order that for any l 2 Rn an equality

¡.ljXu Œt / D ¡.ljE .0; XC


Œt //


would be possible for an appropriately selected ellipsoid E .0; XC Œt /, it is sufficient
that the pair fA.t /; e g would be completely controllable for any k D 1; : : : ; m.
k

Then XC Œt  will be correctly defined when described by Eq. (5.92), with parameters
  0
j .t / and initial condition XC Œt0  D XC D 0; selected due to (5.93), (5.94).
This follows from the definition of complete controllability (see Sect. 1.3). Under

such condition the boundary of set XC Œt  will have no faces (“platforms”) and the
set can be totally described by “tight” ellipsoids, as in Chap. 3, Sects. 3.7 and 3.9
(see Fig. 3.1).
Finally, a parametric presentation of set X  Œt , similar to (5.92), (5.93) can be
produced. Then

x  .t / D x 0 .t / C XC

Œt l  hl  ; XC

Œt l  i1=2 ; (5.97)

with

x  .t0 / D x 0 C XC
0
li ; l  .t / D G.t0 ; t //0 l
0 1=2
lhl; XC (5.98)

.0–/
with either j .t /; k0 or j– .t /; k selected as indicated in Theorem 5.4.5. This

results in an array of external ellipsoids of either type E .x 0 .t /; XC Σ/ which yields
an equality similar to (5.82), namely

¡.l  j X Œ£/ D ¡.l  j E .x 0 ; XC


Œ£/ D hl  ; x  .£/i; (5.99)
238 5 The Comparison Principle: Nonlinearity and Nonconvexity

–
or type E .x 0 .t /; XC Œt /, which yields

¡.l  jE .0; XC

.£; pŒ//  ¡.l  j X Œ£/  –jjl  jj (5.100)

similar to (5.84). Recall that here

Z £
¡.l  j X Œ£/ D ¡.l  jB .0; T0 X 0 //  ¡.l  jB .0; T .t /B.t /P .t ///dt
t0

In Figs. 5.13 and 5.14, we return to Example 3.6 of Chap. 3. Here external
ellipsoidal approximations are used to construct reachability sets emanated from
a box X 0 D X Œt0  D fx W jxi j  1; i D 1; 2g rather than from an ellipsoid,
as in Chap. 3, Sect. 3.3. Calculations are made due to relations (5.90), (5.92), and
Theorems 5.4.4, 5.4.5, with m D 1; n D 2:9

t = 0.5
5
x2

−5
−5 −4 −3 −2 −1 0 1 2 3 4 5
x1

t=1
5
x2

−5
−5 −4 −3 −2 −1 0 1 2 3 4 5
x1

Fig. 5.13 Approximations of reachability sets emanating from a box

9
Illustrations to this example were worked out by M. Kirillin.
5.4 Ellipsoidal Methods for Non-ellipsoidal Constraints 239

5
x2

−5
0 0
0.2
0.4
0.6 x1
0.8 1 −5
t

Fig. 5.14 Trajectory tube emanating from a box

5.4.4 Reach Tubes for Box-Valued Constraints: Internal

Approximations

Following Remark 5.4.1, we recall that the results of Chap. 3, Sects. 3.2–3.4, 3.7,
and 3.8 are all also true for degenerate ellipsoids. We may therefore directly apply
them to box-valued constraints

P .t / D B .u0 .t /; P .t //; X 0 D B .x 0 ; X 0 /;

using relations (1.37), in view of inclusions

X
m X
n
B .u0 .t /; P .t // D E .u0 .t /; Qi i .t //; B .x 0 ; X 0 / D E .x 0 ; Xkk /;
j D1 kD1

where
0 0
Qi i .t / D e.i/ e.i/ qi i ; qi i D 2i ; Xkk
0
D e.k/ e.k/ xkk
0 0
; xkk D k2 ;

and e.i/ ; e.k/ are unit orths in the respective spaces Rm ; Rn . This leads to the
following statements.
240 5 The Comparison Principle: Nonlinearity and Nonconvexity

Theorem 5.4.7. (i) An ellipsoid E .x 0 .t /; X .t // that satisfies relations

X .t / D Y 0 .t /Y.t /; Y 0 .t / D G.t; t0 /X  .t /; (5.101)

where

Z X t !
X
n n
X .t / D 0 1=2 0
.Xkk / S0k C G.t0 ; s/.B.s/Qjj .s/B 0 .s//1=2 Sj0 .s/ds
kD1 t0 j D1

0
and S0k ; Sj are any orthogonal matrices of dimensions n  n; .S0k S0k DI;
0
S Sj Sj D I / is an internal ellipsoidal approximation for the reach set X Œt 
of (5.87).
(ii) In order that for a given “direction” l the equality

¡.l jX Œt / D ¡.l jE .x 0 .t /; X .t ///

would be true, it is necessary an sufficient that there would exist a vector d 2

Rn , such that the equalities

S0k Xkk l D œ0k d; Sj .B.s/Qjj .s/B 0 .s//1=2 G 0 .t0 ; s/l D œj .s/d;

1=2
(5.102)

k D 1; : : : ; n; j D 1; : : : ; m; s 2 Œt0 ; t ;

would be true for some scalars œ0k > 0 œj .s/ > 0.

(iii) The function x 0 .t / is the same as for external approximations, and is given
by (5.90).
We may now express relations for X .t / through differential equations similar
to those of Sect. 3.9.
Differentiating X .t / and using the previous relations, we come to the
proposition.
Theorem 5.4.8. Matrix X .t / of the ellipsoid

E .x 0 .t /; X .t // X Œt ;

satisfies the equation

XP  D A.t /X C X A0 .t / C Y 0 .t /S 0 .t /.B.t /Qjj .t /B 0 .t //1=2 C

C.B.t /Qjj .t /B 0 .t //1=2 S.t /Y .t /

5.5 Ellipsoidal Methods for Zonotopes 241

with initial condition

X
n 0  X
n 
0 1=2 0 0 1=2 0
X .t0 / D .Xkk / S0k .Xkk / S0k ;
kD1 kD1

where

X
m
YP 0 D A.t /Y 0 C .B.t /Qjj B 0 .t //1=2 Sj0 .t /;
j D1

X
n
Y 0 .t0 / D 0 1=2 0
.Xkk / S0k :
kD1

0
and S0k ; Sj are orthogonal matrices of dimensions nn, (S0k S0k D I; Sj Sj0 D I ).
In order that equality ¡.l jX Œt / D ¡.l jE .x 0 .t /; X .t /// would be true, it is
necessary and sufficient that relations of type (5.102) would be satisfied.

5.5 Ellipsoidal Methods for Zonotopes

5.5.1 Zonotopes

In this section we consider hard bounds on control u and initial set X.t0 / D X 0 in
the form of symmetric polyhedrons also known as zonotopes, denoted here as

P .t / D Z .p.t /; P .t //; X 0 D Z .x 0 ; X 0 / (5.103)

and described as follows.

Definition 5.5.1. A zonotope is understood to be a symmetric polyhedron of type

X
m
Z .p; P / D fx W x D p C p i ’i ; ’i 2 Œ1; 1g:
iD1

where p 2 Rm is its center and P D fp 1 ; : : : ; p m g is an array of m vectors

(“directions”) p i which define its configuration.
The support function for Z is

X
m
¡.ljZ .p; P // D hp; li C jhp i ; lij:
iD1

If the interior int Z .p; P / 6D ;, then zonotope Z .p; P / is said to be nondegenerate.

242 5 The Comparison Principle: Nonlinearity and Nonconvexity

0 2
X0 P(t)
0.2 1.5
0.4
1
0.6
0.5
0.8

1 0

1.2 0.5
1.4
1
1.6
1.5
1.8
2 2
0 0.5 1 1.5 2 2 1 0 1 2

Fig. 5.15 Hard bounds on control u and initial set X 0 for Example 5.9

Examples of zonotopes used below in Example 5.9 are given in Fig. 5.15. Note
that a particular case for the zonotope is a box which is initially defined by a set of
orthogonal vectors p i .
Definition 5.5.2. An ellipsoid E .q; Q/ with support function

¡.ljE .q; Q// D hl; qi C hl; Qli1=2 :

is said to be degenerate if its matrix Q D Q0  0 is degenerate.

Similar to Sect. 3.6 we observe that zonotope Z .0;
PP / may be presented as the
sum of m degenerate ellipsoids, namely, Z .0; P / D m iD1 E .0; Pi i /, where Pi i D
0
p i p i ; i D 1; : : : ; m:
Then the hard bounds on u; x.t0 / may be described as

X
mp
X
m0
u.t / 2 p.t / C E .0; Pjj .t //; x.t0 / 2 x 0 C E .0; Xi0i /; (5.104)
j D1 iD1

0 0
Here Pjj .t / D p j .t /p j .t /, Xi0i D x i x i
In order to calculate reachability sets for zonotopes one may follow the schemes
of the previous section as taken for boxes. These may be repeated for zonotopes as a
useful exercise. We formulate the main task beginning with internal approximation.
5.5 Ellipsoidal Methods for Zonotopes 243

5.5.2 Internal Ellipsoidal Tubes for a Zonotope

Problem 5.5.1. (i) For the reachability set X Œt  D X .t; t0 ; X 0 / of system (5.12),
under constraints 5.103 specify the family E D fE g of internal ellipsoidal
approximations such that
[ 
conv E j E 2 E D X Œt :
E


(ii) Indicate a subfamily of tight recurrent approximations El Œt  X Œt , that touch
the reachability set along “good” curves” ensuring equalities

[ 
¡.l  .t /jEl Œt / D ¡.l  .t /jX Œt /; conv El Œt  D X Œt :
l

Recall that such a tight approximation is unique if it is selected among the class
E , in the sense of Definitions 3.2.3, 3.7.3 of Chap. 3. Namely E Œt  is unique if for
any ellipsoid E that satisfies E Œt  E X Œt  it follows that E D E Œt :
Following the reasoning of Sects. 5.4.3 and 5.4.4, but applying it our zonotopes
Z .p.t /; P .t //; Z .x 0 ; X 0 /; we come to a conclusion similar to Theorem 5.4.8.
Theorem 5.5.1. (i) The reachability set X Œt  allows to have the following internal
approximation
Z t
E .x .t /; X .t // G.t; t0 /Z .x ; X /C0 0
G.t; s/B.s/Z .p.s/; P .s//ds D X Œt ;
t0

where X .t / D .X .t //0 X .t / and x .t / satisfy relations

X
m0 Z tX
mp
1 1
X .t /D S0;i .Xi0i / 2 G 0 .t; t0 /C Sj .s/.B.s/Pjj .s/B 0 .s// 2 G 0 .t; s/ds;
iD1 t0 j D1

Z (5.105)
t
x .t / D G.t; t0 /x 0 C G.t; s/B.s/p.s/ds; (5.106)
t0

for any orthogonal matrices S0;i , i D 1; : : : ; m0 ; and continuous orthogonal-

valued matrix functions Sj .t /, j D 1; : : : ; mp .
(ii) At fixed time t the ellipsoid E .x .t /; X .t // touches the surface of the
reachability set X .t; t0 ; Z .x 0 ; X 0 // along direction l.t / D l  .t /; l.t0 / D l 0 ,
namely,

¡.l  jX .t; t0 ; Z .x 0 ; X 0 /// D ¡.l  jE .x .t /; X .t /// D hl  ; x .t /i C hl  ; X .t /l  i1=2 ;

(5.107)
244 5 The Comparison Principle: Nonlinearity and Nonconvexity

t
if only the time-varying parameters S0;i , Sjt ./ are selected such that

1 1
t
S0;i .Xi0i / 2 G 0 .t; t0 /l 0 D œt0;j p t ; Sjt .s/.B.s/Pjj .s/B 0 .s// 2 G 0 .t; s/l  D œtj .s/p t ;
(5.108)
s 2 Œt0 ; t ; i D 1; : : : ; m0 ; j D 1; : : : ; mp ;

for certain scalar functions œtj ./ > 0; œt0;j > 0; j D 1; : : : ; mp ; and
functions p t 2 Rn , p t ¤ 0.
Differentiating relations (5.105), (5.106), and applying equality

XP  .£/ D .XP  .£//0 X .£/ C .X .£//0 XP  .£/;

we come to differential equations for the dynamics of approximating ellipsoids

E .x .t /; X .t //:
8
ˆ
ˆ XP .£/ D A.£/X .£/ C X .£/A0 .£/C
< C.X  .£//0 .Pmp S t .£/.B.£/P .£/B 0 .£// 12 / C .Pmp .B.£/P .£/B 0 .£// 12 S t 0 .£//X  .£/;
ˆ
 j D1 j jj j D1 jj j 
Pm0 P
ˆ
ˆ X .t0 / D . i D1 t
S0;i
1
.Xi0i / 2 /0 . i D1
m0 t
S0;i
1
.Xi0i / 2 /;
:̂
xP  .£/ D A.t /x .£/ C B.£/p.£/; x .t0 / D x0 :
(5.109)

Among these ellipsoidal-valued tubes we now have to specify those that ensure
tight approximations.
We now as usually (see Sects. 3.3 and 3.9) select a “good” curve as a solution to
equation

lP .t / D A0 .t /l  .t /; l  .t0 / D l:0

for a certain direction l 0 , so that l  .t / D G 0 .t0 ; t /l 0 :

For curve l  .t / we then find the ellipsoidal tube E .x .t /; X .t // designed for
l D l  .t /, such that for each t  t0 we have the equality

¡.l  .t /jX .t; t0 ; Z .x 0 ; X 0 // D ¡.l  .t /jE .x .t /; X .t // D hl  .t /; x  .t /i:

(5.110)

In order to satisfy this equality it is necessary to ensure, due to Theorem 5.5.1 (ii),
that orthogonal matrices S0;i t
, Sjt ./, i D 1; : : : ; m0 , j D 1; : : : ; mp would satisfy
conditions (5.108) for certain œt0;i > 0; œtj ./ > 0; p t 2 Rn , p t ¤ 0. Substituting
in these conditions l  .t /, we arrive at
1 1
t
S0;i .Xi0i / 2 l 0 D œt0;i p t ; Sjt .s/B.s/.Pjj .s// 2 l  .s/ D œtj .s/p t ; (5.111)

s 2 Œt0 ; t ; i D 1; : : : ; m0 ; j D 1; : : : ; mp :
5.5 Ellipsoidal Methods for Zonotopes 245

Note that for a good curve l  .s/ functions S0;it

; Sjt .s/; œt0;i ; œtj .s/; p t ; the last
relations do not depend on t , so further on the upper index t or the functions may be
deleted.
Summarizing the above, we conclude the following
Theorem 5.5.2. (i) For the reachability tube X Œt  D X .t; t0 ; Z .x 0 /; X 0 / there
exists an internal approximation

E .x .t /; X .t // X .t; t0 ; Z .x 0 ; X 0 //; t  t0 ;

where X .t / D .X .t //0 .X .t // and x .t / satisfy differential equa-

tions (5.109) and S0;i , Sj ./, i D 1; : : : ; m0 ; j D 1; : : : ; mp ; are orthogonal
matrices that evolve continuously in time.
(ii) The ellipsoidal tube E .x .t /; X .t // touches the boundary @X Œt  of tube X Œt 
from the inside, along direction l  .t / D G 0 .t0 ; t /l 0 ; ensuring equalities (5.107)
whenever relations (5.111) turn out to be true.
The last theorem gives the solution to Problem 5.5.1 described through recur-
rent relations. The internal ellipsoidal tube fE .x .t /; X .t //; t  t0 g touches
the reachability tube fX .t; t0 ; M .x 0 ; X 0 //; t  t0 g along the curve l  .t / D
G 0 .t0 ; t /l 0 ; ensuring for any l 0 the equalities

.l  .t /; x .l/ .t // D ¡.l  .t /jX Œt / D ¡.l  .t /jE .x .t /; X .t ///; (5.112)

where

x .l/ .t / D x .t / C X .t /l  .hl  ; X .t /l  i/1=2 ; (5.113)

These relations represent the maximum principle and the ellipsoidal maximal
principle which hold simultaneously at x D x .l/ .t /; l.t / D l  .t /: They also
indicate that at each time t point x .l/ .t // lies on the support plane to X Œt  generated
by vector l  .t / with equalities (5.112) reached at x .l/ .t /:
Remark 5.5.1. (a) Note that in formula (5.113) vector x .l/ .t / does not depend on
the length of l  .t /. Then, denoting x .l/ .t / D xŒt; l  .t /; we may consider the
surface „ Œt; l 0  D xŒt; l  .t /; where l  .t / depends on l 0 , which together with
t serve as a parameter of dimension n C 1. Hence, for t  given, we have the
n-dimensional surface
[
„ Œt  ;  D fxŒt  ; l0  j kl 0 k D 1g

and
[
„Œt  ;  D @X Œt   D fxŒt  ; l 0  j kl 0 k D 1g;
246 5 The Comparison Principle: Nonlinearity and Nonconvexity

where @X is the boundary of set X . Then X Œt  D convfxŒt  ; l 0  j kl 0 k D 1g is

the convex hull of „ Œt  ; .
(b) Also note that in Eqs. (5.109), (5.111) matrix X .t / depends on the selection of
good curve l D l  .t / and was marked as X .t /, while x .t / does not depend
on l  and is the same for all internal and external ellipsoids within classes
E ; EC : Now, if we take a finite number k of good directions li .t /; i D
1; : : : ; k; then, for a selected l D li .t /, we shall mark matrix X .t / D Xi .t /.
We further introduce set „k Œt  D [kiD1 E .x .t; Xi .t /;   X Œt  which is a
“star-shaped ” set and such that its convex hull convf„k Œt g  X Œt Œ is a convex
internal approximation of X Œt : Sets „k Œt  may be effectively used in related
numerical procedures.
Example 5.9. Consider system

xP 1 D x2 C u1
;
xP 2 D ¨2 x1 C u2

u 2 Z .p; P /

x.0/ 2 Z .x0 ; X0 /

where setsZ .x0 ; X0 /, Z .p; P / are shown in Fig. 5.15.

In Fig. 5.16 we demonstrate the exact reachability tube with one of its tight
internal ellipsoidal approximations. This tube was constructed as the union of an
array of such tight approximations. In Fig. 5.17 we depict a reachability set for a
zonotope with some of its tight internal ellipsoids at time t D 1.10

5.5.3 External Ellipsoidal Tubes for a Zonotope

Problem 5.5.2. (i) For the reachability set X Œt  D X .t; t0 ; X 0 / of system (5.12),
under constraints 5.103 specify the family E D fE Cg of external ellipsoidal
approximations such that for a given © > 0 we have
\ \
h.X Œt ; fEC j EC 2 Eg/  ©; X Œt  fEC j EC 2 Eg:

l
(ii) Indicate a subfamily of tight recurrent approximations EC Œt 
X Œt , that touch
the reachability set along “good” curves ensuring equality
\
l
¡.l.t /jEC Œt /  ¡.l.t /jX Œt / C ©kl.t /k; h.X Œt ; ECl Œt /  ©:
klk1

10
Examples illustrated in Figs. 5.16, 5.17, 5.18, and 5.19 of this section were worked out by M.
Kirillin.
5.5 Ellipsoidal Methods for Zonotopes 247

Fig. 5.16 Reach tube and one tight internal approximation

Fig. 5.17 The reachability set with its internal approximations

248 5 The Comparison Principle: Nonlinearity and Nonconvexity

Fig. 5.18 Reachability tube X©z Œt  with one tight external approximation

Fig. 5.19 Reachability set with its tight external approximations

5.5 Ellipsoidal Methods for Zonotopes 249

To solve this problem we first define some neighborhoods of a zonotope.

Pm
Definition 5.5.3. For a zonotope Z .0; P / D iD1 E .0; Pi i /; with degenerate
matrices
P iiP denote
P D m iD1 Pi i and further

© X m
Pi©i D Pi i C I; P © D Pi©i D P C ©I:
m iD1

Then the ©-neighborhood of Z .0; P / is defined as

X
m
Z© .0; P / D Z .0; P © / D E .0; Pi©i /  Z .0; P /
iD1

Pm
Here with X 0 D iD1 Xi0i ; X 0 C ©I D X 0© ; we have the ©-neighborhood of
Z .0; X 0 / denoted as

X
m
Z© .0; X 0 / D Z .0; X 0© / D E .0; Xi0©i /  Z .0; X 0 /
iD1

To approximate reachability sets X Œt  D X .t; t0 ; Z 0 / we introduce constraints on

u; x.t0 /

u.t / 2 P© D Z .p.t /; P © .t //; x.t0 / 2 X©0 D Z .x 0 ; X 0© / (5.114)

Then the reachability set of system (5.12) under such constraints will be
denoted as
Z t

X© Œt  D x .t / C G.t; t0 /Z .0; X / C
z 0©
G.t; s/B.s/Z .0; P © .s//ds; (5.115)
t0

Z t
x  .t / D G.t; t0 /x 0 C G.t; s/B.s/p.s/ds: (5.116)
t0

Exact reach set X Œt  will be defined through (5.115) where X 0© ; P © .s/ are
substituted by X 0 ; P .s/.
Lemma 5.5.1. For any © > 0 we have
(i) the inclusion X Œt   X©z Œt ,
(ii) the inequality h.X Œt ; X©z Œt /  k.t /©; where
 Z t ˇ 
ˇ
k.t / D sup kG 0 .t; t0 /lk1=2 C kB.s/G 0 .t; s/lk1=2 ds ˇ l 2 B .0/ :
ˇ
t0
 250 5 The Comparison Principle: Nonlinearity and Nonconvexity

Exercise 5.5.1. Prove Lemma 5.5.1

We now calculate the external ellipsoidal ©-approximation to X©z Œt :
Remark 5.5.2. Since Z .p; P / D p C Z .0; P /; the centers of the zonotopes are not
involved in ellipsoidal approximation which is the same as when x 0 D 0; p.t / D 0,
which also yields x.t / D 0. The approximation is then the sum of x  .t / and the
external EC Œt  under x.t / D 0:
Using schemes of Sect. 3.3 with those of Sect. 5.4 of this chapter, we come to the
next assertion11
Theorem 5.5.3. (i) For the ©-neighborhoods of X Œt  there exist an external ellip-
soidal approximation

X Œt  X© Œt z E .x  .t /; XC© .t //; (5.117)

where x  .t /, is given by (5.116) and XC

©
.t / satisfies equations
8
< XP C
©
.t / D A.t /XC ©
.t / C XC © 0
A .t /C
P mp P m 1
C 
j D1 j .t /X ©
C P.t / C p
.
j D1 Pj .t // B.t /Pjj© .t /B 0 .t /; (5.118)
:
XC .t0 / D . iD1 p0;i /. iD1 .p0;i /1 Xi0©i /;
© m0 m0

for any positive p0;i > 0, i D 1; : : : ; m0 ; and any continuous functions j .t / >
0, j D 1; : : : ; mp .
(ii) At each time t ellipsoid E .x  .t /; XC
©
.t // touches the ©-neighborhood X©z Œt  along
 0
direction l .t / D G .t0 ; t /l , so that relation
0

¡.l  .t /jX©z Œt / D ¡.l  .t /jE .x  .t /; XC

©
.t //; t  t0 ; (5.119)

is true as long as the next conditions are satisfied

1
p0;i D hl; Xi0©i li 2 ; i D 1; : : : ; m0 ;

j .s/ D hl  .s/; B.s/Pjj© .s/B 0 .s/l  .s/i1=2 ; s 2 Œt0 ; t ; j D 1; : : : ; mp :

Relation (5.119) yields the next property: set X©z Œt  may be presented as the
following intersection of approximating ellipsoids, namely

X©z Œt  D \fE .x  .t /; XC© .t // j p0;i > 0; j ./ > 0g: (5.120)

11
Note that designing external approximations of reachability sets X Œt  we have to apply them
to nondegenerate ©-neighborhoods of ellipsoids Ei i instead of exact Ei i , as in Sect. 5.4.2. This is
because all ellipsoids Ei i are degenerate, and their approximations may also turn out to be such. But
since we need all externals to be nondegenerate, this will be guaranteed by applying our scheme to
nondegenerate neighborhoods of Ei i .
5.5 Ellipsoidal Methods for Zonotopes 251

For a fixed vector l 0 2 Rn let xl .t / be the related good curve along which
ellipsoid E .x  .t /; XC
©
.t // touches the tube X©z Œt ; ensuring equalities

hl  .t /; xl .t /i D ¡.l  .t /jX©z Œt / D ¡.l  .t /jE .x  .t /; XC

©
.t ///: (5.121)

Then point xl .t / lies on the boundaries of both E .x  .t /; XC

©
.t /// and X©z Œt ,
satisfying an ellipsoidal maximum principle

hl  .t /; xl .t /i D maxfhl  .t /; xi j x 2 E .x  .t /; XC
©
.t //g; (5.122)

along trajectory

XC ©
.t /l  .t /
xl .t / D x  .t / C :
hl  .t /; XC©
.t /l  .t /i1=2

Remark 5.5.3. Similar to Remark 5.5.1(a) we may introduce a two-parameter

surface „Œt; l 0  but with array of functions fl  Œt g calculated for X© .t / rather than
for X Œt , leading to some similar conclusions. However, one of the differences is that
with number of ellipsoids that form zonotope Z.x 0 ; X 0 / being m0 < n, the matrix
P 0
X0 D m iD1 Xi i will be degenerate, and in R set Z.x ; X / will be a “zonotopic
0 n 0 0

cylinder.”
Example 5.10. Is the same as Example 5.9. Here Fig. 5.18 indicates the
©-neighborhood X©z Œt  for the exact zonotopic reachability tube X Œt  calculated as
an intersection of its tight external approximations one of which is shown touching
X©z Œt  along a good curvel  .t /. Figure 5.19 demonstrates the zonotopic cross-section
of the ©-neighborhood X©z Œt  at fixed time t D 1; together with an array of its tight
ellipsoidal approximations.
Chapter 6
Impulse Controls and Double Constraints

Abstract In the first section of this chapter we deal with the problem of feedback
impulse control in the class of generalized inputs that may involve delta functions
and discontinuous trajectories in the state space. Such feedback controls are not
physically realizable. The second section thus treats the problem of feedback control
under double constraints: both hard bounds and integral bounds. Such solutions are
then used for approximating impulse controls by bounded “ordinary” functions.

Keywords Bounded variation • Impulsive inputs • •-Function • Closed-loop

control • Value function • Variational inequality • Double constraints

Control inputs in the form of impulses of the •-function type have been studied since
the conception of control theory, being motivated by designing space trajectories,
automation, biomedical issues, and also problems in economics (see [23, 42, 43]).
However, impulse control problems were initially studied mostly as open-loop
solutions, [120, 219] with detours to closed-loop strategies being rather rare [23].
In the first section we describe the Dynamic Programming approach to problems
of feedback impulse control for any finite-dimensional linear system. This is
reached through variational inequalities that propagate HJB techniques to impulse
control. So, we begin with problems of open-loop control, then concentrate on
closed-loop solutions emphasizing their specifics and the type of HJB equation
that describes them. The solutions arrive as •-functions. However such functions
are ideal mathematical elements which require physically realizable approximation
by “ordinary” functions. This is done in the second section, where we solve the
problem with double constraints—joint hard instantaneous and soft L1 —integral
bounds on the controls. The achieved solution then yields realizable approximations
of impulsive feedback inputs. The contents of this chapter rely on investigations
[55, 152, 154].

© Springer International Publishing Switzerland 2014 253

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__6
254 6 Impulse Controls and Double Constraints

6.1 The Problem of Impulse Controls

Considering impulse control problems in finite time, it could at first seem reason-
able, say, in system (1.2), to minimize the norm ku./g of the control, taking it in the
space L1 . But due to the specifics of this space the minimum may not be attained
within its elements. It is therefore natural to apply the next scheme.
Consider the following generalization of the Mayer-Bolza problem in classical
Calculus of Variations (see [33]).
Problem 6.1.1. Minimize the functional

J.u.// D VarU./ C ®.x.ª C 0// ! inf; (6.1)

over controls U./, due to system

dx.t / D A.t /x.t /dt C B.t /d U.t /; t 2 Œt0 ; ª; (6.2)

with continuous coefficients A.t /; B.t /, under restriction

x.t0  0/ D x 0 : (6.3)

Here ®.x/ is a convex function, VarU./ stands for the total variation of function
U./ over the interval Œt0 ; ªC0, where U./ 2 Vp Œt0 ; ª is the space of vector-valued
functions of bounded variation. The generalized control U.t / attains its values in Rp
which means that each component Ui .t /; .i D 1; : : : ; p/; assumed left-continuous,
is a function of bounded variation on Œt0 ; ª C 0 and B.t / 2 Rnp are taken to be
continuous. The minimum over U./ will now be attained.1
Equation (6.2) with condition (6.3) is a symbolic relation for
Z t
x.t / D G.t; t0 /x 0 C G.t; Ÿ/B.Ÿ/d U.Ÿ/; (6.4)
t0

where the last term in the right-hand side is a Stieltjes or Lebesgue–Stieltjes integral
(see [234]). The terminal time is fixed and the terminal cost function ®.x/ W Rn !
R [ f1g is closed and convex.
A special selection of ®.x/ D I .x jx .1/ / yields
Problem 6.1.2. Steer x.t / from point x 0 D x.t0 / to point x .1/ D x.ª/ with
minimal variation of control U.t /:

VarfU.t / j t 2 Œt0 ; ª C 0g D VarU./ ! inf; (6.5)

1
In this book we give a concise description of impulse controls that are confined only to
•-functions, but not their derivatives. A general theory of impulse control that also involves
derivatives of •-functions is beyond the scope of this book and is presented in [152, 154]. Such
theory leads to the description of “fast” or “ultra fast” control solutions achieved on a quantum
level and in “nano”-time.
6.1 The Problem of Impulse Controls 255

due to system (6.3), under boundary conditions

x.t0  0/ D x .0/ ; x.ª C 0/ D x .1/ : (6.6)

Our main interest in this book lies in closed-loop control, but prior to that we start
with open-loop solutions presenting results in terms of the above. Such type of
problems in terms of the moment problem was indicated in Sect. 1.3 and addressed
in detail in [120, 219].

6.1.1 Open-Loop Impulse Control: The Value Function

Define

V .t0 ; x 0 / D minfJ.u.// j x.t0  0/ D x .0/ g

as the value function for Problem 6.1.1. We shall also use notation V .t0 ; x 0 / D
V .t0 ; x j ª C 0; ®.//; emphasizing the dependence on fª; ®./g:
Let us start by minimizing V1 .t0 ; x 0 / D V .t0 ; x 0 j ª; I . j x .1/ //: Then we first
find conditions for the solvability of boundary-value problem (6.6) under constraint
VarU./  ; with  given.
Since

hl; x.ª C 0/i D hl; x .1/ i 

(Z )
ªC0
hl; G.ª; t0 /x i C max
0
hl; G.ª; Ÿ/B.Ÿ/d U.Ÿ/i j VarU./   ;
t0

and since the conjugate space for vector-valued continuous functions C p Œt0 ; ª is
Vp Œt0 ; ª; then treating functions B 0 .t /G.ª; t /l as elements of C p Œt0 ; ª; we have

hl; x .1/ i  hl; G.ª; t0 /x 0 i C kB 0 ./G 0 .ª; /lkC Œt0 ;ª ; (6.7)

whatever be l 2 Rn :
Here

kB 0 ./G 0 .ª; /lkC Œt0 ;ª D maxfkB 0 .t /G 0 .ª; t /lk j t 2 Œt0 ; ªg D klkV ;
t

and klk in the braces is the Euclidean norm. We further drop the upper index p in
C p ; Vp while keeping the dimension p.
Denote c.t0 ; x .1/ / D x .1/  G.ª; t0 /x .0/ . Moving the first term in the right-hand
side of (6.7) to the left, then dividing both sides by kB 0 ./G 0 .ª; /lkC Œt0 ;ª , we come
to the next proposition.
256 6 Impulse Controls and Double Constraints

Theorem 6.1.1. Problem 6.2 of steering x.t / from x 0 D x.t0 / to x .1/ D x.ª C 0/
under constraint VarU.jŒt0 ; ª/   is solvable iff (6.7) is true for all l 2 Rn :
The optimal control U 0 .t / for this problem is of minimal variation
( ˇ )
hl; c.t0 ; x .1/ /i ˇ
ˇ
VarU ./ D  D sup
0 0
ˇl 2R ; :
n
(6.8)
kB ./G 0 .ª; /lkC Œt0 ;ª
0 ˇ

Note that the criteria of controllability in the class of functions U./ 2 Vp Œt0 ; ª
for system (6.2) are the same as in Sect. 1.3.
Lemma 6.1.1. In (6.8) the maximum (supremum over l) is attainable if system (6.2)
is strongly completely controllable. (Assumption 1.5.2 is true.)
Indeed, if 0 6D 0 and Assumption 1.5.2 requiring strong complete controllability
is true, then klkV defines a finite-dimensional norm in Rn (check this property (!))
and (6.8) is equivalent to

0 D maxfhl; x .1/  G.ª; t0 /x 0 i j klkV  1g D kx .1/  G.ª; t0 /x 0 kV ; (6.9)

where kxkV is the finite-dimensional norm conjugate to klkV :

Exercise 6.1.1. Find the norm kxkV :
Remark 6.1.1. Note that under Assumption 1.5.2 we have 0 D 0 .t0 ; x 0 I ª; x .1/ /
< 1:
Let l 0 be the maximizer in (6.9). Then from (6.7), (6.8) we observe
(Z ˇ )
ªC0 ˇ
ˇ
 D max
0
hl ; G.ª; Ÿ/B.Ÿ/d U.Ÿ/i ˇ VarU./   D
0 0
t0 ˇ

Z ªC0
D hl 0 ; G.ª; Ÿ/B.Ÿ/d U 0 .Ÿ/i: (6.10)
t0

This is the maximum principle for impulse controls which is given in integral
form.
Denote §0 ŒŸ D §.Ÿ; ª; l/ D l 0 G.ª; Ÿ/: Then we can rewrite (6.10) as
(Z ˇ ) Z ªC0
ªC0 ˇ
ˇ
0 D max § ŒŸB.Ÿ/d U.Ÿ/ ˇ VarU./   D
0 0
§0 ŒŸB.Ÿ/d U 0 .Ÿ/:
t0 ˇ t0
(6.11)
Theorem 6.1.2. The optimal impulse control U 0 ./ that minimizes variation
VarU./ satisfies the maximum principle (6.11) under l D l 0 —the maximizer
of (6.9). With 0 > 0 and under Assumption 1.5.2 it is also sufficient for the
optimality of U 0 ./:
6.1 The Problem of Impulse Controls 257

The proof of sufficiency is similar to Sect. 1.5.

Remark 6.1.2. The optimal control U 0 ./ for Problem 6.1.1 may not be unique.
Denote T .l/ D arg maxt fkB 0 .t /G 0 .ª; t /lk j t 2 Œt0 ; ªg which is the set of
points £ where

kB 0 .£/G 0 .ª; £/lk D maxfkB 0 .t /G 0 .ª; t /lk j t 2 Œt0 ; ªg:

t

Lemma 6.1.2. (i) Among optimal controls U 0 ./ there always exists one of type

U.t / D const; £i1 < t  £i ; i D 1; m C 1;
U.t / W
U.£i C 0/  U.£i / D ’i ; i D 1; m:

where ’ 6D 0 each of the instants £i 2 T .l 0 /; so that

d U 0 Œt  Xm
D p .i/ •.t  £i /; (6.12)
dt iD1

where

t0  £1 < £2 < : : : < £k  ª; p .i/ 2 Rp ;

X
m
VarU ./ D
0
jp .i/ j D 0 I
iD1

(ii) the number of such instants £i is m  n where n is the system dimension.

Exercise 6.1.2. Prove Lemma 6.1.2.2
A particular case arises under scalar control U:
Assumption 6.1.1. Coefficients A; B D const; B D b; jAj 6D 0; rankŒAB;
A2 B; : : : ; An B D n:
Lemma 6.1.3. Under Assumption 6.1.1 points £i 2 T .l 0 / are isolated.
This fact facilitates the calculation of U 0 .t /:
Exercise 6.1.3. Prove Lemma 6.1.3.
Example 6.1.1. Consider the motion of a ball which rolls along a planar curve as in
Fig. 6.1. The ball may be controlled by striking it along the direction of the motion

2
The proof of this Lemma may be found in [120, 219].
258 6 Impulse Controls and Double Constraints

Fig. 6.1 Motion of a ball 1

along a planar curve
0.8
0.6
0.4
0.2
0
−1 −0.5 0 0.5 1

with positive or negative impulse (a strike), so as to calm down the motion to a total
stop at given finite time.The problem is to find the total number of strikes with their
intensities and directions.
The equation for the motion of the controlled ball is

sR C ¨2 s D u; (6.13)

where s is the length from the origin along the curve with t 2 Œ0; ™. The stopping
rule is

s.ª/ D 0;
sP .ª/ D 0:

Exercise 6.1.4. 1. What should be the curve that yields equation of motion without
friction to be (6.13)?
2. Write down the equation of motion for the ball when the curve is a circular arc.
Write down (6.13) as

dx1 D x2 dt;
;
dx2 D ¨2 x1 dt C Udt;

where u D d U=dt; VarU  ; t 2 Œ0; =¨; x.0/ D Œx10 ; x20 0 .

Thus we have
   
0 1 0
AD ; B D ;
¨2 0 1

and the fundamental matrix for the homogeneous system is

 
 cos.¨t / ¨1 sin.¨t /
G.=¨; t / D ;
¨ sin.¨t /  cos.¨t /
6.1 The Problem of Impulse Controls 259

Hence
 1

sin.¨t /
G.=¨; t /B D ¨ ;
 cos.¨t /

with
   
0 1
x .1/
D x.=¨/ D ; x .0/
D x.0/ D ;
0 0

Then
 
1
c D x .1/  X.=¨; 0/x .0/ D :
0

Using (6.9) and making an equivalent substitution l1

¨
! l1 ; l2 ! l2 , we
come to

l1 ¨
0 D max :
l max jl1 sin.¨t / C l2 cos.¨t /j
0t ¨

 q 
Introducing ® D arccos l1 = l1 C l2 we get
2 2

q
l1 sin.¨t / C l2 cos.¨t / D l12 C l22 sin.¨t C ®/;

so that

l1 ¨
0 D max q : (6.14)
l
l12 C l22 max j sin.¨t C ®/j
0t=¨

Note that the right hand side of (6.14) depends only on the ratio ® D arctg.l2 = l1 /.
Then, since l1 > 0 is obviously infeasible, we may take l1 D 1, arriving at the
minmax problem of finding
q
min 1 C l22 max j sin.¨t C ®/j;
l2 0t=¨

Its solution gives l20 D 0, 0 D ¨ with only one extremum t D =2¨. Then the
desired control is
  
u D u0 D ’• t  :
2¨
260 6 Impulse Controls and Double Constraints

The constant ’ D ¨ may be found from condition

Z
ªC0
0
hl ; ci D
0
l 0 G.ª; t /B.t /d U 0 .t /;
t0

which follows from (6.9), (6.11).

So we have

Z=¨ Z=¨
sin.¨t /u dt D ¨;
0
cos.¨t /u0 dt H) ’ sin =2 D ¨;
0 0

The specified control u0 D d U 0 =dt ensures VarU./ D min : It is also a time-

optimal control for starting point x .0/ and x .1/ D 0 (prove that).
Exercise 6.1.5. 1. Indicate the set of solutions for this example when the time
interval is 0; 2
¨
, the criterion is VarU./ D min and c is arbitrary.
2. Find the time optimal solution for any starting point ft; xg and terminal point
x.toptimal / D 0.
We now move to solutions in the class of closed-loop (feedback) controls. This
requires us to develop a Dynamic Programming approach within this new class.

6.1.2 Closed-Loop Impulse Control: The HJB Variational

Inequality

Returning to Problem 6.1.1 with arbitrary starting position, we shall look for the
value function V .t; x/; assuming klkV is a norm. (As indicated above this is
ensured if system (6.2) is strongly completely controllable.) Recall that 0 D
0 .t; xI ª; x .1/ /:
We have
V .t; x/ D

inff0 .t; x I ª; x .1/ /C®.x .1/ /g D inffkx .1/ G.ª; t /xkV C®.x .1/ / j x .1/ 2 Rn g:
(6.15)
Denote

ˆ.x .1/ / D kc.t; x .1/ /kV C ®.x .1/ /; c.t; x .1/ / D x .1/  G.ª; t /x;
6.1 The Problem of Impulse Controls 261

and suppose inffˆ.x .1/ /jx .1/ 2 Rn g > 1: Then, since kc.t; x .1/ /kV is a norm,
we have

ˆ .0/ D maxfˆ.x .1/ /jx .1/ 2 Rn g < 1:

Hence 0 2 int Domˆ which indicates that the level sets of ˆ.x/ are compact
(closed and bounded). Then clearly, V .t; x/ < 1; the “infimum” in x .1/ in the
above is attainable (let it be point x  ) and can be substituted for “minimum.” This,
together with Lemma 6.1.2 brings us to the next conclusion.
Theorem 6.1.3. With V .t; x/ < 1 there exists a point x  where

minfˆ.x .1/ /jx .1/ 2 Rn g D 0 .t; xI ª; x  / C ®.x  /:

For every starting position ft; xg there exists an open-loop control U 0 .jt; x/
of type (6.12) which steers system (6.1) from ft; xg to fª; x  g thus solving Prob-
lem 6.1.1.
In order to calculate the conjugate of V .t; x/ in the second variable we apply
formula

V .t; x/ D minfmaxfhl; G.ª; t /x  x .1/ i C ®.x .1/ / j klkV  1; x .1/ 2 Rn g;

x .1/ l

where the min and max are interchangeable (check this property). Changing the
order of these operations and using the definition of convex conjugate functions we
have

V .t; x/ D max minfhl; G.ª; t /x  x .1/ i C ®.x .1/ / j x .1/ 2 Rn ; klkV  1g D

l x .1/

D maxfhl; G.ª; t /xi  maxfhl; x .1/ i  ®.x .1/ / j x .1/ 2 Rn g j klkV  1g D

l x .1/

D maxfhG 0 .ª; t /l; xi  ® .l/  I .ljBV Œt; ª/g; (6.16)

l

with

BV Œt; ª D fl W klkV  1g and klkV D maxfkB 0 .Ÿ/G 0 .ª; Ÿ/lk j Ÿ 2 Œt; ª/g;
Ÿ

where klk is the Euclidean norm.

This is a unit ball in Rn whose definition depends on system parameters and the
interval Œt; ª:
The last formula yields the next result.
262 6 Impulse Controls and Double Constraints

Theorem 6.1.4. The value function V .t; x/ is convex in x and its conjugate in the
second variable is

V  .t; l/ D ® .G 0 .t; ª/l/  I .G 0 .t; ª/ljBV Œt; ª/: (6.17)

Exercise 6.1.6. Check whether V  .t; x/ D V .t; x/:

Exercise 6.1.7. Prove Theorem 6.1.4 without Assumption 1.5.2 on complete con-
trollability of system (6.2). Then klkV will be only a seminorm rather than a norm
in Rn :
Theorem 6.1.5. The value function V .t; x j ª C 0; ®.// satisfies the principle of
optimality as a semigroup property: for any £ 2 Œt; ª the next relation is true

V .t; x/ D V .t; x j ª C 0; ®.// D V .t; x j £; V .£ C 0; jª C 0; ®.///: (6.18)

Proof. If at time £ there is no jump, then V .£; x.£// D V .£ C 0; x.£ C 0//. So, by
applying formulas (6.16), (6.17) we have:

V .£; xjª C 0; ®.// D maxfhl; G.ª; £/xi  ® .l/  I .ljBV Œ£; ª/ j l 2 Rn g;
l

V .t; xj£; V .£; // D maxfhl; G.£; t /xi  V  .£; l/  I .l j BV Œt; £/ j l 2 Rn g D
l

D maxfhG 0 .£; ª/l; G.ª; t /xi  ® .G 0 .£; ª/l/  I .G 0 .£; ª/ljBV Œ£; ª/
l

I .G 0 .£; ª/ljBV Œt; £/ j l 2 Rn g:

After substituting G 0 .£; ª/l D œ we get

V .t; xj£; V .£; // D V .t; xj£; V .£; jª C 0; ®./// D

D maxfhœ; G.ª; t /xi  ® .œ/  I .œ j œ 2 BV Œt; ª/g D V .t; xjª C 0; ®.//:

œ

In the last line we presume BV Œt; ª D fœ W kœkV  1g: t

u
Now, if at time £ there is jump x.£ C 0/  x.£/ 6D 0; we have

V .£; x.£ C 0// D V .£ C 0; x.£ C 0// D V .ª C 0; ®.//;

V .t; x/ D V .£; x.£C0// D V .£; x.£//C’.£/I ’.£/ D V .£; x.£C0//V .£; x.£//:

Using these relations with previous type of reasoning, we come to (6.18).

Remark 6.1.3. For the specific problem of this section note that the inequality

V .ª; xjª; ®.//  ®.x/

6.1 The Problem of Impulse Controls 263

is true, since

V .ª; xjª; ®.// D

D maxfhl; xi  ® .l/ j kB 0 .ª/lk  1g  maxfhl; xi  ® .l/ j l 2 Rn g  ®.x/:

l l

This inequality may even be a strict one. Thus, with ®.x/ D I .xjfx .1/ g/;
B.ª/ D I; we have V .ª; xjª; ®.// D maxl fhl; x  x .1/ ijklk  1g D kx  x .1/ k
< ®.x/:
The principle of optimality may now be used for deriving an analogy of the HJB
equation. However, here we have to count on two types of points £ 2 Œt; ª, namely,
those, where there is no control (U.t / D const; with t 2 Œ£  ©; £ C © for some
© > 0/ and those where there is an impulse u0 •.t  £/:
In the first case take t D £; ¢ 2 Œ0; ©/: Then we come to inequality

v.£/ D V .£; x/  V .£ C ¢; G.£ C ¢; £/x/ D v.£ C ¢/;

where v.¢ C £/ is nondecreasing due to the principle of optimality. Hence V .£; x/

is directionally differentiable at f£; xg along f1; A.£/xg and here

V 0 .£; xI 1; A.£/x/ D d v.¢/=d¢ j¢D0  0:

With V totally differentiable we have

d V .£; x/
H1 .£; x; Vt ; Vx / D D Vt .£; x.£//ChVx .£; x.£//; A.£/xi  0: (6.19)
d£
Further on, in the absence of differentiability the scalar product should be
interpreted as the directional derivative.
In the second case there is a jump at t D £: Then £ does not change and we take
Vt .£; x.£// D 0. Here denote the jump as u D ’h; where the unit vector h defines
the direction of the jump and ’ > 0—the size of the jump.
Now we have

H2 .£; x; 0; Vx / D ’ minfhVx .£; x/; B.£/hi C khk j khk D 1g D 0; (6.20)

h

where the minimizer h0 of unit length gives us the direction of the jump. Clearly,
here the multiplier ’ > 0 may be omitted.
The size ’ of the jump is obtained from condition

H1 .£ C 0; x.£ C 0/; Vt .£ C 0; x.£ C 0//; Vx .£ C 0; xC .£ C 0// D 0: (6.21)

Here x.£ C 0/ D x.£/ C B.£/u0 ; u0 D ’h0 ; ’ > 0: With ®.x/  0 it can also
arrive from the calculation ’ D V .£ C 0; x.£ C 0//  V .£; x.£//:
264 6 Impulse Controls and Double Constraints

We have thus found that V .t; x/ must satisfy the relation (a variational inequality)

H .t; x; Vt ; Vx / D minfH1 .t; x; Vt ; Vx /; H2 .t; x; Vt ; Vx /g D 0; (6.22)

under boundary condition

V .ª; x/ D V .ª; xI ª; ®.ª//: (6.23)

If necessary, V .t; x/ may be always treated as a generalized (viscosity-type)

solution (see [16]).
Denote q D Vt ; p D Vx . The last conclusions are now summarized as follows.
Theorem 6.1.6. The value V .t; x/ function is a solution (classical or generalized)
to the HJB-type Eq. (6.22) under boundary condition (6.23), where

H1 .t; x; q; p/ D q C hp; A.t /xi; H2 .t; x; q; p/ D minfhp; B.t /hi C khk j khk D 1g:
h

Denote H 0 D arg minh fhp; B.t /hi C khk j khk D 1g:

Theorem 6.1.7. The synthesizing control U 0 .t; x/ has the following form:
(i) if H2 .t; x; Vt ; Vx / > 0, then U 0 .t; x/ D f0g;
(ii) if H2 .t; x; Vt ; Vx / D 0, then U 0 .t; x/ D fu0 D ’h W h 2 H 0 g;
with (ii) taken under H2 .t; x.t C 0/; Vt .t; x.t C 0//; Vx .t; x.t C 0/// D 0;
and with H1 .t; x.t C 0/; Vt .t; x.t C 0//; Vx .t; x.t C 0/// D 0; x.t C 0/ D
x C ’B.t /h:
Related assertions are proved by direct calculation. These theorems are illustrated
next.
Example 6.1.2. Consider the problem of minimizing VarU./ due to equation

dx D b.t /d U; x.0/ D x; x.2/ D 0; b.t / D 1  .1  t /2 :

Here x 2 R; t 2 Œ0; 2: Following Theorems 6.1.6, 6.1.7, we have

V .t; x/ D
( ˇ )
hl; xi ˇ jxj
ˇ
D max ˇŸ 2 Œt; 2; l 2 R; l 6D 0 D
l maxŸ jb.Ÿ/lj ˇ maxŸ fb.Ÿ/jŸ 2 Œt; 2g

jxjb 1 .1/; t  1;
D
jxjb 1 .t /; t > 1:
6.1 The Problem of Impulse Controls 265

Let £ D 1: Consider the following cases:

• With t < £ we have Vt .t; x/ D 0; Vx .t; x/ D signxb 1 .1/, so that
H1 .t; x; Vt ; Vx / D Vt .t; x/ D 0 and

b.t /
H2 .t; x; Vt ; Vx / D 1  jb.t /Vx j D 1  D .1  t /2 > 0:
b.1/

There are no jumps at these points.

• With t > £ we have

2.t  1/jxj sign.x/

Vt .t; x/ D ; Vx .t; x/ D :
t 2 .2  t /2 b.t /

Here H1 .t; x; Vt ; Vx / D Vt .t; x/ > 0 with x 6D 0; while H1 .t; x; Vt ; Vx / D 0

with x D 0 and

H2 .t; x; Vt .t; x/; Vx .t; x// D 1  jb.t /Vx j D 1  b.t /b 1 .t / D 0; 8x:

Therefore, if we start moving at t  > £; then there must be an immediate

jump to zero with u0 D x.t  /signx.t  /b 1 .t /j•.t  t  /.
• With t D £ we have

b.£/ D 1; V .£; x/ D jxjb 1 .£/ D jxj; Vt .£; x/ D 0; Vx .£; x/ D signx;

while

H2 .£ C 0; x.£ C 0/; Vt .£ C 0; x.£ C 0//; Vx .£ C 0; x.£ C 0/// D

minfhsign.x/; hi C 1g D h0 C 1 D 0;
h

and

V .£; x.£//  V .£ C 0; x.£ C 0// D ’ D jxj;

H1 .£ C 0; x.£ C 0/; Vt .£ C 0; x.£ C 0//; Vx .£ C 0; x.£ C 0/// D 0:

There is a jump u0 D xsignx•.t  £/:

Finally, the synthesized system now looks like

dx=dt D b.t /x•.t  £/ or dx D b.t /xd U 0 .t; £/; (6.24)

where U 0 .t; £/ D 1£ .t /. (Recall that 1£ .t / D 0; if t 2 Œ0; £ and 1£ .t / D 1; if

t 2 .£; 2:/
266 6 Impulse Controls and Double Constraints

The solution to symbolic Eq. (6.24) is given by the Lebesgue–Stieltjes integral

equation (see [234])

Zt
x.t / D x  b.t /x.t /d U 0 .t; £/;
0

where x.t / is a left-continuous function. 

It now just remains to interpret the synthesized equation in the general case. This
will be done at the end of next section after discussing approximate solutions to
impulse controls.

6.2 Realizable Approximation of Impulse Controls

Impulse controls introduced above are “ideal” elements. In order to allow a physical
realization of the proposed schemes we introduce the next “realistic” scheme. Here
the original control will be subjected to double constraints where in the previous
problem there is an additional hard bound on the norm of the control vector, while
the bounding parameter may tend to infinity.

6.2.1 The Realistic Approximation Problem

Problem 6.2.1. Consider Problem 6.1.1 under additional constraint u.t / 2 B .0/,
so that there arrives the next problem: find
Z ª
J.t0 ; x 0 ju.// D ku.t /kdt C ®.x.ª// ! inf; (6.25)
t0

P / D A.t /x.t / C B.t /u.t /;

x.t t 2 Œt0 ; t1 ; (6.26)
x.t0 / D x ; 0
ku.t /k  : (6.27)

The solution to this problem exists since the set of all admissible controls u./ is now
weakly compact in L2 .Œt0 ; t1 I Rp / and convex, while the cost functional J.u.// is
weakly lower semicontinuous and convex.
The value function for the last problem is

V .t; x; / D minfJ.t; xju.// j t; xg:

u

Using the schemes of Chap. 2 (see Sect. 2.3.1), we come to the next proposition.
6.2 Realizable Approximation of Impulse Controls 267

Theorem 6.2.1. The value function V .t; x/ is a solution to the HJB equation
(* + ˇ )
@V .t; x; / @V .t; x; / ˇ
C min ; A.t /x.t / C B.t /u C kuk ˇˇ kuk   D 0;
@t @x
(6.28)
with boundary condition

V .t; x; / D ®.x/: (6.29)

As we shall see, V .t; x; / is directionally differentiable at each position ft; xg, so

that it satisfies Eq. (6.28) for all t .
Function V .t; x; / may be calculated through methods of convex analysis,
similar to the schemes of Sects. 2.4 and 2.6. Indeed, after applying the minmax
theorem of [70], and a permutation of the minimum in u with the integral, we have

V .t; x; / D
Z ª
D min max .ku.Ÿ/k C hl; G.ª; Ÿ/B.Ÿ/u.Ÿ/i/d Ÿ C hl; G.ª; t /xi
u l t

® .l/ j l 2 Rn ; kuk   D

D maxf¥.t; x; l/ C hl; G.ª; t /xi  ® .l/g (6.30)

l

where
Z ª
¥.t; x; l/ D minfhl; G.ª; s/B.s/ui C kuk j kuk  gds:
t u

Here the subproblem

h.s; l/ D minf§ŒsB.s/u C kuk j kuk  g;

u

with §Œs D l 0 G.ª; s/, has the solution


0 if k§ŒsB.s/k  1
h.s; l/ D
.1  k§ŒsB.s/k/ if k§ŒsB.s/k > 1;

This yields
Lemma 6.2.1. The value function
Z ª
V .t; x; / D maxfhl; G.ª; t /xi   fk§ŒsB.s/k  1gC ds  ® .l/g D
l t

D maxf§Œt x C ¥.t; x; l/  ® .l/ j l 2 Rn g;

l
268 6 Impulse Controls and Double Constraints

where

k§ŒsB.s/  1k; if k§ŒsB.s/k  1 > 0
fk§ŒsB.s/k  1gC D
0; if k§ŒsB.s/k  1  0:

Exercise 6.2.1. (a) Check that function V .t; x; / satisfies Eqs. (6.28), (6.29).
(b) Prove that with  ! 1 function V .t; x; / converges pointwise to V .t; x/
of (6.16).
(c) Prove that with  ! 1 the HJB Eqs. (6.28), (6.29) converges to
Eqs. (6.22), (6.23).
Lemma 6.2.2. The conjugate function to V .t; x; / in the second variable is
Z ª

V .t; l; / D  fkl 0 G.t; s/B.s/k  1gC ds C ® .l 0 G.t; ª//: (6.31)
t

The last formula is derived through direct calculation.

The control strategy u0 .t; x; / for Problem 6.2.1 is obtained from the next
subproblem
 
@V .t; x; /
U 0 .t; x; / D arg min ; B.t /u C kuk :
@x

Denote B 0 .t /Vx .t; x; / D c.t; x; /. Then this gives

8
< 0; if kc.t; x; /k < 1;
U 0 .t; x; / D Œ0; c.t; x; /; if kc.t; x; /k D 1;
:
c.t; x; /=kc.t; x; /k; if kc.t; x; /k > 1:

Theorem 6.2.2. The closed-loop solution to Problem 6.2.1 is given by U 0 .t; x; /

and the optimal trajectories emanating from ft; xg satisfy the differential inclusion

dx
2 A.s/x C B.s/U 0 .s; x; /: (6.32)
ds

Here set-valued function U 0 .s; x; / is upper semicontinuous in fs; xg. Therefore a

solution to (6.32) exists.
Suppose x Œs is a solution to (6.32) emanating from ft; xg. Denote
u0 .s; x Œs; / 2 U 0 .s; x Œs; / to be a measurable selector of U 0 .s; x Œs; /
and take continuous function U 0 .s; x Œs;  to be such that d U 0 .s; xŒs; / D
u0 .s; xŒs; /ds a.e. Then x Œs will also be a solution to the Stieltjes integral
equation
Z s
x Œs D G.s; t /x C G.s; Ÿ/B.Ÿ/d U 0 .Ÿ; x ŒŸ; /:
t
6.2 Realizable Approximation of Impulse Controls 269

With  ! 1 the approximate control U 0 .s; x Œs; / weakly converges to the

ideal U 0 .s; xŒs/, so that with  ! 1
Z s Z s
G.s; Ÿ/B.Ÿ/d U 0 .Ÿ; x ŒŸ; / ! G.s; Ÿ/B.Ÿ/d U 0 .Ÿ; xŒŸ/:
t t

where U 0 .Ÿ/ D U 0 .Ÿ; x.Ÿ// is a piecewise-constant function. The pointwise limits

of trajectories x Œs will be the optimal trajectories xŒs D x 0 Œs for the original
Problem 6.1.1.
Another version of formalizing the limit transition from approximating Prob-
lem 6.2.1 to Problem 6.1.1 is as follows.

6.2.2 The Approximating Motions

We indicate the approximation of a delta impulse u.t; t  ; x/ D ’.t  ; x/•.t  t  /

by a rectangular pulse (a tower) of lower base t 2 Œt  ; t  C ™.t  ; x; / and height
u.t; x; / D ; t 2 Œt  ; t  C ™.t  ; x; / with ™.t  ; x; / D ’ D const ant  0.
Definition 6.2.1. The pair of functions ‡ D fu.t; xI /; ™.t; xI /g (magnitude
and duration), is said to be a feedback control strategy for Problem 6.2.1 if

u.t; xI / 2 @B .0/ [ f0g; and u.t; xI / ! u1 .t; x/; . ! 1/

™.t; xI /  0; and ™.t; xI /  ’.t; x/  0; . ! 1/:

The component u.t; xI / is the approximating control impulse which is issued

on the interval Œt; t C ™.t; xI /. Note that ™ ! 0 as  ! 1, and in the limit one
has a delta-function of direction u1 .t; x/ and intensity ’.t; x/ as the ideal impulse
control.
Definition 6.2.2. Fix a control strategy ‡ , number  > 0 and a partition t0 D £0 <
£1 < : : : < £s D t1 of interval Œt0 ; t1 . An approximating motion of system (6.28)
is the solution to the differential equation

£i D £i1 C ™.£i1 ; x .£i1 /I /;

xP  .£/ D B.£/u.£i1 ; x .£i1 /I /; £i1  £ < £i :

Number ¢ D maxi f£i  £i1 g is the diameter of the partition.

Definition 6.2.3. A constructive motion of system (6.2) under feedback control
‡ is a piecewise continuous function x.t /, which is the pointwise limit of
approximating motions x .t / as  ! 1 and ¢ ! 0.
270 6 Impulse Controls and Double Constraints

Due to [75] the approximating differential inclusion (6.32) under feedback

strategy U 0 .s; x; / does have a solution, a realizable constructive approximating
motion. And thus the ends meet.
An illustrative explanation to the indicated procedure is given by the following
situation. Suppose the on-line position of system (6.1) is ft; xg, and from (6.22) it
N
follows that control U 0 ./ has a jump h•.t  tN/. Then the feedback strategy pair
uN D u.tN; xI N
N /, ™ D ™.tN; xI
N / is to be chosen such that

Z tNC™N 
N
B.t /h D  B.t /dt uN :
tN

In the limit this yields u.tN; xI N hk

N / ! h=k N and ™.tN; xI
N / ! khkN with  ! 1:
N N
That is, an impulse h•.t  tN/ of intensity ’ D h is approximated by a platform
of height (magnitude) , direction h,N and width (duration) 1 khk.
N

Example 6.1.3. We now deal with the same 2-dimensional system as in Exam-
ple 6.1.1, but looking for the solution in the form of feedback control strategies.
This will be done by specifying certain impulse domains which serve as regions
where the instantaneous impulses are activated. They are in a certain sense similar
to switching surfaces for bang-bang control.
We consider system

dx1 D x2 dt;
;
dx2 D x1 dt C d U;

where impulses arrive as generalized derivatives of u D d U=dt and, t 2

Œ0; =2, x.t / D Œx1 ; x2 0 . Our aim will be to minimize VarU.Ÿ/; Ÿ 2 Œt  0; ª C 0;
under x.ª/ D 0; ª D =2:
Following (6.8), (6.16), we have

0 .t; x/ D V .t; x/ D maxfhl; G.=2; t /xi j klkjV  1g:

where
 
sin t cos t
G.=2; t / D ;
 cos t sin t

and

klkV D maxfjhl; G.=2; Ÿ/b/ij j Ÿ 2 Œt; =2g D

q
D l12 C l22 maxfj cos.Ÿ  ®/j j Ÿ 2 Œt; =2g

where b D Œ0; 10 and ® D arcsin l2 .l12 C l22 /1=2 ; ® 2 Œ=2; =2.
6.2 Realizable Approximation of Impulse Controls 271

Further assuming t D 0; we have. Ÿ 2 Œ0; =2 and cos t D 1; sin t D 0: This

gives

1; ª 2 .0; =2/;
max j cos.Ÿ  ®/j D
maxfj sin ®j; j cos ®jg; otherwise;

and
(q
l12 C l22 l1 l2 > 0;
klkV D
maxfjl1 jj; jl2 jg otherwise:

Then
(q
z21 C z22 z1 z2 > 0;
 .0; x/ D kzk D maxfhl; zi
0
jV klkV  1g D
maxfjz1 j C jz2 jg z1 z2 < 0;

where z D G.=2; 0/x D fx2 ; x1 g: The maximizer in the last problem is
8
< .1; 1/ z1 > 0; z2 < 0;
l D .1; 1/ z1 ; 0; z2 > 0
0
:
z=kzk z1 z2 > 0;

and the impulses arrive at £ D argmaxfjb 0 G 0 .; Ÿ/l 0 j j Ÿ 2 Œ0; =2g which here, at
z1 z2 < 0; are £1 D 0; £2 D =2: Then the optimal synthesizing strategy u0 .t; x/ at
point ft 2 .0; ª; xg; for ’1 > 0; ’2 > 0; is

u.t; x/ D ’1 •.t  0/ C ’2 •.t  =2/; ’1 ; ’2  0; ’1 C ’2 D 0 ;

Checking this with terminal condition x.ª/ D 0; we finally get ’1 D x2 ; ’2 D x1 :

This solution is illustrated in the following figures. Namely, Fig. 6.2 demonstrates
the general picture of ideal feedback solutions from position f.t  0/; xg with
optimal discontinuous trajectories with jumps of the velocities x2 shown in Fig. 6.3.
The realizable solution under double constraint is indicated in Fig. 6.4. Here the
plane is divided into four domains three of which, D0 ; D , and D correspond,
respectively, to controls u D 0; u D ; u D : The fourth external domain
D; indicates initial states from which the solution to problem of reaching final
state x D 0 under constraint  is impossible (not solvable). Figure 6.5 illustrates
convergence of solutions from case Fig. 6.4 to case Fig. 6.3 with  ! 1.
272 6 Impulse Controls and Double Constraints

Fig. 6.2 General scheme 10

of feedback impulse control
8

6
wait jump down
4

x2
0

−2

−4
jump up wait
−6

−8

−10
−10 −8 −6 −4 −2 0 2 4 6 8 10
x1

Fig. 6.3 Ideal control 10

trajectories
8

2
x2

−2

−4

−6

−8

−10
−10 −8 −6 −4 −2 0 2 4 6 8 10
x1
6.2 Realizable Approximation of Impulse Controls 273

Fig. 6.4 Double-constraint mu = 5

5
control trajectories
4
Not Solvable
3
u=0
2

u = −µ
1

x2 0

−1
u=µ
−2
u=0
−3
Not Solvable
−4

−5
−5 −4 −3 −2 −1 0 1 2 3 4 5
x1

Fig. 6.5 The convergence of 25

double-constraint controls to
the impulse control 20

15 u=0 u = −µ

10

5
x2

−5

−10

−15 u=µ u=0

−20

−25
−25 −20 −15 −10 −5 0 5 10 15 20 25
x1
Chapter 7
Dynamics and Control Under State Constraints

Abstract The topics of this chapter are problems of reachability and system
dynamics under state constraints in the form of reach tubes. Indicated are general
approaches based on the Hamiltonian formalism and a related Comparison Prin-
ciple. Further emphasis is on the dynamics of linear systems under hard bounds
on the controls and system trajectories. A detailed solution is presented based on
ellipsoidal approximations of bounded trajectory tubes.

Keywords State constraints • Reach tubes • Comparison Principle • Hamilto-

nian approach • Linear-convex systems • Ellipsoidal techniques

The issue of control under state constraints is at the heart of many applied problems.
An abundant literature on such issues is published under the notion of “viability
theory” [5, 6]. The specifics of this book lies in general treatment of such problems
under time-dependent state constraints through Hamiltonian techniques with related
Comparison Principle, while the linear-convex case is covered in detail through
ellipsoidal approximations. Such a move provides a natural merger of theory with
appropriate computation schemes. The material of this chapter uses results of
papers [158, 185]. Its topics are also a useful preparation for further consideration
of problems on feedback control under state constraints and external obstacles
addressed in the next chapter.

7.1 State-Constrained Reachability and Feedback

This section deals with general theory of reach sets and tubes under hard bounds
on controls, and time-dependent geometric (hard) constraints on the states. The
solution relations are given here in Hamiltonian terms, while the linear-convex
case is described through tight external approximations by parametrized ellipsoid-
valued tubes. The result is an exact parametric representation of state-constrained
reachability tubes through families of external ellipsoidal tubes.

© Springer International Publishing Switzerland 2014 275

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__7
276 7 Dynamics and Control Under State Constraints

7.1.1 The Reachability Problem Under State Constraints

We begin with the general nonlinear case. Similar to (1.1) the controlled system is
thus described by the same differential equation, with same properties, namely

xP D f .t; x; u/; t0  t  £; (7.1)

in which the state x 2 Rn and the control u 2 Rp are restricted by

u.t / 2 P .t /; x.t / 2 Y .t /; (7.2)

for all t  t0 . Here P .t /; Y .t / are compact set-valued functions in Rp ; Rn ,

respectively, continuous in the Hausdorff metric. The initial state is restricted by
x.t0 / 2 X 0 , a compact subset of Rn . The function f .t; x; u/ is assumed to ensure
uniqueness and uniform extension of solutions to any finite interval of time for any
x.t0 / D x 0 ; u.t / 2 P .t /; t  t0 .
Definition 7.1.1. Given set-valued positions ft0 ; X 0 g, X 0 \ Y .t0 / 6D ;, the reach
set (or “attainability domain”) X .£; t0 ; X 0 / at time £ > t0 is the set

X Œ£ D X .£; t0 ; X 0 /

of all states xŒ£ D x.£; t0 ; x 0 /; x 0 D x.t0 / 2 X .t0 / D X 0 , that can be reached at

time £ by system (7.1), from some x 0 2 X 0 , using all possible controls u that ensure
constraints (7.2). The set-valued function £ 7! X Œ£ D X .£; t0 ; X 0 / is the reach tube
from ft0 ; X 0 g.
The Basic Problem is simply stated:
Problem 7.1.1. Calculate the reach sets X .£; t0 ; X 0 /; £  t0 .
A fairly general approach is to relate reach sets to an optimization problem [178].
This could be done by calculating certain value functions that may be selected in
several ways. Consider first the value function
( Z )
£
V .£; x/ D min d .x.t0 /; X / C
2 0 2
d .x.t /; Y .t //dt ; (7.3)
u t0

under the restriction x.£/ D x. Here the minimum is over all measurable functions
(controls) u.t / 2 P .t /, function x.t /; .t  t0 /; is the corresponding trajectory
of (7.1), and

d 2 .x; X / D minfhx  z; x  zijz 2 X g

is the square of the distance function d.x; X /.

7.1 State-Constrained Reachability and Feedback 277

Lemma 7.1.1. The following relation is true:

X .£; t0 ; X 0 / D fx W V .£; x/  0g:

This follows from the definition of the reach set X .£; t0 ; X 0 /, which is thus a level
set of V .£; x/.
To state the important semigroup property of the value function, we extend
definition (7.3) to more general boundary conditions, namely,

V .£; x j t0 ; V .t0 ; // D

 Z £ 
D min V .t0 ; x.t0 // C d 2 .x.t /; Y .t //dt j u./ 2 P ./ ; (7.4)
u t0

under the restriction x.£/ D x. Function V .t0 ; / defines a given boundary condition.
In (7.2) this condition is V .t0 ; x/ D d 2 .x; X 0 /.
Theorem 7.1.1. The value function V .£; x/ satisfies the principle of optimality,
which has the semigroup form

V .£; x j t0 ; V .t0 ; // D V .£; x j t; V .t;  j t0 ; V .t0 ; ///; t0  t  £: (7.5)

This property is established through conventional arguments similar to Sect. 2.1,

that also yield similar properties for reach sets. The solution of the reachability
problem can be cast in the form of a solution of the “forward” HJB equation that
follows from (7.5). To develop this approach we further assume that the function
V .t; x/ is continuously differentiable.
A standard procedure, as before, then yields

Vt .t; x/ C maxfhVx .t; x/; f .t; x; u/i  d 2 .x; Y .t // j u 2 P .t /g D 0; t0  t  £;

u
(7.6)

with boundary condition V .t0 ; x/ D d 2 .x; X 0 /. Here Vt ; Vx stand for the partial
derivatives of V .t; x/. Note that the term d 2 .x; Y .t // 6D 0 only outside the state
constraint Y .t /.
An alternative scheme relies on the value function

V .£; x/ D minf®0 .t0 ; x.t0 //g: (7.7)

u

The minimum in (7.7) is over all u.t / 2 P .t /; t 2 Œt0 ; £, under restrictions x.£/ D
x and ®.t; x/  1; t 2 Œt0 ; £. We will use this formulation for the case when
0
®0 .t0 ; x/ D hx  x 0 ; .X 0 /1 .x  x 0 /i1=2 ; X 0 D X 0 > 0;

®.t; x/ D hx  y.t /; .Y .t //1 .x  y.t //i1=2 ; Y.t / D Y 0 .t / > 0; (7.8)

278 7 Dynamics and Control Under State Constraints

so that the initial condition and the state constraints are ellipsoidal:

x.t0 / 2 E .x 0 ; X 0 / D fx W ®0 .t0 ; x/  1g;

x.t / 2 E .y.t /; Y.t // D fx W ®.t; x.t //  1; t 2 Œt0 ; £g:

Once x.£; t0 ; x.t0 // D x.£/ D x is fixed, one may figure out whether x 2
X .£; t0 ; X 0 / by looking at the value V .£; x/. Then the respective vector x.t0 / 2
E .x 0 ; X 0 / iff there exists a control u.t /; t 2 Œt0 ; £; that ensures V .£; x/  1, so
that x.£/ 2 X .£; t0 ; X 0 / iff V .£; x/  1.
Therefore the reach set at time £ is

X .£; t0 ; X 0 / D fx j V .£; x/  1g;

which is thus a level set of V .£; x/, and the previously stated semigroup property of
the value function (the principle of optimality) holds again.
The solution of the reachability problem can again be cast in the form of a
solution of the forward HJB equation—now somewhat different from (7.6). Again
assume the functions V .t; x/ and ®.t; x/ are continuously differentiable.
Denote

H .t; x; V; u/ D Vt .t; x/ C hVx .t; x/; f .t; x; u/i: (7.9)

Lemma 7.1.2. The formal HJB equation is

max H .t; x; V; u/ D 0 if ®.t; x/ < 1; (7.10)

u2P .t/

and

max H .t; x; V; u/ D 0 if ®.t; x/ D 1; (7.11)

u2Ps .t/

with

Ps .t; x/ D P .t / \ fu j H .t; x; ®; u/  0g; (7.12)

and with boundary condition

V .t0 ; x/ D ®0 .t0 ; x/: (7.13)

We sketch a proof of Lemma 7.1.2.

Together with ®.t; x/ < 1, the principle of optimality (7.5), with t0 D £  ¢,
gives

0 D minfV .£  ¢; x.£  ¢//  V .£; x/ j u 2 P .t /; t 2 Œ£  ¢; £g (7.14)

u
7.1 State-Constrained Reachability and Feedback 279

or

max H .t; x; V; u/ D H .t; x; V; u0 / D 0: (7.15)

u2P .t/

With ®.t; x/ D 1 we apply the same principle but only through those controls that
do not allow the trajectory to move outside the state constraint. These are

u.t; x/ 2 Ps .t; x/ D fu 2 P .t / \ fu j H .t; x; ®; u/  0gg:

Let u0 .t; x/ be the optimal control for the trajectory x 0 .t / that emanates from
x.£/ D x and minimizes ®0 .t0 ; x.t0 // under constraints ®.t; x.t //  1; t 2 Œt0 ; £.
Note that with ®.t; x 0 .t // D 1 we have two cases: either H .t; x 0 .t /; ®; u0 / D
0, which means the related optimal trajectory runs along the state constraint
boundary, or

d®.t; x 0 /.t //=dt juDu0 D H .t; x 0 .t /; ®; u0 / < 0;

so the optimal trajectory departs from the boundary, and for ¢ > 0 we have ®.t C
¢; x 0 .t C ¢// < 1. Relations H .t; x 0 .t /; ®; u0 /  0 and (7.15) allow one to find the
control u0 along the state constraint boundary.
If all the operations in (7.6), (7.10), (7.11) result in smooth functions, then these
equations may have a classical solution [51]. Otherwise (7.6), (7.7) form a symbolic
generalized HJB equation, which has to be described in terms of subdifferentials,
Dini derivatives, or their equivalents. However, the typical situation is that V is
not differentiable and as said before, in Chap. 5, the treatment of (7.6), (7.7) then
involves the notion of a generalized “viscosity” solution for these equations or their
equivalents [50, 80, 247]. One approach is to use the method of characteristics as
developed for this type of equation (see [51]) and modified for the nonsmooth
case [248]. But it is a fairly complicated procedure, especially in the nonsmooth
case for which the method requires additional refinement. Another approach is
to look for the solution through level set methods [221, 244]. However, for the
specific “linear-convex” problems of this book the function V .t; x/ is convex in
x, hence directionally differentiable in any direction f.1; x/g, which proves it to be
a generalized solution through classical technique (see [16, 123]). Moreover, in this
case an effective ellipsoidal technique may be applied, which allows one to bypass
the calculation of solutions to the HJB equation.

The Evolution Equation

The next question is: on which evolution equation would the multivalued mapping
X Œt  D X .t; / D X .t; jX .t0 ; // under state constraint be satisfied? An answer is
280 7 Dynamics and Control Under State Constraints

in papers [158, 164], where this was suggested to be taken as the integral funnel
equation for differential inclusion

xP D2 f .t; x; P .t //; x.t0 / 2 X 0 ; (7.16)

under constraint

x 2 Y .t /; t 2 Œt0 ; ª:

Here is one of such equations (see [174] and also [164])

( ˇ )!
[ ˇ
1 ˇ
lim hC Z Œt C ¢; x C ¢.f .t; x; P .t //ˇ x 2 Z Œt  \ Y .t / D 0; Z .t0 / D X 0 ;
¢!C0 ¢ ˇ
(7.17)
where hC is the Hausdorff semidistance:

hC .X 0 ; X 00 / D minf© W X 0 X 00 C ©B .0/g

and B .0/ is a unit ball in Rn . The solution Z Œt  to this evolution equation is a

multivalued function, with Z Œt0  D X 0 ; which satisfies (7.17) almost everywhere.
As a rule, this solution is not unique. However, there exists a solution Xm Œt , sought
for among these Z ./; which coincides with the realization of function X .t; / and is
the inclusion-maximal among all solutions Z Œt  to (7.17). Namely, X Œt 
Z Œt ,
where Z Œt  is any solution that starts from Z .t0 / D X 0 ; see [174]. Note that
Eq. (7.17) makes sense for any upper semi-continuous set-valued functions Y .t /.
Hence in equations of type (7.17) we may allow these functions to be right or left-
continuous.
However, there is a more rigorous version of the funnel equation (see [158]),
namely,
( ˇ )!
[ ˇ
1 ˇ
lim hC Z Œt C ¢; x C ¢.f .t; x; P .t //ˇ x 2 Z Œt  \ Y .t C ¢/ D 0; Z .t0 / D X 0 :
¢!C0 ¢ ˇ
(7.18)

This version demands that support function f .l; t / D ¡.l j Y .t // would be

differentiable in t and its partial derivative @f .l; t /=@dt would be continuous. Then
the set-valued solution Z Œt  satisfies (7.18) everywhere and will be unique.
Theorem 7.1.2. (i) The solution Z Œt  D Z .t; t0 ; X 0 / to differential inclusion (7.16)
satisfies the funnel equation (7.17) almost everywhere and its inclusion-
maximal solution Z Œt  D Xm Œt  is unique.
(ii) The solution Z Œt  D Z .t; t0 ; X 0 / to differential inclusion (7.16) satisfies the
funnel equation (7.18) everywhere and its solution Z Œt  D X Œt  is unique.
In the sequel we always presume the following.
7.1 State-Constrained Reachability and Feedback 281

Assumption 7.1.1. Functions V .£; x/ are continuous in all the variables, with
nonempty compact zero level-sets that have nonempty interiors of full dimension.
Such level sets

X Œ£ D fx W V .£; x/  0g 6D ;

may be gotten due to the last assumption for Lemma 7.1.2. Such are those that solve
our problem for linear systems with continuous coefficients and convex constraints
continuous in time. The indicated class of function V .£; / will be denoted as KV .
Remark 7.1.1. The variety KV may be considered as a metric space with metric:

d.V 0 .£; /; V 00 .£; // D h.X 0 ; X 00 /;

where X 0 ; X 00 —zero level-sets of V 0 ; V 00 , and the Hausdorff distance

h.X ; Z / D maxfhC .X ; Z /; hC .Z ; X /g;

where hC .X ; Z / is the Hausdorff semidistance.

This variety KV may also be endowed with another metric, introducing
d.V 0 .£; /, V 00 .£; // as the distance in the space Cr Œt0 ; £ (of r-dimensional
continuous functions) between respective functions y 0 .t /, y 00 .t / (t 2 Œt0 ; £)
of (7.8) or (7.19), which generated V 0 .£; /; V 00 .£; / due to Eqs. (7.9)–(7.13).
Such a definition may be propagated only to functions y./ 2 Y./. Similar metrics
may be also considered on the variety of functions V .1/ .£; x/.
A detailed discussion of possible metrics for spaces of functions similar to those
generated by value functions of this chapter is given in monograph [98, Chap. 4,
Appendix C.5].
Sets X Œt  may thus be found, as in systems without state constraints, either
through value functions V .t; x/; from their level sets, or through evolution
Eqs. (7.17), (7.18). However, since direct calculation of these solutions is not
simple, we remind that it may suffice to avoid exact solutions to related HJB
equations under state constraints. We may hence apply the comparison principle
which gives upper and lower approximations to value function V .t; x/ for HJB
equation (7.6). Henceforth we may also obtain external and internal ellipsoidal
estimates of sets X . Such issues are described in the next sections.

7.1.2 Comparison Principle Under State Constraints

We shall now indicate the specifics of applying the comparison principle of Sect. 5.1
when the HJB equation is derived under state constraints. We further work with
system (7.1), (7.2), but here the state constraint Y .t / will be presented as

Y .t / D fx W y.t /  g.t; x/ 2 R .t /g; t 2 Œt0 ; £; (7.19)

282 7 Dynamics and Control Under State Constraints

where values y; g lie in Rr ; r  n, while properties of functions y.t /; g.t; x/ as

same as those of x.t /; f .t; x/, respectively, and set-valued function R .t / is taken to
be convex, compact-valued, and Hausdorff-continuous. The Comparison Principle
is further treated within two versions.
Version-A
We first use the approach used for Lemma 7.1.2 relying on relations of type
(7.3)–(7.6) and modifying them in view of (7.19). Then introducing notation

H.t; x; p/ D maxfhp; f .t; x; u/iju 2 P .t /g  d 2 .y.t /  g.t; x/; R .t //;

we observe, as in Sects. 2.3 and 2.4 (see also [178,198]), that the solution V .t; x/ of
the corresponding “forward” HJB equation

Vt C H.t; x; Vx / D 0; V .t0 ; x/ D d 2 .x; X 0 / (7.20)

allows to calculate X Œt  D X .t; t0 ; X 0 / as the level set X Œ£ D fx W V .£; x/  0g:

Here again this property is independent of whether V is a classical or a generalized
solution to Eq. (7.20).
To obtain approximations of V .t; x/ we proceed as follows.
Assumption 7.1.2. Given are functions H C .t; x; p/, wC .t; x/ 2 C1 and .t / 2
L1 , which satisfy the inequalities

H.t; x; p/  H C .t; x; p/; 8ft; x; pg; (7.21)

wC C
t C H .t; x; wx /  .t /; (7.22)

wC .t0 ; x/  V .t0 ; x/: (7.23)

Theorem 7.1.3. Suppose H C .t; x; p/; wC .t; x/; .t / satisfy Assumption 7.1.2.
Then the following estimate for the reachability set X Œt  is true:

X Œt  XC Œt ; (7.24)

where
 Z t 
XC Œt  D x W wC .t; x/  .s/ds C V .t0 ; x.t0 // : (7.25)
t0

Consider a pair x  2 X Œt ; u .s/ 2 P .s/; s 2 Œt0 ; t , so that the corresponding

trajectory x  .s; t; x  j u .// D x  Œs 2 X Œs. Then

d wC .t; x/=dtjxDx  .t/

D wC  C    
t .t; x / C hwx .t; x /; f .t; x ; u /i  d .y.t /  g.t; x .t //; R .t //
2

 wC   C C  C  C
t .t; x / C H.t; x ; wx /  wt .t; x / C H .t; x ; wx /  .t /;
7.1 State-Constrained Reachability and Feedback 283

hence

d wC .s; x/=dsjxDx  .s/  .s/:

Integrating this inequality from t0 to t gives

Z t Z t
wC .t; x/  .s/ds C wC .t0 ; x.t0 //  .s/ds C V .t0 ; x.t0 //g;
t0 t0

which implies x  .t / 2 XC Œt  and the theorem is proved.

We now move to the discussion of internal estimates for sets X Œt : As in the
above, we do not necessarily require differentiability of the value function V .t; x/.
Consider the next assumption.
Assumption 7.1.3. Given are functions H  .t; x; p/ 2 C and w .t; x/ 2 C1 ;
which satisfy the inequalities

.i / H.t; x; p/  H  .t; x; p/; 8ft; x; pg;

.i i / w  
t .t; x/ C H .t; x; wx .t; x//  0;

.i i i / w .t0 ; x/  V .t0 ; x/:

Theorem 7.1.4. Suppose functions H  .t; x; p/; w .t; x/ satisfy Assumption 7.1.3.
Then the following upper estimate for V .t; x/ is true:

V .£; x/  w .£; x/: (7.26)

Indeed, with t; x given, consider an optimal trajectory x.s/ for prob-

lem (7.1), (7.2) with (7.19), t0  s  t , under condition x.t / D x. The definition of
V .t; x/ yields
Z t
V .t; x/ D V .t0 ; x / C 0
d 2 .y.s/  g.s; x.s//; R .s//ds:
t0

Differentiating w along this optimal trajectory and having in view

Assumption 7.1.3, we get

d w .s; x/=ds jxDx.s/ D w 

s .s; x.s// C hwx .s; x.s//; f .s; x.s/; u.s//i

D w 
s .s; x.s// C H.s; x.s/; wx .s; x.s/// C d .y.s/  g.s; x.s//; R .s//
2

 w  
s .s; x.s// C H .s; x.s/; wx .s; x.s/// C d .y.s/  g.s; x.s//; R .s//
2

 d 2 .y.s/  g.s; x.s//; R .s//:

284 7 Dynamics and Control Under State Constraints

Integrating the last equality on Œt0 ; t ; we further have

Z t
w .t; x/  w.t0 ; x 0 / C d 2 .y.s/  g.s; x.s//; R .s//ds
t0

Z t
 V .t0 ; x 0 / C d 2 .y.s/  g.s; x.s//; R .s//ds D V .t; x/:
t0

and the theorem is proved.

Denoting X  Œt  D fx W w .t; x/  0g; t  t0 ; and using the last inequality, we
further come to the conclusion
Corollary 7.1.1. Under Assumption 7.1.3 the following inclusion is true:

X  Œt  X Œt : (7.27)

This statement is similar to those of 7.5–7.7 from [48].

Version-B
We now rely on the approach used for Lemma 7.1.1. After introducing notation

H1 .t; x; p; p0 ; œ/ D maxfhp; f .t; x; v/i  œhp0 ; f .t; x; v/i jv 2 P .t /g;

the corresponding “forward” HJB equation (7.7) transforms into

.1/
Vt C œ™t C H1 .t; x; Vx ; ™x ; œ/ D 0; V .t0 ; x/ D d.x; X 0 / (7.28)

where the multiplier œ is to be found from condition

d ™.t; x/=dt j(7.1) D ™t C h™x ; f .t; x; v/i  0; (7.29)

which is the total derivative of constraint function ™ along trajectories of

system (7.1).
This allows one to calculate X Œt  D X .t; t0 ; X 0 / as the level set X Œ£ D fx W
V .£; x/  1g: This property is independent of whether V .1/ is a classical or
.1/

a generalized solution to Eq. (7.29). The comparison theorems and the respective
proofs for Version-B are similar to Version-A.
Remark 7.1.2. In this subsection we dealt with reach sets that are forward. For the
design of state-constrained feedback controls we may need to use backward reach
sets. These are designed similarly and discussed with detail within related schemes
of Sect. 5.1.
We now pass to the class of linear systems with convex constraints where the
previous results may be developed with greater detail.
7.1 State-Constrained Reachability and Feedback 285

7.1.3 Linear-Convex Systems

Consider the linear system

xP D A.t /x C B.t /u; t0  t  t1 ; (7.30)

in which A.t /; B.t / are continuous, and the system is completely controllable (see
[109]). The control set P .t / is a nondegenerate ellipsoid, P .t / D E .p.t /; P .t //,
and

E .p.t /; P .t // D fu j hu  p.t /; P 1 .t /.u  p.t //i  1g; (7.31)

with p.t / 2 Rp (the center of the ellipsoid) and the symmetric positive definite
matrix function P .t / 2 Rpp (the shape matrix of the ellipsoid) continuous in t .
The support function of the ellipsoid is

¡.l j E .p.t /; P .t /// D maxfhl; ui/ j u 2 E .p.t /; P .t //g D hl; p.t /iChl; P .t /li1=2 :

The state constraint is

x.t / 2 Y .t /; t 2 Œt0 ; t1 ; (7.32)

in which Y .t / is an ellipsoidal-valued function Y .t / D E .y.t /; Y.t //; Y.t / D

Y 0 .t / > 0; Y.t / 2 Rnn , and with y.t /; Y.t / absolutely continuous. Lastly, X 0 D
E .x 0 ; X 0 / is also an ellipsoid.
In [174, 181] it is also assumed that the constraints on the controls and initial
values are ellipsoidal, which makes the explanations more transparent. However,
these methods are applicable to box-valued constraints as well, see Sect. 5.4.3 and
also [160, 182].
Lemma 7.1.3. For the linear system (7.30), with convex-valued restric-
tions (7.31), (7.32) the reach set X .£; t0 ; X 0 / at time £ is convex and compact.
The problem of calculating the reach set is now formulated as follows.
Problem 7.1.2. Calculate the support functions ¡.l j X .£; t0 ; X 0 //; l 2 Rn :
This is equivalent to solving the following optimal control problem with state
constraints:

¡.l j X .£; t0 ; X 0 // D maxhl; x.£/i

subject to x.t0 / 2 X 0 ; u.t / 2 P .t /; x.t / 2 Y .t /; t0  t  £:

286 7 Dynamics and Control Under State Constraints

For each vector l the solution to the optimal control problem is attained at the
terminal point x .0/ .£/ of an optimal trajectory x .0/ .t /; t 2 Œt0 ; £; which starts from
a point x .0/ .t0 / 2 X 0 determined throughout the solution process.
We will solve Problem 7.1.2 by calculating the value function (7.4), namely,

V .£; x/ D minfd.x.t0 /; E .x 0 ; X 0 // j x.£/ D x;

u

u.t / 2 P .t /; x.t / 2 E .y.t /; Y.t //; t 2 Œt0 ; £g;

for system (7.30), using the techniques of convex analysis as in paper [166].
To handle the state constraints X Σ, one usually imposes the following constraint
qualification.
Assumption 7.1.4. There exist a control u.t / 2 P .t /; t 2 Œt0 ; £; x 0 2 X 0 , and
© > 0, such that the trajectory xŒt  D x.t; t0 ; x 0 / D x.t; t0 ; x 0 j u.// satisfies

x.t; t0 ; x 0 / C –B .0/  Y .t /; t 2 Œt0 ; £:

Here B .0/ D fz W .z; z/  1; z 2 Rn g. Under Assumption 7.1.4 this gives

V .£; x/ D supfhl; xi  ‰.£; t0 ; l/ j lg;

where

‰.£; t0 ; l/ D minf‰.£; t0 ; l; ƒ.// j ƒ./g; (7.33)

‰.£; t0 ; l; ƒ.// D s.t0 ; £; l j ƒ.//x 0 C .s.t0 ; £; l j ƒ.//X 0 s 0 .t0 ; £; l j ƒ.///1=2

Z £ 
C s.t; £; l j ƒ.//B.t /p.t / C .s.t; £; l j ƒ.//B.t /P .t /B 0 .t /s 0 .t; £; l j ƒ.///1=2 dt
t0

Z £ 
C dƒ.t /y.t / C .dƒ.t /Y .t /dƒ0 .t //1=2 :
t0

Here, s.t; £; l j ƒ.//; t  £; is the row-vector solution to the adjoint equation

ds D sA.t /dt C dƒ.t /; s.£/ D l 0 ; (7.34)

and ƒ./ 2 Vn Œt0 ; £—the space of n-dimensional functions of bounded variation on

Œt0 ; £. We also used the notation
Z £ Z £ 
.dƒ./Y.t /dƒ0 .//1=2 D max dƒ.t /h.t / j h.t / 2 E .0; Y.t //; t 2 Œt0 ; £ :
t0 t0

This is the maximum of a Stieltjes integral over continuous functions h.t / 2

E .0; Y.t //. The minimum ‰ over ƒ./ is reached because of Assumption 7.1.4.
A direct consequence of formula (7.33) is the solution to Problem 7.1.2.
7.1 State-Constrained Reachability and Feedback 287

Theorem 7.1.5. Under Assumption 7.1.4, the support function ¡.l j X Œ£/ is
given by

¡.l j X Œ£/ D ‰.£; t0 ; l/ D minf‰.£; t0 ; l; ƒ.// j ƒ./ 2 Vn Œt0 ; £g: (7.35)

Let ƒ.0/ ./ 2 arg minf‰.£; t0 ; l; ƒ.// jƒ./ 2 Vn Œt0 ; £g be the minimizer
of (7.14) and let s .0/ Œt  D s.t; £; l j ƒ.0/ .// be the solution to (7.30) with ƒ./ D
ƒ.0/ ./.
The application of formula (7.34) indicates a new approach to the next proposi-
tion. Alternate derivations of this approach can be found in [137,166, pp. 116–121].
Theorem 7.1.6 (The “Standard” Maximum Principle Under State Con-
straints). For Problem 7.1.2 under Assumption 7.1.4, the optimal control u.0/ .t /,
initial condition x .0/ , and trajectory x .0/ Œt  D x.t; t0 ; x .0/ ju.0/ .// must satisfy the
“maximum principle”

s .0/ Œt B.t /u.0/ .t / D maxfs .0/ Œt B.t /u j u 2 E .p.t /; P .t //g; t0  t  £;

(7.36)
and the “maximum conditions”
Z £ Z £
0
.dƒ.0/ .t /y.t / C .dƒ.0/ .t /Y.t /d ƒ.0/ .t //1=2 / D dƒ.0/ .t /x .0/ Œt  (7.37)
t0 t0
Z £ 
D max .dƒ.0/ .t /z.t // j z.t / 2 E .y.t /; Y.t // ;
t0

s Œt0 x Œt0  D s .0/ Œt0 x 0 C .s .0/ Œt0 ; X 0 s .0/ Œt0 /1=2 D

.0/ .0/

maxfs .0/ Œt0 x j x 2 E .x 0 ; X 0 /g: (7.38)

Here we note that the minimum over ƒ./ 2 Vn Œt0 ; £ in (7.33) may be replaced by
the minimum over the pair fM.t /; œ.t /g, with dƒ.t / D l 0 M.t /d œ.t /, in which the
n  n matrix M.t / is continuous, and œ./ 2 V1 Œt0 ; £ is a scalar function of bounded
variation. Moreover, M.t / may be chosen within a compact set C0 of continuous
functions. This new form of the multiplier dƒ.t / is a result of combining the earlier
schemes of [96, 158]. We summarize this result as follows.
Lemma 7.1.4. The multiplier ƒ.0/ .t / allows the representation

dƒ.0/ .t / D l 0 M .0/ .t /d œ.0/ .t /; (7.39)

with M .0/ ./ 2 C0 ; œ.0/ ./ 2 V1 Œt0 ; £.

Denote by SM .t; £/ D S.t; £ j M./; œ.// the solution of the matrix equation

dS D SA.t /dt C M.t /d œ.t /; S.£/ D I: (7.40)

288 7 Dynamics and Control Under State Constraints

This is a symbolic expression for the linear differential equation. Under permutabil-
ity of A.t / with its integral, namely, under condition

Z t Z t
A.t / A.s/ds D A.s/dsA.t /
£ £

the solution to (7.40) for fixed £ is

Z £  Z £  Z s 
SM .t; £/ D exp A.s/ds  exp A.Ÿ/d Ÿ M.s/d œ.s/;
t t n

in which the second integral is a Stieltjes integral. In the absence of the permutation
property the solution is expressed through expanding a matrix series of the Peano
type [32]. Note that with A.t /  const the permutability property is always true.
As in the scheme of [158], the single-valued functional ‰.£; t0 ; l; ƒ.// may be
substituted with the set-valued integral

R.£; t0 ; M./; œ./; X 0 / D S.t0 ; £ j M./; œ.//E .x 0 ; X 0 / (7.41)

Z £ Z £
C S.t; £ j M./; œ.//B.t /E .p.t /; P .t //dt C d œ.t /M.t /E .y.t /; Y.t //dt;
t0 t0

which yields the next result whose analytical form differs from [158].
Lemma 7.1.5. The following equalities hold:

X Œ£ D \fR.£; t0 ; M./; œ./; X 0 / j M./; œ./g D

\fR.£; t0 ; M./; œ./; X 0 / j M./; œ  t g;

for M./ 2 C0 ; œ 2 V1 Œt0 ; £.

Note that l 0 M.t /d œ.t / D dƒ./ is the Lagrange multiplier responsible for
the state constraint, with M.t / continuous in t , and œ.t /, with bounded variation,
responsible for the jumps of the multiplier ƒ./. Such properties of the multipliers
are due to the linearity of the system and the type of the constraints on u; x, taken
in this section (see also [96, 97]). In general, the multipliers responsible for the state
constraints may be represented through a measure of general type and therefore
may contain singular components. This will not be the case, however, due to the
next assumption which is now required.
Assumption 7.1.5. The multipliers ƒ.0/ .t /; œ.0/ .t / do not have singular compo-
nents.
This assumption holds, for example, for piecewise absolutely continuous func-
tions, with finite number of jumps.
7.2 State-Constrained Control: Computation 289

Lemma 7.1.6. Assumption 7.1.5 holds if the support function ¡.q j Y .t // is

absolutely continuous in t , uniformly in q; hq; qi  1, and for each vector l the
optimal trajectory for Problem 7.1.2 visits the boundary @Y .t / for a finite number
of intervals.
Note that Lemma 7.1.5 gives the reach set at time £ as an intersection of sets
R.£; t0 ; M./; œ./; X 0 / parametrized by functions M./ 2 C0 . Lemma 7.1.5 may be
used to calculate the reach set X Œ£ for any fixed time £.
However, on many occasions our objective is to recursively calculate the whole
tube X Œ£; £  t0 . This means that while solving Problem 7.1.2 for increasing values
of £, we want a procedure that does not require one to solve the whole problem
“afresh” for each new value of £, but allows the use of the solutions for previous
values.
There is a difficulty here, namely, given M./; œ./, one may observe that
in general, R.£; t0 ; M./; œ./; X 0 / 6D R.£; t; M./; œ./; R.t; t0 ; M./; œ./; X 0 //,
which means R given by (7.41) does not satisfy the semigroup property. Therefore
Theorem 7.1.5 and Lemmas 7.1.4 and 7.1.5 have to be modified to meet the
recursion requirements. Such a move would yield a modified version of the
maximum principle for linear systems under state constraints that eliminates the
last difficulty.

7.2 State-Constrained Control: Computation

In this section we emphasize computational approaches. We start with a modified

maximum principle. Then present an ellipsoidal technique to calculate reach sets
for linear systems with constraints on the control and state. The suggested scheme
introduces parametrized families of tight ellipsoidal-valued tubes that approximate
the exact reach tube from above, touching the tube along specially selected “good”
curves of Sect. 2.3 that cover the entire exact tube. This leads to recursive relations
that compared to other approaches, simplify calculations. The proofs that rely on
the mentioned “recurrent” version of the maximum principle under state constraints
are followed by an example. Finally given is a description of specific properties
used for proving the assertions stated in Sects. 7.1, 7.2 and crucial for easing the
computational burden in calculation.

7.2.1 The Modified Maximum Principle

To achieve the desired results, in the following sections we shall use some properties
related to the structure of the optimal controls and the state constraints. These
properties and additional assumptions are summarized below in Sect. 7.2.1 and
are typical of the problems under discussion. They are borrowed from previous
investigations.
290 7 Dynamics and Control Under State Constraints

Starting with Assumptions 7.1.4 and 7.1.5 and presuming the conditions of
Lemma 7.1.6 are true, let us first restrict ƒ in (7.39) to satisfy the relation
dƒ.t / D l 0 M.t /d œ.t /, in which œ.t / is absolutely continuous. This means
dƒ.t / D l 0 M.t /˜.t /dt and ˜.t / D d œ.t /=dt  0 for t 2 TY D ft j xl .t / 2
.0/

.0/
intY .t /g, when xl .t / is the optimal trajectory for Problem 7.1.2, for the given l.
Denoting L.t / D S 1 .t; £ j M./; ˜.//M.t /˜.t /, we may replace

dS=dt D SA.t / C M.t /˜.t /; S.£; £ j M./; l.// D I; (7.42)

whose solution is S.t; £ j M./; l.//, with

dSL =dt D SL .A.t /  L.t //; SL .£; £/ D I;

whose solution is SL .t; £/. Then S.t; £ j M./; l.//  SL .t; £/; t 2 Œt0 ; £.
In this case, R.£; t0 ; M./; ˜./; X 0 / transforms into
Z £
R .£; t0 ; L.// D SL .t0 ; £/E .x 0 ; X 0 / C SL .t; £/.E .B.t /p.t /; B.t /P .t /B 0 .t //
t0

CL.t /E .y.t /; Y.t ///dt D X .£; t0 ; L./; X 0 / D XL Œ£;

and XL Σ turns out to be the solution to the differential inclusion

P / 2 .A.t /  L.t //x C L.t /E .y.t /; Y.t // C E .B.t /p.t /; B.t /P .t /B 0 .t //;
x.t
(7.43)
t  t0 ; x 0 2 E .x 0 ; X 0 /: (7.44)

Here also the compact set C0 of functions M.t / transforms into a compact set C00
of functions L.t /.
We thus arrive at the following important property, which is similar to those
proved in [158].
Lemma 7.2.1. The reach set X Σ is the intersection

X Œ£ D \fXL .£; t0 ; E .x 0 ; X 0 // j L./g (7.45)

of the “cuts” or “cross-sections” XL .£; t0 ; E .x 0 ; X 0 // of the reach tubes (solution

tubes) XL ./ D fXL Œt  W t  t0 g to the differential inclusion (7.43), (7.44). The
intersection is over all L./ 2 C00 , a compact set of continuous matrix functions
L.t /; t 2 Œt0 ; £.
A further calculation using convex analysis yields the next formula.
Theorem 7.2.1. The support function

¡.l j X .£; t0 ; E .x 0 ; X 0 /// D inff¡.l j XL .£; t0 ; E .x 0 ; X 0 /// j L./g; (7.46)

7.2 State-Constrained Control: Computation 291

with

¡.l j XL .£; t0 ; E .x 0 ; X 0 /// D hl; xL? Œ£i C hl; GL .£; t0 /X 0 GL0 .£; t0 /li1=2 (7.47)
Z £
C hl; GL .£; s/B.s/P .s/B 0 .s/GL0 .£; s/li1=2 ds
t0

Z £
C hl; GL .£; s/L.s/Y.s/L0 .s/GL0 .£; s/li1=2 ds:
t0

Here
Z £
xL? Œ£ D GL .£; t0 /x C 0
GL .£; s/.B.s/p.s/ C L.s/y.s//ds; (7.48)
t0

and GL .£; s/ D SL .s; £/ is the transition matrix for the homogeneous system

xP D .A.t /  L.t //x; GL .t; t / D I; L./ 2 C00 :

The significance of the last result is that in this specific problem the support
function of the intersection (7.45) is equal to the pointwise infimum (7.46) of the
support functions rather than to their infimal convolution as given by general theory
[237].
It is not unimportant to specify when the infimum in (7.46) is attained; that is,
it is actually a minimum. Indeed, it may happen that for a given l, the minimum
over L./ is in the class C00 (this, for example, is the case when œ.t / turns out to
be absolutely continuous, as in the above). But to ensure the minimum is always
reached, we have to broaden the class of functions L./.
To illustrate how to continue the procedure we will assume the following.
Assumption 7.2.1. For each l 2 Rn , the optimal trajectory xl0 .t / of Problem 7.1.2
visits the boundary @Y .t / only during one time interval Œt1 ; t2 ; t0  t1  t2  £.
(The case of finite or countable collection of such intervals is treated in a similar
way.)
Then, instead of the product M.t /˜.t /, in (7.42) we must deal with multipliers
of the form M .t / D M.t /˜.t / C M1 •.t  t1 / C M2 •.t  t2 /, where M1 ; M2 are
n  n matrices.
By introducing a new multiplier L .t / D L.t / C L1 •.t  t1 / C L2 •.t  t2 / under
transformation L .t / D S 1 .t; £ j M .//M .t /, we shall match the formulas for
R.£; t0 ; M .// and its transformed version R .£; t0 ; L .//.
Applying the schemes of [96, 97, 158], to the specific case of this paper, it is
possible to rewrite the preceding assertions.
292 7 Dynamics and Control Under State Constraints

Lemma 7.2.2. The support function

(7.49)
¡.l j X Œ£/ D ¡.l j X .£; t0 ; E .x ; X /// D minf¡.l j XL .£; t0 ; E .x ; X // j L ./g;
0 0 0 0

with

¡.l j XL .£; t0 ; E .x 0 ; X 0 // D ˆ.l; L ./; £; t0 /; (7.50)

and

ˆ.l; L ./; £; t0 / D hl; x Œt i C hl; G .£; t0 /X 0 G 0 .£; t0 /li1=2 (7.51)

Z £
C hl; G .£; s/B.s/P .s/B 0 .s/G 0 .£; s/li1=2 ds
t0
Z £
C hl; G .£; s/L.s/Y.s/L0 .s/G 0 .£; s/li1=2 ds
t0
Z £
C hl; G .£; s/L1 Y.s/L01 G 0 .£; s/li1=2 d¦.s; t1 /
t0
Z £
C hl; G .£; s/L2 Y.s/L02 G 0 .£; s/li1=2 d¦.s; t2 /:
t0

Here G .£; s/ is the transition matrix for the homogeneous system

dx D .A.t /  L.t //xdt  L1 xd¦.t; t1 /  L2 xd¦.t; t2 /;

so that

x.t / D G .£; t0 /x 0 ;

and ¦.s; t 0 / is the step function,

¦.s; t 0 /  0; s < t 0 I ¦.s; t 0 /  1; s  t 0 ; d¦.s; t 0 /=ds D •.s  t 0 /:

The vector x Σ in (7.48), (7.51) may be described by equation

dx Œt  D ..A.t /  L.t //x Œt  C B.t /p.t / C L.t /y.t //dt;

 L1 .x Œt   y.t //d¦.t; t1 /  L2 .x Œt   y.t //d¦.t; t2 /; x Œt0  D x 0 : (7.52)

Remark 7.2.1. Note that in the transition function G .t; s/ we have the difference
of a Riemann integral and a Riemann–Stieltjes integral. On the other hand, the last
7.2 State-Constrained Control: Computation 293

two integrals in (7.51) should be interpreted as Lebesgue–Stieltjes integrals [234].

This does not cause additional difficulty since the multipliers L1 ; L2 are among the
optimizers in (7.47).
A result similar to (7.45) is all the more true for the sets XL .£; t0 ; E .x 0 ; X 0 //.
Corollary 7.2.1. The following intersection holds:

X Œ£ D X .£; t0 ; E .x 0 ; X 0 // D \fXL .£; t0 ; E .x 0 ; X 0 // j L ./g:

The difference between Lemmas 7.2.1 and 7.2.2 is that in the former it is
not guaranteed that the boundary @X Σ is touched at each point by one of the
intersecting sets XL .£; t0 ; E .x 0 ; X 0 //, whereas in the latter the boundary @X Œ£ is
indeed touched at each point by one of the sets XL .£; t0 ; E .x 0 ; X 0 //. This is because
the minimum in (7.47) is attained.
Under Assumptions 7.1.4, 7.1.5, 7.2.1, and Assumption 7.2.3 given below, in
Sect. 7.2.5, the reasoning of the above leads to the next result.
Lemma 7.2.3. The reach set X Œt  D X .t; t0 ; E .x 0 ; X 0 // is a convex compact set in
Rn which evolves continuously in t.
The boundary of the reach set X Œt  has an important characterization. Consider a
point x  on the boundary @X Œ£ of the reach set X Œ£ D X .£; t0 ; E .x 0 ; X 0 //.1
Then there exists a support vector z such that

hz ; x  i D ¡.z j X Œ£/:

Let L 0 ./ be the minimizer for the problem (see (7.47)):

¡.z j X Œ£/ D minfˆ.z ; L ./; £; t0 / j L ./g D ˆ.z ; L 0 ./; £; t0 /: (7.53)

Then the control u D u .t /, the initial state x.t0 / D x 0 2 E .x 0 ; X 0 /; and the

corresponding trajectory x  .t /, along which system (7.30) is transferred from state
x  .t0 / D x 0 to x.£/ D x  , are specified by the following “modified maximum
principle”.
Theorem 7.2.2 (The Modified Maximum Principle Under State Constraints).
For Problem 7.1.2 suppose state x  is such that

hz ; x  i D ¡.z j X Œ£/:

Then the control u .t /, which steers the system (7.30) from x  .t0 / D x 0 to x  .£/ D
x  under constraints u.t / 2 E .p.t /; P .t //; x.t / 2 E .y.t /; Y.t // while ensuring

hz ; x  i D maxfhz ; xi j x 2 X Σg;

1
The boundary @X Σ of X Σ may be defined as the set @X Σ D X Σ n intX Σ. Under the
controllability assumption, X Œ£ has a nonempty interior, intX Œ£ 6D ;; £ > t0 .
294 7 Dynamics and Control Under State Constraints

satisfies the following pointwise “maximum principle” for the control .s 2 Œt0 ; £/:

0 0
z G 0 .£; s/B.s/; u .s/ D maxfz G 0 .£; s/B.s/u j u 2 E .p.s/; P .s//g (7.54)
u
0
D hz ; G 0 .£; s/B.s/p.s/iChz ; G 0 .£; s/B.s/P .s/B 0 .s/G 0 .£; s/z i1=2 ; s 2 Œt0 ; £;

and the “maximum conditions” for the system trajectory (pointwise),

0 0
z G 0 .£; s/L 0 .s/x  .s/ D maxfz G 0 .£; s/L 0 .s/p j p 2 E .y.s/; Y.s//g (7.55)
p
0 0 0
D z G 0 .£; s/L 0 .s/y.s/ C hz ; G 0 .£; s/L 0 .s/Y.s/L 0 .s/G 0 .£; s/z i1=2 ;

and the initial state,

.z ; G 0 .£; t0 /x 0 / D maxfhz ; xi j x 2 G 0 .£; t0 /E .x 0 ; X 0 /g D ¡.z j G 0 .£; t0 /E .x 0 ; X 0 //

(7.56)
0
D hz ; G 0 .£; t0 /x 0 i C hz ; G 0 .£; t0 /X 0 G 0 .£; t0 /z i1=2 :

Here G 0 .£; s/ stands for the matrix function G .£; s/ taken for L 0 .s/—the minimizer
of problem (7.53).
The function h.£; s/ D z 0 G 0 .£; s/B.s/ is taken right-continuous.2
Remark 7.2.2. Suppose we want to solve Problem 7.1.2, seeking ¡.l  .t / j X Œt /
along a curve l  .t /; t > t0 : Then, taking l  .t / D z 0 .G 0 /1 .t; s/, one may observe
that the integrands in functional ˆ.l  .t /; L 0 ./; t; t0 / of (7.47)–(7.51) will be
independent of t . This property ensures the existence of a recurrent computational
procedure, as indicated in the next section (see also Sect. 2.3). The modified
maximum principle of this section thus allows a solution in recurrent form. This
is not the case for the standard maximum principle.
We now pass to the construction of ellipsoidal approximations for the reach sets.

7.2.2 External Ellipsoids

Despite the linearity of the system, the direct calculation of reach sets is rather
difficult. Among effective methods for such calculations are those that rely on
ellipsoidal techniques. Indeed, although the initial set E .x 0 ; X 0 / and the control
set E .p.t /; P .t // are ellipsoids, the reach set X Œt  D X .t; t0 ; E .x 0 ; X 0 // will of
course not generally be an ellipsoid.

2
In the general case, under Assumption 7.2.1, the optimal trajectory may visit the smooth boundary
of the state constraint for a countable set of closed intervals, and function L0 ./ allows not more
than a countable set of discontinuities of the first order.
7.2 State-Constrained Control: Computation 295

As shown in Chap. 3, in the absence of state constraints the reach set X Œt  may be
approximated both externally and internally by ellipsoids E and EC , with E
X Œt  EC . Here we deal only with external approximations, but taken under state
constraints.
An approximation E .xC ; XC / is said to be tight if there exists a vector z 2 Rn
such that ¡.z j E .xC ; XC // D ¡.z j X Œt / (the ellipsoid E .xC ; XC / touches X Œt 
along direction z). We shall look for external approximations that are tight, on the
one hand, and are also recursively computable, on the other.
In order to apply ellipsoidal techniques to state-constrained problems, recall
Lemma 7.2.1 and Corollary 7.2.1, which indicate how the reach set X Œt  D
X .t; t0 ; X 0 / may be presented as an intersection of reach sets XL or XL for
system (7.30) without state constraints. We first study how to approximate sets XL
by ellipsoids.
To demonstrate the nature of the procedures suggested in this section we shall
introduce the ellipsoidal technique in two stages. First, transform the system
coordinates in (7.30) from x to z according to the formula z.t / D S.t; t0 /x.t /,
with

dS.t; t0 /=dt D S.t; t0 /A.t /; S.t0 ; t0 / D I:

Then system (7.30) and state constraint x.t / 2 Y .t / transform into

zP D S.t; t0 /B.t /u D BN .t /u; (7.57)

z.t / 2 S.t; t0 /E .y.t /; Y.t //; z.t0 / 2 S.t; t0 /E .x 0 ; X 0 /;

while the constraint on control u 2 P .t / D E .p.t /; P .t // remains the same.

Returning to the old notation, we will deal with system

xP D B.t /u; x.t0 / 2 E .x 0 ; X 0 /; y.t / 2 Y .t /: (7.58)

Thus, without loss of generality, in the forthcoming formulas we assume

A.t /  0.
Problem 7.2.1. Given a vector function l  .t /, continuously differentiable in t , find
external ellipsoids ELC Œt  D E .xL .t /; XL .t //  XL Œt  such that for all t  t0 ; the
equalities

¡.l  .t / j XL Œt / D ¡.l  .t / j ELC Œt / D hl  .t /; xL .t /i

hold, so that the supporting hyperplane for XL Œt  generated by l  .t /, namely, the

plane hxxL .t /; l  .t /i D 0 which touches XL Œt  at point x  .t /, is also a supporting
hyperplane for ELC Œt  and touches it at the same point.
Remark 7.2.3. Under Assumption 7.2.1 for the optimal trajectories x .0/ ./ of
Problem 7.1.2 the boundary @X Σ of the reach set X Σ consists of three types
296 7 Dynamics and Control Under State Constraints

of points, namely (I) those that are reached by x .0/ .£/ without having visited the
boundary of the state constraint, then (II) those that are reached by x .0/ .£/ after
having visited the boundary of the state constraint, and (III) those that lie on the
boundary of the state constraint. Since case (I) has been investigated in detail in
Chap. 3, Sects. 3.2–3.4, and case (III) is trivial, our present investigation actually
deals only with case (II).
Apart from Assumptions 7.1.4 and 7.1.5, let us assume in this section that the
functions L./ in what follows do not have any delta function components. (The
case when such components are present is treated in the next section.)
The solution to Problem 7.2.1 is given within the following statement.
Theorem 7.2.3. With l  .t / given, the solution to Problem 7.2.1 is an ellipsoid

ELC Œt  D EL .xL .t /; XL Œt /, in which
Z t 
XL Œt  D .pu .t; s/ C pY .t; s//ds C p0 .t / (7.59)
t0
Z t
 .pu .t; s//1 GL .t; s/B.s/P .s/B 0 .s/GL0 .t; s/ds
t0
Z t 
0
C .pY .t; s//1 GL .t; s/Lt .s/Y.s/Lt .s/GL0 .t; s/dsCp01 .t /GL .t; t0 /X 0 GL0 .t; t0 / ;
t0

and

pu .t; s/ D .l  .t /; GL .t; s/P .s/GL0 .t; s/l  .t //1=2 ; (7.60)


pY .t; s/ D .l .t /; GL .t; s/L .s/Y.s/L t t
GL0 .t; s/l  .t //1=2 ;
p0 .t / D .l.t0 /; GL .t; t0 /X 0 GL0 .t; t0 /l.t0 //1=2 ;

with xL .t / D xL? Œt :

This result follows from Chap. 3, Sect. 2.3. Since the calculations have to be made
for all t , the parametrizing functions pu .t; s/; pY .t; s/; s 2 Œt0 ; t ; p0 .t / must also
formally depend on t .
In other words, relations (7.59), (7.60) need to be calculated “afresh” for each t . It
may be more convenient for computational purposes to have them in recursive form.
As indicated in Sects. 3.3–3.4, in the absence of state constraints this could be done
by selecting function l  .t / in an appropriate way. For the case of state constraints
we follow Remark 7.2.2, arriving at the next assumption.
0
Assumption 7.2.2. The function l  .t / is of the form l  .t / D l0 GL .t0 ; t /, with
l 2 Rn given.
Under Assumption 7.2.2 the function l  .t / is the solution to the differential
equation

lP D L0 .t /l  ; l  .t0 / D l : (7.61)

7.2 State-Constrained Control: Computation 297

The tight external ellipsoids to XL .t /, which are

E .xL .t /; XL .t //
XL .t /
X Œt ;
may be described applying formulas of Sect. 3.3 to system (7.43). Each of these
ellipsoids will be an external estimate for X Œt .
But to describe the collection of tight ellipsoids for X Œt  we have to ensure the
following property:

¡.ljX Œt / D minfhl; xL .t /i C hl; XLC


.t /li1=2 j L./ 2 C00 g: (7.62)

Under our assumptions the minimum over L./ in (7.47) is attained for any l D
l  .t / D GL0 .t0 ; t /l 2 Rn , the minimizers being denoted as Lt ./ 2 C00 :3
But prior to moving ahead we have to investigate the following. Suppose element
Lt .s/ is the minimizer of functional

ˆ.GL0 .t0 ; t /l ; L./; t; t0 / D ˆ.l0 ; L./; t; t0 /; l0 D .GL0 .t0 ; t /l ;

The question is, if we minimize function ˆ.l0 ; L./; t C ¢; t0 /; ¢ > 0; over a

larger interval Œt0 ; t C ¢ than before, will the minimizer LtC¢ .s/, for the latter
problem, taken within s 2 Œt0 ; t , be the same as the minimizer Lt .s/; s 2 Œt0 ; t 
for ˆ.l0 ; L./; t; t0 /?
The answer to this question is given by the next lemma.
Lemma 7.2.4. Taking l D l  .t / according to Assumption 7.2.2, suppose
Lt .s/; s 2 Œt0 ; t  and LtC¢
 .s/; s 2 Œt0 ; t C ¢; ¢ > 0, are the minimizers of
functionals ˆ.l0 ; L./; t; t0 / and ˆ.l0 ; L./; t C ¢; t0 /, respectively. Then

Lt .s/  LtC¢

 .s/; s 2 Œt0 ; t :

The proof is achieved by contradiction. Note that under Assumption 7.2.2 and
due to Property 7.2.1 of Sect. 7.2.5 we may always take L.t /  0 when t < £1 ; t >
£2 , where £1 ; £2 are the points of arrival and departure at the state constraint.
Following Assumption 7.2.2, we proceed further by selecting L./ to be the
minimizer in L of functional ˆ.l0 ; L./; t; t0 /, namely, we now take L.s/ D
Lt .s/ D L .s/; s 2 Œt0 ; t ; which depends on l ; s, but, as indicated in
Lemma 7.2.4, does not depend on t .
Let G .t; s/ D GL .t; s/ under conditions A.t /  0; L.t /  L .t /.
Then pu .t; s/; pY .t; s/; p0 .t / of (7.60) transform into

pu .t; s/ D hl ; G .t0 ; s/B.s/P .s/B 0 .s/G0 .t0 ; s/l i1=2 D pu .s/; (7.63)
pY .t; s/ D 0
hl ; G .t0 ; s/L .s/Y.s/L .s/G0 .t0 ; s/l i1=2 D pY .s/;

p0 .t / D hl ; X l i
0 1=2
D p0 I

3
The same minimum value is also attained here in classes of functions broader than C00 .
298 7 Dynamics and Control Under State Constraints

matrix XL Œt  transforms into

Z t 

XC Œt  D .pu .s/ C pY .s//ds C p0  (7.64)
t0
Z t
 .pu .s//1 G .t0 ; s/B.s/P .s/B 0 .s/G0 .t0 ; s/ds
t0
Z t 
1
C .pY .s// G .t0 ; s/L .s/Y.s/L .s/G0 .t0 ; s/ds
0
C p01 X 0 ;
t0

and the functional ˆ.l  .t /; L ./; t; t0 / transforms into

ˆ.l  .t /; L ./; t; t0 / (7.65)

Z t
D hl ; x  .t /i C hl ; X 0 l i1=2 C hl ; G .t0 ; s/B.s/P .s/B 0 .s/G0 .t0 ; s/l i1=2 ds
t0
Z t
C hl ; G .t0 ; s/L .s/Y.s/L0 .s/G0 .t0 ; s/l i1=2 ds;
t0

with xL .t / transformed into

Z t

xC Œt  D x 0 C G .t0 ; s/.B.s/p.s/ C L .s/y.s//ds:
t0

 
We may now differentiate XC Œt ; xC Œt . According to Sect. 7.2.5 “Specifics”,
below, in Sect. 7.2.5, the necessary condition (7.84) for a jump in L .t / is not
fulfilled, and therefore L .t /  L 0 .t /.
Denoting
Z t 1
u .t / D pu .t / .pu .s/ C pY .s//ds C p0 ;
t0
Z t 1
Y .t / D pY .t / .pu .s/ C pY .s//ds C p0 ; (7.66)
t0

 
and differentiating XC Œt ; xC Œt , we arrive at

XP C
 
Œt  D .u .t / C Y .t //XC Œt  C .u .t //1 G .t0 ; t /B.t /P .t /B 0 .t /G 0 .t0 ; t /
C. .t //1 G .t ; t /L .t /Y.t /L0 .t /G 0 .t ; t /; (7.67)
Y  0    0


xP C Œt  D G .t0 ; t /.B.t /p.t / C L .t /y.//; (7.68)


XC Œt0  D X 0 ; x  Œt0  D x 0 : (7.69)
7.2 State-Constrained Control: Computation 299

Returning to function l  .t / D GL0 .t0 ; t /l with GL .t; s/ D G .t; s/, we have

XC Œt  D G .t; t0 /XC Œt G0 .t; t0 /

and

XP C Œt  D L .t /XC Œt   XC Œt L .t /0 C G .t; t0 /XP C


Œt G0 .t; t0 /; (7.70)

so that

XP C Œt  D L .t /XC Œt   XC Œt L0 .t / (7.71)

1 0 1
C.u .t / C Y .t //XC Œt  C .u .t // B.t /P .t /B .t / C .Y .t // L .t /Y.t /L 0 .t /

and

xP C D L .t /xC C B.t /p.t / C L .t /y.t /:

Finally, returning to the case A.t / 6D 0, we have

XC Œt  D S.t0 ; t /XC Œt S 0 .t0 ; t /; x.t / D S.t0 ; t /xC .t /;

P C Œt  D .A.t /  L .t //XC Œt  C XC Œt .A0 .t /  L0 .t //

X (7.72)


C .u .t / C Y .t //XC Œt  C .u .t //1 B.t /P .t /B 0 .t /

C .Y .t //1 L .t /Y.t /L 0 .t /; XC Œt0  D X 0 ;

and

xP D .A.t /  L .t //x C B.t /p.t / C L .t /y.t /; x.t0 / D x 0 : (7.73)

Rewriting u .t /; Y .t / following schemes of [181], p. 189, for these items (see

also below: Remark 7.2.6 of Sect. 7.2.5), we have

u .t / D hl  .t /; B.t /P .t /B 0 .t /l  .t /i1=2 =hl  .t /; XC .t /l  .t /i1=2 ; (7.74)


Y .t / D .l .t /; L .t /Y.t /L0 .t /l  .t //1=2 =.l  .t /; XC .t /l  .t //1=2 :

Summarizing the results we come to the following.

Theorem 7.2.4. With shape matrix Y .t / of the state constraint being nondegen-
erate and if, under Assumptions 7.1.4, 7.2.1, and 7.2.2, the multipliers L .t /
responsible for the state constraint have no jumps (dƒ.t / D œ.t /dt /, then the
external ellipsoids E .xŒt ; XC Œt /
X Œt  have the form (7.72), (7.73) with XŒt0  D
X 0 ; xŒt0  D x 0 .
300 7 Dynamics and Control Under State Constraints

Moreover, with l  .t / D G 0 .t0 ; t /l ; l 2 Rn , the ellipsoids E .xŒt ; XŒt / touch

the set X Œt  along the curve x  .t / which satisfies the condition

hl  .t /; x  .t /i D ¡.l  .t / j X Œt /; t  £1 ; t  £2 ;

so that

hl  .t /; x  .t /i D ¡.l  .t / j X Œt / D ¡.l  .t / j E .xŒt ; XC Œt //: (7.75)

On the state constraint boundary, with t 2 Œ£1 ; £2 , the trajectory which

satisfies the first equality of (7.75) may be uniquely determined by the maximum
condition (7.55).
The overall procedure of calculation for Problem 7.1.2 starting from a point in
the interior of the state constraint is as follows.
(i) Consider an array of initial vectors l located on the unit ball B .0/ in Rn . Then
for each l proceed as follows (possibly, in parallel).
(ii) For each l construct the array of external ellipsoids EC Σl for Problem 7.1.2
of system (7.30) without state constraints following the explicit formulas of
Sects. 3.2–3.4.
(iii) Construct the array of ellipsoids E .xŒt ; XŒt /, namely,
(iii-a) for a given l0 find related l  0 .t / D l0 G .t0 ; t / and, further on, the related
curve x .0/ .t / for the reachability problem without state constraints. Note
that on the interval Œt0 ; £1 /, where L .t /  0, we have

x .0/ .t / D x  .t / C XC .t /l  .t /hl  .t /; XC .t /l  .t /i1=2 ;

where explicit relations for elements x  .t /; XC .t / of this formula are

indicated in Remark 7.2.6 of Sect. 7.2.5. Follow curve x .0/ .t / until the
state constraint boundary is reached at time, say, £1 ;
(iii-b) for l  .£1 / solve the optimization problem (7.49) over L./. Use
Lemma 7.2.5 of the appendix for additional information on optimizer
L ./, taking £2 as a parameter which varies within interval Œ£1 ; £;
(iii-c) given optimizer L ./, which depends on the selected l , solve
Eqs. (7.72), (7.73), determining the related external ellipsoid
El .xŒt ; XC Œt /.
(iv) In view of Remark 7.2.3, the intersection
\ \ \
fEC Œ£l jl 2 B .0/g fEl .xŒt ; XC Œt / j l 2 B .0/g E .y.£/; Y .£// D Xa Œ£
X Σ

is an external approximation of X Σ. Loosely speaking, the greater is the

number of vectors l used as starting points of the calculations, the more
accurate will be the approximation Xa .
7.2 State-Constrained Control: Computation 301

Remark 7.2.4. If it is not necessary to find the tight external approximations for the
exact reach set X Σ, but a conservative estimate of X Σ using only one ellipsoid
would suffice, then there is no need to solve optimization problem (7.46) in L, since
ellipsoids E .xŒt ; XC Œt /, defined by (7.72), (7.73), happen to be external estimates
of X Σ for all L./. One may single out one of these through some appropriate
criterion.
An obviously conservative estimate may also be obtained by intersecting the
reach set without state constraints with the ellipsoid that defines the state constraint.
The objective of this paper, however, is to indicate the calculation of the exact reach
set under state constraints, especially when the state constraint produces a reach set
different from the one without state constraints.
We now pass to the more general case, in which the multipliers may have delta
function components.

7.2.3 Generalized Multipliers

In this section, Assumptions 7.1.4 (of Sect. 7.1.3) and 7.2.1 (of Sect. 7.2.1) and the
conditions of Lemma 7.1.6 are taken to be true together with Assumption 7.1.5. This
allows us to consider ellipsoidal approximations without the additional requirement
of Property 7.2.2 of Sect. 7.2.5.
Now functional ˆ.l; L ./; £; t0 / has the form (7.65), and its minimizer for a given
l D l  .£/ is of the form

L 0 .t / D L0 .t / C L01 •.t  £1 / C L02 •.t  £2 /

for all l  .£/ with L0 .t / absolutely continuous and L0 .t /  0 whenever x  .t / 2
intY .t /. Then the respective transition matrix is G .t; s/ D G0 .t; s/ and l  .t / D
0
G0 .t; s/l  .
Here L 0 ./ may be interpreted as the generalized derivative of a generalized
Lagrange multiplier ƒ00 ./ similar to ƒ.0/ ./ of Sect. 7.2.1. ƒ00 ./ is piecewise
absolutely right-continuous, with possible jumps at points £1 ; £2 and possible jumps
at t0 and/or £ (when it happens that t0 D £1 and/or £ D £2 ). ƒ00 .t /  const,
whenever x  .t / 2 intY .t /.
Following the reasoning of the previous section, we may derive equations
for the approximating external ellipsoids similar to (7.72), (7.73). The necessary
prerequisites for such a move are similar to Lemma 7.2.4 and Theorems 7.2.3
and 7.2.4.
Thus, we come to equations for tight external ellipsoids:

X2
dxC D ..A.t /L0 .t //xC C.B.t /p.t /CL0 .t /y.t //dt  L0i .xC y.t //d¦.t; £i /;
iD1
(7.76)
0 0
dXC Œt  D ..A.t /  L0 .t //XC Œt  C XC Œt .A .t /  L0 .t ///dt (7.77)
302 7 Dynamics and Control Under State Constraints

!
X
k
C u .t / C Y .t / C i XC Œt dt C .u .t //1 B.t /P .t /B 0 .t /dt
iD1

0
X
2
0
C.Y .t //1 L0 .t /Y.t /L0 .t /dt C i1 L0i Y .t /L0i d¦.t; £i /;
iD1

XC Œt0  D X 0 ; xC .t0 / D x 0 :

The i > 0 are additional parametrizing coefficients similar to Y .t /.

The last equation may be interpreted as before (see Remark 7.2.1).
Theorem 7.2.5. Under Assumptions 7.1.5, 7.2.1, 7.2.2, and those of Lemma 7.1.6
0
with l  .t / D G0 .t0 ; t /l ; l 2 Rn , the external ellipsoids E .xC .t /; XC Œt /
X Œt 
have the form (7.76), (7.77) with XC Œt0  D X 0 ; xC .t0 / D x 0 . One may select
the parametrizing coefficients u ; Y.t/ ; i so that ellipsoids E .xC .t /; XC Œt / would
touch set X Œt  along the curve xC .t /, which satisfies condition (7.75).
If the optimal trajectories visit the boundary of the state constraint m > 1 times,
then the reasoning is the same as in the above, and the sums in (7.76), (7.77) will
have additional 2m terms.
Remark 7.2.5. The results of Sects. 7.2.2 and 7.2.3 remain true if the state constraint
is applied to a system output z.t / D T x.t /; z 2 Rk ; k < n; rather than to the whole
state vector x. The state constraint Y is then given by an elliptical cylinder in Rn .
The proofs may be achieved either by directly following the scheme of this Chap. 7
or by substituting the elliptical cylinder with an ellipsoid having n  k axes of size
r followed by limit transition r ! 1.

7.2.4 An Example

The system is the double integrator:

xP 1 D x2 ; xP 2 D u; x.0/ D 0; (7.78)

under constraints

juj  ; jx2 j  : (7.79)

We wish to find the reach set X Œ£ at time £ > 0.

The state constraint may be treated either directly or as a limit as – ! 0 of the
ellipsoid ©2 x12 C x22  2 .
7.2 State-Constrained Control: Computation 303

The treatment of this example indicates the following boundary types:

1. The boundary of the reach set X Œ£ at each time £ consists of three types of points
as follows:
(a) points reached without visiting the boundary of the state constraint;
(b) points reached after visiting the boundary of the state constraint for some time;
(c) points on the boundary of the state constraint.
2. Each optimal trajectory x .0/ .t /, for given l 2 R2 , visits the state constraint not
more than once.
3. The necessary conditions for a jump of the multiplier responsible for the state
constraint are not fulfilled, so there are no jumps.
4. Assumption 7.2.3 of Sect. 7.2.5 is fulfilled (see below)
Case 1(a) is treated according to Chap. 3, while case 1(c) is trivial. Therefore we
concentrate on case 1(b).
The exact parametric equations for case 1(b) may be derived, by minimizing over
˜ the functional
 Z £ Z £ 
¡.ljX Œ£/ D min  js2 .t /jd Ÿ C  j˜.t /jd Ÿ j ˜./ ; (7.80)
0 0
R£
where s2 .t / D l1 .£  t / C l2  t ˜.Ÿ/d Ÿ is the solution to the adjoint equation

sP1 D 0; sP2 D s1 C ˜.t /; s1 .£/ D l1 ; s2 .£/ D l2 :

This gives parametric equations for a part of the boundary @X Σ, which is:

x1 D .£  ¢/2 =2 C c; x2 D ¢ C d; £1  ¢  £:

Here £1 D = < £ is the first instance when the trajectory reaches the boundary
of the state constraint, while ¢ is a parameter that indicates the instance of £2 when
the trajectory leaves this boundary. Also c D £  2 =2I d D   £: Note that
we have used Assumption 7.1.4 of Sect. 7.2.1, which implies that optimizer ˜0 .t /
satisfies relations

˜0 .t /  l1 ; t 2 Œ£1 ; £2 ; ˜0 .t /  0; t 62 Œ£1 ; £2 ;

and s1 .t /  l1 .
A typical trajectory for case 1(b) is when it reaches the boundary of the state
constraint at time £1 and then runs along the boundary and leaves it at time £2 . Later
it runs toward the boundary @X Σ while staying in the interior of the constraint.
For the interval Œ0; £, the adjoint system given above, when written in matrix
form (7.42), is

SP D SA.t / C M.t /˜.t /: (7.81)

304 7 Dynamics and Control Under State Constraints

To solve the problem through a numerical procedure involving external ellip-

soids, using recursive formula and following Sect. 7.2.2, transform the adjoint
Eq. (7.81) of the above into the modified form

SPL D SL .A.t /  L.t //; SL .£/ D I:

Taking l D l .0/, we will have l0 M.t /˜0 .t / D .0; 1/˜0 .t / when t 2 Œ£1 ; £2 .
This yields M.t / D M0 .t / depending on ˜0 . Direct calculation then allows one to
find for each l the relation
for the optimizer L .t / of (7.53). This is

L .t / D SM0 .t /1 M0 .t /; t 2 Œ£1 ; £2 ;

with L .t /  0 when t < £1 ; t > £2 . Here SM0 .t / is the solution to (7.81) with
initial condition SM0 .£/ D I and M.t /  0; t; £1 ; t > £2 .
Recalling the ellipsoidal equations, we have

XP D .A.t /  L.t //X C X.A.t /  L.t //0 (7.82)

C.u .t / C Y .t //X C u1 .t /BP .t /B 0 C Y1 .t /L.t /Y.t /L0 .t /:

Here one should take L.t / D L .t / and u ; Y according to (7.74).

The algorithm for this example involves the following steps:
1. For the given starting direction l find time £1 D = of the first exit (i.e.,
encounter with the boundary of the state constraint).
2. Solve extremal problem (7.80), determining l1 D ˜0 D const and l2 , for each
£2 D ¢ 2 Œ£1 ; £.
3. Construct the ellipsoidal approximations for the system without constraint t 
£1 , taking L.t /  0. Denote these as E .0; XlC .t //.
4. For each £2 D ¢ 2 Œ£1 ; £ construct the ellipsoidal approximation follow-
ing (7.82).
The part of the boundary related to case (b) is described by the boundary of the
intersection E .0; X.£// \ L./ where L./ is actually reduced to parameter ¢ D £2 .
Denote this intersection as E .0; X¢C .£//.
The total reach set is the intersection of approximations for each group 1(a), 1(b),
and 1(c), namely,
\ \ \
X Œ£ D X .£; 0; 0/ D E .0; XlC .£// E .0; X¢C .£// E .0; Y /:
l ¢

Figure 7.1 shows the structure of the reach set boundary, given here by a thick
line. The system trajectory OAG reaches its end point G without visiting the
boundary of the state constraint. Point G is attained at time £. It lies on the reach
set boundary of type 1(a) (see beginning of this Sect. 7.2.4). Trajectory OBF visits
7.2 State-Constrained Control: Computation 305

Fig. 7.1 Reach set: structure of the boundary

the boundary of the state constraint at only one point £1 D £2 < £. Its end point
F is of both types 1(a) and 1(b). Trajectory OBCE lies on the boundary of the state
constraint during the interval Œ£1 ; £2 / with £1 < £2 < £, and its end point E lies on
the reach set boundary of type 1(b). Finally, trajectory OBD reaches the boundary
of the state constraint at time £2 < £ and lies on it until final time £, so that point D
is of both types 1(b) and 1(c). Thus, the part of the reach set boundary along points
BDEFGH consists of segments BD [of type 1(c)], DF [of type 1(b)], and FGH [of
type 1(a)].
Figure 7.2 shows the structure of the generalized Lagrange multiplier ˜.t / for the
case of trajectory x2 .t / of type OBCE shown in the top of the figure. Here ˜  0 on
intervals Œ0; £1 / and Œ£2 ; £ and ˜  const on Œ£1 ; £2 /. Note that the related trajectory
x2 .t / runs along the boundary of the state constraint during the interval Œ£1 ; £2 /,
where ˜.t / 6D 0. Points £1 ; £2 potentially could have •-functions as components
of ˜. But the necessary conditions for the existence of such components are not
fulfilled in this example.
Figure 7.3 shows an external ellipsoid (thin line) for the boundary of the reach
set under state constraints (thick line), indicating its difference from the reach set
without state constraints (dash-dotted line). Figure 7.4 illustrates the final reach tube
with and without state constraints for this example.4

4
The computer illustrations for this subsection were made by M.N. Kirilin.
306 7 Dynamics and Control Under State Constraints

Fig. 7.2 Structure of multiplier ˜

Fig. 7.3 External ellipsoid at boundary point of type (b)

7.2 State-Constrained Control: Computation 307

Fig. 7.4 Reach tubes with and without state constraints

7.2.5 Specifics: Helpful Facts for State-Constrained

Control Design

In this subsection we present some additional facts useful for proving the main
assertions of this paper.
One difficulty in solving the control problem under state constraints (Prob-
lem 7.1.2) is to determine the set of times ft W x.t / 2 @Y .t /g. This is a union of
closed intervals during which the optimal trajectory is on the boundary of the state
constraint.
Here are some helpful facts. For Problem 7.1.2 the Properties 7.2.1–7.2.3 are
given below under Assumptions 7.1.4 and 7.2.1,
Property 7.2.1. For any l 2 Rn , the minimizer of (7.35) is ƒ.0/ .t / D const and
in (7.39) the corresponding minimizer is œ.0/ .t / D const during time intervals for
which x.t / 2 intY .t /, the interior of Y .t /.
Since l 0 M .0/ .t /d œ.0/ .t / D dƒ.0/ .t /; we may track whether the trajectory is on the
boundary of Y .t / by the multiplier œ.0/ .t /. Thus, we need not be interested in values
of M .0/ .t / when the trajectory is not on the boundary @Y .t /.
Property 7.2.2. Suppose l 2 Rn is given and x .0/ .t / is the solution of
Problem 7.1.2. For the function ƒ.0/ .t / to have a jump at t  , under the smoothness
conditions on the state constraint of Lemma 7.1.6, t  must be a time of arrival
308 7 Dynamics and Control Under State Constraints

or departure from the boundary of the tube E .y.t /; Y.t //; t0  t  £, and the
trajectory x .0/ .t / must be tangent to the tube E .y.t /; Y.t // at t  . Thus, x .0/ .t / is
differentiable at t  and

h¦; xP .0/ .t   0/i D h¦; xP .0/ .t  C 0/i D 0: (7.83)

Here ¦ is the support vector to the state constraint Y .t  / at x .0/ .t  /. In general, if

the jump is ƒ.t  C 0/  ƒ.t   0/ D ¦, then the necessary condition for such a
jump is

h¦; xP .0/ .t  C 0/  xP .0/ .t   0/i  0: (7.84)

For example, if x .0/ .t   ¢/; ¢ > 0, lies inside the interior of the constraint set and
x.t  C ¢/ lies on the boundary, then

h¦; xP .0/ .t  C 0/  xP .0/ .t   0/i  0;

and (7.84) will be fulfilled only if this inner product equals zero.
We also need the following assumption.
Assumption 7.2.3. For Problem 7.1.2, with given l, there exists no control u .s/
that satisfies the maximum principle

l 0 S.s; £ j M .0/ ./; œ.0/ .//B.s/u .s/

D maxfl 0 S.s; £ j M .0/ ./; œ.0/ .//B.s/u j u 2 E .p.s/; P .s//g

for fs j l 0 S.s; £ j M .0/ ./; œ.0/ .//B.s/ 6D 0g and at the same time ensures for these
values of s that the corresponding trajectory x  .s/ 2 @Y .s/.
Property 7.2.3. Assumption 7.2.3 holds.
This means that if the control u.0/ .s/ is determined by the maximum principle,
with h.0/ .£; s/ D l 0 S.s; £ j M .0/ ./; œ.0/ .//B.s/ 6D 0, and therefore attains its
extremal values under given hard bounds, then this control cannot also keep the
corresponding trajectory x .0/ .s/ along the boundary @Y .s/. In other words, in this
case the maximum principle is degenerate along the state constraint, i.e. h.0/ .£; s/ 
0 and does not help to find the control when the trajectory lies on the boundary
@Y .s/.
This assumption excludes the case when solution of Problem 7.1.2 without state
constraints already satisfies the given state constraints, which therefore turn out to
be passive.
Lemma 7.2.5. Under Assumption 7.83 applied to Problem 7.1, the function

h.0/ .£; s/ D l 0 S.s; £ j M .0/ ./; œ.0/ .//B.s/  0

whenever x .0/ .s/ 2 @Y .s/.

7.2 State-Constrained Control: Computation 309

Remark 7.2.6. Recall that equations for tight external ellipsoids without state
constraints are explained in detail in Chap. 3, Sects. 3.2–3.3, being described by
equations (see (3.59), (3.60), (3.17))

xP  D A.t /x  C B.t /p.t /; x  .t0 / D x 0 ;

XP C

D A.t /XC C XC A0 .t / C .t /XC C  1 .t /P .t /; XC .t0 / D X 0 ;

l  .t / D A.t /0 l  .t /; l  .t0 / D l ;

and

.t / D hl  .t /; B.t /P .t /B 0 .t /l  .t /i1=2 =hl  .t /; XC .t /l  .t /i1=2 :

Exercise 7.2.1. Compare equations of Remark 7.2.6 with (7.72)–(7.73) derived

under state constraints.
Chapter 8
Trajectory Tubes State-Constrained
Feedback Control

Abstract This chapter begins with the theory of trajectory tubes which are
necessary elements of realistic mathematical models for controlled processes and
their evolutionary dynamics. We then deal with the evolution in time of state-
constrained forward and backward reachability tubes also known as “viability
tubes.” The backward tubes are then used to design feedback controls under state
constraints that may also appear in the form of obstacles to be avoided by system
trajectories.

Keywords Trajectory tubes • Viable solutions • Funnel equations • Feedback

control • Obstacle problems

In this chapter we study the evolution of trajectory tubes. These tubes are needed
for problems of dynamics and control, especially those that deal with incompletely
described systems. Such a theory of trajectory tubes was introduced in [158]
and is described here in appropriate form for the topics of this book. The theory
is then applied to state-constrained reachability (“reach”) tubes, also known as
“viability tubes.” We introduce the technical tools in set-valued dynamics that are
used for calculating such tubes. These backward tubes are then used to define state-
constrained closed-loop strategies for target control. Finally we study similar issues
for obstacle problems that generate a class of more complex state constraints: the
target-oriented trajectories should develop within a bounded control set while also
avoiding obstacles that lie on route to the target set. Throughout the investigations
we assume that the notions of continuity and measurability of single-valued and
multivalued maps are taken in the sense of [41, 238].

8.1 The Theory of Trajectory Tubes: Set-Valued

Evolution Equations

We start with a definition of trajectory tubes that are at the object of further
discussion.

© Springer International Publishing Switzerland 2014 311

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__8
312 8 Trajectory Tubes State-Constrained Feedback Control

8.1.1 Trajectory Tubes and the Generalized Dynamic System

Consider the nonlinear differential inclusion derived from Sect. 7.1 , (7.1), (7.2),

xP 2 F .t; x/; t 2 T; x.t0 / D x 0 2 X 0 2 comp Rn ; (8.1)

with F .t; x/ D f .t; x; P .t //,

Assume that F .t; x/ satisfies the Lipschitz condition with constant k > 0

h.F .t; x 0 /; F .t; x 00 //  kkx 0  x 00 k; 8x 0 ; x 00 2 Rn : (8.2)

Taking set X 0 2 comp Rn to be given, denote, as before, xŒt  D x.t; t0 ; x 0 /, t0 2

T D Œt0 ; t1 ; to be a solution of (8.1) (an isolated trajectory) that starts at point
xŒt0  D x 0 2 X 0 .1
We regard the solution to be a Caratheodory-type trajectory xŒ, namely, an
absolutely continuous function xŒt  (t 2 T ) that satisfies the inclusion

P  2 F .t; xŒt /; xŒt0  D x 0

dxŒt =dt D xŒt (8.3)

for almost all t 2 T .

We require all solutions fxŒt  D x.t; t0 ; x0 / j x0 2 X0 g to be extendible up to
time t1 [7, 26, 75].
Let Y .t / be a continuous multivalued map (Y W T ! conv Rn ) of (7.2), that
defines the state constraint

x.t / 2 Y .t /; t 2 T: (8.4)

The map Y .t / may be obtained from measurement equations (considered further,

in the next chapter)

y.t / D G.t /x C Ÿ.t /; Ÿ.t / 2 R .t /;

so that

G.t /x.t / 2 Y .t / D y.t /  R .t /; (8.5)

with y.t / given.

A more general form could be

y.t / 2 G.t; x/ (8.6)

1
Recall that a trajectory x.t / of system (8.1) generated by given initial condition ft0 ; x 0 g is marked
by square brackets, as xŒt , in contrast with an unspecified trajectory x.t /:
8.1 The Theory of Trajectory Tubes: Set-Valued Evolution Equations 313

where G.t; x/ is a multivalued map (G W T  Rn ! conv Rp ). Other requirements

on G.t; x/ will be indicated below, along with specific theorems.
The problem will now consist in describing the tube Xy Πof solutions to
system (8.1) under state constraints (8.4), (8.5), (or (8.6)) and also the equations
for the evolution of tube Xy Œt .
As mentioned in the Preface to Chap. 7, the problem of jointly solving the system
of inclusions (8.4), (8.5), could be also treated as a version of the viability problem
[5]. The solution set to this system will be called the viable tube and defined more
precisely below.
As we will observe, the differential inclusion (8.1) with viability constraints (8.5)
may generate a class of generalized dynamic systems that are relevant for describing
tube-valued solutions to problems of dynamics and control.
We now proceed with a rigorous formulation and a constructive theory for
problems under discussion.
Definition 8.1.1 (See [5, 158]). A trajectory xŒt  D x.t; t0 ; x 0 /; .x 0 2 X 0 ; t 2 T /
of the differential inclusion (8.1) is said to be viable on Œt0 ; £ £  t1 , if

xŒt  2 Y .t / for all t 2 Œt0 ; £: (8.7)

We proceed under assumption that there exists at least one solution x  Œt  D

x .t; t0 ; x0 / of (8.4) (together with a starting point x  Œt0  D x0 2 X 0 ) that satisfies


condition (8.7) with £ D t1 . The trajectory x  Œt  is therefore assumed to be viable

on the whole segment Œt0 ; t1 . Conditions for the existence of such trajectories may
be given in terms of generalized duality concepts [137, 238]. Known theorems on
viability also provide the existence of such trajectories x  Œt  (see [5]).
Let X .; t0 ; X0 / be the set of all solutions to inclusion (8.4) without the state
constraint (8.5) that emerge from X 0 (the “solution tube”) With X Œt  D X .t; t0 ; X 0 /
being its cross-section at time t . Define
[
„D fX .t; t0 ; X 0 / j t0  t  t1 g

to be the integral funnel for (8.1), [224]. One may observe that under our
assumptions „ 2 comp Rn [8, 26].
The subset of X .; t0 ; X0 / that consists of all solutions to (8.1) viable on Œt0 ; £
is denoted by Xy .; £; t0 ; X 0 / and its s-cross-sections as Xy .s; £; t0 ; X 0 /, s 2 Œt0 ; £.
We also use the notation

Xy Œ£ D Xy .£; t0 ; X 0 / D Xy .£; £; t0 ; X 0 /:

As indicated earlier, it is not difficult to observe that sets Xy Σ are actually
the reachable sets at instant £ for the differential inclusion (8.1) with state
constraint (8.7).
314 8 Trajectory Tubes State-Constrained Feedback Control

As in Chap. 7, the map Xy .t; t0 ; X 0 / (Xy W T T comp Rn ! comp Rn ) satisfies

the semigroup property

Xy .t; £; Xy .£; t0 ; X 0 // D Xy .t; t0 ; X 0 /; t0  £  t  t1 ;

and therefore defines a generalized dynamic system with set-valued trajectories

Xy Œt  D Xy .t; t0 ; X 0 /. The multivalued functions X Œt , Xy Œt , t 2 T , are the
trajectory tube and the viable trajectory tube, respectively.

8.1.2 Some Basic Assumptions

We will usually work under one of the two following groups of hypotheses, unless
otherwise noted.
Denote the graph of map F .t; / as grapht F (t is fixed):

grapht F D ffx; zg 2 Rnn j z 2 F .t; x/g:

The next assumption is related to convex compact-valued tubes.

Assumption 8.1.1. 1. For some D 2 conv Rn such that „ int D the set
.D  Rn / \ grapht F is convex for every t 2 T .
2. There exists a solution x Œ to (8.4) such that x Œt0  2 X 0 and

x Œt  2 int Y .t /; 8t 2 T:

3. The set X 0 2 conv Rn .

In order to formulate the next group of assumptions, related to star-shaped sets,
we recall the following notion.
Definition 8.1.2. A set St Rn is said to be star-shaped (with center at c) if c 2 St
and .1  œ/c C œSt St for all œ 2 .0; 1.
The family of all star-shaped compact sets St Rn with center at c is denoted
as S t .c; Rn / and with center at 0 2 Rn as S t Rn .
Assumption 8.1.2. 1. For every t 2 T , one has grapht F 2 St.R2n /.
2. There exists © > 0 such that ©B.0/ Y .t / for all t 2 T; (B.0/ D fx 2
Rn j kxk  1g/.2
3. The set X 0 2 St.Rn /.

2
If Y .t / has no interior points, being located in space Rm ; m < n, then Y .t / may be substituted
by its neighborhood Yr .t / D Y .t / C rB.0/ > 0; with some additional regularization.
8.1 The Theory of Trajectory Tubes: Set-Valued Evolution Equations 315

Observe that Assumptions 8.1.1 and 8.1.2 are overlapping, but in general neither
of them implies the other.
One important property of trajectory tubes and viable trajectory tubes is to
preserve some geometrical characteristics of sets X Œt , Xy Œt  along the system
trajectories, as in the following assertions.
Lemma 8.1.1. Under Assumption 8.1.1 the cross-sections X Œt , Xy Œt  are convex
and compact for all t 2 T (X Œt , that is, Xy Œt  2 conv Rn .
Lemma 8.1.2. Under Assumption 8.1.2 the cross-sections X Œt , Xy Œt  are star-
shaped and compact for all t 2 T , that is, X Œt , Xy Œt  2 St.Rn /.
Loosely speaking, a differential inclusion with a convex grapht F generates
convex-valued tubes X Œt , Xy Œt , while a star-shaped grapht F generates tubes with
star-shaped cross-sections X Œt , Xy Œt . Convexity and the star-shape property are
therefore the two simplest basic geometrical invariants for cross-sections of the
trajectory tubes and viable trajectory tubes.

8.1.3 The Set-Valued Evolution Equation

Having defined the notion of viable trajectory tube and observed that mapping
Xy Œt  D Xy .t; t0 ; X 0 / defines a generalized dynamic system, we come to the
following natural question: does there exist some sort of evolution equation that
describes the tube Xy Œt  as solutions to a related generalized dynamic system?
It should be emphasized here that the space K D fcomp Rn g of all compact
subsets of Rn to which the “states” Xy Œt  belong is only a metric space with
a rather complicated nonlinear structure. In particular, there does not exist even
an appropriate universal definition for the difference of sets A; B 2 comp Rn .
Hence the evolution equation for Xy Œt  is devised to avoid using such “geometric”
differences. It is mainly due to this reason that the construction of an infinitesimal
generator for set-valued transition maps generated by such generalized system
dynamics is cumbersome.
In this subsection we rely on an approach to the evolution of trajectory tubes
through funnel equations for set-valued functions in contrast with Hamiltonian
techniques where this is done, as we have seen in Sect. 7.1, through single-valued
functions described by PDEs of the HJB type.
We shall further require that one of the following assumptions concerning the
mapping Y ./ is fulfilled.
Assumption 8.1.3. The graph graph Y 2 conv RnC1 .
Assumption 8.1.4. For every l 2 Rn the support function f .l; t / D ¡.l j Y .t // is
differentiable in t and its derivative @f .l; t /=@t is continuous in fl; t g.
316 8 Trajectory Tubes State-Constrained Feedback Control

Theorem 8.1.1. Suppose Assumption 8.1.1 or 8.1.2 holds and map Y satisfies
either Assumption 8.1.3 or 8.1.4. Then the multifunction Xy Œt  D Xy .t; t0 ; X 0 / is
a set-valued solution to the following evolution equation
 [ 
lim ¢ 1 h Xy Œt C ¢; fx C ¢ F .t; x/ j x 2 Xy Œt g \ Y .t C ¢/ D 0;
¢!C0
(8.8)
t0  t  t1 ; Xy Œt0  D X 0 :

Rigorous proofs of this theorem in terms of the Hausdorff distance (see Sect. 2.3)
and of the following uniqueness theorem are rather lengthy and are given in detail
in [158, Sects. 6, 7]. The uniqueness of solution to Eq. (8.8) is indicated in the next
theorem.
Denote Z Œt0 ; t1  to be the set of all multivalued functions Z Œt  (Z W Œt0 ; t1  !
comp Rn ) such that
 [ 
lim ¢ 1 h Z Œt C ¢; fx C ¢ F .t; x/ j x 2 Z Œt g \ Y .t C ¢/ D 0; (8.9)
¢!C0

Z Œt0  D X 0 ; t0  t  t1 ;

where the Hausdorff limit in (8.9) is uniform in t 2 Œt0 ; t1 .

Under assumptions of Theorem 8.1.1 we have

Xy Œ D Xy .; t0 ; X 0 / 2 Z Œt0 ; t1 :

Theorem 8.1.2. Suppose the assumptions of Theorem 8.1.1 hold. Then the mul-
tivalued function Xy Œ£ D Xy .£; t0 ; X 0 / is the unique solution to the funnel
equation (8.8) in the class Z Œt0 ; t1  of all the set-valued mappings Z ./ that satisfy
this equation uniformly in t 2 Œt0 ; t1 .

8.1.4 The Funnel Equations: Specific Cases

In this subsection we indicate some examples of differential inclusions for which

Assumptions 8.1.1 and 8.1.2 are fulfilled, presenting also the specific versions of
the funnel equations.

Linear Systems

Consider the linear differential inclusion

xP 2 A.t /x C B.t /Q .t /; x.t0 / D x 0 2 X 0 ; t0  t  t1 ; (8.10)

8.1 The Theory of Trajectory Tubes: Set-Valued Evolution Equations 317

where x 2 Rn , A.t / and B.t / are continuous n  n- and n  m-matrices, Q .t / is a

continuous map (Q W Œt0 ; t1  ! conv Rm ), X 0 2 conv Rn .
Here Assumptions 8.1.1 (1), 8.1.1 (3) are fulfilled automatically. So to retain
Assumption 8.1.1 (2) we introduce
Assumption 8.1.5. There exists a solution x Πof (8.10) such that

x Œt0  2 X 0 ; x Œt  2 int Y .t /; 8t 2 Œt0 ; t1 :

The following result is a consequence of Theorem 8.1.1.

Theorem 8.1.3. Suppose Assumptions 8.1.5 and 8.1.3 or 8.1.4 hold. Then the set-
valued function Xy Œt  D Xy .t; t0 ; X 0 / is the solution to the evolution equation

lim ¢ 1 h.Xy Œt C¢; ..I C¢A.t //Xy Œt C¢B.t /Q .t //\ Y .t C¢// D 0 (8.11)
¢!C0

t0  t  t1 ; X Œt0  D X 0 :

The solution to this equation may be obtained using ellipsoidal techniques

described earlier in Chap. 3 and in Chap. 7, Sect. 7.2.

Bilinearity: Uncertainty in Coefficients of a Linear System

Consider a differential equation of type (8.1) where

f .t; x; v/ D A.t; v/x C h.t /; v.t / 2 Q .t /;

with A.t; Q .t // D A .t / and h.t / 2 P .t / unknown but bounded (P .t / 2 conv Rnn

being continuous in t ). This yields

xP 2 A .t /x C P .t /; x.t0 / D x 0 2 X 0 ; x.t / 2 Y .t /; t0  t  t1 ; (8.12)

and the right-hand side F .t; x/ D A .t /x C P .t / depends bilinearly upon the state
vector x and the set-valued map A .
We assume that A ./ is a continuous mapping from Œt0 ; t1  into the set conv Rnn
of convex and compact subsets of the space Rnn of n  n-matrices.
Equation (8.12) is an important model of an uncertain bilinear dynamic system
with set-membership description of the unknown matrices A.t / 2 A .t /, and inputs
h.t / 2 P .t / and with x 0 2 X 0 ; [158, Sect. 3].
It is not difficult to demonstrate that here Assumption 8.1.1 does not hold, so that
sets X Œt  D X .t; t0 ; X 0 / (and also Xy Œt  D Xy .t; t0 ; X 0 /) need not be convex.
Example 8.1.1. Indeed, consider a differential inclusion in R2 :

xP 1 2 Œ1; 1x2 ; xP 2 D 0; 0  t  1;
318 8 Trajectory Tubes State-Constrained Feedback Control

with initial condition

x.0/ D x0 2 X 0 D fx D .x1 ; x2 / j x1 D 0; jx2 j  1=2g

Then

X Œ1 D X .1; 0; X 0 / D X 1 [ X 2 ;

where

X 1 Dfx D .x1 ; x2 / j jx1 j  x2  1=2g; X 2 Dfx D .x1 ; x2 / j jx1 j  x2  1=2g:

It is obvious that set X Œ1 is not convex.

However one can verify that though Assumption 8.1.1 is not fulfilled for the
system (8.12), the Assumption 8.1.2 (1) is indeed satisfied, provided 0 2 X 0 and 0 2
P .t / for all t 2 Œt0 ; t1 . Assumption 8.1.2 may hence be rewritten in the following
reduced form.
Assumption 8.1.6. 1. Condition 0 2 P .t / is true for all t 2 Œt0 ; t1 .
2. There exists an © > 0 such that ©B.0/ Y .t /, 8t 2 Œt0 ; t1 .
3. Set X 0 2 St.Rn / and 0 2 X 0 .
To formulate an analog of Theorem 8.1.1 we introduce an additional notation:
M  X D fz D M x 2 Rn j M 2 M ; x 2 X g, where M 2 conv Rnn , X 2
conv Rnn .
Theorem 8.1.4. Under Assumptions 8.1.6 and 8.1.3 or 8.1.4 the set-valued map
Xy Œt  D Xy .t; t0 ; X 0 / is the solution to the following evolution equation

lim ¢ 1 h.Xy Œt C ¢; ..I C ¢ A .t //  Xy Œt  C ¢ P .t // \ Y .t C ¢// D 0; (8.13)

¢!C0

t0  t  t1 ; X Œt0  D X 0 :

A Nonlinear Example

We indicate one more example, which is given by a set-valued function F .t; x/ with
a star-shaped grapht F .
Namely, let F .t; x/ be of the form

F .t; x/ D G.t; x/U C P .t /; (8.14)

where the n  n-matrix function G.t; x/ is continuous in t , Lipschitz-continuous

in x and F .t; x/ is positively homogeneous in U 2 conv Rn . The map P .t / is the
same as before. One may immediately verify that now Assumption 8.1.2 (1) holds,
so that the cross-section Xy Œt  of the solution tube to Eq. (8.13) is star-shaped.
8.2 Viability Tubes and Their Calculation 319

8.1.5 Evolution Equation Under Relaxed Conditions

We now mention an approach similar to the described, but with the evolution
equation (8.8) written using the Hausdorff semidistance

hC .A; B/ D max minfka  bk j a 2 A; b 2 Bg; A; B 2 comp Rn ;

a b

rather than the Hausdorff distance (a metric) h.A; B/ D maxfhC .A; B/; hC .B; A/g.
Here is such an equation (see [174] and also [163]) for system (7.1), (7.2):
( ˇ )!
[ ˇ
1 ˇ
lim hC ZŒt C ¢; x C ¢.f .t; x; P .t //ˇ x 2 ZŒt  \ Y .t / D 0; Z.t0 / D X 0 ;
¢!C0 ¢ ˇ
(8.15)

Remark 8.1.1. The solution ZŒt  to this evolution equation is a multivalued function
with ZŒt0  D X 0 ; which satisfies (8.15) almost everywhere. As a rule, this solution
is not unique. However, we may single out a solution X Œt  which is the inclusion-
maximal among all solutions ZŒt  to (8.15). Namely, X Œt   ZŒt , where ZŒt 
is any solution that starts from Z.t0 / D X 0 : Note that Eq. (8.15) makes sense
for any piecewise continuous function y.t /. As mentioned above, when dealing
with information tubes, we presume these functions to be right-continuous. The
hC -techniques may be more adequate for dealing with discontinuous set-valued
functions Xy Œt  D Xy .t; t0 ; X0 /.
The evolution equation (8.15) delivers a formal model for the generalized
dynamic system generated by mapping Xy Œt  D Xy .t; t0 ; Xy0 /. However, if the sets
Xy Œt  are convex, there exists an alternative presentation of this evolutionary system.
Namely, each of the sets Xy Œt  could be described by its support function

®.l; t / D ¡.l j Xy Œt / D maxf.l; x/ j x 2 Xy Œt g:

The dynamics of Xy Œt  would then be reflected by a generalized partial

differential equation for ®.l; t /. The description of this equation is the next issue
to be discussed.

8.2 Viability Tubes and Their Calculation

8.2.1 The Evolution Equation as a Generalized Partial

Differential Equation

In this section we restrict our attention to a system for which Assumption 8.1.1 is
satisfied. In this case the tube Xy .; £; t0 ; X 0 / will be a convex compact subset of
C n Œt0 ; t1 . With set Xy Œ£ D Xy .£; t0 ; Xy0 / 2 conv Rn for every £ 2 Œt0 ; t1 .
320 8 Trajectory Tubes State-Constrained Feedback Control

Since there is an equivalence between an inclusion x 2 X and the system of

inequalities .l; x/  ¡.l j X /, 8l 2 Rn , we may now describe the evolution of sets
Xy Œ£ through differential relations for the support function ®.l; t / D ¡.l j Xy Œt /.
The primary problem on the law of evolution for multivalued “states” Xy Œt  will now
be replaced by one of evolution in time of a distribution ¥t .l/ D ®.l; t / which is
defined over all l 2 Rn , and is positively homogeneous and convex in l.
We now calculate the directional derivative

@C ®.l; t /=@t D lim ¢ 1 .®.l; t C ¢/  ®.l; t //

¢!C0

for each instant t 2 Œt0 ; t1  and for any fixed l 2 Rn .

The next result will be proved as a direct consequence of Theorem 8.1.1.
Theorem 8.2.1. Suppose Assumptions 8.1.1 together with either 8.1.3 or 8.1.4
hold. Then the support function ®.l; t / D ¡.l j Xy Œt / is right-differentiable in t
and its directional derivative in time is

@C ®.l; t /=@t D min maxf§.q; x; l; t / j q 2 Q.l; t /; x 2 @l ®.l; t /g D

q x

D max minf§.q; x; l; t / j x 2 @l ®.l; t /; q 2 Q.l; t /g; (8.16)

x q

where

§.q; x; l; t / D ¡.q j F .t; x// C @¡.l  q j Y .t //=@t; (8.17)

Q.l; t / D fq 2 R j ®.l; t /  ®.q; t / D ¡.l  q j Y .t //g; @l ®.l; t /
n

D fx 2 Xy Œt  j .l; x/ D ®.l; t /g:

Proof. From Theorem 8.1.1 we deduce

®.l; t C ¢/ D ¡.l j R.¢; t; Xy Œt / \ Y .t C ¢// C o.¢/klk;

where ¢ 1 o.¢/ ! 0 (¢ ! C0),

[
R.¢; t Xy Œt / D fx C ¢ F .t; x/ j x 2 Xy Œt g; t0  t  t C ¢  t1 ;

and R.¢; t; Xy Œt / is convex due to Assumption 8.1.1.

Taking the infimal convolution of support functions for respective sets
in the above intersection (under assumptions of Theorem 8.2.1 the inf-
convolution (8.16), (8.17) is exact, [194]), we come to
8.2 Viability Tubes and Their Calculation 321

®.l; t C ¢/ D (8.18)

D minf¡.q j R.¢; t; Xy Œt // C ¡.l  q j Y .t C ¢// j q 2 rB.0/ C o.¢/; kqk  1g

q

for a certain r > 0.

Hence we have to verify only the right-differentiability in ¢ of the following
function

H.¢; l; t / D min maxfh.¢; t; l; q; x/ j q 2 rS; x 2 Xy Œt g; (8.19)

q x

where

h.¢; t; l; q; x/ D .q; x/ C ¢¡.q j F .t; x// C ¡.l  q j Y .t C ¢// (8.20)

with fl; t g fixed.

The differentiation of this minimax relation may be realized using the rules
given in [60, Theorem 5.3]. Applying this theorem, all the conditions of which are
satisfied, we arrive at (8.16).
Let us elaborate on this result. Due to properties of Xy Œt  we have Xy Œt  Y .t /,
so that (8.17) yields (with q 2 Q.l; t /)

®.l; t /  ®.q; t / D ¡.l  q j Y .t //  ®.l  q; t / (8.21)

On the other hand, since the function ®.l; t / is convex and positively homogeneous
in l 2 Rn with t fixed, we also observe

®.l; t /  ®.q; t / C ®.l  q; t / (8.22)

Comparing this with (8.21), we come to the equality

®.l; t /  ®.q; t / D ®.l  q; t /: (8.23)

Without loss of generality we could have assumed 0 2 Xy Œt .

Suppose we now require

0 2 int Xy Œt ; 8t 2 Œt0 :t1 ; (8.24)

which is an additional assumption. It is not difficult to prove the following sufficient

condition for the sets X Œt  to be symmetric with respect to 0.3

3
Recall that a set A Rn is defined to be symmetric with respect to 0 if A D A .
322 8 Trajectory Tubes State-Constrained Feedback Control

Lemma 8.2.1. Suppose assumption (8.24) holds and sets X 0 , grapht F , Y .t / are
symmetric for all t 2 Œt0 ; t1 . Then set X Œt  is also symmetric whatever be the instant
t 2 Œt0 ; t1 .
Under the assumptions of Lemma 8.2.1 the inequality (8.22) turns into an
equality if and only if the vectors l, q are collinear (l D ’q and ’  0). Then (8.17)
turns into

.1  ’/.¡.l j X Œt /  ¡.l j Y.t /// D 0

so that with ¡.l j X Œt / < ¡.l j Y .t // we have ’ D 1 and in (8.16) the set Q.l; t / D
flg. Otherwise, q D ’l, ’ 2 Œ0; 1 and

§.q; x; l; t / D ’¡.l j F .t; x// C .1  ’/@¡.l j Y.t //=@t;

so that the minimum in (8.16) is to be taken over ’ 2 Œ0; 1.

Finally, we have
Theorem 8.2.2. Under assumptions of Theorem 8.1.1 the next relations are true:

@C ®.l; t / ¡max ; if ®.l; t / < ¡.l j Y .t //I
D (8.25)
@t min f¡max ; @¡.l j Y .t //=@t g ; if ®.l; t / D ¡.l j Y .t //;

¡max D maxf¡.l j F .t; x// j x 2 @l ®.l; t /g:

For a linear system this turns into


@C ®.l; t / ¡ ; if ®.l; t / < ¡.l j Y .t //I
D (8.26)
@t minf¡ ; @¡.l j Y .t //=@t g; if ®.l; t / D ¡.l j Y .t //;

¡ D ¡.A0 l j @l ®.l; t // C ¡.l j B.t /Q .t //:

Each of the relations (8.25), (8.26) is actually a generalized partial differential

equation which has to be solved under boundary condition

®.l; t0 / D ¡.l j X Œt0 /: (8.27)

The last result will later be used in Sect. 8.4 in describing feedback solution
strategies for the problem of closed-loop control under state constraints.

8.2.2 Viability Through Parameterization

In this subsection the description viable trajectory tubes is reduced to the treatment
of trajectory tubes for a variety of specially designed new differential inclusions
8.2 Viability Tubes and Their Calculation 323

without state constraints. These new inclusions are designed depending upon certain
parameters and have a relatively simple structure. The overall solution is then
presented as an intersection of parallel solution tubes for the parametrized inclusions
over all the parameters.

The General Case

We start with the general nonlinear case, namely, with differential inclusion (8.1)
and set-valued state constraint (8.4), repeated here as

xP 2 F .t; x/; x.t0 / 2 X 0 ; t0  t  t1 ; (8.28)

with constraints

x.t / 2 Y .t /; t0  t  £; (8.29)

supposing also that the basic conditions of Sect. 8.1 on maps F , Y and set X 0 are
satisfied.
In this section we do not require however that either of the Assump-
tions 8.1.1, 8.1.2 or 8.1.3, 8.1.4 would hold.
We further need to introduce the restriction FY .t; x/ of map F .t; x/ to a set
Y .t / (at time t ). This is given by

F .t; x/; if x 2 Y .t /I
FY .t; x/ D
;; if x 62 Y .t /:

The next property follows directly from the definition of viable trajectories.
Lemma 8.2.2. An absolutely continuous function xŒt  defined on the interval Œt0 ; £
with x 0 2 X 0 is a viable trajectory to (8.28) for t 2 Œt0 ; £ if and only if the inclusion
P  2 FY .t; xŒt / is true for almost all t 2 Œt0 ; £.
xŒt
We will now represent FY .t; x/ as an intersection of certain multifunctions. The
first step to achieve that objective will be to prove the following auxiliary assertion.
Lemma 8.2.3. Suppose A is a bounded set, B a convex closed set, both in Rn . Then

\ 
A if 0 2 B;
fA C LB j L 2 Rnn g D
; if 0 62 B;

Proof. First assume 0 2 B. Then

\ \
A fA C Lf0g j L 2 Rnn g fA C LB j L 2 Rnn g A C 0  B D A:
324 8 Trajectory Tubes State-Constrained Feedback Control

T
Hence, A D fACLB j L 2 Rnn g. If 0 62 B, then there exists a hyperplane in Rn
that strictly separates the origin from B. By the boundedness of A, for a sufficiently
large number œ > 0, we then have

.A C œB/ \ .A  œB/ D ;:

Setting LC D œI and L D œI we conclude that

\
fA C LB j L 2 Rnn g .A C LC B/ \ .A C L B/ D ;:

The lemma is thus proved.

From the above two lemmas above we obtain the following description of viable
trajectories.
Theorem 8.2.3. An absolutely continuous function x./ defined on an interval
Œt0 ; t1  with x 0 D x.t0 / is a viable trajectory of (8.28) for Œt0 ; £ iff the inclusion
\
P /2
x.t f.F .t; x/  Lx.t / C LY .t // j L 2 Rnn g

is true for almost all t 2 Œt0 ; £.

We introduce a family of differential inclusions that depend on a matrix
parameter L 2 Rnn . These are given by

zP 2 F .t; z/  Lz C LY .t /; z.t0 / 2 X 0 ; t0  t  t1 : (8.30)

By zŒ D z.; £; t0 ; z0 ; L/ denote the trajectory to (8.30) defined on the interval

Œt0 ; £ with zŒt0  D z0 2 X 0 . Also denote
[
Z.; £; t0 ; X 0 ; L/ D fZ.; £; t0 ; z0 ; L/ j z0 2 X 0 g;

where Z.; £; t0 ; z0 ; L/ is the tube of all trajectories zŒ D z.; £; t0 ; z0 ; L/ issued at

time t0 from point z0 and defined on Œt0 ; £. The cross-sections of set Z.; £; t0 ; X 0 ; L/
at time t are then denoted as Z.£; t0 ; X 0 ; L/.
Theorem 8.2.4. For each £ 2 Œt0 ; t1  one has
\
Xy .; £; t0 ; X 0 / D fZ.; £; t0 ; X 0 ; L/ j L 2 Rnn g:

Moreover, the following inclusion is true

\
Xy Œ£ D Xy .£; t0 ; X 0 / fZ.£; t0 ; X 0 ; L/ j L 2 Rnn g:
8.3 Control Synthesis Under State Constraints: Viable Solutions 325

The Linear-Convex Case

For the case of linear system (8.10)

xP 2 A.t /x C B.t /Q .t /; x.t0 / D x 0 2 X 0 ; t0  t  t1 ;

inclusion (8.30) turns into

zP 2 .A.t /L.t //zC Q .t /CL.t /Y .t /; z.t0 / D z0 2 X 0 ; t0  t  £: (8.31)

whose solution tubes parametrized by L./ are Z.£; t0 ; X 0 ; L.// However, the main
point for a linear system is that the second inclusion of Theorem 8.2.4 actually turns
into an equality, producing the next result.
Theorem 8.2.5. The following equality is true for any £ 2 Œt0 ; t1 :
\
Xy Œ£ D fZ.£; t0 ; X 0 ; L.// j L./ 2 Rnn Œt0 ; £g: (8.32)

The proof of this theorem is a rather cumbersome procedure which relies on

direct calculations of Sects. 8.1 and 8.2, including the modified maximum principle
of Sect. 7.2.1, is given in detail in [158] (see Sects. 10–13 of that paper).
Remark 8.2.1. Dealing with nonlinear systems we could look at a broader class of
functions that parametrize (8.30), taking L D L.; / D fL.t; x/ t 2 Œt0 ; t1 ; x 2
Rn g: Question: does there then exist a class of nonlinearities and a related class of
functions L.t; x/ that would ensure an equality similar to (8.32)?

8.3 Control Synthesis Under State Constraints:

Viable Solutions

The techniques of Chaps. 2 and 3 with those of Sects. 7.1 and 7.2 allow to approach
the problem of control synthesis under state constraints (see [136]). Such issues
were treated later under the term “viability problems,” with some renaming of earlier
terminology [5]. Therefore we may also interpret our problem as that of terminal
control “under viability constraints,” meaning that solutions subject to the state
constraint are “viable solutions.” As in the case of systems without state constraints,
the synthesized solution strategy described here will be nonlinear.
326 8 Trajectory Tubes State-Constrained Feedback Control

8.3.1 The Problem of State-Constrained Closed-Loop Control:

Backward Reachability Under Viability Constraints

We now return to system (7.30), namely,

xP D f .t; x; u/ D A.t /x C B.t /u; u 2 P .t / D E .p.t /; P .t // (8.33)

x 2 Y .t / D E .y.t /; Y.t // (8.34)

x 0 D x.t0 / 2 X 0 ; (8.35)

where p.t /, P.t / are continuous and y.t /, Y .t / are absolutely continuous. The target
set is also ellipsoidal: M D E .m; M /
We may present constraints (8.34) and x 2 E .m; M / as

®.t; x/ D hx  y.t /; Y 1 .t /.x  y.t //i  1; ®M .x/ D hx  m; M 1 .x  m/i  1:

The following problems are considered.

Problem 8.3.1. Given interval Œ£; t1 , system position f£; xg and convex compact
target set M  Rn , find
(i) solvability set W Œ£ D W .£; t1 ; M / and
(ii) feedback control strategy u D U .t; x/, U .; / 2 UY such that all the solutions
to the differential inclusion

xP 2 A.t /x C B.t /U .t; x/ (8.36)

that start from a given position f£; xg, x D xŒ£, xŒ£ 2 W .£; t1 ; M /, £ 2
Œt0 ; t1 , would satisfy the inclusion (8.34), £  t  t1 , with x.t1 / 2 M .
Here UY is an appropriate class of functions defined below, in (8.42). The given
problem is non-redundant, provided W Œ£ D W .£; t1 ; M / ¤ ;.
The value function for Problem 8.3.1 may be defined as follows
 ˇ 
ˇ
V s .£; x/ D min max maxf®.t; xŒt / j t 2 Œ£; t1 g; ®M .xŒt1 / ˇˇ xŒ£ D x :
u t
(8.37)
Then

W Œ£ D fx W V s .£; x/  1g

will be the backward reach set relative to M under state constraint Y .t /, which
is the set of points x for which there exists some control u.t / that steers trajectory
xŒt  D x.t; £; x/ to M with x.t / 2 Y .t /; t 2 Œ£; t1 .
8.3 Control Synthesis Under State Constraints: Viable Solutions 327

Denote V s .t; x/ D V s .t; x j V s .t1 ; //, emphasizing the dependence of V s .t; x/

on the boundary condition V s .t1 ; x/ D maxf®.ª; x/; ®M .x/g.
Lemma 8.3.1. The value function V s .t; x/ satisfies the principle of optimality,
which has the semigroup form

V s .£; x j V s .t1 ; // D V s .£; x j V s .t;  j V s .t1 ; ///: (8.38)

As in Sect. 2.1 this property is established through conventional reasoning and

implies a similar property for the corresponding reachability sets. Namely, if we
take W Œ£ D W .£; t1 ; M /, we have W .£; t1 ; M / D W .£; t; W .t; t1 ; M //:
To proceed further, denote

H .t; x; p; u/ D p0 C hp; f .t; x; u/i; p D fp0 ; pg:

Consider the .n C 1/-dimensional vector p D fVts ; Vxs g: Then, under control u

the total derivative of V s .t; x/ along a trajectory x.t / of (8.33) will be

d V s =dt ju D H .t; x; p; u/; p0 D Vts ; p D Vxs :

This relation is true for linear systems of type (8.33), with a differentiable V s .t; x/.
Otherwise, the property of differentiability for V s may be relaxed to directional
differentiability which in our case always holds.
The class of solution strategies. As in Chap. 2, we shall look for the solution in the
class UY of all set-valued functions with convex compact values that are measurable
in t and upper semicontinuous in x, ensuring thus the solvability and extendability
of solutions to (8.36) for any x 0 2 X 0 2 conv Rn . But the difference will be in that
now UY of Problem 8.3.1 will be
8
ˆ
ˆ Pw .t / D E .p.t /; P .t // \ fu W d®.t; x/=dt ju  0g;
ˆ
ˆ
< if ®.t; x/ D 1;
UY D fU .t; x/g; U .t; x/ D Pw .t / D E .p.t /; P .t //;
ˆ
ˆ
ˆ if ®.t; x/ < 1:
:̂
(8.39)
Assume

V s .t; x/ > ®.t; x/:

Then the HJB equation for V s will be

Vts .t; x/ C minhVxs ; A.t /x C B.t /ui D 0; u 2 Pw .t /; (8.40)

u

under boundary condition V s .t1 ; x/ D maxf®.t1 ; x/; ®M .x/g:

328 8 Trajectory Tubes State-Constrained Feedback Control

Note that under control u the total derivative along the trajectory of (8.33) is

d V s =dt ju D H .t; x; p; u/; p D fVts ; Vxs g;

and d®.t; x/=dt ju D H .t; x; ®t;x ; u/:

Lemma 8.3.2. The backward reachability set, at time £  t1 ; from target set M ;
under state constraint x.t / 2 Y is the level set

W .£; t1 ; M / D W Œ£ D fx W V s .£; x/  1g:

The proof of this assertion is similar to the reasoning of Sect. 7.1.2

Exercise 8.3.1. Prove Lemma 8.3.2.
Now consider the second part (ii) of Problem 8.3.1 on closed-loop control
synthesis. Without loss of generality we shall assume A.t /  0, then indicate
formal changes for A.t / 6 0: In order to do that we will need to know set W Σ
and may therefore use the description given above, repeating ellipsoidal techniques
of Sect. 7.1.2. This route relies on reaching the solutions through single-valued
functions, as thoroughly discussed in Sects. 7.1 and 7.2.
However we shall now rely on the alternative vision of the same problem which
involves set-valued representations.

8.3.2 State-Constrained Closed-Loop Control

We thus deal with system

xP D B.t /u; (8.41)

u 2 P .t / D E .p.t /; P .t //; x 2 Y .t / D E .y.t /; Y .t //; x£ D x.£/ 2 X£ D E .x£ ; X£ /

(8.42)
Following the scheme of Sect. 2.6 we will look for the closed-loop solution
strategy U  by using solvability (or backward reach) sets introduced in Chap. 2,
Sects. 2.3 and 2.6.
We recall that apart from Lemma 8.3.2, the backward reach set W Σ D
W .£; t1 ; M / is also given by a funnel equation, similar to equations derived for
forward reach in Sect. 8.1 (see (8.8) for the general case and (8.11) for the linear
case):
Lemma 8.3.3. The set-valued function W Œt  satisfies the following evolution
equation

lim ¢ 1 h W Œt  ¢; .W Œt   ¢B.t /P .t // \ Y .t  ¢/ D 0; (8.43)
¢!0

t0  t  t1 ; W Œt1  D M :
8.3 Control Synthesis Under State Constraints: Viable Solutions 329

The tube W Œt  will now be used to introduce a scheme for finding the
synthesizing control strategy U .t; x/.
Given W Œt , t 2 Œ£; t1 , let
ˇ
dhC .x.t /; W Œt /=dt ˇ.8:41/ (8.44)

denote the total derivative at time t C 0 of the distance d.x; W Œt / D hC .x; W Œt /

due to Eq. (8.41).
Define
n ˇ ˇ o
U  .t; x/ D u ˇˇ dhC .x; W Œt /=dt ˇ.8:41/  0 :

We now have to prove that the set-valued strategy U  .t; x/ ¤ ; for all ft; xg and
that it solves the problem of control synthesis.
Calculating hC .x; W Œt /, we have

hC .x; W Œt / D maxfhl; xi  ¡.l j W Œt / j klk  1g;

so that the maximizer l 0 D l 0 .t; x/ ¤ 0 yields

hC .x; W Œt / D hl 0 ; xi  ¡.l 0 j W Œt /

if hC .x; W Œt / > 0 (otherwise l 0 D 0 and hC .x; W Œt / D 0).

To calculate the derivative (8.41) we need to know the partial derivative of
¡.l j W Œt / at time t C 0, which is

@ @
¡.l 0 j W Œt / D  ¡.l 0 j W Œs/; s D t:
@.t C 0/ @.s  0/

Here the right-hand part can be calculated similarly to [158, Sect. 9], as done for
@¡.l j X Œt /[email protected] C 0/, but now in backward time.
Since W Œt  Y .t /, this gives

@¡.l 0 j W Œt /
@.t  0/
8
ˆ
< ¡.l 0 j B.t /P .t //; if ¡.l 0 j W Œt  / < ¡.l 0 j Y .t //;
D (8.45)
:̂ minf¡.l 0 j B.t /P .t //; @=@t .¡.l 0 j Y .t ///g; if ¡.l 0 j W Œt  / D ¡.l 0 j Y .t //:

Exercise 8.3.2. Check the formula of (8.45).

The last relation indicates that

U 0 .t; x/ D fu W hl 0 ; B.t /ui D ¡.l 0 j B.t /P .t //g ¤ ;;

and therefore U  .t; x/ ¤ ;, since U 0 .t; x/ U  .t; x/. Thus we come to

330 8 Trajectory Tubes State-Constrained Feedback Control

Lemma 8.3.4. The strategy U  .; / 2 UY .

The necessary properties that ensure U  .; / 2 UY may be checked directly.
A standard type of reasoning yields the following
Theorem 8.3.1. The strategy u D U  .t; x/ solves the Problem 8.3.1 of target
control synthesis under “viability” constraint Y .t / for any position ft; xg that
satisfies the inclusion x 2 W Œt .
This theorem follows from
Lemma 8.3.5. For any solution xŒt  D x.t; £; x£ / to the inclusion

xP 2 B.t /U  .t; x/; xŒ£ D x£ 2 W Œ£; £  t  t1 ; (8.46)

one has xŒt  2 W Œt , £  t  t1 , and therefore xŒt  2 Y .t /, £  t  t1 , xŒt1  2 M .

Indeed, if we suppose xŒ£ 2 W Œ£ and hC .xŒt  ; W Œt / > 0 for some t  >£,
then there exists a point t  2 .£; t   where dhC .xŒt  ; W Œt  /=dt > 0 in
contradiction with the definition of U  .t; x/.
The theory of trajectory tubes and the results presented particularly in this section
allow to give a further description of the synthesized solution to (8.36) under
U .t; x/  U  .t; x/. Namely, it is now necessary to describe the tube of all solutions
to (8.36) that start with xŒ£ 2 X£ . The main point is that this tube may be described
without knowledge of the strategy U  .t; x/ itself, but only on the basis of the
information given just for the original Problem 8.3.1. This is due to the property
of colliding tubes, described in Sect. 2.6, which is also true under state constraints.
Exercise 8.3.3. Prove Corollary 2.5.1 under state constraint x.t / 2 Y .t /; t 2
Œt0 ; t1 .
Together with (8.43) consider the following evolution equation

lim ¢ 1 h..t C ¢/; ..t / C ¢ P .t // \ W .t C ¢// D 0; (8.47)

¢!0

.£/ D X£ ; X£ 2 conv Rn ; X£ Y .£/; £  t  t1 :

Assumption 8.3.1. (i) The support function f .t; l/ D ¡.l j Y .t // is differentiable

in t for each l and the derivative @f .t; l/=@t is continuous.
(ii) The set graph Y ./ D ft; x W x 2 Y .t /; t 2 Œt0 ; t1 g is convex and compact.
Suppose one of the conditions (i), (ii) of Assumption 8.3.1 holds. Then it is
possible to prove the following assertions (see [158], Sects. 6–8).
Theorem 8.3.2. The solution tube Œt  D .t; £; X£ / to the synthesized sys-
tem (8.46), X Œ£ D X£ under the viability constraint (8.34) is the tube of all solutions
to the evolution equation (8.47).
8.3 Control Synthesis Under State Constraints: Viable Solutions 331

Corollary 8.3.1. The tube of all synthesized trajectories from a given point
x£ 2 W Œ£ may be defined as the tube of all solutions to the differential inclusion

xP 2 B.t /P .t /; x.£/ D x£ ; t 2 Œ£; t1  (8.48)

that satisfy x.t / 2 W .t /, £  t  t1 , and are therefore viable relative to W .t /,

where W Œt  is the solution to (8.43).
The synthesized trajectory tube is therefore a set-valued solution to the “two-set
boundary value problem” (8.43), (8.47) due to the respective evolution equations.
The calculation of this solution obviously does not require the knowledge of the
strategy u D U  .t; x/ itself.
Remark 8.3.1. Observe that set W Σ is weakly invariant relative to M ; Y ./ due
to differential inclusion (8.48) and strongly invariant relative to M ; Y ./ due to
differential inclusion (8.46).
Exercise 8.3.4. A theorem of 8.3.2 type was proved for a nonlinear differential
inclusion [158] under conditions different from Assumption 8.3.1. Prove this
theorem under Assumption 8.3.1 of this section which imply original equation (8.33)
to be linear.
Note that with A.t / 6D 0 the funnel equation for W Œt  is as follows


lim ¢ 1 h W Œt  ¢; ..I  ¢A.t //W Œt   .B.t /P .t //¢/ \ Y .t  ¢/ D 0;
¢!0
(8.49)
t0  t  t1 ; W Œt1  D M :

Exercise 8.3.5. Prove the results of this section for A.t / 6D 0, using (8.34) and the
results of Sect. 2.6.
Exercise 8.3.6. Calculate W Σ using the techniques of convex analysis, similar to
Sect. 2.4.

8.3.3 Example

Consider the double integrator system of Sect. 7.2.4, taken on the time interval t 2
Œ0; 2; namely,

xP 1 D x2 ; xP 2 D u; (8.50)

with control juj  k. Let m D .m1 ; 0/, ®M .x/ D hxm; M.xm/i; M D M 0 > 0
and ®.x2 / D jx2 j2 .
332 8 Trajectory Tubes State-Constrained Feedback Control

Fig. 8.1 Growth of the k = 1, m = [ 0 0.8 ], M = [ 12 −4; −4 12 ]

backward reach set subject to 2.5
convex constraints for a
double integrator (solid lines; 2
final time is shown thicker).
The target set (dotted line) 1.5
and constraints (shaded
region) are shown as well.
The unconstrained backward 1
reach set at the same final
time is shown for comparison x2 0.5
(dashed line)
0

−0.5

−1

−1.5
−3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1
x1

The objective is to calculate at time £ 2 Œ0; 2/ the backward reach set W Œ£ D
W .£; 2; M / from target set M D fx W hx  m; M.x  m/i1=2  1g under state
constraint jx2 Œt j  1; t 2 Œ£; 2.
Such calculations may be done using schemes of Sect. 7.2.4 or following funnel
equation (8.43).
Based on such a scheme Fig. 8.1 shows a particular version of this example
worked out by Ian Mitchell, with solution approximated numerically, using level
set methods of [221, 244]. The chosen parameters are
 
12 4
k D 1; m D .0; 0:8/0 ; M D :
4 12

This figure shows final reach set at t1 D 2; without constraint (dashed line) and
with convex constraints jx2 .t /j  1 (thick solid line). The constrained reach set is
much smaller than the intersection of the state constraints and the unconstrained
reach set.
To present system (8.50) in the form of (8.41), we apply transformation z.t / D
G.0; t /x.t /
 
1 t
G.0; t / D ;
0 1
8.4 Obstacle Problems 333

keeping for variable z the original notation x. We then come to equations


xP 1 D t u
;
xP 2 D u

or

xP D b.t /u; b 0 .t / D .t; 1/

which is of the form (8.41). Hence the related funnel equation (8.43) will be


lim ¢ 1 h W Œt  ¢; .W Œt   ¢B.t /P .t // \ Y .t  ¢/ D 0; (8.51)
¢!0

t0  t  t1 ; W Œt1  D M :

Here the state constraint Y is a stripe which may be approximated internally by an

ellipsoid E .0; Y– /, where ¡.l j E .0; Y– / D .©2 l12 C l22 /1=2 .
Exercise 8.3.7. Present W Σ as an intersection of several ellipsoids.
Exercise 8.3.8. (a) Indicate the control that connects a given point on the
boundary of W Σ with a point given on the boundary of M at time t1 :
(b) Solve similar problem when both points need not be on the related boundaries.

8.4 Obstacle Problems

8.4.1 Complementary Convex State Constraints:

The Obstacle Problem

Consider function §.t; x/ D hx  z; Z 1 .t /.z  x/i with Z .t / D fx W §.t; x/  1g.

Then inclusion x.t / 2 Z .t / will be referred to as the complementary convex state
constraint. The next problem has such type of state constraint.
Problem 8.4.1. Given time interval Œ£; t1 , and functions §.t; x/; ®M .x/, find set
 
Wc Œ£ D x W 9u./; 8t 2 Œ£; ª §.t; xŒt /  1; ®M .xŒª/  1I xŒ£ D x :

Wc Œ£ D fx W V c .£; x/  1g is the level set for the value function

 ˇ 
ˇ
V c .£; x/ D min max maxf§.t; xŒt / C 2 j t 2 Œ£; t1 g; ®M .xŒt1 / ˇˇ xŒ£ D x :
u t
334 8 Trajectory Tubes State-Constrained Feedback Control

This is the set of points x from which some controlled trajectory xŒt  D x.t; £; x£ /,
starting at time £, reaches M at time t D t1 under state constraint §.t; xŒt /  1 or,
equivalently, xŒt  2 Z .t /; 8t 2 Œ£; ª.
Note that Problem 8.4.1 requires x.t / to stay outside the interior of convex
compact set Z .t /. So function V c .t; x/ and sets W Œt  in general lack the property
of convexity.
The value function V c .t; x/ also satisfies a semigroup property similar to (8.43).
This gives, for any u.s/ 2 E .p.s/; P .s//; s 2 Œt; t C ¢; ¢  0; the inequality

max maxf§.s; xŒs/ C 2 j s 2 Œt; t C ¢g  V c .t; x/;
s
ˇ 
ˇ
c ˇ
V .t C ¢; xŒt C ¢/  V .t; x/ ˇ xŒt  D x  0;
c

with equality reached along the optimal trajectory, and

§.t; x/ C 2  V c .t; x/:

We assume that V c .t; x/ and §.t; x/ are differentiable.

Case (a) . Assuming

§.t; x/ C 2 < V c .t; x/;

we have

Vtc .t; x/ C minhVxc ; f .t; x; u//i D 0; u 2 E .p.t /; P .t //: (8.52)

u

Case (b). Assuming

§.t; x/ C 2 D V c .t; x/;

with u0 .t /; x 0 .t / being the optimal solution to Problem 8.4.1, we have, through

reasoning similar to the above

maxfH .t; xŒt ; Vtxc .t; xŒt /; u/; H .t; xŒt ; §tx .t; xŒt /; u/  0;

0 D H .t; x 0 Œt ; Vtxc .t; x 0 Œt ; u0 /  H .t; x 0 Œt ; §tx .t; x 0 Œt /; u0 /: (8.53)

The boundary condition is

V c .t1 ; x/ D maxf§.t1 ; x/ C 2; ®M .x/g: (8.54)

8.4 Obstacle Problems 335

Theorem 8.4.1. The solution to Problem 8.4.1 is given by

Wc Œ£ D fx W V c .£; x/  1g;

in which V c .£; x/ is the solution to (8.52), (8.54).

8.4.2 The Obstacle Problem and the Reach-Evasion Set

The next obstacle problem combines the previous two state constraints, namely
those of Problems 8.3.1 and 8.4.1. We thus have
Problem 8.4.2. Given time interval Œ£; t1  and functions ®.t; x/; §.t; x/; ®M .x/,
find the set

Wo Œ£ D x 2 Rn W 9u./; 8t 2 Œ£; t1 ; ®.t; xŒt /  1;

§.t; xŒt /  1; ®M .xŒt1 /  1I xŒ£ D x :

Hence Wo Œ£ D fx W V o .£; x/  1g is the level set of the value function

n
V o .£; x/ D min max maxfmaxf®.t; xŒt /; §.t; xŒt / C 2g j t 2 Œ£; t1 g ;
u t
ˇ 
ˇ
®M .xŒt1 / ˇˇ xŒ£ D x :

This is the set of points x from which some controlled trajectory xŒt  D x.t; £; x/,
starting at time £, reaches M at time t D t1 and also satisfies the state constraints
xŒt  2 Y .t /; xŒt  2 Z .t /; 8t 2 Œ£; t1 .
Set W Σ is known as the reach-evasion set [203, 204].
Finally, in Problem 8.4.2, the value function V o .t; x/ satisfies an analog of
Lemma 8.3.1. We have, for any u.s/ 2 P .s/; s 2 Œt; t C ¢, ¢  0, the relations

max maxfmaxf®.s; xŒs/; §.s; xŒs/ C 2g j s 2 Œt; t C ¢g  V o .t; x/;
s

V .t C ¢; xŒt C ¢/  V .t; x/ j xŒt  D x  0;
o o
(8.55)

with equality along the optimal trajectory, and

®.t; x/  V o .t; x/; §.t; x/ C 2  V o .t; x/:

336 8 Trajectory Tubes State-Constrained Feedback Control

Case (a-o). Assuming V o .t; x/ > ®.t; x/; V o .t; x/ > §.t; x/ C 2; we have
the HJB equation

Vto .t; x/ C minhVxo ; f .t; x; u/i D 0; u 2 P .t /I (8.56)

u

Case (b-o). Assuming V o .t; x/ D ®.t; x/, but V 0 .t; x/ > §.t; x/ C 2; we have

maxfH .t; x; Vtxo ; u/; H .t; x; ®tx ; u/g  0;

0 D H .t; x .0/ Œt ; Vtxo .t; x .0/ Œt /; u0 /  H .t; x .0/ Œt ; ®tx .t; x .0/ Œt /; u0 / (8.57)

and lastly,
Case (c-o). Assuming V o .t; x/ D §.t; x/ C 2; but V 0 .t; x/ > ®.t; x/; we have

maxfH .t; x; Vtxo ; u/; H .t; x; §tx ; u/g  0;

0 D H .t; x; Vtxo ; u0 /  H .t; x; §tx ; u0 /: (8.58)

The boundary condition is

V o .t1 ; x/ D maxf®.t1 ; x/; §.t1 ; x/ C 2; ®M .x/g (8.59)

Theorem 8.4.2. The solution to Problem 8.4.1 is given by

Wo Œ£ D fx W V o .£; x/  1g;

in which V o .£; x/ is the solution to (8.56)–(8.59).

Here we have presumed Z .t / \ Y .t / 6D ;; 8 t 2 Σ; t1 .

8.4.3 Obstacle Problem: An Example

Here we consider an illustrative example for the obstacle problem worked out by I.
Mitchell. We introduce a complementary convex constraint of Problem 8.4.1, then
add a convex constraint coming to Problem 8.4.2. On a finite interval t 2 Œ0; t1  we
consider system

xP 1 D u1 ; xP 2 D u2 ; (8.60)

with controls jui j  k; i D 1; 2; k  1:

Now let m D .0; m2 / be a terminal point, with m2 > 1 and ®M .x/ D hx  m;
x  mi; ®.x/ D x2 ; §.x/ D .x12 C x22 /. Our objective is to calculate at time
8.4 Obstacle Problems 337

£ 2 Œ0; ª/ the backward reach set W Œ£ D W .£; t1 ; M / from the terminal set
M D fx W hx  m; x  mi1=2  1g under state constraints ®.x/  2; §.xŒt / 
1; t 2 Σ; t1 .
For our system (8.60) we can calculate the related value function V o .t; x/ as
n
V .£; x/ D min max maxfmaxf®.xŒt /  1;
u t
ˇ 
®2 .xŒt / C 2g j t 2 Œ£; t1 g; ®M .t1 / ˇ xŒ£ D x : (8.61)

We then have Wo Œ£ D fx W V o .£; x/  1g.

A calculation of this function through methods of convex analysis is given in
detail in [161].
The calculations through a level set algorithm, due to Mitchell, are indicated in
Fig. 8.2.
This figure shows a particular version of this example approximated numerically
in the same manner as for Fig. 8.1. The parameters are

k D 1; m D .0; 1:3/0 :

k = 1, m = [ 0 1.3 ]

2.5

1.5

1
x2

0.5

−0.5

−1

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

x1

Fig. 8.2 Growth of reach W Œt  set subject to convex and nonconvex constraints (solid lines; final
time is shown thicker). The target set (dotted circle) and constraints (shaded regions) are shown as
well. The unconstrained reach set at the same final time is shown for comparison (dashed line)
338 8 Trajectory Tubes State-Constrained Feedback Control

The figure shows the final reach set at t1 D 1 with no constraint (dashed line)
and subjectqto the combination of convex constraint jx2 Œt j  2 and nonconvex
constraint x12 C x22  1 (thick solid line).

8.4.4 Closed-Loop Control for the Obstacle Problem

Once the backward reachability sets Wo Œt  are described, one may proceed to the
solution of the control synthesis problem, to obtain the closed-loop control.
Problem 8.4.3. Given set Wo Σ; find control strategy ue .t; x/.Ue .t; x// that steers
system (8.33) from any position f£; xg; x 2 Wo .t / to a position ft1 ; x.t1 /g under
following additional constraints:
(i=1) ®.t; x.t //  1; t 2 Œ£; t1 I ®M .x.t1 /  1;
(i=2) §.t; x.t //  1; t 2 Œ£; t1 I ®M .x.t1 //  1;
(i=3) ®.t; x.t //  1; §.t; x.t //  1; t 2 Œ£; ªI ®M .x.ª//  1.
Each of the strategies ue .t; x/.Ue .t; x// may be sought for directly, from the
respective HJB-type equations or inequalities given above for calculating Vo .t; x/.
However we will apply the generalized “aiming” scheme, used in Sect. 8.1.2 and
introduced earlier, for systems without state constraints, in Sect. 2.6 (see [123,247]).
Namely, considering function Vo .t; x/ D d 2 .x; Wo .t //, introduce either single-
valued strategies

ue .t; x/ 2 Ue .t; x/ D

arg minfexp.2œt /h.Vo /x .t; x/; f .t; x; u/i j u 2 P .t /g; (8.62)

œ > 0, or the set-valued strategies

Ue .t; x/ D Ue .t; x/;

depending on the type of system and the definition of solutions used. Here œ is the
Lipschitz constant in ft; xg for function f .
Theorem 8.4.3. The closed-loop strategy Ue .t; x/ for Obstacle Problem 8.4.2 is
given by (8.62).
The problem is that the proposed strategy ue .t; x/ or Ue .t; x/ must satisfy in
some appropriate sense the equation

xP D f .t; x; ue .t; x//; (8.63)

8.4 Obstacle Problems 339

or the differential inclusion

xP 2 f .t; x; Ue .t; x//: (8.64)

For a general nonlinear system of type (8.63) the solution may be defined as
a “constructive motion” introduced in [123], while in the case of linear systems
with convex compact constraints on the controls the solutions Ue .t; x/ may be
taken in the class of upper semi-continuous set-valued strategies with synthesized
system (8.64) treated as a differential inclusion [121, 123, 174].
Chapter 9
Guaranteed State Estimation

Abstract This chapter deals with the problem of set-membership or “guaranteed”

state estimation. The problem is to estimate the state of a dynamic process from
partial observations corrupted by unknown but bounded noise in the system and
measurement inputs (in contrast with stochastic noise). The problem is treated in
both continuous and discrete time. Comparison with stochastic filtering is also
discussed.

Keywords State estimation • Bounding approach • Information states • Informa-

tion tubes • Hamiltonian techniques • Ellipsoidal approximations

The problem of model and system state estimation through incomplete observa-
tions under non-probabilistic noise (the “theory of guaranteed estimation”) was
introduced and developed in [25, 45, 119, 135–137, 210, 241, 271]. Various versions
of the problem had been worked out using different tools and serving numerous
applications. The specifics of this problem are that the system operates under
unknown but bounded input disturbances while the available observations, corrupted
by similar set-membership noise, satisfy the measurement equation which yields an
on-line state constraint (in contrast with such constraints given in advance). Hence
the problem to be considered is actually one of finding the reachability set for a
system subjected to state constraints which arrive on-line.
The solution to the state estimation problem (an “observer”) is set-valued, with
output given by “information sets” of states consistent with the system dynamics
and the available measurements. The calculation of such sets—the observer outputs,
which evolve in time as an “information tube,” pose a challenging computational
problem, especially in the nonlinear case.
In this chapter we proceed with a further application of Hamiltonian techniques
indicated previously in Chap. 2 for systems without state constraints and in Chap. 7
for those with state constraints given in advance. This gives the solution to the
guaranteed (set-membership) state estimation problem by describing information
tubes using HJB equations. The idea applied here is that these information sets may
be expressed as level sets of related value functions, called “information states”,
which are the solutions to special types of the HJB equation. Such a turn yields

© Springer International Publishing Switzerland 2014 341

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__9
342 9 Guaranteed State Estimation

a deeper insight into the investigated problem and opens new routes for designing
computational algorithms. It applies to both linear and nonlinear systems.
Since calculating solutions to HJB equations is not simple, we proceed, as in
Sect. 7.1, with substituting the original HJB equations by variational inequalities
due to a related comparison principle. As in Sects. 5.1 and 7.1, the comparison
theorems generated now by the guaranteed state estimation (filtering) problem are
applicable to both smooth and non-smooth solutions to related HJB equations. It is
also shown that in case of linear systems with convex constraints on the disturbances
this approach may lead to effective external and internal approximations of the
information sets and tubes using ellipsoidal techniques.
Further on, a later detour is to discrete-time systems introducing the reader to a
collection of formulas useful for computation. This is followed by interrelations
between guaranteed state estimation theory of this book, as taken for linear
systems, and the well-known stochastic approaches to state estimation based on
the Kalman filtering theory. Finally mentioned is the important issue of dealing
with discontinuous measurements which extend previous results of this chapter to a
broader array of applied problems.

9.1 Set-Membership State Estimation: The Problem.

The Information Tube

Consider the n-dimensional system

xP D f .t; x; v/ (9.1)

where function f .t; x; v/ is similar to (1.1), namely, continuous in all variables

and ensuring standard conditions of uniqueness and extendibility of solutions
throughout a finite interval Œt0 ; ª for any initial condition x 0 D x.t0 / 2 Rn , and also
for any admissible disturbances v.t /, restricted by geometrical constraints (“hard
bounds”), so that

v.t / 2 Q .t /; t 2 Œt0 ; ª; and x.t0 / 2 X 0 : (9.2)

Here Q .t / is a multivalued function with values in the set compRq of all compacts
of space Rq , continuous in the Hausdorff metric, and set X 0 is compact. The pair
ft0 ; X0 g is the “initial position” of system (9.1).
We also assume that set F .t; x/ D f .t; x; Q .t // is convex and compact. Then,
due to the indicated properties, the set-valued function F .t; x/ will be Hausdorff-
continuous in all the variables.
The on-line information on vector x is given through observations which arrive
through one of the two following schemes.
9.1 Set-Membership State Estimation: The Problem. The Information Tube 343

Scheme I. State x is measured via continuous observations

y.t / D g.t; x/ C Ÿ.t /; t 2 Œt0 ; ª; (9.3)

where y.t / 2 Rr is the measurement and Ÿ.t /—the unknown but bounded
disturbance (“noise”), which is restricted to the inclusion

Ÿ.t / 2 R .t /; t 2 Œt0 ; ª; (9.4)

while the properties of set R .t / are similar to Q .t /. Function g.t; x/ is taken to

be continuous over both variables.
Scheme II. State x is measured via discrete observations

y.£i / D g.£i ; x.£i // C Ÿ.£i /; (9.5)

where y.£i / 2 Rr ; x.£i / 2 Rn , £i < £iC1 , £i 2 Œt0 ; ª, and noise Ÿ.£i / is
unknown but bounded, restricted by (9.4). The measurement times £i are taken
to be given.
Remark 9.1.1. An interesting situation is the case of communication-type con-
straints where the measurement signals y.£i / are assumed to arrive at random time
instants £i distributed, (say) as a Poisson process, with frequency œ > 0. Such case
lies beyond the scope of this book and is treated in [56].
Taking Scheme I, we assume that given are: the initial position ft0 ; X 0 g, functions
f .t; x; v/; g.t; x/; set-valued functions Q .t /; R .t /, and available on-line measure-
ments y£ .¢/ D y.£ C ¢/ (¢ 2 Œt0  £; 0).
Definition 9.1.1. The information set X .£; y£ ; / D X Œ£ of system (9.1)–(9.4) is
the collection of all its states xŒ£ D x.£; t0 ; x.t0 //; x.t0 / 2 X 0 , consistent with its
parameters and with observed measurements y£ .¢/. 
Thus, the actual on-line position of the system may be taken as the pair f£; X Œ£g.
Problem 9.1.1. Calculate sets X Œt ; t 2 Œt0 ; ª; and derive an equation for their
evolution in time.
The set-valued function X Œt  is called the information tube. Such tubes are estimates
of the system dynamics due to on-line measurements. They will be calculated
through two approaches—Hamiltonian techniques and set-valued calculus.
A similar definition and a problem are introduced within Scheme II under
available on-line measurements yŒt0 ; £ D fy.£i / W £i 2 Œt0 ; £g. Then one may
define a similar information set Xd .£; yŒt0 ; £/ D Xd Œ£. The information tube Xd Œt 
may turn out to be discontinuous. This depends on the properties of the function
y.t /. Under Scheme I the discontinuities may be caused by discontinuous noise
while under Scheme II they arrive naturally, since measurements y.£i / are made at
isolated times.
Concentrating on Scheme I, note that the information set X Σ is a guaranteed
estimate of the unknown actual state x.£/ of system (9.1), so we always have
344 9 Guaranteed State Estimation

x.£/ 2 X Œ£; whatever be the unknown noise. Given starting position ft0 ; X 0 g and
measurement y.t /; t 2 Œt0 ; £, it makes sense to construct for system (9.1), (9.2) a
forward reachability tube X Œt  D X .t I t0 ; X 0 /; t  t0 ; which consists of all those
solutions to the differential inclusion xP 2 F .t; x/ that emanate from the set-valued
initial position ft0 ; X 0 g and develop in time under the on-line state constraint

g.t; x/ 2 yŒt   R .t /; t  t0 ; (9.6)

generated by the available measurement y.t / D yŒt . Then function X Œt  is precisely

the information tube that solves Problem 9.1.1 of guaranteed filtering.
We now proceed with solving the announced problem.

9.2 Hamiltonian Techniques for Set-Membership

State Estimation

Problem 9.1.1 indicated above is not the one of optimization. But as before we
shall solve it through alternative formulations of dynamic optimization. Following
Sect. 7.1.1 these are given in two versions.

9.2.1 Calculating Information Tubes: The HJB Equation

Version A

Problem 9.2.1. Given system (9.1)–(9.3), with available measurement y.s/ D

yŒs; s 2 Œt0 ; t ; and set-valued starting position ft0 ; X 0 g; find function
Z t
V .t; x/ D minfd 2 .x.t0 /; X 0 / C d 2 .yŒs  g.s; x.s//; R .s//ds j x.t / D xg
v t0
(9.7)
over the trajectories of system (9.1), under constraints (9.2).
The information set X Œ£ D X .£; / may then be expressed as the level set

X Œ£ D fx W V .£; x/  0g: (9.8)

Here V .t; x/ is the value function related to Problem 9.2.1. This function V .t; x/
is henceforward referred to as the information state of system (9.1)–(9.3). Its level
set or “cross-section” X Œ£ at level 0, namely (9.8), is the reachability set for (9.1)
under on-line state constraints (9.3), given y.t /. Relation

V .t0 ; x/ D d 2 .x; X 0 / (9.9)

9.2 Hamiltonian Techniques for Set-Membership State Estimation 345

defines the boundary condition for V .t; x/. And as in Theorem 7.1.1 we have the
“semigroup” property

V .t; xj V .t0 ; // D V .t; xjV .£; jV .t0 ; ///; t0  £  t;

which implies the HJB type equation

Vt C maxfhVx ; f .t; x; v/i j v 2 Q .t /g  d 2 .y.t /  g.t; x/; R .t // D 0; (9.10)

with boundary condition (9.9). Note that the term d 2 .y.t /g.t; x/; R .t // 6D 0 only
if y.t /  g.t; x/ is outside of the constraint R .t /.
Here again V .t; x/ is assumed to be differentiable and if not, then (9.10) is
a formal symbolic notation for an equation whose solution should be considered
in a generalized “viscosity” sense (see [16, 17, 50, 80]) or equivalent “minimax”
sense (see [247]). For linear systems with convex value functions V .t; x/ the total
derivative d V =dt D Vt C hVx ; f .t; x; v/i may be substituted for a directional
derivative of V .t; x/ along the direction f1; f .t; x; v/g. This directional derivative
here exists for any direction f1; f g.
The value function V may be also defined under additional assumptions on
smoothness and convexity of the constraints. Given proper continuously differen-
tiable functions ®0 .t; x/; ®.t; Ÿ/, convex in x; Ÿ respectively, the initial set and the
constraint on the disturbance Ÿ in (9.3), may be presented in the form

X 0 D fx W ®0 .t0 ; x/  1g; R .t / D fŸ W ®.t; Ÿ/  1g: (9.11)

The second inequality, generated by the measurement equation, defines an on-

line state constraint for system (9.1). In particular, functions ®0 ; ® may be defined
through equalities

®0 .t0 ; x/ D d 2 .x; X 0 /C1; ®.t; Ÿ.t // D d 2 .Ÿ.t /; R .t //C1; Ÿ.t / D y.t /g.t; x/;

with set-valued mapping R .t / being Hausdorff-continuous and function y.t / being

right-continuous. As before, here d.y  ; Y / D minfky   yk y 2 Y g is the
Euclidean distance function for set Y . Such types of constraints are common for
both linear and nonlinear systems. Using these constraints in what follows will allow
to present the solutions in a more transparent form.

Version B

Assume given is the realization y./ D yΠof the observation y on the interval
Œt0 ; t .
Problem 9.2.2. Given are system (9.1) and starting position ft0 ; X 0 g; according
to (9.11). Find the value function
346 9 Guaranteed State Estimation

V .1/ .t; x/ D

D minf®0 .t0 ; xŒt0 / j xŒt  D x; ®.s; y.s/  g.s; xŒs//  1; s 2 Œt0 ; t g: (9.12)
v

.1/ .1/
Denoting hVx ; f .t; x; v/i D H .t; x; Vx ; v/; consider the equation

.1/
Vt CmaxfH .t; x; Vx.1/ ; v/ j v 2 Q ™ .t; x/g D 0; V .1/ .t0 ; x/ D ®0 .t0 ; x/ (9.13)
v

where

Q .t /; ™.t; x/ < 1;
Q ™ .t; x/ D
Q .t / \ fv W d ™.t; x/=dt jv  0g; ™.t; x/  1

and ™.t; x/ D ®.t; yŒt   g.t; x//.

Theorem 9.2.1. Function V .1/ is a solution to Eq. (9.13), with indicated boundary
condition.
The proof of this assertion follows the lines of (7.9)–(7.15) and the proof of
Lemma 7.1.4. Similar to our previous conclusions we have
Lemma 9.2.1. If for y D yŒs (s 2 Œt0 ; t ) Eq. (9.13) has a solution V .1/ .t; x/,
(classical or generalized), then the following equality is true

X Œ£ D fx W V .1/ .£; x/  1g: (9.14)

Remark 9.2.1. The suggested Versions A and B lead to different HJB equations
and serve to solve the original Problem 9.1.1 under different assumptions on the
problem data. Namely, Version-B requires additional assumptions on smoothness
and convexity of the state constraints produced by the measurement equation. In
the general nonlinear case this difference may affect the smoothness properties of
respective solutions V .t; x/; V .1/ .t; x/. On the other hand, the Comparison Principle
of the next subsection does not depend on the smoothness properties of original HJB
equations for Versions A and B.

9.2.2 Comparison Principle for HJB Equations

Version A

Introduce the notation

H.t; x; p/ D maxfhp; f .t; x; v/ijv 2 Q .t /g  d 2 .y.t /  g.t; x/; R .t //:

9.2 Hamiltonian Techniques for Set-Membership State Estimation 347

Then the related HJB equation (9.10) transforms into

Vt C H.t; x; Vx / D 0; V .t0 ; x/ D d 2 .x; X 0 /: (9.15)

We will now obtain an external approximation of X Œt  which in the linear case will
be exact. We proceed as follows.
Assumption 9.2.1. Given are functions H.t; x; p/, wC .t; x/ 2 C1 and .t / 2 L1 ,
which satisfy the inequalities

H.t; x; p/  H.t; x; p/; 8ft; x; pg; (9.16)

wC C
t C H.t; x; wx /  .t /: (9.17)

Theorem 9.2.2. Suppose H.t; x; p/; wC .t; x/; .t / satisfy Assumption 9.2.1. Then
the following estimate for the information set X Œt  holds:

X Œt  XC Œt ; (9.18)

where
 Z t 
XC Œt  D x W wC .t; x/  .s/ds C maxfwC .t0 ; x/ j x 2 X 0 g : (9.19)
t0

The proof is similar to Sect. 7.1.2, Theorem 7.1.1.

Remark 9.2.2. Condition (9.19) may be complemented by the following:

wC .t0 ; x/  V .t0 ; x/:

Then the last term in (9.19) may be substituted by V .t0 ; x/.

Version-B

After introducing notation

H1 .t; x; p; p0 ; œ/ D maxfhp; f .t; x; v/i  œhp0 ; f .t; x; v/i jv 2 Q .t /g;

the corresponding “forward” HJB equation (9.13) transforms into

.1/
Vt C œ™t C H1 .t; x; Vx.1/ ; ™x ; œ/ D 0; V .1/ .t0 ; x/ D ®0 .t0 ; x/ (9.20)

where the multiplier œ is to be found from the condition

d ™.t; x/=dt j(9.1) D ™t C h™x ; f .t; x; v/i  0; (9.21)

348 9 Guaranteed State Estimation

which allows one to calculate X Œt  D X .t; t0 ; X 0 / as the level set X Œ£ D fx W

V .1/ .£; x/  1g: This property is independent of whether V .1/ is a classical or a
generalized solution to Eq. (9.20). The comparison theorem and the respective proof
are similar to Sect. 7.1.2, Theorem 7.1.1.
We now pass to the class of linear systems with convex constraints where the
previous results may be developed with greater detail.

9.2.3 Linear Systems

Consider system (9.1) to be linear, so that

xP D A.t /x C C.t /v; (9.22)

under same constraint (9.2) on v. The measurement equation is also taken to be

linear

y.t / D G.t /x C Ÿ.t /; (9.23)

with constraint (9.4) on Ÿ. The functions Q .t /; R .t / are assumed ellipsoidal-valued,

described by nondegenerate ellipsoids,

.a/ v 2 Q .t / D E .q.t /; Q.t //I .b/ Ÿ.t / 2 R .t / D E .0; R.t //; (9.24)

with matrix-valued functions Q0 .t / D Q.t / > 0; R0 .t / D R.t / > 0 and Hausdorff-

continuous. The bound on the initial vector x.t0 / is also an ellipsoid:

x.t0 / 2 X 0 D E .x 0 ; X 0 /;

so that the starting position is ft0 ; E .x 0 ; X 0 /g.

With y.t / D yŒt  being the available measurement that arrives on-line, we come
to the on-line state constraint

G.t /x.t / 2 yŒt  C E .0; R.t // D Ry .t /: (9.25)

To solve Problem 9.1.1 for the linear case we now approach the versions for
Problems 9.2.1 and 9.2.2 of dynamic optimization.
Denote

k 2 .t; x/ D hyŒt   G.t /x; R1 .t /.yŒt   G.t /x/i;

and

¦.t / D 1; if k 2 .t; x.t //  1 > 0I ¦.t / D 0; if k 2 .t; x.t //  1  0:

Further denote .f .x//C D f .x/ if f  0 and f .x/ D 0 otherwise.

9.2 Hamiltonian Techniques for Set-Membership State Estimation 349

Version LA

The information set X Œt  may be expressed as the level set

X Œt  D fx W V .t; x/  0g; (9.26)

for value function

Z t
V .t; x/ D minf®0 .x/ C .k 2 .s; x.s//  1/C ds j x.t / D xg; t  t0 : (9.27)
v t0

where ®0 .x/ D .hx  x 0 ; .X 0 /1 .x  x 0 /i  1/C and .k 2 .t; x.t //  1/C D

¦.t /.k 2 .t; x.t //  1/: Then Eq. (9.10) transforms into

Vt C hVx ; A.t /x C C.t /q.t /i C hVx ; C.t /Q.t /C 0 .t /Vx i1=2  ¦.t /.k 2 .t; x/  1/ D 0;
(9.28)
with boundary condition

V .t0 ; x/ D ®0 .x/ D .hx  x 0 ; .X 0 /1 .x  x 0 /i  1/C : (9.29)

Version LB

To track the on-line state constraint generated by measurement y.t / we have to

calculate the total derivative of function k 2 .t; x/ D ™.t; x/ along our system (9.22).
So

d ™.t; x/=dt D ™t .t; x/  2hR1 .t /.yŒt   G.t /x/; G.t /.A.t /x C C.t /v/i

and Eq. (9.13) transforms into

.1/
Vt C Ϫt C maxfhVx.1/ C Ϫx ; A.t /x C C.t /vi j v 2 E .q.t /; Q.t //g D 0;
v

or, with additional notations G 0 .t /R1 .t /y.t / D z.t /; G 0 .t /R1 .t /G.t / D G.t /,
into
.1/
Vt C œ™t C œhVx.1/  2œ.z.t /  G.t //x; A.t /x C C.t /q.t /i C H 0 .t; x; œ/ D 0;
(9.30)
where

H 0 .t; x; œ/ D maxfhVx.1/  2œ.z.t /  G.t /x/; C.t /vi j v 2 E .0; Q.t //g D
v

D hVx.1/  2œ.z.t /  G.t /x/; C.t /Q.t /C 0 .t /.Vx.1/  2œ.z.t /  G.t /x//i1=2 :
350 9 Guaranteed State Estimation

The multiplier œ D œ.t; x/ should be calculated from relation

™t .t; x/  2 hz.t /  G.t /x; A.t /x C C.t /v i  0; (9.31)

where

v D Q.t /C 0 .t /.Vx.1/  2œ.z.t /  G.t /x//H 0 .t; x; œ/1 :

The next move is to approximate the exact solutions given in this subsection by
using the comparison principle. This is done through an application of ellipsoidal
approximations. The next scheme will rely on a deductive approach to the derivation
of ellipsoidal estimates (see also Sect. 5.1.2) in contrast with Chap. 7, where such
estimates were achieved due to inductive procedures introduced in Chap. 3.

9.3 Ellipsoidal Approximations

These approximations are given in two versions.

9.3.1 Version AE

Recalling (9.15) with function

H.t; x; p/ D maxfhp; A.t /x C C.t /vi j v 2 Q .t /g  ¦.t /.k 2 .t; x/  1/

we use relation

hp; C.t /Q.t /C 0 .t /pi1=2  ” 2 .t / C .4” 2 .t //1 hp; C.t /Q.t /C 0 .t /pi; 8p 2 Rn ;
(9.32)
with equality reached for ” 2 .t / D .1=2/hp; C.t /Q.t /C 0 .t /pi1=2 . Denoting

H.t; x; p/ D hp; A.t /x C C.t /q.t /i C ” 2 .t / C .4” 2 /1 hp; C.t /Q.t /C 0 .t /pi

¦.t /.k 2 .t; x/  1/;

we then have

H.t; x; p/  H.t; x; p/

for all values of ft; x; pg.

We shall further approximate the function V .t; x/ in the domain x 2 Rn ; t  t0
by one of type
9.3 Ellipsoidal Approximations 351

w.t; x/ D hx  x  .t /; P .t /.x  x  .t //i C h2 .t /  1; (9.33)

with differentiable h2 .t /; P .t / D P 0 .t / > 0. We assume k 2 .t; x/  1; so that

¦.t / D 1 explaining later that here this is the main case. Then

wt C H.t; x; wx / D

D wt C hwx ; A.t /x C C.t /q.t /iC

Chwx ; C.t /Q.t /C 0 .t /wx i1=2  ¦.t /.k 2 .t; x/  1/ 

 wt C H.t; x; wx / D wt C hwx ; A.t /x C C.t /q.t /iC

C” 2 .t / C .4” 2 .t //1 hwx ; C.t /Q.t /C 0 .t /wx i

¦.t /.hy.t /  G.t /x; R1 .t /.y.t /  G.t /x/i  1/;

and substituting w.t; x/; into these relations we get

wt C H.t; x; wx /  hx  x  .t /; PP .t /.x  x  .t //i

2hxP  .t /; P .t /.x  x  .t //i C 2hP .t /.x  x  .t //; A.t /x C C.t /q.t /iC

C.” 2 .t //1 hP .t /.x  x  .t /; C.t /Q.t /C 0 .t /P .t /.x  x  .t //i C ” 2 .t /

 ¦.t /.hy.t /  G.t /x; R1 .t /.y.t /  G.t /x/i  1/ C ¦.t /d.h2 .t //=dt; (9.34)

with ¦.t / D 1.
Continuing further with ¦.t / D 1, we demand that the right-hand side in (9.34)
is treated as follows. We separately equate with zero the terms with multipliers of
second order in x  x  , then those of first order in the same variable, then free
terms, excluding ” 2 .t / C ¦.t /. We finally observe that the equated elements of the
right-hand side of (9.34) will be zeros if and only if the following equations are true:

PP C P 0 A.t / C A0 .t /P C .” 2 .t //1 P C.t /Q.t /C 0 .t /P  ¦.t /G 0 .t /R.t /G.t / D 0;

(9.35)
xP  D A.t /x  .t /CC.t /q.t /C¦.t /P 1 .t /G 0 .t /R.t /.y.t /G.t /x  .t //; (9.36)

hP2 D hy.t /  G.t /x  .t /; R.t /.y.t /  G.t /x  .t //i; (9.37)

with boundary conditions

P .t0 / D .X 0 /1 ; x  .t0 / D x 0 ; h2 .t0 / D 0:

352 9 Guaranteed State Estimation

Relations (9.33)–(9.37) together with some further calculation bring us the

inequality

d w=dt  ” 2 .t / C ¦.t /: (9.38)

Functions H.t; x; p/; w.t; x/ satisfy Assumption 9.2.1. Hence we are in the position
to apply Theorem 9.2.2. We proceed as follows.
Definition 9.3.1. A measurement y.t / is said to be informative on the interval
Œt; t C ¢; ¢ > 0; if constraint (9.25) is active throughout this interval.
Let X 0 Œs D X 0 .s; t; X Œt / stand for the reachability set at time s  t; for
system (9.22) without state constraints, emanating at time t from X Œt  and recall
that X Œs D X .s; t; X Œt / is the information set gotten under active measurement
output state constraints. Then Definition 9.3.1 means that for all s 2 Œt; t C ¢; it
should be X Œs  X 0 Œs: Namely, taking into account the measurement y.s/ must
diminish set X 0 Œs towards X Œs.
Assumption 9.3.1. The available measurement y.s/ D yŒs is informative on any
interval t < s  t C ¢ taken within Œt0 ; ª.
Remark 9.3.1. Since we are actually approximating the reach set from above,
we always deal with informative measurements which yield solutions that satisfy
Assumption 9.3.1.
This means we further assume ¦.t /  1 throughout Œt0 ; ª.
Integrating inequality (9.38) from t0 to t; along a trajectory xŒs D x.s; t0 ; x 0 /;
xŒt  D x; s 2 Œt0 ; t  that runs as xŒs 2 X Œs; under disturbance v.s/ 2 E .q.s/;
Q.s//; with measurement y.s/ D yŒs under disturbance Ÿ.s/ 2 E .0; R.s//; we get
Z t
w.t; x/  .” 2 .s/ C 1/ds  w.t0 ; xŒt0 / D V .t0 ; xŒt0 /: (9.39)
t0

Further substituting w.t; x/ and taking into account (9.33), we have

Z t
 
hx  x .t /; P .t /.x  x .t //i C h .t /  1 
2
.” 2 .s/ C 1/ds C V .t0 ; x.t0 //:
t0

Then, keeping (9.26) in mind, we come to inequality

Z t
 
hx  x .t /; P .t /.x  x .t //i  1  h .t / C 2
.” 2 .s/ C 1/ds D “.t /; (9.40)
t0

where P .t / depends on parameter ” 2 .t /.

This leads to the final conclusion
9.3 Ellipsoidal Approximations 353

Theorem 9.3.1. An external set-valued estimate of information set X Œt  is the

ellipsoid

E .x  .t /; P 1 .t /“.t //;

defined by (9.40), where functions P .t /; x  .t /; h.t / are described by Eqs. (9.35)–

(9.37) under boundary conditions

P .t0 / D .X 0 /1 ; x  .t0 / D x 0 ; h2 .t0 / D 0:

This estimate is true for any Lebesgue-integrable function ” 2 .t /.

This theorem gives us an array of ellipsoids E” Œt  D E .x  .t /; “.t /P 1 .t //,
parameterized by ”, each of which is an external estimate for X Œt . Moreover, the
schemes of Chap. 3 yield the next property
Lemma 9.3.1. The information set X Œt  D \fE” Œt  j ” 2 ./g.
The exact information set is therefore presented as an intersection of all external
ellipsoidal estimates of the form given above. Among these one may of course seek
for an optimal one or the one most appropriate for a related problem of control.
Remark 9.3.2. (i) Condition ¦.t / D 0 indicates that measurement yŒt  brings no
innovation into the estimation process and is redundant, so X 0 Œs D X Œs. Then,
instead of the information set under on-line state constraint, the formulas achieved
above describe the reachability set X 0 Œs without state constraints. You just have to
take ¦.t / D 0; h.t / D 0 in all these relations.

9.3.2 Version-BE

Consider Eq. (9.30). Applying inequality (9.32) to H 0 .t; x; œ/; we get

.1/
Vt C œ™t C œhVx  2œ.z.t /  G.t /x/; A.t /x C C.t /qi C H 0 .t; x; œ/ 

Cœ™t CœhVx 2œ.z.t /G.t /x/; A.t /xCC.t /qiC” 2 .t /C.4” 2 .t //1 .H 0 .t; x; œ//2 :
.1/
 Vt
(9.41)

Then we again take w.t; x/ in the form (9.33). Substituting w.t; x/ into the right-
hand side of the last relation we further follow the previous scheme of Version-A
now applying it to (9.41). This time we come to equations

PP C P .A.t / C œG.t //0 C .A.t / C œG.t //P C

C .” 2 .t //1 P C.t /Q.t /C 0 .t /P C œ2 .t /G 0 .t /R1 .t /G.t / D 0; (9.42)

354 9 Guaranteed State Estimation

xP  D A.t /x  .t / C C.t /q.t /  œ.z.t /  G.t /x  .t //; (9.43)

hP2 D hz.t /  G.t /x  ; R1 .t /.z.t /  G.t /x  /i; (9.44)

with boundary conditions

P .t0 / D .X 0 /1 ; x  .t0 / D x 0 ; h2 .t0 / D 0:

These equations should be taken with any multiplier œ < 0; that satisfies (9.31).
The set of all such multipliers is further denoted as ƒ. Therefore now P .t / D
P”œ .t /. The further procedures are similar to Version AE, transforming P into
P”œ .t / and thus allowing to formulate in the next form.
1
Theorem 9.3.2. The set X Œt  E .x  .t /; “.t /P”œ .t //; where

E .x  .t /; “.t /P”œ
1
.t // D fx W hx  x  .t /; “1 .t /P”œ .t /.x  x  .t //i  1g;

and functions P”œ .t /; xP  .t / are described by Eqs. (9.42)–(9.44), with ” 2 .t / > 0 and
œ 2 ƒ.

9.3.3 Example: Information Set for a Linear System

Given is the system

xP 1 D x2 ;
(9.45)
xP 2 D x1 C v; jvj  1

at t 2 Œ0; ª, with observation

y.t / D x1 C Ÿ.t /; Ÿ.t / 2 R D Œ1; 1 (9.46)

and initial set

˚ ˇ 
x.0/ 2 X 0 D x ˇ x12 C x22  1 : (9.47)

Following (4.5) the reachability set without measurement state constraint will be
described as X Œt  D X .t; 0; X 0 / D X0 Œt  C Xv Œt  where
 
cos.t  s/ sin.t  s/
X0 Œt  D G.t; 0/X ; G.t; s/ D
0
 sin.t  s/ cos.t  s/
9.4 Discrete Observations 355

and
Z t
Xv Œt  D fx W x D G.t  s/Q .s/dsg; Q .s/ D Q D fq W q D bv; jvj  1g; b 0 D f0; 1g:
0

The measurement state constraint is x.t / 2 Y .t / D fx W x1 .t / 2 y.t /  R g.

Thus the reachability set Xy Œt  under constraint Y .t / is the solution to the funnel
equation

lim ¢ 1 h.Xy Œt C ¢; f.I C ¢A/Xy Œt  C ¢ Q .t /g \ Y .t // D 0; Xy Œ0 D X 0 :

¢!0C0

The illustrations below are done for y.t /  0. Then Y .t / D R and the recurrent
relation for calculating Xy Œt  is

Xy Œt C ¢ D ..I C ¢A/Xy Œt  C ¢ Q .t // \ R C o.¢/:

The boundary of the reachability set X .t / D X .t; 0; X 0 / at time t D  without

measurement constraint in the illustrations is green. The measurement constraint Y
is a stripe bounded by parallel red lines. The other sets are illustrated for measure-
ment y.t /  0 generated under actually realized inputs x 0 D 0; jv.t /j  1; jŸ.t /j 
1, which we do not know. Then, at t D  the boundary of the exact reachability
set Xy Œt˚ at t D  under such measurement  ˚ consists of four parts: two circu-

lar arcs .1  2 cos ™; 2 sin ™/ ; ™ 2 Œ0; 2  , .2 cos ™  1; 2 sin ™/ ; ™ 2 Œ0; 2  and
two intervals f1g  Œ2; 0, f1g  Œ0; 2 located on the red lines. This boundary is
shown in orange. Also shown are the ellipsoidal approximations of Xy Œ.
Exercise 9.3.1. Find the boundaries for sets X Œt  (green) and Xy Œt  (orange) shown
in Fig. 9.1 by straightforward calculation.
We now make a detour to systems with discrete observations under continuous
and discrete dynamics. Related formulas may be useful for calculations.

9.4 Discrete Observations

9.4.1 Continuous Dynamics Under Discrete Observations

Let y.£i /; £i 2 Œt0 ; t  ; i D 1; : : : ; N.t /; be the sequence of measurements (9.5)

for system (9.1). Consider
Problem 9.4.1. Given system (9.1), (9.5) with starting position ft0 ; X 0 g and mea-
surements y.£i /, find function
356 9 Guaranteed State Estimation

4 4
3 3
2 2
1 1
0
x2

x2
0
−1 −1
−2 −2
−3 −3
−4 −4
−4 −3 −2 −1 0 1 2 3 4 −4 −3 −2 −1 0 1 2 3 4
x1 x1
4
3 8
2 6
1 4
0 2
x2

x2

−1 0

−2 −2
−4
−3
−6
−4
−4 −3 −2 −1 0 1 2 3 4 −6 −4 −2 0 2 4 6 8
x1 x1

Fig. 9.1 Reachability set X Πwithout constraint Y in green. Reachability set Xy Πwith
constraint Y in orange. State constraint Y in red

X
N.t/
V .t; x/ D minfd 2 .x.t0 /; X 0 / C d 2 .y.£i /  g.£i ; x.£i //; R .£i // j x.t / D xg:
v
iD1
(9.48)
Following (9.26) , (9.25) denote

X Œ£ D fx W V .£; x/  0g; Xy Œ£ D fx W g.£; x/ 2 y.£/  R .£/g:

The solution to the last problem is to be taken as a recurrent procedure with V .s; x/
being the solution to equations

Vs C maxfhVx ; f .s; x; v/i j v 2 Q .s/g D 0; V .t0 ; x/ D d 2 .x; X 0 /; s 2 Œt0 ; £1 ;

so that Xd Σ1  D X Σ1  \ Xy Σ1  and with further relations

Vs CmaxfhVx ; f .s; x; v/i j v 2 Q .s/g D 0; s 2 Œ£i ; £iC1 ; V .£i ; x/ D V .t; x/jtD£i ;

(9.49)
9.4 Discrete Observations 357

and

Xd Œ£iC1  D X Œ£iC1  \ Xy Œ£iC1 ; X Œ£iC1  D fx W V .£iC1 ; x/  0g: (9.50)

This results in
Theorem 9.4.1. The information set for Problem 9.4.1 is

Xd Œt  D X .t; £N.t/ ; Xd .£N.t/ //; (9.51)

where with t D £N.t/ we have Xd .t / D X .£N.t/ /.

For the linear case, keeping in mind the formal relation

X
N.t/ Z t X
N.t/
d 2 .y.£i /  g.£i ; x.£i //; R .£i // D d 2 .y.s/  g.s; x.s//; R .s// •.s  £i /ds;
iD1 t0 iD1

we may derive for V .t; x/ a related HJB-type equation, similar to (9.10), presenting
it in symbolic form as an equation in distributions, [242],

X
N.t/
Vs CmaxfhVx ; f .s; x; v/ijv 2 Q .s/g •.s£i /d 2 .y.s/g.s; x.s//; R .s// D 0;
iD1
(9.52)
seeking V .t; x/, according to the theory of distributions, as a weak solution to this
equation.
In the last case the formal procedures for deriving tight external ellipsoids may
follow those of Version-A. However, here we indicate a slightly different route by
directly applying Theorem 9.4.1. Namely, for s 2 Œt0 ; £1 ; take

PP C PA.s/ C A0 .s/P   2 .s/P  . 2 .s//1 P C.s/Q.s/C 0 .s/P D 0; P .t0 / D .X 0 /1 ;

xP  D A.s/x  .s/ C C.s/q.s/; x  .t0 / D x 0 ; h.s/  0:

Then at s D £1 we should find an external ellipsoidal approximate EdC Œ£1  D

E .xd .£1 /; Pd .£1 // for
E .x  .£1 /; P .£1 // \ Xy .£1 / E .xd .£1 /; Pd .£1 //;
continuing at s > £1 with P .£1 C 0/ D Pd .£1 /; x  .£1 C 0/ D xd .£1 /.
Proceeding further, we have, for s 2 .£i ; £iC1 :

PP CPA.s/CA0 .s/P  2 .s/P . 2 .s//1 P C.s/Q.s/C 0 .s/P D 0; P .£i C0/ D Pd .£i /;

xP  D A.s/x  .s/ C C.s/q.s/ D 0; x  .£i C 0/ D xd .£i /;

358 9 Guaranteed State Estimation

where

EdC Œ£iC1  D E .xd .£iC1 /; Pd .£iC1 //

E .x  .£i /; P .£i // \ Xy .£i /; i D 1; : : : ; N.t /  1;

and the array of external ellipsoids EdC may be found through one of the schemes
given in Chap. 3, Sect. 3.2.
The treatment of ellipsoidal techniques for internal approximations of informa-
tion sets in linear systems may be achieved along the schemes of Sects. 3.7 and 3.12.

9.4.2 Discrete Dynamics and Observations

A frequent approach is to restrict the investigation to discrete-time models of system

dynamics and observations. Here are the related equations.

Discrete-Time Processes

Consider a multistage process described by a recurrent equation

x.k C 1/ D f .k; x.k// C C.k/v.k/; k  k0 ; x.k0 / D x 0 ; (9.53)

Here f .k; x/ is a given map from N  Rn into Rn (N is the set of integers), C.k/
are given matrices. The vector-valued inputs v.k/ are taken in Rp ; p  n. These
vectors, together with x 0 , are unknown but bounded, with hard bounds

x 0 2 X0 ; v.k/ 2 P.k/; (9.54)

where X0 , P.k/ are convex, compact sets in spaces Rn and Rp , respectively.

The available information on x.k/ is confined to measurement outputs given by

y.k/ D G.k/x.k/ C Ÿ.k/; k D k0 C 1; : : : ; N; (9.55)

where observations y.k/ 2 Rm and matrices G.k/ are known while measurement
disturbances Ÿ.k/ are unknown but bounded, with

Ÿ.k/ 2 Q.k/; (9.56)

where Q.k/  Rm is convex and compact.

Here yŒk; l D fy.k/; : : : ; y.l/g will stand for the sequence of measurements
that arrive due to Eq. (9.55) throughout stages s, from k to l. Similarly hŒr; s D
fh.r/; : : : ; h.s/g will stand for a sequence of vectors hi ; i D r; : : : ; s, taken,
respectively, from set-valued sequence FŒr; s D fF.r/; : : : ; F.s/g . Then hŒr; s 2
FŒr; s will stand for a sequence of inclusions h.i / 2 F.i /, i D r; : : : ; s. Hence, with
q q q
h.i / 2 Rq we will have hŒ1; s 2 Rqs D R1  : : :  Rs , where Ri D Rq , for
i D 1; : : : ; s.
9.4 Discrete Observations 359

Symbol xŒk D x.k; vŒk0 ; k  1; x 0 / will be the end of trajectory x.j / of (9.53),
that develops through Œk0 ; k with x 0 and vŒk0 ; k  1 given.
As before the information set Xy Œs D Xy .s j t; k; F/ is the collection of all
points xŒs 2 Rn that arrive at time s (k  s) due to such trajectories of system
(9.53) that emanate from state x.k/ D x  ; evolve throughout interval Œk; t  and also
produce the measured realization yŒk; s for some triplet fx  ; v; Ÿg constrained by
inclusions

x  2 F; vŒk; t  1 2 PŒk; t  1; ŸŒk C 1; t  2 QŒk C 1; t : (9.57)

We also denote Xy .t; k; F/ D Xy .t j t; k; F/.

Lemma 9.4.1. Whatever be s, l, £, given realization yŒk; l given, the following
relations are true

Xy .t; k; F/ D Xy .t; s; Xy .s; k; F//; t  s  k; (9.58)

Xy .s j t; k; F/ D Xy .s j t; l; Xy .l j £; k; F//:

The given conditions ensure that the mapping Xy .t; k; F/ W comp Rn ! comp Rn
defines a generalized dynamic system similar to Xy .t; t0 ; X 0 / of Sect. 8.3.1, but
under discrete time.
In the general case sets Xy Œt  defined here may turn out to be nonconvex and
even disconnected. However the next property is true.
Lemma 9.4.2. Let F be a closed, convex set in Rn , and f .k; x/ D A.k/x be a
linear map. Then sets Xy .s j t; k; F/ are closed and convex.
As before, the problem is to describe information sets Xy Œs and the related
information tube Xy Œk; s; s 2 Œk0 ; t —now a sequence of sets Xy Œs.

Multistage Systems: Information Tubes

Returning to system (9.53)–(9.56), consider set Xy Œs D Xy .s; k0 C 1; X0 / omitting

further the lower index y.
Together with XŒs consider two other sets, namely,

X Œs D X .s; k0 C 1; X0 / and X Œs D X .s; k0 C 1; X0 /:

Here are their definitions. Taking

z.k C 1/ 2 f .k; co Z .k// C C.k/P.k/;

y.k C 1/ 2 G.k C 1/z.k C 1/ C Q.k C 1/; Z .k0 / D X0 ; k  k0 ; (9.59)

360 9 Guaranteed State Estimation

where Z .s/ D Z .s; k0 C 1; X0 / D fz.s/g are solutions to (9.59) at stage s 

k0 C 1, we get

X Œs D co Z Œs D co Z .s; k0 C 1; X0 / D X .s; k0 C 1; X0 /:

Similarly, define system

z .k C 1/ 2 co f .k; Z  .k// C C.k/P.k/;

y.k C 1/ 2 G.k C 1/z .k C 1/ C Q.k C 1/; (9.60)

with solution Z  .s/ D fz .s/g D Z  .s; k0 C 1; X0 /, s  k0 C 1, under Z  .k0 / D

X0 . Then we get Z  .s/ D X Œs.
Sets X Œs, X Œs are obviously convex and the next inclusions are true;

XŒs X Œs X Œs: (9.61)

Remark 9.4.1. Note that apart from X Œs, X Œs one may introduce other sequences
which at each respective stage may be chosen as either (9.59), or (9.60).
We first describe sets X Œs D X.s; k0 C 1; X0 /, using notation

Y.k/ D fx 2 Rn j y.k/  G.k/x 2 Q.k/g;

and X? .s; j; F/ as the solution X? .s/ to equation

X? .k C 1/ D co f .k; X? .k// C C.k/P.k/; j k s1

with X? .k0 / D F. Then the recurrent evolution equation will be as follows.

Lemma 9.4.3. Let yŒk0 C 1; k be the sequence of realized measurements y due to
system (9.53)–(9.56). Then the next equalities are true:

X Œk D X .k; k0 C 1; X0 / D X? .k; k  1; X Œk  1/ \ Y.k/; k0  k; (9.62)

The last equation indicates that the innovation introduced by each new
measurement y.k/ arrives through an intersection of set X? .k; k  1; X Œk  1/—
the estimate at stage k before arrival of measurement y.k/ with Y.k/ derived
from y.k/.

A One-Stage System. For further calculation we need some additional details.

Consider the one-stage system (z 2 Rn ; y 2 Rm )

z 2 f .X/ C C P; Gz  y 2 Q; (9.63)
9.4 Discrete Observations 361

and

z 2 co f .X/ C C P; Gz  y 2 Q; (9.64)

S
where f .X/ D ff .x/ j x 2 Xg and as before, co F is the closure of the convex
hull of set F.
Let Z and Z  stand for the sets of all solutions fzg and fz g to the last system.
Obviously we have

Z coZ Z 

where set Z  is convex.

Exercise 9.4.1. Give an example indicating when sets Z; coZ; Z  do not coincide
(see [113]).
Applying methods of convex analysis, we come to the following property.
Lemma 9.4.4. The next relations are true:

Z R.M; f .X//; Z  R.M; co f .X/; 8M 2 Rmn ;

where

R.M; F/ D .In  M G/.F  C P/ C M.y  Q/:

These relations allow to approach exact solutions due to the next assertion, given in
a form similar to Chaps. 3, 7, and 8.
Theorem 9.4.2. The exact relations for Z; Z  are:
\
ZD fR.M; f .X// j M 2 Rmn g; (9.65)
\
Z D fR.M; co f .X// j M 2 Rmn g: (9.66)

Multistage Information Tubes: Evolution Equations

These relations may now be propagated to multistage recurrent equations that

describe information tubes XŒk, X Œk, X Œk. Consider system

Z.k C 1/ D .En  M.k C 1/G.k C 1//ŒF.k; D.k// C C.k/P.k/C

C M.k C 1/.y.k C 1/  Q.k C 1//; (9.67)

\
D.k C 1/ D fZ.k C 1/ j M.k C 1/ 2 Rmn g; D.k0 / D X0 ; (9.68)
362 9 Guaranteed State Estimation

where F.k; D/ is a mapping from N  Rn into Rn . Solving these equations for

k0  k  s and having in mind Theorem 9.4.2, we get

Z.s/ D XŒs; if F.k; D.k// D f .k; D.k//; (9.69)

Z.s/ D X  Œs; if F.k; D.k// D co f .k; D.k//: (9.70)

If we now substitute (9.70) for

\
D.k C 1/ D co fZ.k C 1/ j M.k C 1/ 2 Rmn g; (9.71)

then solutions to (9.67), (9.71) yield the following result

Z.s/ D X Œs; if F.k; D.k// D f .k; D.k//: (9.72)

We finally come to the conclusion

Theorem 9.4.3. (i) Inclusions (9.61) are true.
(ii) Equalities (9.69), (9.70) are true due to system (9.67) and (9.68).
(iii) Equality (9.72) is true due to relations (9.67) and (9.71).
Remark 9.4.2. The last theorem indicates relations that may be used to approach
exact set-valued estimates of XŒk; X Œk; X Œk for k0  k  s. They require
intersections over M at each stage “k” of the evolution process. However the
intersections may all be made at the end k D s of the process, being taken over
all sequences M Œk0 ; s. Such procedures may be especially convenient for linear
systems in conjunction with ellipsoidal calculus.
Exercise 9.4.2. Compare the computational complexity for both types of intersec-
tions indicated in previous remark.
Following the last remark, consider linear map f .k; Z.k// D A.k/Z.k/ D
cof .k; Z.k//. Now take the set-valued equation

Z.k C 1/ D .En  M.k C 1/G.k C 1//.F.k; Z.k// C C.k/P.k//C

C M.k C 1/.y.k C 1/  Q.k C 1//; Z.k0 / D X0 : (9.73)

Denote the solution tube to this equation as

Z.s/ D Z.s; k0 ; F./; M./; X0 /; where F./ D F.k; Z.k//;

M./ D M Œk0 ; s; where M Œk0 ; s D fM.k0 C 1/; : : : ; M.s/g

9.5 Viability Tubes: The Linear-Quadratic Approximation 363

Denoting
\
Z.s; F./; X0 / D fZ.s; k0 ; F./; M./; X0 / j M Œk; s 2 Rmn g; (9.74)

we come to the conclusion

Theorem 9.4.4. For a linear mapping F.k; Z.k// D A.k/Z.k/ , (k D k0 ; : : : ; s),
with convex X0 , we have

Z.s; F./; X0 / D XŒs D X Œs D X Œs: (9.75)

9.5 Viability Tubes: The Linear-Quadratic Approximation

Though the problem considered further is formulated under hard bounds on the
system inputs and system trajectories, the calculation of related trajectory tubes may
be also achieved through quadratic approximations.
Let us return to the problem of finding viability tube Xy Œt  D Xy .t; t0 ; X 0 / for
system

xP 2 A.t /x C Q .t /; x.t0 / 2 X 0 ; (9.76)

G.t /x 2 Y .t /; (9.77)

where set-valued maps Q .t /, Y .t / and set X 0 are the same as in Sect. 9.1.
The support function for convex compact set Xy Œt  may be calculated directly, as
a linear-quadratic problem, similar to Sects. 1.6 and 2.2.This gives

¡.l j Xy Œt / D inff‰t .l; œ.// j œ./ 2 Lm

2 Œt0 ; £g; (9.78)

where
 Z ˇ 
t ˇ
0
‰t .l; œ.// D ¡ S .t0 ; t /l  S .t0 ; £/G .£/œ.£/d £ ˇˇ X0 C
0 0
t0

Z  Z t ˇ 
t ˇ
C ¡ S 0 .£; t /l  S 0 .£; s/G 0 .s/œ.s/ds ˇ Q .£/ d £C
ˇ
t0 £

Z t
C ¡.œ.£/ j Y .£//d £:
t0

We shall rewrite system (9.76), (9.77) in the form

xP D A.t /x C v.t /; x.t0 / D x 0 ; G.t /x D w.t /; (9.79)

364 9 Guaranteed State Estimation

where measurable functions v.t /, w.t / and vector x 0 satisfy the inclusions

v.t / 2 Q .t /; w.t / 2 Y .t /; x0 2 X 0: (9.80)

The viable information tube Xy Œt  may then be described as a multivalued map

Xy .t; t0 ; X 0 / generated by the set (the bundle) of all trajectories x.t; t0 ; x 0 / of
system (9.79) that are consistent with constraints (9.80).
The calculation of Xy Œt  for a given instant t D £ will now run along the
following scheme. Let us fix a triplet k  ./ D fv ./; w ./; x0 g where v ./, w ./,
x0 (t 2 Œt0 ; £) satisfy the constraints (9.80):

k  ./ 2 Q ./  Y ./  X 0 ; Y .t / D y.t /  R .t /; (9.81)

as in (9.2), (9.3).
Instead of handling (9.80) we shall now consider a “perturbed” system, which is

zP D A.t /z C v .t / C ˜.t /; z.t0 / D x 0 C —0 ; t0  t  £; (9.82)

G.t /z D w .t / C Ÿ.t /;

Elements d./ D f—0 ; ˜./; Ÿ./g represent the unknown disturbances bounded
jointly by quadratic inequality
Z £ Z £
0
—0 M —0 C ˜0 .t /R.t /˜.t /dt C Ÿ0 .t /H.t /Ÿ.t /dt  2 ; (9.83)
t0 t0

where fM; R./; H./g 2 = and symbol = stands for the product space

= D Rnn
C  RC Œt0 ; £  RC
nn mm
Œt0 ; £;

with RC rr
Œt0 ; £ denoting the class of all r  r-matrix functions N./ 2 Rrr Œt0 ; £
whose values N.t / are symmetric and positive definite.
For every fixed k  ./, ƒ D fM; R./; H./g and  denote ZŒ£ D Z.£; k  ; ƒ; /
to be the set of all states z.£/ of system (9.82) that are consistent with con-
straint (9.83). The support function of this set is

¡.l j ZŒ£/ D maxfhl; z.£/i j d./ W (9.83)g (9.84)

d./

It is well known that ZŒ£ D Z.£; k  ; ƒ; / is an ellipsoid with center z0 Œ£ D

z0 .£; k  ; ƒ/ that does not depend on  [25, 137]. This can be observed by direct
calculations (see Exercise 1.6.5) which also indicate that z0 Œs satisfies the linear
differential equation
9.5 Viability Tubes: The Linear-Quadratic Approximation 365

zP D .A.s/  †.s/G 0 .s/H.s/G.s//z C †.s/G 0 .s/H.s/w .s/ C v .s/; (9.85)

z.t0 / D x0 ; t0  s  £;

where ƒ D fM; R./; H./g 2 = are fixed and †./ is the matrix solution to Riccati
equation

P D A.s/† C †A0 .s/  †G 0 .s/H.s/G.s/† C R1 .s/;

† (9.86)

†.t0 / D M 1 ; t0  s  £:

Let us now introduce the set

[
Z0 .£; ƒ/ D fz0 .£; k./; ƒ/ j k./ 2 Q ./  Y ./  X 0 g

which is the union of centers z0 Σ over all triplets k./ D k  ./ from (9.81).
Set Z0 .£; ƒ/ is convex and compact, being the reachability set for system (9.85)
under constraints (9.80), or in other words, of the differential inclusion

zP 2 .A.t /  †.t /G 0 .t /H.t /G.t //z C †.t /G 0 .t /H.t /Y .t / C Q .t /; (9.87)

Y .t / D y.t /  R .t /; z.t0 / 2 X 0 ; t0  t  £;

The support function for Z0 .£; ƒ/, which gives a complete description of this set, is
now to be calculated as in Sect. 2.4. It is a closed convex, positively homogeneous
functional

¡.l j Z0 .£; ƒ// D ‡ .£; l; ƒ/: (9.88)

where

‡ .£; l; ƒ/ D .Dƒ l/.£/ C Lƒ1 l

with linear operators Dƒ W Rn ! Lm

2 Œt0 ; £ and Lƒ W L2 Œt0 ; £ ! L2 Œt0 ; £ defined
m m

by relations

.Dƒ l/.t / D G.t / S.t0 ; t /M 1 S 0 .t0 ; £/C
Z t 
C S.s; t /R1 .s/S 0 .s; £/ds l; l 2 Rn ; t0  t  £;
t0

Lƒ œ./ D .K1 C K2 /œ./; œ./ 2 Lm

2 Œt0 ; £;
366 9 Guaranteed State Estimation

Z £
.K1 œ.//.t / D K.t; s/œ.s/ds;
t0

K.t; s/ D G.t /.S.t0 ; t /M 1 S 0 .t0 ; s/C

Z minft;sg
C S.¢; t /  R1 .¢/S 0 .¢; s/d¢/G 0 .s/;
t0

.K2 œ.//.t / D H 1 .t /œ.t /; t0  t  £: (9.89)

Here Lƒ is a nondegenerate Fredholm operator of the second kind, so that functional

‡ .£; l; ƒ/ is defined correctly for all l 2 Rn and ƒ 2 = [110].
Exercise 9.5.1. Calculate given formulas for operators Dœ and Lœ .
Lemma 9.5.1. The following inequality is true

¡.l j Xy Œ£/  ¡.l j Z0 .£; ƒ// (9.90)

for all l 2 Rn and ƒ 2 = .

From here we immediately come to the next assertion.
Corollary 9.5.1. The viable information tube Xy Σ may be estimated from above
as follows
\
Xy Œ£ fZ0 .£; ƒ/ j ƒ 2 =g: (9.91)

Since here we deal with a linear system, the further objective is to emphasize, as
in Theorem 8.2.5 of Sect. 8.2.2, that inclusion (9.91) is actually an equality.
Lemma 9.5.2. Suppose the m  n-matrix G.t / is of full rank: r.G.t // D m for any
t 2 Œt0 ; £. Then for every l 2 Rn the following equalities are true

¡.l j Xy Œ£/ D inff‰£ .l; œ.// j œ./ 2 Lm

2 Œt0 ; £g D (9.92)

D co inff‡ .£; l; ƒ/: j ƒ 2 =g:

Combining formula (9.78), Lemma 9.5.2 and taking into account (9.91) as an
equality, we get
Theorem 9.5.1. Let r.G.t // D m for every t 2 Œt0 ; £. Then
\
Xy Œ£ D fZ0 .£; ƒ/ j ƒ 2 =g: (9.93)
9.6 Information Tubes vs Stochastic Filtering Equations. Discontinuous. . . 367

Theorem 9.5.1 gives a precise description of Xy Œ£ through solutions Z0 .£; ƒ/ of

the linear-quadratic problem (9.82)–(9.83) by varying the matrix parameters ƒ D
fM; R./; H./g in the joint integral constraint (9.83).
Finalizing this section we emphasize that described above is a parameterized
family of set-valued estimators Z0 .£; ƒ/ which together, in view of Theorem 9.5.1,
exactly determine the set Xy Σ.
Theorem 9.5.2. Sets Z0 .£; ƒ/ are the “cuts” (cross-sections) at time £ of the
solution tube (the integral funnel) to the differential inclusion (9.87) where ƒ D
fM; R./; H./g 2 = and †./ is the matrix solution to Riccati equation (9.86).
Each of sets Z0 .£; ƒ/ is a guaranteed external estimator of Xy Œ£. This property
opens the route for parallel calculations that yield description of Xy Σ with high
accuracy.
Remark 9.5.1. The last result of Theorem 9.5.2, together with Theorem 9.5.1,
provides a special structure of matrix functions L./ for these relations. Namely,
here we may set L.t / D †.t /G 0 .t /H.t /G.t /, t0  t  £, where †./ is defined
by (9.86) with ƒ D fM; R./; H./g varying within set =.
Note that several techniques based on using auxiliary uncertain systems under
quadratic integral constraints for various classes of system models have been
discussed in this context in [113, 140, 167].
One of these is the guaranteed state estimation problem of this chapter. Apart
from deterministic techniques in earlier lines of this section the problem may also
be solved through a stochastic scheme which ends up with results of similar type.

9.6 Information Tubes vs Stochastic Filtering Equations.

Discontinuous Measurements

The two approaches to the state estimation problem, namely the stochastic and the
set-membership filtering, may seem rather different. However it turns out that, apart
from differences, there are useful connections between the techniques of calculating
the results.

9.6.1 Set-Valued Tubes Through Stochastic Approximations

Suppose that system (9.76), (9.77) is specified as follows

xP 2 A.t /x C Q .t /; t  t0 ; x.t0 / 2 X 0 ; (9.94)

y.t / 2 G.t /x C R .t /; (9.95)

368 9 Guaranteed State Estimation

where (9.95) describes the measurement (observation) equation while continuous

set-valued function R .t / W T D Œt0 ; t1  ! conv Rm reflects the unknown but
bounded noise w in the observations.
Given measurement y.t / D yŒt , t 2 Œt0 ; £; the guaranteed state estimation
problem is to specify information sets Xy Œt  and their evolution as an information
tube Xy Œ which is to be viable relative to inclusion (9.95), when yŒt  arrives on-
line. The evolution of Xy Œt  was described by relations (9.93), (9.84), (9.86) under
appropriate assumptions on Y .t /.
It is well known, however, that a conventional stochastic filtering technique for
similar linear systems subjected to stochastic Gaussian noise are given by equations
of the “Kalman filter” [108]. Our next question therefore will be to see whether
equations of the Kalman filter type could be also used to describe the deterministic
information tubes Xy Œt  for the guaranteed estimation problem of the above.
This question is justified by the fact that, on the one hand, the tube Xy Œt  D
Xy .t; t0 ; X 0 / may be described through the linear-quadratic approximations of
Sect. 9.5, while, on the other, by the well-established connections between the
Kalman filtering equations and the solutions to the linear-quadratic problem of
control.
Using solutions of the previous section, fix a triplet k./ D k  ./ D
fv ./; w ./; x0 g with k  ./ 2 Q ./  .y./  R .//  X 0 and consider the stochastic


differential equations

d z D .A.t /z C v .t //dt C ¢.t /d Ÿ; (9.96)

dq D .G.t /z C w .t //dt C ¢1 .t /d ˜; (9.97)

z.0/ D x0 C —; q.0/ D 0; (9.98)

where Ÿ, ˜ are standard normalized Brownian motions with continuous diffusion

matrices ¢.t /, ¢1 .t / and det.¢.t /¢ 0 .t // ¤ 0 for all t 2 T , — is a Gaussian vector
with zero mean and variance M  D ¢0 ¢0 0 .
Denoting ¢.t /¢ 0 .t / D R .t /, ¢1 .t /¢1 0 .t / D H  .t / and treating q D q.t / as the
available measurement we may find equations for the minimum variance estimate
z .t / D E.z.t / j q.s/; t0  s  t / (the respective “Kalman filter”).
These are
1
d z .t / D .A.t /  †.t /G 0 .t /H  .t /G.t //z .t /dt C (9.99)

C†.t /G 0 .t /w .t /dt C v .t /dt; z .t0 / D x0 ;

P / D A.t /†.t / C †.t /A0 .t /

†.t (9.100)
1
†.t /G 0 .t /H  .t /G.t /†.t / C R .t /; †.t0 / D M  :

The estimate z .t / thus depends on the triplets k  ./ and ƒ DfM  ; R ./;

H ./g2=.
9.6 Information Tubes vs Stochastic Filtering Equations. Discontinuous. . . 369

Now let us consider the set

[
Z  .t / D Z  .t; ƒ / D fz .t / j k  ./ 2 Q ./  Y ./  X 0 g

which, with a given realization q.t /, is the reachability set for Eq. (9.99).
Theorem 9.6.1. Assume equalities

M  D M 1 ; R .t /  R1 .t /; H  .t /  H 1 .t / (9.101)

to be true and Y .t / D y.t /  R .t /, t 2 T . Also assume

Z t
q.t /  y.£/d £: (9.102)
t0

Then sets Z0 .t; ƒ/ of Sect. 9.5 and Z  .t; ƒ / of this section coincide, namely,

Z0 .t; ƒ/  Z  .t; ƒ /; t 2 T; ƒ D fM; R./; H./g:

Corollary 9.6.1. Under the assumptions of Theorem 9.6.1, the following equality
is true
\
Xy Œ£ D fZ  .£; ƒ / j ƒ 2 =g: (9.103)

The proof of Theorem 9.6.1 follows from the fact that under the assumptions
of the last theorem equation (9.99), with k  ./ 2 fQ ./  Y ./  X 0 g, and
the system (9.79)–(9.81), have the same reachability sets under constraint (9.83).
Corollary 9.6.1 then follows from Theorem 9.5.1.
Remark 9.6.1. The last results describe a clear connection between solutions to
the linear-quadratic Gaussian filtering problem (the Kalman filter) and those for
deterministic guaranteed state estimation under unknown but bounded “noise”
which may satisfy not only soft integral quadratic bounds on the uncertain items
but also the non-quadratic instantaneous (hard) bounds on the unknown items.
A more detailed discussion of this approach may be found in [138].
One should recall that while the theorems discussed in Sect. 9.1 were proved
under Assumptions 8.1.1, 8.1.3, or 8.1.4, or 8.1.6, those of Sect. 9.2.1 contained
additional conditions on smoothness and convexity of the on-line continuous state
constraints, whereas those of Sects. 9.2.2 and 9.2.3, allowed Y .t / to be only
continuous in time t . A certain relaxation of this condition would be to demand that
y.t / would be piecewise continuous. We shall now recall a technique of singular
perturbations for differential inclusions which will eventually allow to relax the
requirements on Y ./, accepting state constraints, when Y .t / is only measurable
on T .
A nonlinear filtering version of these results could be pursued through a
combination of the reasoning of Sect. 8.1 and of [14].
370 9 Guaranteed State Estimation

9.6.2 The Singular Perturbation Approach: Discontinuous

Measurements in Continuous Time

In this section we consider linear-convex differential inclusions of type

xP 2 A.t /x C Q .t /; y.t /  G.t /x 2 R .t /; (9.104)

with x.t0 / 2 X 0 ; t0  t  £.
Here the nn- and mn-matrices A.t /, G.t / are taken measurable and bounded
almost everywhere on Œt0 ; t1 , with set-valued maps Q W Œt0 ; t1  ! conv Rn and
R W Œt0 ; t1  ! conv Rm measurable and bounded.
Further on we consider a system of type

xP 2 A.t /x C Q .t /; L.t /rP 2 R .t /  y.t / C G.t /x; (9.105)

parameterized by functions L./, with

t0  t  £; x 0 D x.t0 / 2 X 0 ; r 0 D r.t0 / 2 R 0 ;

and

L./ 2 Rmm Œt0 ; £; Z 0 D X 0  R 0 2 conv RnCm

Here Rmm Œt0 ; £ is the class of all continuous invertible matrix functions L./ 2
Rmm Œt0 ; £.
Denote z D fx; rg 2 Rn  Rm and zŒt  D z.t I £; t0 ; z0 ; L/ to be the solution
to (9.105) that starts at point zŒt0  D z0 D fx 0 ; r 0 g. Symbol Z.I £; t0 ; Z 0 ; L/ will
now stand for the solution tube to system (9.105) with ZŒ£ D Z.£; t0 ; Z 0 ; L/ D
Z.£I £; t0 ; Z 0 ; L/ and Z 0 2 Rn  Rm .
Let x W denote the projection of set W Rn Rm into space Rn of x-variables.
Then the next assertion is true
Theorem 9.6.2. Under assumption x Z 0 D X 0 the following formula is true for
any £ 2 Œt0 ; t1 
\ 
Xy Œ£ D x fZ.£I t0 ; Z 0 ; L/ j L 2 Rmm Œt0 ; £ :

Set Xy .£/  Rn therefore arrives as the intersection over all functions L./ 2
R mm
Œt0 ; £ of the projections of sets ZŒ£  R.nCm/.nCm/ on the space Rn of
x-variables. The proof of the last theorem is given in detail in [158, Sect. 16], with
examples given in [174, Sect. 4.6] (see also [159, 168]).
It is clear that conditions of smoothness or continuity of Y .t / in time are not
required in such setting.
Chapter 10
Uncertain Systems: Output Feedback Control

Abstract This chapter finalizes some results of earlier chapters for systems that
operate under set-membership uncertainty. Its aim is to emphasize a successful
application of previously described techniques to such systems. The chapter thus
gives a concise presentation of solution techniques for the problem of output feed-
back control based on available measurements under set-membership uncertainty.

Keywords Set-membership uncertainty • Bounding approach • Measurement

output • Information space • Information tube • Feedback control strategy
• Linear-convex systems • Separation theorem • Ellipsoidal techniques

This chapter gives a concise description of controlled systems that operate under
uncertainty—incomplete information on the system dynamics and uncertainty in
the system and measurement inputs due to unknown but bounded disturbances.
Considered are systems with mostly hard bounds on the uncertain items. The issue
of uncertainty is illustrated on the problem of output feedback control (OFC) for
such systems. The aim of the chapter is to demonstrate that techniques of the present
book may be successfully used in solving problems of feedback control under
imperfect knowledge of its model. A detailed discussion of feedback control under
realistic (inaccurate or incomplete) information on the system model, its on-line
dynamics and accompanying on-line measurement process is a broad topic left for
separate publication.
The basic problem here is of closed-loop target control under realistic
information. As everywhere in this book, the forthcoming results range from
theoretical issues to computational schemes. But the specifics of the derived solution
schemes lie in the combination of both approaches discussed in the previous
chapters, namely, of Hamiltonian methods in the form of dynamic programming
and of techniques taken from set-valued analysis and minmax approaches. The
overall general solution for the considered problem then appears as a combination
of two parts, which deal firstly with a finite-dimensional problem of guaranteed
state estimation and secondly—with an infinite-dimensional problem of feedback
control under set-membership uncertainty. For the first problem new types of set-
valued observers were introduced. For the second problem, which is especially
difficult to formalize and solve, the achieved solution is reduced to one of

© Springer International Publishing Switzerland 2014 371

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__10
372 10 Uncertain Systems: Output Feedback Control

finite-dimensions which facilitates calculation. For systems with linear structure and
convex constraints the solution is more precise. Here the computational procedure
is based on new developments in ellipsoidal calculus, which proved to be effective
and allowed to design complemental software, producing complete solutions with
useful examples (see [133]). Calculations for the nonlinear case are achieved by
using specifics of the problem in combination with comparison principles that allow
to relax the original equations or variational inequalities of the Hamilton–Jacobi
type to simpler, finite-dimensional relations.

10.1 The Problem of OFC

One key problem in realistic control synthesis is to design closed-loop feedback

control laws based on available on-line observations under noisy measurements.
One must assess how the level of uncertainty and the available incomplete, but
realistic information affect the values of the achievable cost, optimizing guaranteed
performance. Related problems of measurement output control were well developed
in a stochastic setting, [4, 18, 59, 122, 273]. However, many applications require
solving this problem under other information conditions that need not rely on prob-
abilistic models but require other approaches which are the topics of this book. Such
approaches naturally require other techniques and new formalization schemes.
This may include other types of observers and other models for disturbances. The
solutions to such problems were investigated mostly within the H1 approach,
[13, 18, 98], with soft-type integral costs, while problems with hard bounds on the
noise, in the set-valued perspective, were less developed [147, 189].
The following text begins with a rigorous problem formulation and emphasizes
the crucial role of properly selecting the on-line generalized state of the system
in the form of information states, described by either value functions for related
problems of optimization, or by information sets—the outputs of related set-valued
observers. Further developed is the detailed overall solution scheme, which is a
combination of two: a finite-dimensional problem of guaranteed state estimation
and an infinite-dimensional problem of feedback control under set-membership
uncertainty. Both problem solutions are described in detail. The second one is
especially difficult to formalize and solve. The novelty of the developed solution is
that it indicates how to reduce the second problem to finite-dimensions, facilitating
calculation. An optimization scheme is then introduced for the overall problem.
A further important aspect is to indicate how to solve the problems completely
(“to the end”), through appropriate computation. For systems with linear structure
and convex constraints a computational procedure is indicated, based on ellipsoidal
calculus, which proved to be effective for many problems and is used with comple-
mentary software which is applicable to systems of large dimensions (see ellipsoidal
10.2 The System: The Generalized State and the Rigorous Problem Formulation 373

toolbox [132]). The procedure also allows one to apply polyhedral techniques,
[114, 115, 117]. For the nonlinear case calculations may be facilitated by using
the specifics of the problem and applying modifications of the earlier suggested
comparison principles that allow to relax the original equations or variational
inequalities of the Hamilton–Jacobi type to simpler relations (see Sects. 5.1 and 7.1
of this book and also [149]).1
The results of this chapter follow the lines of [189].

10.2 The System: The Generalized State and the Rigorous

Problem Formulation

The problem of OFC under set-membership noise is formulated for the next control
system model and available information.
The System Model. Given is an n-dimensional differential equation

xP D f1 .t; x; u/ C f2 .t; x; v/; x.t0 / D x 0 ; t 2 Œt0 ; ª; (10.1)

where the functions f1 .t; x; u/, f2 .t; x; v/ are continuous with respect to the triplets
of independent variables and such that their sum satisfies standard conditions of
uniqueness and extendability of solutions throughout the interval Œt0 ; ª, for any
initial condition x 0 2 Rn , any admissible, bounded, closed-loop control strategies

u D U .t; / 2 P .t /; (10.2)

and uncertain disturbances v.t / from appropriate classes indicated below. Here,
as before, P .t / 2 compRp is a set-valued function, with compact values,
continuous in the Hausdorff metric. And as before, the system trajectory is denoted
xŒt  D x.t; t0 ; x 0 /.
The Measurement Equation and the Noise. The observations of vector x are due
to measurement equation

y.t / D g.t; x.t // C Ÿ.t /; (10.3)

where g.t; x/ is continuous in both variables, y.t / 2 Rr is the measurement, and

Ÿ.t / is the unknown noise.
The Uncertain Items—starting position x.t0 / D x 0 and right-continuous input
disturbances v.t /; Ÿ.t /, are taken as either

1
Effective results may be reached for some classes of problems, especially those on an infinite
horizon, by Lyapunov-type methods [198].
374 10 Uncertain Systems: Output Feedback Control

(a) unknown but bounded, with given bounds

x 0 2 X 0 ; v 2 Q .t /; Ÿ 2 R .t /; (10.4)

where X 0 is compact, while Q .t /; R .t / are set-valued functions with properties

similar to P .t /; or
(b) unknown, with bounds also unknown.
The present discussion further deals with case (a). However, it is also applicable
to case (b) within a unified framework that combines H1 approaches with set-
valued interpretations [14, 103]. The aim of the control is to steer the system to
a prespecified neighborhood of a given target set M , whatever are the unknown
but bounded uncertain items. This is to be done in fixed time by feedback control
strategy U .t; / on the basis of available information:
– the system and measurement model: Eqs. (10.1) and (10.3),
– the given constraints on control u and uncertain inputs x 0 ; v.t /; Ÿ.t /: inclu-
sions (10.2), (10.4),
– the starting position ft0 ; X 0 g,
– the available measurement y.t / D yŒt  (arrives on-line).
Sets X 0 ; M are assumed convex and compact. The feedback control strategy
U .t; / depends on time “t” and on the generalized state Xy Œt  or V .t; / described
further within a precise setting of the problem.
For a precise problem formulation we first need to define a new (generalized)
state of the system. Namely, what should it be now, that the on-line measurement
information is incomplete and corrupted by set-membership noise?
To propagate earlier schemes to OFC the new state may be taken as
(a) ft; yt .; t0 /g; where yt .; t0 / D fy.t C ¢/ j ¢ 2 Œt0  t; 0g is the memorized
measurement. This memorization is done through observers and filters.
(b) ft; Xy Œt g—the information set of states consistent with measurements yt .; t0 /
and constraints (10.4) on the uncertain items

—t .; t0 / D fx 0 ; vt .; t0 /; Ÿt .; t0 /g:

The single-valued trajectories of the classical case should then be substituted by

set-valued information tubes, introduced in Chap. 9
(c) ft; V .t; /g—the information state, which is the value function V .t; x/—the
solution to an appropriate HJB (Hamilton–Jacobi–Bellman) PDE equation,
designed such that

Xy Œt  D fx W V .t; x/  ’g; ’  0
is the level set of V .t; x/.
The overall basic problem of OFC now reads as follows.
10.3 The Overall Solution Scheme: General Case 375

Problem OFC. Specify a control strategy U .t; / P .t / as

U .t; / D U .t; Xy Œt / or U .t; / D U .t; V .t; //;

dependent on the state ft; Xy Œt g or ft; V .t; /g, which for any starting position
ft0 ; X 0 g; t0 < ª; would bring xŒª D x.ª; t0 ; x 0 / to a preassigned neighborhood
M” of the given target set M ; at given time t D ª, whatever be the uncertain items
—t .; t0 /; constrained by (10.4). Here M” D M C ” BM .0/; where BM .0/ D fx W
hx; M xi  1g; M D M 0 > 0.
Note that control strategy U is a functional of the information set (or information
state) that depends on measurement y.t; / and given constraints on the uncertain
items. The notion of information sets and information states, as well as the
description of their properties, are the subject of a separate theory of guaranteed
estimation treated in Chap. 9.
Depending on what we use—the information set or the information state, there
are two basic interconnected approaches, both considered in Chap. 9. The first one
is to use the information state V .t; x/; calculated as the value function for a
Dynamic Programming problem of minimaximizing a functional borrowed from
H1 theory and producing solutions in the form of control strategies, as functionals
u D U .t; V .t; // (see [18, 27, 103, 123]). The second is to use the informa-
tion set described through set-valued calculus in terms of differential inclusions,
involving further the notion of invariant sets and calculating set-valued strategies
u D U .t; Xy Œt / through the “aiming rule” and its analogies (see related work at
[6,122,136,137,153,204]). We will use both approaches within a unified framework.

10.3 The Overall Solution Scheme: General Case

Suppose that on the interval Œt0 ; £ the control u D uŒt  and the observation y D yŒt 
have been realized and are therefore known.2
Problem OFC will be to find

V .t0 ; Xy0 / D min maxfd 2 .x.ª/; M” / j x.t0 / 2 Xy0 g; Xy0 D X 0 \ Y .t0 /;

U —./
(10.5)
under constraints (hard bounds) (10.1)–(10.4) in the class of controls u D
U .t; Vy / D U .t; Xy /.

2
Here and in the sequel square brackets in u D uŒt ; y D yŒt  mean that for the interval under
consideration these functions are known, otherwise we use round brackets.
376 10 Uncertain Systems: Output Feedback Control

Under such hard bounds this will be achieved through functionals

 
V .t0 ; X 0 / D min max min max  d 2 .x.£/; Xy Œ£/ (10.6)
uŒ —Œ u —./

Z ª 
 d .y.t /  g.t; x.t //; R .t //dt C d .x.ª/; M /
2 2
£

where

Xy Œ£ D Xy .£ j y£ Œ; t0 ; u D u£ Œ; t0 / D fx W V .£; x j Œy; u/  0g; (10.7)

and

V .£; x j Œy; u/ D maxfd 2 .x.t0 /; X 0 / (10.8)

—

Z £
 d 2 .yŒt   g.t; x.t //; R .t //dt j x.£/ D x; u D u£ Œ; t0 g
t0

so that V .£; x j Œy; u/ D d 2 .x.£/; Xy Œ£/.

Here x.t0 /; x.ª/ are the two ends of the system trajectory x.t /; while, as before,
—ª .; t0 / is the triplet of uncertain items, bounded by (10.1)–(10.4).
Combining the above into one functional, we get
 
V .t0 ; X 0 / D min max min max V .£; xŒ£ j Œy; u/ (10.9)
uŒ yŒ u —

Z ª 
 d .y.t /  g.t; x.t //; R .t //dt C d .x.ª/; M /
2 2
£

Note that on the interval Œt0 ; £; where the realization of the measurement noise
is ŸŒt  D yŒt   g.t; x.t //; the triplet — D —£ .; t0 / is constrained by inclusions
x 0 2 X 0 ; v.t / 2 Q.t /; yŒt   g.t; x.t / 2 R .t /; with observation yŒt  and control
uŒt  being known (such triplets are denoted as —Œ), while on the interval .£; ª; the
measurements y.t / are unknown, so that — D —ª .; £/ has to be bounded only by
Xy Œ£; Q .t /; R .t /; t  £; to satisfy all the possible future realizations of y.t /.
Then, on the interval .£; ª; functional (10.6) has to be maximized over all such
—ª .; £/ and minimized over u 2 P .t / in the class of strategies described further.
One may now observe that the overall Problem OFC may be separated into
two, namely, Problem E on guaranteed estimation (calculation of the information
state V .t; x/ D V .t; x/), solved within the interval Œt0 ; £ and Problem C of
feedback control (finding strategy U .t; V .t; // as a functional of the information
state), solved within the interval .£; ª. Note that V .£; x/ depends on the available
measurement y£ Œ; t0  D y.t C ¢/; ¢ 2 Œt0  £; 0; and that its role is similar
10.4 Guaranteed State Estimation Revisited: Information Sets and Information. . . 377

to sufficient statistics in stochastic control. Also note that the control strategy
U .t; V .t; // has to be selected within a class of functionals that ensure existence
and extendability of solutions to Eq. (10.1) with u D U .t; V .t; //. Therefore of
importance will also be a third goal which is to achieve a proper mathematical
formalization for the obtained results, ensuring the existence of solutions to the basic
Eq. (10.1) under strategies U .t; V .t; // obtained in the space of information states.
In the previous paragraph the information state V .t; x/ may be substituted by
information set Xy Œt  D Xy .t; / due to the following: with set-valued hard bounds
on fx 0 ; v; Ÿg being given, it turns out that

Xy Œt  D fx W V .t; x/  0g; (10.10)

therefore the information set X Œt  would be the level set for the information state
V .t; /. This property thus justifies the fact that the generalized state is selected,
as indicated above, either as a function (the information state, which is the value
function V .t; / for an appropriate problem of Dynamic Programming) or as a
set (the information set). The dynamics of the overall OFC problem may be thus
described by either PDEs of the HJBI type (which give the dynamics of V .t; /
and its level sets ) or, for example, through a related integral funnel equation—an
evolution equation with set-valued solution which gives the dynamics of Xy Œt  (see
Chaps. 8, 9, and also [158]). We further rely on either of these two definitions. The
fact is that to achieve effective solutions with natural geometrical interpretations it
appears necessary to combine both approaches. This allows us to work out a unified
vision of the general problem and open new routes to a detailed solution of specific
Problem OFC, especially with hard bounds and working within computation in
finite-dimensional space. Conditions for the latter to suffice, as given later, also
allow us to avoid the common restriction that u; v should satisfy certain matching
conditions. It is thus possible to solve the linear case completely and to apply
new computation methods to systems with set-valued solutions, enabling effective
computer animation of solution examples. Recall that under feedback control the
original linear system becomes nonlinear. We now describe the solution schemes
for Problems E and C the subproblems of the overall Problem OFC.

10.4 Guaranteed State Estimation Revisited: Information

Sets and Information States

We consider this Problem E in two versions: E and E0 , dealing with continuous

measurements.
Problem E. Given are system (10.1) with measurement Eq. (10.3). Also given are
starting position ft0 ; X 0 g, available measurement yŒt , and realization of control
u D uŒt ; t 2 Œt0 ; £. One is to specify the information set Xy Œ£ D Xy .£; / of
all states x.£/, generated by system (10.1), (10.3), that are consistent with given
realizations uŒt ; yŒt  and on-line state constraints (10.4).
378 10 Uncertain Systems: Output Feedback Control

This definition of information set Xy Σ is similar to Definition 9.1.1, but the
difference lies in the system Eq. (10.1), where also present is the realization uŒt 
known within Œt0 ; £/.
Therefore, the generalized state of the system may be taken as ft; Xy Œt g.
An equivalent definition of such a state may be also served by the triple
f£; y£ .; t0 /; u£ ./g. Then the passage from y£ .; t0 / to Xy Œ£ (now with given
u£ ./, is achieved through set-valued observers described in Chap. 9, Sects. 9.2
and 9.6 (see also [137, 151, 158]).
To solve basic problems E and C one should start by describing the evolution
of information sets Xy Œ£ D Xy .£; / in time, which now is also influenced by the
selected control u. Here we first explain this process through differential inclusions,
along the lines of Sects. 8.2, 9.2 and 9.6. Denote

Y .t / D fx W y.t /  g.t; x/ 2 R .t /g; Yª .; £/ D fY .t /; t 2 Œ£; ªg:

We further assume Y .t / to be closed and convex-valued. Let D be a convex compact
domain in Rn sufficiently large to include projections on Rn of all possible set-
valued trajectory tubes of system (10.1), (10.4) and all possible starting sets and
target sets that may occur under our considerations. Then set YD .t / D Y .t / \ D
will be convex and compact.
Assumption 10.4.1. Set-valued function YD .t / satisfies the Lipschitz condition:

h.YD .t /; YD .t 0 //  œy jt 00  t 0 j: (10.11)

Consider system (10.1), (10.4), with functions f1 ; f2 ; g continuously differen-

tiable in ft; xg. Also assume given are the realizations of the control u D uŒt  and
measurement y D yŒt ; t 2 Œt0 ; £. Denote

F .t; x; uŒt / D f1 .t; x; uŒt / C f2 .t; x; Q .t //; f2 .t; x; Q / D [ff2 .t; x; v/ j v 2 Q .t /g:

Then the information set Xy Œ£ will be the cross-section (cut) at time £ of the solution
tube (the “information tube”) Xy Œt  D Xy .t; t0 ; X 0 / for the differential inclusion
(DI)

xP 2 F .t; x; uŒt /; x.t0 / 2 X 0 ; (10.12)

under state constraint x 2 Y Œt ; t 2 Œt0 ; £.3 In the coming propositions we presume

Q .t / 2 convRq ; R .t / 2 convRr .
According to Lemma 8.2.3 and Theorem 8.2.3 (see also [158], Sect. 10) the
information tube Xy Œ£;  D fXy Œt ; t 2 Œt0 ; £g will be the solution tube to the
differential inclusion (DI):
\
xP 2 fF .t; x; uŒt / C L.yŒt  C g.t; x/ C R .t // j L 2 Rnr g; x.t0 / 2 X 0 ; t 2 Œt0 ; £;

3
For Y .t / generated by given yŒt  we use notation Y D Y Œt  with square brackets.
10.4 Guaranteed State Estimation Revisited: Information Sets and Information. . . 379

where L may be either constant matrices of dimension n  r or continuously

differentiable n  r matrix functions L.t; x/; L.; / 2 Cnr
1 Œt0 ; £ or L.t /; L./ 2
Cnr
1 Œt0 ; £.
Taking the DI

zP 2 F .t; z; uŒt / C L.t; z/.yŒt  C g.t; z/ C R .t //; z.t0 / 2 X 0 ; (10.13)

denote its solution tube, emanating from X 0 ; as Zu Œt  D Zu .t; t0 ; X 0 ; L.; //.

Theorem 10.4.1. Under Assumption 10.4.1 the next inclusion will be true:
\
Xy Œ£ D Xy .£; t0 ; X 0 / fZu .t; t0 ; X 0 ; L/ j L D L.; / 2 C1nr Œt0 ; £g
\
fZu .t; t0 ; X 0 ; L/ j L D L./ 2 Cnr
1 Œt0 ; £g: (10.14)

Corollary 10.4.1. For a linear system with

xP 2 A.t /x C B.t /uŒt  C C.t /Q .t /; yŒt  2 G.t /x C R .t /; x.t0 / 2 X 0 ;

and with convex, compact X 0 ; Q .t /; R .t / the DI (10.13) turns into

zP 2 A.t /x CB.t /uŒt CC.t /Q .t /CL.t /.yŒt CG.t /zC R .t //; z0 D z.t0 / 2 X 0 ;
(10.15)
and inclusion (10.14) into an equality
\
Xy Œ£ D Xy .£; t0 ; X 0 / D fZu .t; t0 ; X 0 ; L/ j L D L./ 2 Cnr
1 Œt0 ; £g:
(10.16)
The last formula will be used below to obtain more detailed solutions under system
linearity. DIs (10.13) and (10.15), taken under given u£ ./; are actually the related
set-valued observers.
Assumption 10.4.1 implies that measurement y.t / is continuous. This require-
ment may be relaxed to piecewise continuous measurements y.t / D yŒt  by
applying another set-valued observer generated due to the relaxed set-valued funnel
equation of Sect. 8.1.5. For solution tube Xy Œ£;  of system (10.12), under x 2 Y Œt 
the set-valued function Y .t / may then be taken as Hausdorff upper semi-continuous.
T Xy Œt  then satisfies the evolution equation of the
With uŒt ; yŒt  known, the tube
funnel type, with X Œt0  D X 0 Y Œt0  :
 [ \ 
1
lim ¢ hC X Œt C ¢; fx C ¢ F .t; x; uŒt / j x 2 X Œt  Y Œt C 0g D 0:
¢!0C0
(10.17)
Passing to the Hamiltonian approach, the dynamics of information sets Xy Σ
may be now calculated by solving the alternative problem E0 , formulated in terms
of value functions.
380 10 Uncertain Systems: Output Feedback Control

Problem E0 . Given are the starting position ft0 ; X 0 g, the available measurements
y£ Œ; t0 , and the control realization uŒt ; t 2 Œt0 ; £/. One is to specify
Z £
V .£; x/ D maxfd .xŒt0 ; X /
2 0
d 2 .yŒt g.t; x/; R .t //dt j xŒ£ D x 2 Rn ;
v t0
(10.18)
v.t / 2 Q .t /; t 2 Œt0 ; £g;

due to system (10.1).

Here xŒt  D x.t; £; x/; t  £ stands for the backward trajectory, emanating from
position f£; xg. Function V .£; x/ D V .£; x j V .t0 ; /; y£ Œ; t0 ; u£ Œ; t0 / turns out to
be precisely the information state of system (10.1), (10.3) introduced in the above.
Remark 10.4.1. If one presents functional (10.6) as the sum of two parts: the first
defined for Œt0 ; £, and the second, defined for .£; ª, then (10.18) will coincide with
the first part, reflecting the problem of state estimation, namely, the calculation of
the on-line information state.
Using function V .£; / is convenient for describing the evolution of information
states in differential form, since V .£; / may be represented in more conventional
terms of partial differential equations. Note that in (10.7) we actually have an
equality X Œ£ D fx W V .£; x/  0g. However, we prefer to write an inequality
in (10.7), since this will also allow us to describe the neighborhoods of Xy Σ as

Xy© D fx W V .£; x/  ©g; (10.19)

without introducing new notation.

Function V .t; x/ satisfies an HJB-type equation for time t 2 Œt0 ; £:

@V @V 
D  max ; f1 .t; x; u Œt / C f2 .t; x; v/
@t v @x
ˇ 
ˇ
ˇ
d .yŒt   g.t; x/; R .t //ˇ v.t / 2 Q .t / ; (10.20)
2

under boundary condition

V .t0 ; x/ D d 2 .x; X 0 /: (10.21)

V .t; x/ may be non-smooth, hence the solution to Eq. (10.20) under condi-
tion (10.21) may not be classical, but must be redefined in a generalized sense,
for example, as a “viscosity” or “minmax” solution [48, 50, 80, 247].
The existence of a generalized solution to Eq. (10.20), (10.21) is necessary and
sufficient for the existence of solution to Problem E0 . In this case the mapping
V .t; / D V .t; jV .t0 ; // satisfies the evolution equation of type

Vt .t; / D ˆ.t; Vx .t; /;  j uŒt ; yŒt /; (10.22)

10.5 Feedback Control in the Information Space 381

whose right-hand side is identical to (10.20). In other words, (10.22) is an “equation

of motion” for the information state V .t; /. The evolution of V .t; /’s level set X Œt 
(see (10.18)) is therefore implicitly also described by this equation.
A fairly broad case is when V .£; x/ are continuous in all variables, with
nonempty compact zero level-sets X Œ£ D fx W V .£; x/  0g 6D ;. These are neces-
sarily those that solve Problem OFC for (10.1), (10.3) being linear, with continuous
coefficients and convex constraints. The indicated class of functions V .£; / will be
denoted as KV and treated as a metric space with metric hV .V 0 .£; /; V 00 .£; // D
h.X 0 Œ£; X 00 Œ£/; where X Œ£ D fx W V .£; x/  0g and h.X 0 ; X 00 / is the Hausdorff
metric. The variety of closed compact sets X Σ may be also considered as a metric
space KX , equivalent to KV , with metric h.X 0 Σ; X 00 Σ/. A discussion of possible
related metrics is given in monograph [98, Chap. 4, Appendix C.5].
Having specified the spaces of information states and sets (also referred to as
the respective information space), we may proceed with the problem of feedback
control itself. The difficulty is that we will now have a problem in the information
space, with states taken as either functions V .t; / or sets X .t; /. We therefore return
to Eq. (10.22), observing, that for t > £ the realization y.t / is unknown in advance,
arriving on-line, while for t  £ the control, taken, for example in the form of
u D u.t; V .t; //; is to be selected due to this equation treated in the space of states
V .t; /. Further on, we indicate, when possible, the routes for overcoming these scary
difficulties through simpler techniques.

10.5 Feedback Control in the Information Space

As indicated in the above, the information space may be defined in either the space
of information sets or the space of value functions—the information states. We first
begin with the space of information sets.

10.5.1 Control of Information Tubes

The feedback control problem in the space of information sets reads as follows.
Problem C. Given starting position f£; Xy Œ£g; target set M and number ” > 0;
specify a control strategy u D u.t; X /; t 2 Œt0 ; £ that steers system (10.20), (10.21)
under u D u.t; X / into a ”-neighborhood M” of given terminal target set M ;
whatever be the function

yª .; £/ 2 Yª .; £/ D fy.t / W —ª .; £/ 2 Zª .; £/g;

S items —ª .; £/ that generated yª .; £/.

and the related uncertain
Here Zª .; £/ D f—ª .; £/ W x 2 Xy Œ£; v.t / 2 Q.t /; Ÿ.t / 2 R .t /; t 2 Œ£; ªg.
382 10 Uncertain Systems: Output Feedback Control

Problem Copt . Among values ” > 0; for which Problem C is solvable, find
smallest ”0 .4
With ” > 0 given, Problem C is solvable if and only if Xy Œ£ belongs to
W Œ£ 6D ;, that is the set of subsets X£  Rn for which this problem with starting
position f£; Xy Œ£ D X£ g, does have a solution. Further we indicate that W Œ£ is
understood to be weakly invariant relative to target set M” .
Similarly, but for the information space of functions, we have the next problem.5
Problem Cv . Find value function
 Z ª
V .£; V .£; // D min max max  V .£; x/  d 2 .y.t /  g.t; x.t //; R .t //dt C
u y x;v £
ˇ 
ˇ
C d .xŒª; M” /ˇˇu 2 U ; y./ 2 Yª .; £/; v./ 2 Qª .; £/; x 2 X Œ£ ;
2
(10.23)

with X Œ£ D fx W V .£; x/  0; V .£; / 2 KV g:

Here the minimum is taken in the class U of feedback strategies u D u.t; V /,
described below. The maximum is taken over all y./ 2 Yª .; £/ D fY .ª C ¢/ j ¢ 2
Œ£  ª; 0g described above, and further, over all x 2 X Œ£; Qª .; £/ D fv.t / 2
Q .t /; t 2 Œ£; ªg.
Functional (10.23) represents the second part of (10.6), which defines the overall
control process. With £ D ª it must ensure the boundary condition

V .ª; V .ª; // D maxfd 2 .x; M” / j V .ª; x/  0g: (10.24)

x

Number ” 2 D V .ª; V .ª; // is the square of “size” ” of the neighborhood

M” D M C ” B .0/ which at time ª entirely includes X Œª, so that X Œª M” .
Functional V .£; V .£; // is defined on the product space RC  KV , where RC D
Œt0 ; 1/. Note that in (10.23) the operations maxy ; maxx;v are interchangeable.
After this interchange one may observe that for every y./ 2 Yª .; £/ the maxx;v
results in d 2 .y.t /  g.t; x.t //; R .t //  0. This shows that in (10.23) operations
minu maxy maxx;v reduce to just minu maxx;v .
Value function V .£; V .£; // satisfies the generalized “backward” equation of the
HJBI type, which is well known in dynamic games with complete information6 and
may be formally written as
 ˇ 
d V .£; V .£; // ˇˇ
min max ˇ u 2 P .£/; v 2 Q .£/ D0 (10.25)
u v d£

4
Note that if for ” D 0 Problem C is not solvable, then ”0 > 0:
5
Here and in (10.6) we use equivalent notations V .£; V .£; // and V .£; X Œ£/; since X Œ£ is the
zero-level set for V .£; /.
6
See [18] and [123] where this is done in finite-dimensional space.
10.5 Feedback Control in the Information Space 383

where d V .£; V .£; //=d £ is the total derivative of functional V .t; V .t; // due to
evolution equation (10.20), (10.21), or (10.22), which is the same. Equation (10.22)
may have a smooth solution, when the total derivative exists in the strong or weak
sense and the equation holds everywhere. Otherwise it has to be dealt with in terms
of generalized solutions, as mentioned above.
The solution strategy should then be formally determined through minimization
over u in (10.25), as u0 D u0 .t; V .t; // P .t /. Substituting this into (10.20),
instead of uŒt  with y.t / instead of yŒt ; we have
˚ ˇ 
Vt C max Vx ; f1 .t; x; u0 .t; V .t; /// C f2 .t; x; v/ ˇ v 2 Q .t / 
v

 d 2 .y.t /  g.t; x/; R .t // D 0 (10.26)

The last relation may be treated as an evolution equation in the metric space
KV of functions fV .t; /g. The “trajectories” V Œt  D V .t; /, issued at time t0
from V .t0 ; / D d 2 .x; X 0 /, may then be interpreted as “constructive motions”
V .t; I u0 .t; //; that arrive as a result of limit transition from infinite-dimensional
analogies of Euler broken lines, constructed under all possible partitions of the
interval Œt0 ; ª, by selecting piecewise-constant realizations uŒt  2 P .t / (see [123],
p. 11). Another option could be to treat fV .t; /g as a generalized (viscosity) solution
in infinite-dimensional space KV (see [98]).
Being based on dynamic programming, the overall Problem OFC allows to
consider linked solutions of problems E and C, where E is finite-dimensional, while
C is infinite-dimensional. In the linear case, as we shall see, these solutions are even
independent and may be separated. Such a scheme produces a good insight into the
problem and its complete solution. But it is difficult to calculate. However, once we
need to solve concrete problems, our interest is in computation schemes feasible for
applications and such that would allow procedures that do not go beyond finite-
dimensional dynamics with computational burden not greatly above the one for
problems with complete information.

10.5.2 The Solution Scheme for Problem C

As indicated above, the solution to Problem OFC is given through two types of
HJB, HJBI equations (see (10.20), (10.25))—one for state estimation and one for
feedback control. However, what we actually need here are not the value functions,
but only their level sets. Hence, instead of directly solving the infinite-dimensional
equation (10.25), we will use the notion of invariant sets (or backward reachability
sets) relative to target set M” . Continuing the previous subsection we deal with the
space of information sets. But to calculate them we shall use relations derived in the
space of information states. We follow the scheme below.
Formally, the set WV Œ£ D fV .£; / W V .£; V .£; //  0g that we need is the
(weakly) invariant set defined as follows.
384 10 Uncertain Systems: Output Feedback Control

Definition 10.5.1. (a) Set WV Œ£; weakly invariant relative to target set M” , is the
union of value functions V .£; / (in space KV ) for each of which Problem C is
solvable from initial position f£; Xy Œ£g, with Xy Œ£ D fx W V .£; x/  0g.
Multi-valued function WV Œt ; t 2 Œ£; ª is the solvability tube in KV (an
infinite-dimensional analogy of the “Krasovski bridge”) for Problem C.
(b) Set W Œ£ weakly invariant relative to target set M” , is the union ( in space KX )
of compact sets X D X Σ for each of which Problem C is solvable from initial
position f£; X Œ£g.
Set-valued function W Œt ; t 2 Œ£; ª is the solvability tube in KX (another
infinite-dimensional analogy of “Krasovski bridge,” equivalent to case (a)).
Combining alternative descriptions in terms of value functions with those given
through set-valued calculus, we now pass from space KV to KX ; applying to
Problem C an infinite-dimensional modification of the rule of extremal aiming
towards weakly invariant sets W Σ (see finite-dimensional version in [123]). With
on-line position f£; X Œ£g given, we now have to calculate the Hausdorff semi-
distance HC .Xy Σ; W Σ/ between the actual state Xy Σ and the solvability set
W Σ, taken in the metric of space KX ; and arriving at

VH .£; Xy Œ£/ D HC .Xy Œ£; W Œ£/;

HC .X Œ£; W Œ£/ D inffinff© W X£ C ©B .0/  X Œ£g j X£ 2 W Œ£g

©

where the first (external) infimum is to be taken over all X£ 2 W Œ£. (This Hausdorff
semi-distance is similar to hC .X 0 ; X 00 / in Rn which was defined earlier.)
Then the solution to Problem C would be

U 0 .£; Xy Œ£/ D fu W d.exp.2œ£/VH .£; Xy Œ£//=d £ ju  0 j u 2 P .£/g:

(10.27)
Namely the union of controls that allow a non-positive derivative in (10.27) would
be the desired solution. Here œ is the Lipschitz constant for (10.1), uniform in
t; x; u; v.
This is an infinite-dimensional problem, which in the general case in not simple.
However, the solution turns out to be simpler than may be expected in that it may
suffice to consider instead of VH .£; Xy Œ£/ the value

Vh .£; Xy Œ£/ D hC .Xy Œ£; WŒ£/ D minf© > 0 W Xy Œ£  WŒ£ C ©B .0/g;

where
ˇ
[ [ ˇ

WŒ£ D ˇ
fx W x 2 X£ g ˇ X£ 2 W Œ£

is a set in finite-dimensional space Rn .

10.5 Feedback Control in the Information Space 385

The next proposition is true with set WΣ being convex.

Theorem 10.5.1. With set WΣ 2 convRn the Hausdorff semi-distances given by
HC .Xy Σ; W Σ/ and hC .Xy Σ; WΣ/ are equal.
The last relation is useful in the linear case with convex compact constraints
where both Xy Σ; WΣ turn out to be convex and compact.
Then, since WΣ coincides with the backward reachability set for system
(10.1), (10.4) without additional state constraint (10.3), this set WΣ will be the
level set for the solution of a finite-dimensional HJBI equation.
Relation (10.27) now reduces to
ˇ
ˇ
U 0 .£; Xy Œ£/ D fu W d.exp.2œ£/h2C .X Œ£; WŒ£//=d £ ju  0 ˇˇ u 2 P .£/g:
(10.28)
Theorem 10.5.2. Under conditions of Theorem 10.5.1 the control solution
U 0 .£; X Œ£/ to Problem C is given by relation (10.28).
We thus have two types of control solutions: u0 .t; V .t; // and U.t; Xy Œt / which
have to be realized through related equations or differential inclusions similar
to (10.26), according to the specific definitions of their solutions for each control
type.

10.5.3 From Problem C to Problem Copt

The previous results described solution schemes that ensure inclusion X Œª M” ;
whatever be the uncertain items —./. Here M” is the guaranteed neighborhood of
the target set. The final problem is now to minimize this neighborhood M” under
uncertainty and output feedback. Then we need to solve the following subproblem.

Solving Problem Copt of Minimizing the Guaranteed Solution

Consider minimax problem (10.6) with £ D t0 and starting position ft0 ; X 0 g,

where M is substituted for M” and W Œ£; V .£; / for W” Œ£; V” .£; /. Then the
solution to Problem C will be to find the smallest ” D ” 0  0 among those that
ensure X 0 W” Œt0  D W” Œt0 :
Here V” .t0 ; X 0 / will depend on the boundary condition (10.24), so that W” Œt0  D
fW W V” .t0 ; W /  0g D W” Œt0 . Introducing

¡.”; t0 / D H” .X 0 ; W” Œt0 / D h” .X 0 ; W” Œt0 /;

we reduce the problem of optimization to calculating

min ¡.”; t0 / D ¡.” 0 ; t0 / D ¡0 .t0 /:

”
386 10 Uncertain Systems: Output Feedback Control

The final step is to solve Problem C for ” D ” 0 by finding the closed-loop

control strategy U 0 .t; Xy / that for the optimal neighborhood M 0 D M”0 of the
target set M , ensures the inclusion X λ M 0 , whatever be the unknown but
bounded uncertain items —./.
Remark 10.5.1. The uncertain items may deviate from their maximizing “worst
case”) values. This allows us to improve the final result by recalculating (rebooting)
the solution and decreasing ” 0 by diminishing sets M” ; W” Œt  beyond their initial
sizes.
Remark 10.5.2. Problem Copt may be also solved by integrating the HJB–HJBI
equations (10.20)–(10.26), but this direct approach appears to impose a far greater
computational burden than the suggested schemes. It also requires appropriate
interpretations of solution classes for these equations.
The ultimate aim of treating Problem OFC is to solve it completely through adequate
computation. We shall now indicate some relaxation schemes for finding Xy Σ and
WΣ through related computation methods, restricting ourselves to linear systems.
The nonlinear case may be approached through the comparison principle of Chap. 5,
Sects. 5.1–5.3.

10.6 More Detailed Solution: Linear Systems

We shall now discuss how far can the OFC problem be solved for linear systems
of type

xP D A.t /x C B.t /u C C.t /v; (10.29)

y.t / D G.t /x C Ÿ; (10.30)

under continuous coefficients and convex compact constraints u 2 P .t / and (10.4).

10.6.1 The “Linear-Convex” Solution

As indicated in the previous section, we must find the feedback control u0 .£; Xy Œ£/,
having calculated Xy Σ and WΣ; where in the linear case both sets are convex
and compact. These sets may be expressed explicitly, through duality methods of
convex analysis, as the level sets of related value functions similarly to formulas of
Chap. 2, Sect. 2.4 and Chap. 3. And it would also allow us to calculate derivatives
of such value functions. Here, however, we indicate another scheme for such
calculations by using related evolution equations of the funnel type introduced in
10.6 More Detailed Solution: Linear Systems 387

Chap. 2, Sect. 2.3.1 and Chap. 8, Sect. 8.1.3. This move brings us, using notation
hC .X£ ; WŒ£/ D minf© > 0 W X£ 2 WŒ£ C ©B .0/g; to problem

Vh .£; Xy Œ£/ D maxf¡.l j G.ª; £/Xy Œ£/  ¡.l j G.ª; £/WŒ£/ j < l; l > 1g:
(10.31)
Then, following (10.28) for VH D Vh , we find the overall control solution u0 .
Hence, in order to calculate u0 we first need to calculate the total derivative of
Vh .£; Xy Œ£/ and ensure that it is non-positive under control u0 . Related evolution
equations for Xy Œt ; WŒt  may be written as

lim ¢ 1 hC .Xy Œt C¢; .I C¢A.t //.Xy Œt \ Y .t //C¢B.t /u.t /C¢C.t /Q .t // D 0;

¢!0C0
(10.32)
lim ¢ 1 h.WŒt  ¢ C ¢C.t /Q .t /; .I  ¢A.t //WŒt  C ¢B.t /P .t // D 0;
¢!0C0
(10.33)
with boundary conditions Xy Œt0  D X ; WŒª D M” . For the first equation we use
0

the unique inclusion maximal solution, for the second we use the ordinary solution,
which is always unique.
Denote the unique maximizer in (10.31) as l 0 . We have

d Vh .t; Xy Œt /=dt D d¡.l 0 j G.ª; t /Xy Œt /=dt  d¡.l 0 j G.ª; t /WŒt /=dt:
(10.34)
Calculating along the lines of Sect. 8.3, we get

d¡.l 0 j G.ª; t /Xy Œt /=dt  hl 0 ; G.ª; t /B.t /ui C ¡.l 0 jG.ª; t /C.t /Q .t //;

d¡.l 0 j G.ª; t /WŒt /=dt D ¡.l 0 j G.ª; t /B.t /P .t // C ¡.l 0 j G.ª; t /C.t /Q .t //:

Substituting these derivatives into (10.34) we come to the next statement.

Theorem 10.6.1. The total derivative

d Vh .t; Xy Œt /=dt  hl 0 ; G.ª; t /B.t /ui C ¡.l 0 j G.ª; t /B.t /P .t //: (10.35)

Finally, selecting

u0 .t; Xy / D arg maxfhl 0 ; G.ª; t /B.t /ui j u 2 P .t /g; (10.36)

and integrating d Vh .t; Xy Œt /=dt juDu0 from t0 to ª, we ensure the next condition.
Lemma 10.6.1. The control strategy (10.36) ensures the inequality

Vh .ª; X Œª/  Vh .t0 ; Xy Œt0 / D ”;

whatever are the uncertain items fx 0 ; v./; Ÿ./g.

388 10 Uncertain Systems: Output Feedback Control

The next question is how to calculate the solutions numerically. Here we first
recall that Xy Σ; WΣ may be indicated in explicit form. Then, using this
knowledge, we suggest a computational approach based on approximating convex
sets by parametrized families of ellipsoids that evolve in time, following the
approximated sets.

10.6.2 The Computable Solution: Ellipsoidal Approximations

Set Xy Σ may be effectively calculated numerically through ellipsoidal techniques.

Using notation of Chap. 3, where an ellipsoid E .p; P / D fu W hup; P 1 .up/i 
1g; P D P 0 > 0, we assume that the sets

X 0 D E .x 0 ; X 0 /; P .t / D E .p.t /; P .t //; Q .t / D E .q.t /; Q.t //;

R .t / D E .0; R.t //; M D E .m; M /
are ellipsoidal. Since every convex compact set may be presented as an intersection
of ellipsoids, the solutions and methods for nonellipsoidal symmetric constraints,
like parallelotopes and zonotopes, could be designed as a modification and extension
of those for ellipsoidal constraints (see Chap. 5, Sects. 5.4 and 5.5).
Then the information set Xy Σ has an external ellipsoidal approximation Xy Σ
EL Œ£ D E .xL .£/; XL .£//, where, throughout Œt0 ; ª; with x.t0 / 2 EL Œt0  D
E .x 0 ; X 0 /; we have
xP D A.t /x C B.t /uŒt  C C.t /v.t /; yŒt  D G.t /x C Ÿ.t /;

xP L D A.t /xL C B.t /uŒt  C C.t /q.t / C L.t /.yŒt   G.t /xL /; xL .t0 / D x 0 ;
(10.37)
XP L D .A.t /  L.t /G.t //XL C XL .A.t /  L.t /G.t //0 C ..t / C ¦.t //XL C

C .¦.t //1 Q.t / C  1 .t /L.t /G.t /R.t /G 0 .t /; XL .t0 / D X 0 : (10.38)

The parametrizing functions ¨.t / D f.t / > 0; ¦.t / > 0; L.t /g may be taken
piecewise continuous.
Theorem 10.6.2. The next relation is true
\
Xy Œ£ D fEL Œ£ j ¨./g:

Among these parametrizers, one may select some optimal or some tight ones when
one could have, for some direction l 2 Rn ; a related triplet ¨0 ./, that ensures the
equality ¡.l j Xy Œ£/ D ¡.l j fEL Œ£ j ¨0 g/.
For WΣ we may also apply ellipsoidal approximations of the types indicated
in Chap. 5 (see also paper [149]), or use discretized versions for reachability under
uncertainty (see [133]). Here available are internal and external ellipsoidal bounds
10.6 More Detailed Solution: Linear Systems 389

E .w .£/; W .£// WŒ£ E .w .£/; WC .£//

that depend on internal and external parametrizing functions .t /; S.t / and tuning
parameter r.t / > 0. The equations for w .t /; WC .t /; W .t / are of the type (B.t / D
.B.t /P .t /B 0 .t //1=2 ; C.t / D .C.t /Q.t /C 0 .t //1=2 /:

wP  D A.t /w C B.t /p.t / C C.t /q.t /; w .ª/ D m; (10.39)

WP C D A.t /WC C WC A0 .t /  .t /WC   1 .t /B.t /

C r.t /.WC S.t /C.t / C C0 .t /S 0 .t /WC /; WC .ª/ D M: (10.40)

WP  D A.t /W C W A0 .t / C .t /W C  1 .t /C.t /

 r.t /.W S.t /B.t / C B0 .t /S 0 .t /W /; W .ª/ D M: (10.41)

With .t / > 0; r.t / > 0 and orthogonal matrices S.t /S 0 .t / D I; we have

E .w .t /; WC .t // D WC0 Œt 
WŒt 
W0 Œt  D E .w .t /; W .t //;

where

WC0 Œt  D fw W hw  w .t /; WC1 .t /.w  w .t /i  1g;

W0 Œt  D fw W hw  w .t /; W1 .t /.w  w .t /i  1g;

whatever be the functions .t / and the orthogonal matrices S.t /. Following [180],
we have the next proposition.
Theorem 10.6.3. The next property is true
[
WŒ£ D fE .w .£/; W .£// j ./; S./g:

The parameter r.t / in (10.35) may be taken as r.t / D hle ; W Œt le i1=2 .
Suppose le is a unique support vector for Xy Œt ; WŒt . Then, along the lines of
Chap. 3, Sects. 3.2 and 3.7, one may describe the values of e .t /; ¦e .t /; Le .t /
for (10.33) and .t /; S.t /; r.t / for (10.35) that ensure the equalities
e .t/ D hle ; C.t/le i1=2 hle ; XL Œtle i1=2 ; ¦.t/ D hle ; L0e R.t/Le le i1=2 hle ; XL Œtle i1=2 ;
(10.42)

.t / D hle ; C.t /le i1=2 hle ; W Œt le i1=2 ;

hle ; W .t /S.t /B.t /P 1=2 .t /le i1=2 D hle ; B.t /le i1=2 hle ; W Œt le i1=2 ; (10.43)

We now pass to the computation scheme.

390 10 Uncertain Systems: Output Feedback Control

We return to problem (10.31), in view of relations


Xy Œ£ EL Œ£ D E .xL .£/; XL .£//; WŒ£
ES Σ D E;S .w .t /; W .t //:

Substituting sets Xy Σ; WΣ in (10.31) by their ellipsoidal approximations, we

take the externals for Xy Œ£ and the internals for WŒ£. Then, instead of Vh .£; Xy Œ£/
we get its approximation VhE .£; /; that also depends on the parametrizing functions
for approximating ellipsoids. We may then “tune” the parametrizers, selecting them
so as to minimize the error of approximation (which, with proper tuning, may even
tend to zero). Denote lE0 to be the unique optimizer for problem

VhE .£; EL Œ£/ D maxf¡.l j G.ª; £/EL Œ£/  ¡.l j G.ª; £/ES Œ£/ j < l; l > 1g:
(10.44)
The ellipsoidal-valued states involved here are finite-dimensional elements, of type
fx; X g; fw; W g, whose evolution is explicitly described by ordinary differential
equations. The approximate control strategy UL0 may then finally be taken as the
minimizer in u0 D Ul0 D U.t; X / of the total derivative d VhE .t; EL Œt /=dt ju .
The question is: will the approximate control strategy UL0 perform as successfully
as the exact solution U 0 ? And what would be the suboptimal neighborhood ML
of set M reachable through strategy UL0 despite the uncertainties as compared to
M 0 D M” ; ” D ” 0 ?
In order to reach the answers we shall calculate the derivatives of functions

¡.l j G.ª; t /EL .t //; ¡.l j G.ª; t /ES .t // involved in (10.38), using notation
0
le .t / D G .ª; t /lE . Then, taking l D lE0 , using relations (10.42), (10.43), and
0 0

the Cauchy–Bunyakovsky inequality

hl; L0 GXC li  hG 0 Ll; XC G 0 Lli

1=2
hl; XC li1=2 ;

after some calculations, we have

d¡.l 0 j EL .t //=dt D .hl 0 ; L0 G.t /XC .t /G 0 .t /L0 l 0 i1=2 Chl 0 ; L00 G.t /.x  x  /i/
Chl 0 ; L0 Ÿ.t /i  hl 0 ; L00 R.t /L0 l 0 i1=2 C hl 0 ; C.t /vŒt i C hl 0 ; B.t /uŒt i: (10.45)

Similarly, we get

d¡.l 0 jES .t //=dt D hl 0 ; B.t /p.t / C C.t /q.t /i C hl 0 ; C.t /l 0 i1=2  hl 0 ; B.t /l 0 i:
(10.46)
Here x; Ÿ are those that generated the realized measurement y.t / D G.t /xCŸ.t /.
Substituting this in (10.44), we find the total derivative
d VhE .£; EL Œ£/=dt D hl 0 ; L0 G.t/XC .t/G 0 .t/L00 G.t/l 0 i1=2 Chl 0 ; L00 G.t/.xx  /iC

Chl 0 ; B.t /.u  p.t //i C hl 0 ; B.t /l 0 i1=2 C hl 0 ; L0 Ÿ.t /i  hl 0 ; L00 R.t /L0l i1=2 C

C hl 0 ; C.t /.vŒt   q.t //i  hl 0 ; C.t /l 0 i1=2 : (10.47)

10.7 Separation of Estimation and Control 391

Selecting

UL0 D U 0 .t; EC .x  ; XC .t // D arg minfhl 0 ; B.t /ui j u 2 E .p.t /; P .t //g D

D arg maxfhl 0 ; B.t /ui j u 2 E .p.t /; P .t //g; (10.48)

we minimize the derivative in (10.47), ensuring

ˇ
ˇ
d VhE .t; EL0 Œt /=dt ˇˇ  0:

Integrating the last inequality from t to ª; we come to the guaranteed condition

” 0  V .ª; EL0 Œª/  V .t; EL0 Œt /: (10.49)

The final result is given by

Theorem 10.6.4. The solution to Problem OFC is given by control strategy (10.48),
where l 0 is the optimizer in problem (10.44).
Exercise 10.6.1. Find the error estimates for (10.49)

10.7 Separation of Estimation and Control

In order to simplify calculations for linear systems we shall treat the given Problem
OFC in another coordinate system. Following Sect. 1.1 and taking G.t; £/ to be the
fundamental transition matrix

@G.t; £/
D A.t /G.t; £/; G.£; £/ D I;
@t

for homogeneous system xP D A.t /x from (10.29), we introduce the transformation

x D G.ª; t /x. which brings us to a new variable x. Making the necessary changes
and returning after that to the original notations, we transform system (10.29), to the
next one

xP D B.t /u C C.t /v.t /; (10.50)

y.t / D H.t /x C Ÿ.t /; (10.51)

under constraints u 2 P .t /, (10.4).

For passing to the solution of Problem OFC we present system (10.50), (10.51)
as the array

dx  =dt D B.t /u; x  .t0 / D 0; (10.52)

392 10 Uncertain Systems: Output Feedback Control

and

d w=dt D C.t /v.t /; w.t0 / 2 X 0 ; (10.53)

with

z.t / D H.t /w C Ÿ.t /; (10.54)

where
Z t

x C w D x; z.t / D y.t /  H.t / B.s/u Œsds:
t0

With u D uŒs; s 2 Œt0 ; t / given, there will be a one-to-one mapping

between realizations yŒs and zŒs. We may now define the information set of
system (10.53), (10.54), denoting it as Wz Œt  D Wz .t; /. Then we have Xy Œt  D
x  Œt  C Wz Œt .
Then x  .t / will define an isolated undisturbed controlled trajectory, while the
set-valued observer Wz Œt  will give a joint estimate of the existing disturbances in
system (10.53), (10.54). Tube Wz Œt  t 2 Œt0 ; £ may be calculated on-line, separately
from the problem of specifying the control itself.
Applying such separation to procedures of Sects. 10.5 and 10.6 may be useful for
solving concrete problems of OFC.
The chapter is now concluded by a few examples.

10.8 Some Examples

These examples illustrate the solution of Problem OFC under various types of
unknown disturbances. The system equation is

xP 1 D x2 C ©1 v
xP 2 D u C ©2 v

with measurement output

y.t / D x1 .t / C Ÿ.t /;

constraints juj  ; jvj    ; 0 < ©1 < ©2 < 1; jŸj  ˜; starting set

X 0 D E .0; I /; and target set E .m; M /; m 6D 0; M 6D I .
The actual disturbances which are unknown are modelled here by the following
types of functions:
in Fig. 10.1 by v.t / D  cos 5t; Ÿ.t / D ˜ sin 4t ,
Concluding Remarks 393

Fig. 10.1 Unknown oscillating disturbances

in Fig. 10.2 by
8
< ™1 ; with j™1 j  ; ™1 N .0; 1/;
v.t / D ; with ™1 < ;
:
; with ™1 > ;

where N .0; 1/ stands for the normal distribution with zero mean and unit
variance,
in Fig. 10.3 by v.t /  0; Ÿ.t /  0 which means the disturbances are zero-valued,
but we do not know that. This is the worst-case disturbance.

Concluding Remarks

This chapter describes theoretical tools and computation approaches for the problem
of optimizing OFC under set-membership bounds on controls and uncertain items. It
indicates solution routes, which allow one to cope with uncertainty and incomplete
measurements within finite-dimensional techniques, with effective computational
schemes given for linear systems without imposing matching conditions for the hard
bounds on controls and input disturbances. It also demonstrates the applicability of
tools presented in this book to significant problems of control. The chapter is also
an invitation to a broad research on challenging problems in feedback control under
realistic information with increasing number of new motivations.
394 10 Uncertain Systems: Output Feedback Control

Fig. 10.2 Unknown random disturbances

Fig. 10.3 Unknown disturbances are zero-valued

Chapter 11
Verification: Hybrid Systems

Abstract This chapter deals with a specific class of hybrid systems which combine
controlled continuous dynamics through switching from one available motion
to another due to discrete-time logical commands. Solutions to the reachability
problem and their verification are indicated, followed by computational schemes,
The application of impulse controls to the switching process is described. Examples
of various difficulty are worked out. The chapter is to demonstrate applicability of
methods of this book to hybrid systems.

Keywords Hybrid system • Switching • Reachability • Verification • Branching

trajectory tubes • Ellipsoidal methods • Impulse feedback control

The main part of this chapter deals with a specific class of hybrid systems
(see [39, 91, 204, 228, 258]). Their performance is due to an array of standard
systems with acting motion generated by only one of them and with instantaneous
switchings from one to another. The switchings are governed by a discrete time
process that generates logically based commands when crossing some prespecified
spatial domains (“the guards”). The reachability problem for such processes leads
to branching trajectory tubes. Their description is complemented by verification
problems whose solution, loosely speaking, should show which branch intersects
with a given target zone or misses. Hence the chapter starts with description of the
verification problem and methods of its solution, then passes to described target
controlled hybrid dynamics and its verification. Ellipsoidal computation schemes
for problems of reachability and verification in hybrid processes for isolated linear
systems are indicated. Finally discussed is the application of impulse controls to
such dynamics. Several examples of various difficulty are demonstrated. The aim
of this chapter is to demonstrate the applicability of methods given in this book to
hybrid processes.

© Springer International Publishing Switzerland 2014 395

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1__11
396 11 Verification: Hybrid Systems

11.1 Verification Problems

The problems of reachability and solvability are closely related to that of verification
for the achieved solutions. The aim of verification is to check whether suggested
algorithms do solve the intended problem.

11.1.1 The Problems and the Solution Schemes

Problem 11.1.1. Given time ª, target set M 2 compRn and set X 0 D X Œt0 , verify
which of the following relations is true:
(i) X .ª; t0 ; X 0 / M (all the reachable points are in M ),
(ii) X .ª; t0 ; X 0 / \ M 6D ; (some of the reachable points are in M ),
(iii) X .ª; t0 ; X 0 / \ M D ; (set M is not reachable from ft0 ; X 0 g at time ª).
Since X .ª; t0 ; X 0 / is the level set of the value function at time t D ª (see Chap. 2,
Sect. 2.3):
V .t; x/ D V .t; x j t0 ; X 0 / D min fd 2 .x.t0 /; X 0 / j x.t / D xg;
u./;x.t0 /

V .t0 ; x/ D d 2 .x; X 0 /;
we may check conditions (i)–(iii) using the following relations:
Theorem 11.1.1. Following conditions (i)–(iii) of Problem 11.1.1 are, respectively,
equivalent to the next relations, with X .ª; t0 ; X 0 / D fx W V .ª; x/  0g W

maxfd 2 .x; M /jV .ª; x/  0g D 0; (11.1)

x

minfd 2 .x; M /jV .ª; x/  0g D 0; (11.2)

x

minfd 2 .x; M /jV .ª; x/  0g > 0: (11.3)

x

Corollary 11.1.1. If M is the level set for a certain convex function ¥, such that

M D fx W ¥.x/  0g;
then conditions (11.1)–(11.3) are, respectively, equivalent to

maxf¥.x/jV .ª; x/  0g  0;
x

minf¥.x/jV .ª; x/  0g  0;
x

minf¥.x/jV .ª; x/  0g > 0:

x

In the previous propositions we dealt with verification of the solutions at fixed

time ª. These may be further extended in the following way.
11.1 Verification Problems 397

Theorem 11.1.2. Recalling Problem 11.1.1, suppose

sup maxfd 2 .x; M /jV .t; x/  0I t  t0 g D 0

t x

then property (i) is ensured for all t  t0 ;

if min maxfd 2 .x; M /jV .t; x/  0I t  t0 g D 0;

t x

then (i) is true for some t  t0 ;

if sup minfd 2 .x; M /jV .t; x/  0I t  t0 g D 0;

t x

then (ii) will be true for all t  t0 ;

if min minfd 2 .x; M /jV .t; x/  0I t  t0 g D 0;

t x

then (ii) will be true for some t  t0 ;

if inf minfd 2 .x; M /jV .t; x/  0I t  t0 g > 0;

t x

then (iii) will be true for all t  t0 ;

if sup minfd 2 .x; M /jV .t; x/  0I t  t0 g > 0;

t x

then (iii) will be true for some t  t0 .

Exercise 11.1.1. Check the validity of Theorems 11.1.1, 11.1.2 and Corollary
11.1.1
Examples of solutions to this problem are given in Sect. 11.1.2.
An array of problems comes up in backward time. We mention some of these.
Problem 11.1.2. Given starting set F , and fixed time t , determine whether target
set M is reachable from position ft; xg; at any time ª  t
(i) for all states x 2 F ,
(ii) for some state x 2 F ,
(iii) is not reachable from any state x 2 F .
The conditions that ensure the solutions to this Problem are now formulated through
value function

V0 .t; x/ D V0 .t; xjª; M / D minfd 2 .x.ª/; M / j x.t / D xg; ª  t;

u./
398 11 Verification: Hybrid Systems

Theorem 11.1.3. The set M is reachable from position ft; xg at some ª

(i) for all states x 2 F if

V0 .t; F / D min maxfV0 .t; xjª; M /j ª  t; x 2 F g D 0; (11.4)

ª x

(ii) for some state x  2 F if

V0 .t; F / D min minfV0 .t; xjª; M /jª  t; x 2 F g D 0; (11.5)

ª x

(iii) is not reachable from any state x 2 F if

V0 .t; F / D minfV0 .t; x; M / j x 2 F g D inf minfV0 .t; xjª; M /jª  t; x 2 F g > 0:

x ª x
(11.6)

Remark 11.1.1. Note that if in the above and in the sequel we write minª over
domain ª  t0 , then this presumes that the minimum should be attained at some
finite time ª .

11.1.2 Ellipsoidal Techniques for Verification Problems

Consider system (3.1) under ellipsoidal constraints (3.4), (3.6), (3.7) and ellipsoidal
target set M D E .m; M /. Dealing with formulation of the verification Prob-
lem 11.1.1 (i)–(iii) we shall solve it through ellipsoidal methods, further naming
it then as a new Problem 11.1-E (i)–(iii).
Recall that due to (3.23) we have
\
X λ D fE .x ? λ; XCl λ/j l W hl; li D 1g: (11.7)

where EC Œl; ª D E .x ? Œª; XC

l
λ/ are tight external approximations of X λ along
direction l.
At the same time Theorem 3.7.3 indicates
[
X λ D fE .x ? λ; Xl λ/j l W hl; li D 1:g (11.8)

where E Œl; ª D E .x ? Œª; Xl Œª/ are tight internal approximations of X Œª along
direction l.
Beginning with Problem 11.1.1-E(i) we have to check the inclusion

X λ E .m; M /: (11.9)

Then to ensure (11.9) we need to have either

minf¡.l j E .m; M //  ¡.l j EC Œl; ª/ j hl; li  1g  0: (11.10)

l
11.1 Verification Problems 399

or

minf¡.l j E .m; M //  ¡.l j E Œl; ª/ j hl; li  1g  0: (11.11)

l

T
For Problem 11.1.1-E(ii) we T have to check condition X λ E .m; M / 6D ;.
This will be ensured if E Œl; ª E .m; M / 6D ; for some tight internal approx-
imation E Œl; ª D E .x  Œª; Xl .ª// of X Œª. The last requirement will be ensured
iff for some such E Œl; ª the inequality

¡.l j E Œl; ª/  ¡.l j E .m; M //  0;

for all l will be true or, in other terms, if

minf¡.l j E Œl; ª/ C ¡.l j E .m; M // j hl; li  1g  0; (11.12)

l

Finally, for Problem 11.1.1-E(iii) we have to check that X λ \ E .m; M / D ;. This
will be ensured if EC Œl; ª \ E .m; M / D ; for some tight external approximation
EC Œl; ª of X Œª; or in other terms, if the distance d.EC Œl; ª; E .m; M // > 0 for
some such EC Œl; ª. The latter holds iff

maxfmaxf¡.p j EC Œl; ª/  ¡.p j E .m; M // jhp; pi  1 g j hl; li  1 g > 0:

l p
(11.13)
Theorem 11.1.4. The solutions to (i)–(iii) of Problem 11.1.1, as written in terms of
approximating ellipsoids, are satisfied iff the following relations are true: (11.10)
or (11.11) for (i); (11.12) for (ii) and (11.13) for (iii).
Now introduce some new notation, namely,

VC .ª; x; l/ D hx  x  Œª; XC
l
λ.x  x  λ/i:

Then

EC Œl; ª D fx W VC .ª; x; l/  1g; E Œl; ª D fx W V .ª; x; l/  1g:

Denote
VC .ª; x/ D maxfVC .ª; x; l/ j hl; li D 1g; V .ª; x/ D minfV .ª; x; l/ j hl; li D 1g:
l l

Here both functions VC .ª; x; l/; V .ª; x; l/ may serve as quadratic approximations
of value function V .ª; x/ that defines X Œª D X .ª; t0 ; X 0 / (see Chap. 2, Sect. 2.3).
Namely,

VC .ª; x; l/  VC .ª; x/ D V .ª; x/ D V .ª; x/  V .ª; x; l/: (11.14)

Further denote Vm .x/ D hx  m; M.x  m/i; so that E .m; M / D fx W Vm .x/  1g.

Theorem 11.1.4 may then be rewritten in terms of functions VC ; V ; Vm .
400 11 Verification: Hybrid Systems

Theorem 11.1.5. Suppose one of the conditions is true:

(i) Vm .x/  VC .ª; x/ D maxfVC .ª; x; l/ j hl; li D 1g; 8x;

l

O x D xO W Vm .x/
.i i / f9 l D l; O  V .ª; x; O
O l/g;

O  1g > 1g:
.i i i / f9 l D lO W minfVm .x/ j VC .ª; x; l/

Then, respectively, the requirements (i)–(iii) of Problem 11.1.1 will be fulfilled.

This theorem implies the following conclusions (in view of V .ª; x/ D
VC .ª; x/ D V .ª; x/ ).
Corollary 11.1.2. The solutions to (i)–(iii) of Problem 11.1.1, as written in terms
of approximating ellipsoids, are satisfied iff, respectively, the following relations are
true:

.i / maxfd 2 .x; E .m; M // j V .ª; x/  1g D 0;

x

.ii/ minfd 2 .x; E .m; M // j V .ª; x/  1g D 0;

x

.iii/ minfd 2 .x; E .m; M // j V .ª; x/  1g > 0:

x

An ellipsoidal version of Theorem 11.1.2 then reads as follows.

Theorem 11.1.6. If

sup maxfd 2 .x; E .m; M // j V .t; x/  1; t  t0 g D 0;

t x

then property (i) will be true for all t  t0 ;

if min maxfd 2 .x; E .m; M // j V .t; x/  1; t  t0 g D 0;

t x

then (i) will be true for some t  t0 ;

if sup minfd 2 .x; E .m; M // j V .t; x/  1; t  t0 g D 0;

t x

then (ii) will be true for all t  t0 ;

if min minfd 2 .x; E .m; M // j V .t; x/  1; t  t0 g D 0;

t x

then (ii) will be true for somet  t0 ;

if inf minfd 2 .x; E .m; M // j V .t; x/  1; t  t0 g > 0;

t x
11.2 Hybrid Dynamics and Control 401

then (iii) will be true for all t  t0 ;

if sup minfd 2 .x; E .m; M // j V .t; x/  1; t  t0 g > 0;

t x

then (iii) will be true for some t  t0 .

Exercise 11.1.2. Formulate and prove an ellipsoidal version of Theorem 11.1.6.

11.2 Hybrid Dynamics and Control

The previous chapters of this book deal with various types of problems on control for
systems whose mathematical models are described by standard ordinary differential
equations. However, recent applications may require treating such problems in a
more complicated setting, under complex dynamics. In the present chapter such
systems are of the hybrid type. The notion of hybrid system has various definitions
(see [36,39,204,229,258,259]). However, the main idea is that the system is defined
by an array of standard systems such that the active motion is due to only one of
them, with instantaneous switching from one to another. The process of switching
is usually logically controlled in discrete times, in such way that a possible switch
may occur only when passing through some spatial domains (the “guards”), or may
not occur. The performance range of such systems is then obviously broader than
that of standard systems. Hence, the overall controlled motions will develop in
time as those generated by alternating isolated continuous motions due to systems
whose sequence is controlled through logically based discrete commands, while
their individual contribution is designed by its own controllers. The switching from
one system to another may be also accompanied by an instantaneous change of some
phase coordinates.
Despite the fairly complicated overall dynamics, efficient computation of control
solutions for hybrid processes appears to be also available through ellipsoidal-
valued approximations for systems composed of individual participants of the types
described above. The discussion given below is devoted to linear controlled systems,
though the overall schemes may be also applied under individual nonlinearities.

11.2.1 The Hybrid System and the Reachability Problem

The System

The considered overall system H is governed by an array of linear subsystems

(i D 1; : : : ; k)

xP 2 A.i/ .t /x C B .i/ .t /u.i/ .t / C C .i/ .t /v.i/ .t /; (11.15)

402 11 Verification: Hybrid Systems

with continuous matrix coefficients A.i/ .t /; B .i/ .t /; C .i/ .t / and x 2 Rn . Here

u.i/ 2 Rp are piecewise open-loop controls u.i/ .t / restricted by inclusions
u.i/ .t / 2 P .i/ .t /, where P .i/ .t /, are set-valued functions with convex compact
values, continuous in time. Functions v.i/ are taken to be given.
In this section we consider hard bounds that are ellipsoidal-valued, with

P .i/ .t / D E .p .i/ .t /; P .i/ .t //: (11.16)

In the phase space Rn given are k hyperplanes (affine manifolds)

Hj D fx W hc .j / ; xi  ”j D 0g; c .j / 2 Rn ; ”j 2 R ; j D 1; : : : ; k:

which are the enabling zones (the guards).

Here is how the system operates.
At given time t0 the motion initiates from starting set X 0 D E .x 0 ; X 0 / and
follows one of the equations of the above (take i D 1, to be precise), due to one of
the controls u.t I 1/ until at time £0 Œ1 it reaches some Hj D H.j Œ1/—the first of the
hyperplanes along its route (we assume j 6D 1). Now a binary operation interferes—
the motion either continues along the “old” subsystem (a “passive” crossing), with
system number i D 1 or switches to (is reset to) a “new” subsystem (an “active”
crossing), with system number indexed as i D i Œ1 D j Œ1 6D 1. (Otherwise, if
j Œ1 D 1, we presume there is a passive crossing, with no reset.)
Some Notation. Each number Œ› D Œ1; Œ2 : : : ; in square brackets is the sequential
number of crossing a hyperplane Hj , so that the range of fj Œ›; › 2 ZC g is j 2
f1; : : : ; kg. Here and further ZC is the set of positive integers. The range of system
numbers i Œ›; › 2 ZC ; after each crossing is also i 2 f1; : : : ; kg.
Then, before crossing Hj Œ1 D H.j Œ1/, the state of the system is denoted as
ft; x; Œ1C g while after the binary operation it is either ft; x; Œ1C ; i Œ1C g—if the
crossing was active (a reset), or ft; x; Œ1C ; i Œ1 g, if it was passive (no reset).
We assume, for the sake of brevity, that an active crossing a hyperplane Hj is a
switching to system with i D j .
So, after first crossing, the further motion develops along i D 1 or i D i Œ1 until
crossing the next hyperplane H.j Œ2/ where a similar binary operation is applied—
the motion either continues along the previous subsystem or switches to (is reset to)
subsystem with index i Œ2 D j Œ2. After the second crossing the state of the system
is ft; x; Œ1C ; i Œ1s ; i Œ2s g, where the boolean index “s” is either s D C or s D .
Note that the state of the system has a part ft; xg without memory related to the
running on-line position of the continuous-time variable and a part Œ1C ; i Œ1s ; i Œ2s 
with memory, related to the discrete event variable i s Œ›, which describes the
sequence of switchings made earlier by the system.1

1
The part with memory may be important for making the decision—“to switch” or “not to switch,”
for example, if the number of switchings is restricted.
11.2 Hybrid Dynamics and Control 403

At each new crossing a new term is added to this sequence. Thus the following
general rules should be observed:
(i) that crossing each hyperplane Hj results either in a reset to subsystem with
number j or in no reset at all,
(ii) that the crossing takes place in direction of support vectors c .j / , and at points
of crossing we have

minfhc .j / ; zijz 2 F .i/ .t; x/g  © > 0; 8i; j D 1; : : : ; k; 8x 2 Hj

(11.17)
where

F .i/ .t; x/ D A.i/ .t /x C B .i/ .t /E .p .i/ .t /; P .i/ .t // C C .i/ .t /v.i/ .t /I

(iii) that the state space variable after › crossings is ft; x; i Œ1s1 ; : : : ; i Œ›s› g where
each “boolean” index “s” is either C or .
(iv) that at a crossing with hyperplane Hj the sequence of type fi Œ1s ; : : : ; i Œ›C g,
describing the “discrete event” part of the state is complemented by a new term,
which is either i Œ› C 1C if there is a switching to system, or i Œ› C 1 , if there
is no switching.
Such a notation allows to trace back the array of subsystems used earlier from any
current position ft; xg. Thus, if the state is ft; xI 1C ; i Œ1 ; : : : ; i Œ› g with s D 
for all i Œ1; : : : ; i Œ›, then the trajectory did not switch at all throughout any of the ›
crossings, having followed one and the same subsystem with i D 1 throughout the
whole process. Note that at each state ft; x; Œ1C ; i Œ1s ; : : : ; i Œ›s g the system is to
follow the subsystem whose number coincides with that of the last term with index
s D C.
We will be interested in the reachability problem for such systems.
Remark 11.2.1. The system under consideration is one of the possible types of
hybrid systems. It differs from the so-called switching systems in that the time
instants for crossing are not given, but are located during the spatial course of the
trajectory as intervals of crossing some specified domains (the “guards”) where it is
also possible to reset the phase coordinates. The suggested scheme also allows to be
propagated to a broad variety of options with different information requirements.
Remark 11.2.2. In this chapter the guards are taken as hyperplanes and the time
of each crossing is unique for every trajectory. An example of propagating the
suggested scheme is when the guards are taken as a domain bounded by two parallel
hyperplanes.The switching is then assumed possible at any time within this domain.
This situation with an example is indicated at the end of the present section.
The problems considered in this chapter deal with reachability under piecewise—
open-loop controls with possible resets of controlled systems at given guards,
taken as hyperplanes, so that between these zones the control is open-loop. For
further considerations the restriction on starting set X 0 is of ellipsoidal type:
X 0 D E .x 0 ; X 0 /.
404 11 Verification: Hybrid Systems

Reachability Under Resets

The reachability problem consists of two versions.

Problem 11.2.1. Find all vectors fxg reachable from starting position ft0 ; X 0 g at
given time t D £ through all possible controls: “the reach set X .£I t0 ; X 0 / at time £
from ft0 ; X 0 g”.
Problem 11.2.2. Find all vectors fxg reachable at some time t within interval t 2
Œt 0 ; t 00  D T through all possible controls: “the reach set X .t 0 ; t 00 I t0 ; X 0 / within
interval T from ft0 ; X 0 g”.
One may observe that the problem consists in investigating branching trajectory
tubes, in describing their cross-sections (“cuts”) and the unions of such cross-
sections. The reach sets may therefore turn out to be disconnected sets. We next
discuss reach sets at given time t .
We first describe the reach set for a given sequence Œi0 ; i Œ1s1 ; : : : ; i Œlsl  of
crossings, from position ft0 ; X 0 ; i0 g, assuming i0 D 1 to be precise. Here index
si is either  or C.

The Reach Set After One Crossing

(a) Suppose that before reaching Hj ; i Œ1 D j; we have

X .1/ Œt  D X .1/ .t I t0 ; X 0 ; Œ1C / D

Z t
G .1/ .t; t0 /X 0 C G .1/ .t; s/.B.s/P .1/ .s/ C C.s/v.1/ .s//ds
t0

where G .i/ .t; s/ is the transition function for the i -th subsystem.
(b) To be precise, suppose that before reaching Hj we have

maxfhc .j / ; xi j x 2 X .1/ Œt g D ¡C
j .t / < ”j :

T
The first instant of time when X .1/ Œt  Hj 6D ; is £0j . It is found as the
smallest root of the equation

”j  ¡C
j .t / D 0; t  t0 :

Introducing the function

minfhc .j / ; xi j x 2 X .1/ Œt g D ¡
j .t /;
T
we may also observe that condition X .1/ Œt  Hj 6D ; will hold as long as
C
¡
j .t /  ”j  ¡j .t /;
11.2 Hybrid Dynamics and Control 405

and the point of departure from Hj is the smallest positive root £00j of the
equation

”j  ¡ 0
j .t / D 0; t  £j :

Condition (11.17) ensures that points £0j ; £00j are unique. Note that £00j is the
time instant when the entire reach set X .1/ Œt  leaves Hj .
T .1/
Denote X .1/ Œt  Hj D Zj .t /.
(c) After the crossing we have to envisage two branches:
() with no reset—then nothing changes and

X .t I t0 ; X 0 ; Œ1C ; j  / D X .1/ .t I t0 ; X 0 ; Œ1C /

(C) with reset—then we consider the union

[
fX .j / .t I s; Zj .s//js 2 Œ£0j ; £00j g D X .t I t0 ; X 0 ; Œ1C ; j C /; t  £00j :
.1/

Here in case () the reach tube develops further along the “old” subsystem (1),
while in case (+) it develops along “new” subsystem (j ).

A Branch of the Reachability Set

For each new crossing we may now repeat the described procedure. Thus,
we may obtain the reach set X .t; t0 ; X 0 I 1C ; i Œ1s1 ; : : : ; i Œlsl / for a branch
f1C ; i Œ1s1 ; : : : ; i Œlsl g.
We further assume the next condition


0 00
Assumption 11.2.1. The neighboring intervals of crossing £› ; £› ; › D 1;
: : : ; l; do not intersect. It is presumed that £00› < ª; where Œt0 ; ª is the interval
under consideration. 
Taking an interval Œt0 ; t , with £00iŒk1  t  £0iŒk and £00iŒm1  t  £0iŒm ; i Œl <
i Œm, one may observe the following semigroup type property.
Lemma 11.2.1. Each branch f1C ; i Œ1s1 ; : : : ; i Œksm g yields the following superpo-
sition property (1  k  m)

X .t; t0 ; X 0 I 1C ; i Œ1s1 ; : : : ; i Œmsm / D

D X .t; t  ; X .t  ; t0 ; X 0 I 1C ; i Œ1s1 ; : : : ; i Œksl /I i ŒkslC1 ; : : : ; i Œmsm /: (11.18)


The computation of a branch may be done as a sequence of one-stage crossing
transformations.
406 11 Verification: Hybrid Systems

Recall that the continuous-time transition between crossings, along system

j D j1 from a continuous-time position f£; X g with £  £00j1 to a position at time
t  £0j2 ; is X .j1 / .t; £; X /.
Then, for example, given position (state) f£0j ; X I Œ1C g, at start of crossing Hj ,
we may define a “one-stage crossing” transformation

Tj f£0j ; X ; Œ1C g D f£00j ; X .1/ .£00j ; £0j ; X /; Œ1C ; j  g; if s D ;

.s/

and

Tj f£0j ; X ; Œ1C g D f£00j ; Z .j / Œ£00j ; Œ1C ; j C g; if s D C:

.s/

Here
[
Z .j / Œ£00j  D fX .j / .£00j ; t; Hj \ X .1/ .t; £0j ; X //j t 2 Œ£0j ; £00j g
t

The Branch of One-Stage Transformations

We may now represent a branch f1C ; i Œ1s1 ; : : : ; i Œksk g through a sequence of
.s/
interchanging operations of type Tj and X .j / .
For example, the reach set for branch f1C ; i Œ1C ; i Œ2 g; from starting position
f£; X ; Œ1C g; £  £0i1 , at time t 2 Œ£00i2 ; £0i3 ; may be represented as the output of the
following sequence of mappings. (Here and further, to clarify when necessary the
notation of lower indices, we denote i› D i ›)

Ti1C f£0i1 ; X .1/ .£0i1 ; £; X /; Œ1C g D f£00i1 ; Z .i1/ Œ£00i1 ; Œ1C ; i Œ1C g

X .i1/ Œ£0i2  D X .i1/ .£0i2 ; £00i1 ; Z .i1/ Œ£00i1 /;

Ti2 f£0i2 ; X .i1/ Œ£0i2 ; Œ1C ; i Œ1C g D f£00i2 ; X .i1/ .£00i2 ; £0i2 ; X .i1/ Œ£0i2 /; Œ1C ; i Œ1C ; i Œ2 g

Then, for t > £00i2 , the desired set of positions is given as

ft I X .i1/ .t; £0i2 ; X .i1/ .£0i2 ; £00i1 ; Z .i1/ Œ£00i1 //; Œ1C ; i Œ1C ; i Œ2 g:

This is a branch of the overall reach set. The continuous variables for this branch
at t 2 Œ£00i2 ; £0i3 / produce the equality

X .t I £; X / D X .i1/ .t I £0i2 ; X .i1/ Œ£0i2 /:

Lemma 11.2.2. A branch of type X .t I t0 ; X 0 / W i D 1˙ ; i Œ1s1 ; : : : ; i Œ›s› ; may be

described by a superposition of interchangings one-stage crossing transformations
Ti›s and continuous maps X .›/ ; › D 1; : : : ; l.
11.2 Hybrid Dynamics and Control 407

An alternative scheme for calculating reach sets is to describe them through value
functions of optimization problems. Its advantage is that it is not restricted to linear
systems.

11.2.2 Value Functions: Ellipsoidal Approximation

Reachability Through Value Functions

As indicated in [175], the “reach” (reachability) sets for “ordinary” (nonhybrid)

systems may be calculated as level sets of solutions to HJB (Hamilton–Jacobi–
Bellman) equations for some optimization problems. We will follow this scheme
for the hybrid system under consideration. Consider first a one-stage crossing.
(a) Before the crossing HiŒ1 D Hj , we assume that the system operates from
position ft0 ; X 0 ; Œ1C g and the next crossing starts at £1 . Then, for t < £1 , we
have

X .1/ Œt  D fx W V .1/ .t; x/  0g;

where

V .1/ .t; x/ D minfd 2 .x .1/ .t0 /; X 0 /jx .1/ .t / D xg;

u.1/

and x .i/ .t / D x.t / is the trajectory of the i-th system. We further also use the
notation V .1/ .t; x/ D V .1/ .t; x1 ; x2 ; : : : ; xn /.
.1/
(b) At the crossing we have X .1/ Œt  \ Hj D Zj .t / which can be calculated as
follows.
.j /
Without loss of generality we may presume c1 D 1. Then

X
n
Zj.1/ .t / D fx W x1 D —.x/; V .1/ .t; —.x/; x2 ; : : : ; xn /  0g \ Hj ; —.x/ D ”1  .j /
c i xi :
i D2

In particular, if the hyperplane Hj D fx W x1 D ”1 g, then

Zj.1/ .t / D fx W x1 D ”1 ; V .1/ .t; ”1 ; x2 ; : : : ; xn /  0g:

C
and the set Zj .t / 6D ; iff ¡
.1/
j .t /  ”j  ¡j .t /, where

¡C
j .t / D maxf.c
.j /
; x/ j V .1/ .t; x/  0g;

¡
j .t / D minf.c
.j /
; x/ j V .1/ .t; x/  0g:
C 0
This happens within the time interval Œ£0j ; £00j ; ¡ 00
j .£j / D ”j ; ¡j .£j / D ”j .
408 11 Verification: Hybrid Systems

(c) After the crossing we envisage two branches:

() with no reset—then X .t I t0 ; X 0 ; Œ1C ; j  / D X .1/ .t I t0 ; X 0 ; Œ1C /,
(C) with reset—then we have to calculate the union
[
fX .j / .t I s; Zj .s//js 2 Œ£0j ; £00j g D X .t I t0 ; X 0 ; Œ1C ; j C /; t  £00j :
.1/

With t > £00j this union may be calculated as the level set for function

V .t; x j Œ1C ; j C / D minfV .j / .t; s; x/ j s 2 Œ£0j ; £00j g

s

where

V .j / .t; s; x/ D minfV .1/ .s; —.x.s//; x2 .s/; : : : ; xn .s//j s 2 Œ£0j ; £00j ; x.t / D xg;
u.j /

so that

X .t I t0 ; X 0 ; Œ1C ; j C / D fx W V .t; xjŒ1C ; j C /  0g D

[
fX .j / Œt; s j s 2 Œ£0j ; £00j g;
s

X .j / Œt; s D X .j / .t I s; Zj.1/ .s// D fx W V .j / .t; s; x/  0g:

Remark 11.2.3. Note that in general the union above is nonconvex.

(d) Repeating the procedure for each new crossing, we may obtain the reach set
X .t I t0 ; X 0 ; Œ1C ; i Œ1.s1 / ; : : : ; i Œk.sk / / for a branch Œ1C ; i Œ1.s1 / ; : : : ; i Œk.sk / .
These are the general schemes to compute reachability sets for the given class of
hybrid systems. We further indicate some ellipsoidal techniques for problems of
this chapter.

Reachability Through Ellipsoidal Approximations

Let us first calculate the reachability set after a one-stage crossing transformation.
(a) Starting with X 0 D E .x 0 ; X 0 /; i D 1; the reach set X .t I t0 ; X 0 ; Œ1C / is due
to equation

xP D A.1/ .t /x C B .1/ .t /E .p .1/ .t /; P .1/ .t // C C .1/ .t /v.1/ .t /;

11.2 Hybrid Dynamics and Control 409

which yields the following ellipsoidal approximations (see Chap. 3,

Sects. 3.2, 3.3, 3.7 and 3.9).

E .x .1/ .t /; X.1/ .t // X .t I t0 ; X 0 ; Œ1C / E .x .1/ .t /; XC.1/ .t //;

where
0 0
XP C D A.1/ .t /XC CXC A.1/ .t /C.t /XC C..t //1 B .1/ .t /P .1/ .t /B .1/ .t /;
.1/ .1/ .1/ .1/

(11.19)
0
XP .1/ D A.1/ .t /X.1/ C X.1/ A.1/ .t /C

0 .1/ 0
CX S.t /B .1/ .t /.P .1/ /1=2 .t / C .P .1/ /1=2 .t /B .1/ .t /S 0 .t /X ;
.1/

xP .1/ D A.1/ .t /x .1/ C B .1/ .t /p .1/ .t / C C .1/ .t /v.1/ .t /;

.t/ > 0; S.t/S 0 .t/ D I; X X

0 .1/
D X ; XC .t0 / D X
.1/
.t0 / D X 0 ; x .1/ .t0 / D x 0 :

Here

.t / D hl.t /; B.t /P .t /B 0 .t /l.t /i1=2 .l; XC .t /l/1=2 ;

.1/

S.t /B.t /P 1=2 .t /l.t / D œ.t /S0 .X 0 /1=2 l;

œ.t / D hl.t /; B.t /P .t /B 0 .t /l.t /i1=2 hl; X 0 li1=2 ; S00 S0 D I:

As indicated in Chap. 3, these approximations, which depend on l; will be

0
tight along a given good curve l.t / D G .1/ .t0 ; t /l; l 2 Rn , generated by
G .i/ .t0 ; t /—the fundamental transition matrix for the homogeneous system
.1/
(11.15). Namely, for such l D l.t / we have X.1/ .t / D X .1/ .t jl/, XC .t / D
.1/
XC .t jl/ and

.t jl/// D ¡.l j X .t I t0 ; X 0 ; Œ1C // D ¡.l j E .x .1/ .t /; XC .t jl///:

.1/
¡.l j E .x .1/ .t /; X
.1/

(11.20)
Moreover,
[ \
E .x .1/ .t /; X.1/ .t jl// D X .t I t0 ; X 0 ; Œ1C / D E .x .1/ .t /; XC.1/ .t jl//:
l l
(11.21)
over all fl W hl; li  1g.
.j / .1/
(b) Let us now discuss the crossings E1 .t jl/ D E .x .1/ .t /; XC .t jl// \ Hj . Let
.i/
e be the unit orths for the original coordinate system.
Introducing the linear map

Tc .j / D e.1/ I Te.i/ D e.i/ ; i D 1; : : : ; k; i 6D j; jTj 6D 0;

410 11 Verification: Hybrid Systems

Without loss of generality we may transform hyperplane Hj into hyperplane

x1 D ” and take ” D 0; marking Hj as H1 .j / and defining it by the equality
x1 D 0. Then, keeping previous notations in the new coordinates, we have2

.j / .j /
E1 .t jl/ D E .z.j / .t /; Z1C .t j l// D

.1/ .1/
D E .x .1/ .t /; XC .t j l// \ H1 .j / D fx W VC .t; x j l/  1; x1 D 0g;

where

VC .t; x j l/ D hx  x .1/ .t /; .XC /1 .t j l/.x  x .1/ .t //i:

.1/ .1/
(11.22)

.j /
The intersection E1 .t jl/ is a degenerate ellipsoid whose support function
is
.j / .j /
¡.ljE1 .t j l// D hl; z.j / .t /i C hl; Z1C .t j l/li1=2 ;

which may be calculated through standard methods of linear algebra.

Exercise 11.2.1. Calculate the parameters of support function for ellipsoid
.j /
E1 .t jl/.
Now, in the n-dimensional space H1 .j / D fx W x1 D 0g, we may consider an
.j /
array of ellipsoids E .z.j / .t /; Z1C .t j l//, and
[n \ o
t 2 Œ£0j ; £00j  D T D Tl Œ£0 ; £00  j hl; li  1 ;

where

Tl D ft W 9x D .0; x2 ; : : : ; xn / W hx  x .1/ .t /; .XC.1/ /1 .t j l/.x  x .1/ .t //i  1g:

Then for any l we have the relations

\
E .z.j / .t /; Z1.jC/ .t j l//
.j / .j /
fE .z.j / .t /; Z1C .t j l// j hl; li  1g D Z1 Œt :
l

where the equality follows from (11.20).

When propagated after the reset along the new subsystem .j /, with ª  £00j , each
.j /
of the ellipsoids E .z.j / .t /; Z1C .t j l// is transformed into

2
From here on it is important to emphasize the dependence of ellipsoids, reach sets, and value
functions on l. Therefore we further include l in the arguments of respective items.
11.2 Hybrid Dynamics and Control 411

XC.j / Œª; t j l  D XC.j / .ª; t; E .z.j / .t /; Z1.jC/ .t j l/// D

.j /
G .j / .ª; t /E .z.j / .t /; Z1C .t j l//C
Z ª
C G .j / .t; s/.B.s/E .p .j / .s/; P .j / .s// C C.s/v.j / .s//ds:
t

Note that though generated by an ellipsoid, the set X .j / λ; t  in general is not an
ellipsoid. However, it may be externally approximated by an array of parametrized
ellipsoids according to Sects. 3.2 and 3.3. Namely the following property is true.
.j /
Lemma 11.2.3. The exact reachability set from position ft; Z1C Œt g is
\ .j /
XC.j / λ; t  D fXC λ; t j l  j hl; li  1g:

Here for each l there is an ellipsoidal approximation. For example, taking t D £ 2

Œ£0 ; £00  in the previous line gives

XC.j / Œª; £ j l  E£ .x .j / .ª/; XC.j / .ª j q; l//:

.j /
Elements x .j / .t /; XC .t /; t 2 Œ£; ª of this relation satisfy the following equations

0 0
XP C D A.j / .t /XC CXC A.j / .t /Cq .t /XC C.q .t //1 B .j / .t /P .j / .t /B .j / .t /;
.j / .j / .j / .j /

(11.23)
xP .j / D A.j / .t /x .j / C B .j / .t /p .j / .t / C C .j / .t /v.j / .t /;

with starting conditions

.j / .j /
x .j / .£/ D z.j / .£/; XC .£/ D Z1 Œ£:

.j /
Note that the approximating sets E£ .x .j / .ª/; XC .ª j q; l// depend on two vector
parameters: l 2 Rn (calculated with  D l ; from Eq. (11.20) and responsible for
.j /
finding Z1 Σ) and q 2 Rn1 (calculated with q ; from Eq. (11.19) and responsible
for finding X .j / Œª; £). Function q .t / is calculated by formulas similar to l .
We finally come to the following conclusion
Theorem 11.2.1. The following equality is true:
\
XC.j / λ; t j Z1.j / Σ D .j /
fE£ .x .j / .ª/; XC .ªjq; l// j klk  1; kqk  1g:

The last formula indicates the possibility of using parallel calculations through an
array of parametrized identical procedures and synchronization of their results.
412 11 Verification: Hybrid Systems

The final move is now to find the union

[
X .ª; t0 ; X 0 / D fXC.j / .ª; £ j Z1.j / Œ£/ j £ 2 T g: (11.24)

which produces the final nonconvex reachability set.

Theorem 11.2.2. The final reachability set X .ª; t0 ; X 0 / after one crossing with an
active switching is given by the union (11.24).
Hence the final reach set may be presented as the union of intersections, namely

X .ª; t0 ; X 0 / D
[\ .j /
fE£ .x .j / .ª/; XC .ª j q; l// j klk  1; kqk  1; £ 2 T g: (11.25)
£ l;q

The above relation is written in the form of set-valued functions. But it may be
also written in terms of single-valued functions. Indeed, since

E£ .x .j / .ª/; XC.j / .ª j q; l// D fx W V .ª; £; x j q; l/  1g;

and

V .ª; £; x j q; l/ D fx W hx  x .j / .t /; .XC .ª; £ j q; l//1 .x  x .j / .t //i  1g;

.j /

define

V.ª; t0 ; x/ D min maxfV .ª; £; x j q; l/ j klk  1; kqk  1; £ 2 T g: (11.26)

£ l;q

Then
Theorem 11.2.3. The reachability set X .ª; t0 ; X 0 / after one crossing with an active
switching is the level set

X .ª; t0 ; X 0 / D fx W V.ª; t0 ; x/  1g: (11.27)

The above value functions thus allow both external approximations and an exact
description of the reach set. A similar scheme is true for internal approximations.
Recall that calculation of unions of reachability sets was discussed in Sect. 5.2
where an example was also given. For computational purposes we refer to following
considerations from this subsection.
According to the theory of minimax problems we have

V.ª; t0 ; x/ D min maxfV .ª; £; x j q; l/ j klk  1; kqk  1; £ 2 T g 

£ l;q

max minfV .ª; £; x j q; l/ j klk  1; kqk  1; £ 2 T g D V] .ª; t0 ; x/

l;q £
11.2 Hybrid Dynamics and Control 413

which in terms of sets yields

[\ .j /
X .ª; t0 ; X 0 / D fE£ .x .j / .ª/; XC .ª j q; l// j klk  1; kqk  1; £ 2 T g
£ l;q

\[ .j /
fE£ .x .j / .ª/; XC .ª j q; l// j klk  1; kqk  1; £ 2 T g D XC .ª; t0 ; X 0 /
l;q £

Set XC .ª; t0 ; X 0 / is therefore an external approximation of the nonconvex exact

set X .ª; t0 ; X 0 / which captures the nonconvexity of the exact one.
Example 11.1 (Replacing Example 5.8).

xP 1 D x2 C r cos ¨ t; xP 2 D u C r sin ¨ t; t 2 Œ0; ª; juj  1;

we shall look for the reach set

[
X .‚; 0; X 0 / D X .ª; 0; 0; X 0 / D fX .t; 0; X 0 / j t 2 ‚g; ‚ D Œ0; ª;
t

where
\
X .t; 0; X 0 / D fE .x .c/ .t /; XC .t j l// j hl; li  1g:

.1/
is an intersection of tight ellipsoids produced using (11.19), with x .1/ D x .c/ ; XC D
XC .t j l/.
Denote Vl .t; x/ D fx W hx  x .c/ ; XC .t j l/.x  x .c/ /i  1g. Then, due to
relations

V.ª; 0; x/ D min maxfVl .t; x/ j hl; li  1; t 2 Œ0; ªg 

t l

 max minfVl .t; x/ j hl; li  1; t 2 Œ0; ªg D V] .ª; 0; x/;

l t

we may find the exact nonconvex reachability set or its external approximation (also
nonconvex), namely,

fx W V.ª; 0; x/  1g D X .ª; 0; X .0/ / XC .ª; 0; X .0/ / D fx W V] .ª; 0; x/  1g:

Figure 11.1 demonstrates the intersection of nonconvex level sets over all fl W
hl; li  1g; which coincides with the external approximation XC .ª; 0; X .0/ /.
414 11 Verification: Hybrid Systems

Fig. 11.1 Nonconvex level sets of function minfVl .t; x/ j t 2 ‚g for fixed values of l and their
intersection

Another Type of Enabling Zone: A Gap

Consider a family of parallel hyperplanes Hj .”/ D fx W hc .j / ; xi D ”g parametrized

by variable ”. The enabling zone will now be Dj —the domain (a “gap”) between
hyperplanes Hj .” .1/ / and Hj .” .2/ /, so that Dj D Dj .” .1/ ; ” .2/ / D fx 2 Hj .”/j
” 2 Œ” .1/ ; ” .2/ g.
We shall discuss one crossing of the gap Dj . Then at time t , before reaching Dj ,
we have

maxfhc .j / ; xi j x 2 X Œt g D ” C .t / < ” .1/ ;

T
and the first instant of time when X Œt  Dj 6D ; is £0j —the smallest positive root
of equation ” .1/  ” C .t / D 0:
Now, with t  £0 there may be either no resets or a reset to system j while
passing Dj . If there are no resets, then introducing the function

minfhc .j / ; xi j x 2 X Œt g D ”  .t /;
T
we observe that condition X Œt  Dj 6D ; will hold as long as ” C .t /  ” .1/ and
”  .t /  ” .2/ , and the point of departure from Dj will be the smallest root £00j  £0j
of equation ” .2/  ”  .t / D 0.
11.2 Hybrid Dynamics and Control 415

If there is a reset from X Œt  0 to X  Œt , it may happen at any time £  £0j ,

with respective Hj D Hj .”/; ” D ”.£/; provided ”.£/  ” .2/ . Hence, denoting
X Œ£ \ Hj .”.£// D Zj .”.£//; we further have the union
[
X  Œt  D fX  .t I s; Zj .”.s///js 2 Œ£; t g: (11.28)

Then, for every ”.t / 2 .” .1/ ; ” .2/ ; we introduce

minfhc .j / ; xi j x 2 X  Œt g D ”r .t /;

and the time of departure of X  Œt  from Dj will be the smallest root £00jr of equation
” .2/  ”r .t / D 0.
Condition (11.17) ensures that the points £0j ; £00jr are unique. What follows is now
similar to what was written above for guards taken as hyperplanes.
We discuss an example.
Exercise 11.2.2. A branching reachability tube.
Consider the system

xP .i/ D A.i/ x .i/ C B .i/ u.i/ C f .i/ .t /; i D 1; 2; (11.29)

with parameters
   
0 1 0 2
A .1/
D ; A D
.2/
; B .1/ D B .2/ D I;
1 0 2 0
   
7 0
f .t / 
1/
; f .t / 
.2/
;
2 20

p .1/ D p .2/ D 0; P .1/ D P .2/ D 0:005I:

The motion starts from position x.0/ 2 E .x 0 ; X 0 / ,

   
10 0 1
X0 D ; x0 D : (11.30)
0 0 1

and moves throughout interval t 2 Œ0; ª due to system i D 1.

The reset to system i D 2 is possible upon intersecting with the “gap”

D D D .c; ”/ D fx W hc; xi D ”; ” 2 Œ” .1/ ; ” .2/  W

1
c D p .1; 1/0 ; ” .1/ D 3; ” .2/ D 5:
2
416 11 Verification: Hybrid Systems

Fig. 11.2 Reachability set for each branch

Fig. 11.3 Reachability tubes for both branches (left) and the overall branching reachability set
(right)

Indicated in Figs. 11.2 and 11.3 are external approximations for the two branches of
the reachability set after crossing the gap D at time t D 3. Here for i D 1 there is
no reset and the reachability set is a convex set X Œt , while for i D 2 there is a reset
from i D 1 to i D 2 and X  Œt  is a nonconvex union of convex sets of type (11.28).
11.3 Verification of Hybrid Systems 417

11.3 Verification of Hybrid Systems

We apply the results of Sect. 11.1 to hybrid systems.

11.3.1 Verification: Problems and Solutions

Suppose M D E .m; M /; M D M 0 > 0; is the given target and M \ Hi D ;; 8i 2

f1; : : : ; kg.
Problem 11.3.1. Given starting position ft0 ; X 0 ; Œ1C g; target set M and time t >
t0 ; verify whether there exists a branch I .k/ D Œ1C ; i1 1 ; : : : ; ik k  for which one of
.s / .s /

the following conditions is true (these cover all possible situations)

(i) X .t; t0 ; X 0 ; I .k// \ M D ;;

(ii) X .t; t0 ; X 0 ; I .k// \ M 6D ;;

(iii) X .t; t0 ; X 0 ; I .k// M :

Let us investigate the solution of this problem for a given branch I .j / D Œ1C ;
.s /
i1 1 ; : : : ; ij j , assuming k D j; t > £00j .
.s /

Theorem 11.3.1. The following conditions are true.

S
(A) Suppose set X .j / Œt  D X .t; t0 ; X 0 ; I .j // D fX .j / Œt; s j s 2 Tj D Œ£0 ; £00 g is
convex.
Then case (i) of Problem 11.3.1 holds if and only if

maxf¡.l j M /  ¡.l j X .j / Œt / j hl; li  1g D • > 0: (11.31)

Case (ii) holds if and only if

maxf¡.l j M /  ¡.l j X .j / Œt / j hl; li  1g  0: (11.32)

Case (iii) holds if and only if

minf¡.l j M /  ¡.l j X .j / Œt / j hl; li  1g  0: (11.33)

(B) Suppose the sets X .j / Œt; s are convex, but their union set X .j / Œt  D
[fX .j / Œt; s j s 2 Tj D Œ£0 ; £00 g is not convex. Then case (i) of Problem 11.3.1
holds if and only if

min maxf¡.l j M /  ¡.l j X .j / Œt; s/ j hl; li  1; s 2 Tj g D • > 0:

s l
(11.34)
418 11 Verification: Hybrid Systems

Case .i i / holds if and only if

min maxf¡.l j M /¡.l j X .j / Œt; s/ j hl; li  1; s 2 Tj g  0: (11.35)

s l

Case (iii) holds if and only if

min minf¡.l j M /  ¡.l j X .j / Œt; s/ j hl; li  1; s 2 Tj g  0: (11.36)

s l

If the same properties have to be verified for nonconvex sets of type

[
X Œª0 ; ª00  D fX Œt  j t 2 Œª0 ; ª00 g;

reachable within interval Œª0 ; ª00 ; ª00  £00 ; then one has to repeat operations similar
to (11.34)–(11.36), but for X Œt  taken within this interval.
If there is more than one crossing, then each new crossing adds a new parameter
s D sk ; with range within related interval of type Œ£0k ; £00k . So, after j such active
intervals, it would be necessary to compute related unions of nonconvex sets involv-
ing optimization of parametrized value functions over parameters sk ; k D 1; : : : ; j .
The realization of such computations brings us to ellipsoidal approximations.

11.3.2 Ellipsoidal Methods for Verification

We now introduce some ellipsoidal procedures for problems of the previous

subsubsection. T
Suppose X .k/ Œt; s D fEŒt; s; l hl; li  1g; where EŒt; s; l is an ellipsoidal
function defined for all fs 2 Œ£0 ; £00  D Tk g and all fl W hl; li  1g, continuous in all
the variables. and such that for all q we have
 \ 
¡ qj fEŒt; s; l j hl; li  1g D minf¡.q j EŒt; s; l/ j hl; li  1g:
l

Due to (11.20), (11.21)

S this property is true for system (11.19).
Denote X .k/ Œt  D s X .k/ Œt; s;  D E.m; M /.
Lemma 11.3.1. The next conditions are true.
(i) For a given s 2 Tk condition X .k/ Œt; s \ E .m; M / D ; holds if and only if

d1 Œt; s D max maxf¡.q j EŒt; s; l/¡.q j E.m; M // j hq; qi  1g  • > 0

l q
11.4 Impulse Controls in Hybrid System Models 419

and X .k/ Œt  \ E .m; M / D ; if and only if

d1 Œt  D minfd1 Œt; s j s 2 T g  • > 0:

s

(ii) For a given s 2 Tk condition X .k/ Œt; s \ E .m; M / 6D ; holds if and only if

d2 Œt; s D max maxf¡.q j EŒt; s; l/  ¡.q j E.m; M // j hq; qi  1g  0;

l q

and X .k/ Œt  \ E .m; M / 6D ; if and only if

d2 Œt  D minfd1 Œt; s j s 2 T g  0:
s

(iii) For a given s 2 Tk condition X .k/ Œt; s E .m; M / holds if and only if

d3 Œt; s D min maxf¡.q j E.m; M //  ¡.q j EŒt; s; l/ j hq; qi  1g  0;

q l

and X .k/ Œt  E .m; M / if and only if

d3 Œt  D minfd1 Œt; s j s 2 T g  0:
s

Relations of this lemma allow us to compute the union X .k/ Œt  in all the considered
cases.
If it is further required to compute
[
fX .ª; t0 ; X 0 / j ª 2 Œª0 ; ª00 g; ª  £00 ;

then the previous schemes have to be applied once more, involving one more
parametrized array of ellipsoids.
Such overall procedures should be repeated after each active crossing, designing
a branching process, whose calculation would involve effective parallelization.
Computing sequential arrays of ellipsoids that correspond to related directions q; l;
through procedures of such parallelization and increasing the number of directions,
one may approach the exact solutions with any preassigned degree of accuracy.

11.4 Impulse Controls in Hybrid System Models

In Sect. 11.2 we considered resets occurring only in the system model. We now
indicate how to treat resets in both system model and the state space variables. This
leads to the use of impulse controls for describing the resets.
420 11 Verification: Hybrid Systems

11.4.1 Hybrid Systems with Resets in Both Model

and System States

We now discuss situations when the hybrid system involves resets both in the system
number (marked as Rs .j / ) and in the phase coordinates (marked as Rcrd .i /). Upper
index s D  with no crossing, s D C with reset in system number and s D  with
reset in phase coordinates.
Returning to (11.19), consider hybrid system

xP 2 A.i/ .t /x C B .i/ .t /u.i/ C C .i/ .t /v.i/ .t /; i D 1; : : : ; k; (11.37)

with resets of system model at hyperplanes at

Hj D fx j hc .j / ; xi  ”j D 0g; c .j / 2 Rn ; ”j 2 R; j D 1; : : : ; m:

Suppose an isolated motion, starting at ft0 ; X 0 g, developed due to system i1 ; under

fixed control u D u.t /; reaches the first hyperplane Hj1 at time £.j1 /. Then the
following events may occur:
– at time £.j1 / there is a system reset for i1 ; so it switches to system i2 D Rs .j1 /i1 ;
due to transformation Rs .j1 / from I .k/ D fi D 1; : : : ; kg to I .k/; and develops
further due to i2 .
– at the same time £.j1 / there is a reset of the phase coordinates due to
transformation x.£.j1 / C 0/ D Rcrd .j1 /x.£.j1 /  0/ from Rn ! Rn . If there
are no resets, then the respective transformations are identities.
So, if at instant of intersecting Hj with both types of resets active, the position
(state) of the system will change from ft; x; Œi1 g; t D £.j1 /  0;—before
crossing, to another after crossing, which is f£.j1 / C 0; x.£.j1 / C 1/; Œi1 ; i2 g; where
x.£.j1 / C 0/ D Rcrd .j1 /x, i2 D Rs .j1 /i1 , i2 D i2s , s D fCg; then the system
will further develop due to i2 , from boundary condition f£.j1 /; x.£.j1 / C 0/g.
After yet one more crossing, with only one active reset Rcrd at £.j2 /; the state
at t 2 .£.j2 /; £.j3  0// will be ft; x; Œi1 ; i2 ; i3 g; with i3 D i3s ; s D fg. The
on-line state is therefore composed of a part ft; xg without memory, responsible for
the continuous component of the hybrid system and a part Œi1 ; i2 ; i3  with memory,
responsible for the discrete part.3
Remark 11.4.1. The difference between the systems considered here and standard
switching systems is that here the switchings depend on spatial rather than temporal
parameters and also that there may be instantaneous change of coordinates which
leads to discontinuity of trajectories in the phase space.
Before introducing some general schemes we begin with illustrative examples.

3
In more complicated problems, for example, with complex constraints on the number of
switchings or other outputs of resets, this memorized part may be logically controlled and its
knowledge may be important. Such components are also important in the design of feedback
controls and computation of backward reach sets for hybrid systems. Situations mentioned in this
footnote mostly lie beyond the scope of this book.
11.4 Impulse Controls in Hybrid System Models 421

11.4.2 Two Simple Examples

We give some examples of hybrid systems with reset of their phase coordinates.
In order to formalize such situations we will introduce additional system inputs in
the form of exogenous impulse controls. We begin with two simple examples.
Example 11.2. The two-dimensional bouncing ball.
Consider a small freely falling heavy ball which bounces when striking the floor
(a plane, inclined to horizontal level), due to an elastic impact. It then moves further
along a trajectory which depends on the inclination angle of the floor at each new
bounce. Such angles, which act as controls, should be selected so that the ball ends
up in a fixed hole on the horizontal axis—the target. The process of bouncing may
cause some small losses of energy.
Following is an analytical, two-dimensional description of such bouncing
motion. A more complicated three-dimensional model of a bouncing motion is
indicated later.
The system equations are

xP 1 D y1 ; yP1 D v1 .x; y; ®/•.x2 /;

xP 2 D y2 ; yP2 D kv2 .x; y; ®/•.x2 /  g; 0 < k < 1;

where ® D ®.x; y/ are the values of the discrete control inputs which are presented
as inclination angles of the floor to horizontal line x2 D 0 at impact times ti . Such
inputs are modeled as delta functions of type v•.x2 .ti //.
The last equations are symbolic. Another representation of the same system,
according to Sect. 9.3.1 is

dx1 D y1 dt; dy1 D v1 .x; y; ®/d U 0 .x2 ; 0/;

dx2 D y2 dt; dy2 D kv2 .x; y; ®/d U 0 .x2 ; 0/  gdt; 0 < k < 1;

where function U 0 .x; h/ D 1h .x/ (with scalar variable x) is a unit jump, namely,
1h .x/ D 0; if x < h and 1h .x/ D 1; if x  h.
The solution of the last equations may be interpreted as a solution of the
next vector-valued integral equation with integral taken in the Lebesgue–Stiltjes
sense [234].
Z t Z t
x1 .t / D x10 C y1 .s/ds; y1 .t / D y10 C v1 .x.t /; y.t /; ®.x.t /; y.t ///d U 0 .x2 .t /; 0/;
t0 t0

Z t Z t
x2 .t / D x20 C y2 .s/ds; y2 .t / D y20  k v2 .x.t /; y.t /; ®.x.t /; y.t ///d U 0 .x2 .t /; 0/:
t0 t0
(11.38)
422 11 Verification: Hybrid Systems

Fig. 11.4 The trajectory of a bouncing ball

Here, Fig. 11.4 shows the bouncing trajectory where at times of impact, with
x2 .ti / D 0; the vertical velocity y2 of the ball instantaneously changes its sign while
velocities y1 ; y2 change their values. This leads to an instantaneous jump (a reset)
of the values of vector fxP 1 .t /; xP 2 .t /g as soon as x2 .t / D 0. The inclination angle ®
of the floor at x2 .ti / D 0 therefore depends on the state space position of the system
and generates a feedback impulse control, thus reflecting the discrete component of
the hybrid process. The reset of variables fxP 1 .t / D y1 .t /; xP 2 .t / D y2 .t /g is due to
virtual •-valued controls of intensities v1 .x; y; ®/ and kv2 .x; y; ®/.
Example 11.3. One inflow for two water tanks.
Another is the well-known simple example of controlling two water tanks
through one inflow [104]. The feedback dynamics of this system is now formalized
using impulsive inputs.
This system is a model with two “motions”: (q1 ), (q2 ), with alternating resets
from one to the other. Here

.q1 / W xP 1 .t / D w  v1 ; xP 2 D v2 ; x2  V2 ;

.q2 / W xP 1 .t / D v1 ; xP 2 D w  v2 ; x1  V1 :

These equations describe a water supply at speed w through inflow from only
one pipe directed alternatingly to two water tanks from which water leaves with
11.4 Impulse Controls in Hybrid System Models 423

respective speeds v1 and v2 . The water levels to be maintained at the tanks are V1
and V2 . At each time t the pipe fills one of the tanks, then switching to the other
according to the following feedback rule.
Suppose at first the water is filling tank q2 with x1 > V1 ; x2 < V2 . Then, when
x1 decreases to level V1 the pipe switches from q2 to q1 (at that time it should be at
x2 > V2 ). Then later, after x2 had decreased to V2 , the pipe is reversed again to q2 .
The speeds and the water levels have to be coordinated to ensure sustainability of
the required water levels V1 ; V2 .
Exercise 11.4.1. Indicate the range of speeds w; v1 ; v2 and water levels V1 ; V2 when
these levels are sustainable.
The two systems may be combined into one, switching from one to the other through
two impulsive control inputs generated due to feedback control signals. Denoting
the right-hand sides of systems (q1 ) and (q2 ) by F1 .x/; F2 .x/ and introducing a
third variable — that has only two possible values ˙1; consider the following system
in symbolic form
8    
< xP D F .x/ 1Csign— C F .x/ 1sign—
1 2 2 2 ;
: P
— D 2•.x1  V1 C 0/  2•.x2  V2 C 0/
or as
8     
< dx D F .x/ 1Csign— C F .x/ 1sign— dt
1 2 2 2 :
:
d — D 2d U .x1  V1 ; 0/  2d U .x2  V2 ; 0/
0 0

The solution of this equation is defined through integral equations taken in the sense
of the Lebesgue–Stiltjes integral and similar to the type (11.37) (Fig. 11.5).

11.4.3 Impulse Controls in Hybrid Systems

We now indicate a scheme involving impulse controls to formalize the mathematical

model of hybrid systems.
Consider equations

xP D A.t; —/x C B.t; —/u C I u.imp/ ; (11.39)

—P D u— ; (11.40)

x 2 Rn ; — 2 f1; : : : ; kg:

Here u D u.t; x; —/ is the on-line p-dimensional “ordinary” control in the class of

bounded functions, the one responsible for the continuous component of system
424 11 Verification: Hybrid Systems

Fig. 11.5 One inflow for two water tanks

trajectory x.t /; the n-dimensional vector u.imp/ D .u1 ; : : : ; un /0 is the

.imp/ .imp/

impulse control responsible for reset of the phase coordinates and has the form

.imp/ .imp/
u.imp/ D u.imp/ .t; x; —/I uj D uj .t; x; —/ D ’.t; x; —.t  0//•.f .x; —//:

The scalar u— controls the number — of system Eq. (11.39), and is the input in
Eq. (11.40) for finding the integer — 2 f1; : : : ; kg and has the form

u— D u— .x; —/ D “.x; —.t  0//•.fd .x; —.t  0///:

The piecewise-constant, integer-valued function —.t / is responsible for switching the

system at crossing time £ from —.£0/ D i to —.£C0/ D j . This is realized through
Eq. (11.40) with “.x; i / D j  i .
The resets occur at instants of crossing surfaces f .x; —/ D 0 for phase
coordinates and f— .x; —/ D 0 for the number — D i of Eq. (11.39). In system (11.37)
these surfaces are presented as hyperplanes H.j /.
Remark 11.4.2. The application of dynamic programming techniques to hybrid
systems was discussed in [23, 36, 170].
11.4 Impulse Controls in Hybrid System Models 425

11.4.4 More Complicated Example: The Three-Dimensional

Bouncing Ball 4

Consider a three-dimensional Newtonian motion of a heavy ball under gravity “g.”

Starting with a free fall, the ball bounces back when reaching a plane H inclined
to the horizontal level. The phase state of the ball is defined by a six-dimensional
vector x D .x1 ; : : : ; x6 /0 2 R6 , where .x1 ; x2 ; x3 /0 are coordinates of its center
in R3 (x3 stands for its vertical position), and .x4 ; x5 ; x6 / are coordinates of the
respective velocity vector. At each time t , with no force other than gravity, it satisfies
Eq. (11.37), where, with Oij standing for a zero matrix of dimensions i  j; we
have
   
O33 I33 O51
A.t / D , B.t / D 0 , C.t /f .t / D f .t / D : (11.41)
O30 O33 g
This system is treated within

t 2 R; x 2 x D f.x1 ; : : : ; x6 / W xi 2 R; x3  0g:

If it is in horizontal position, plane H may rotate around axes l1 and l2 , that

may emanate from any point along directions of vectors e1 D .1; 0; 0/0 and
e2 D .0; 1; 0/0 . At each time it may be rotated around only one of these axes: to angle
®1 2 Œ’1 ; ’1  (’1 2 .0; =2/) for l1 , and angle ®2 2 Œ’2 ; ’2  (’2 2 .0; =2/) for
l2 . Such rotations incline the plane to a new position. So, when the ball now reaches
H , it bounces back causing a reset of the system trajectory in phase space R6 ,
demonstrating the hybrid nature of the process. At an instant of reset H may had
been turned around one of the axes l1 and l2 that emanate from the point of impact.
Let the state of the ball be x  just before the reset and x C right after that.
Introduce the additional notations
0 1 0 1
1 0 0 cos.®/ 0  sin.®/
C1 .®/ D @ 0 cos.®/ sin.®/ A , C2 .®/ D @ 0 1 0 A:
0  sin.®/ cos.®/ sin.®/ 0 cos.®/
If by time of reset the plane H was turned around axis l1 to angle ®1 ; and also
around l2 to angle ®2 ; then vectors x  and x C are linked by the next relation
 
I33 O33
x C D C x  , where C D C.®1 ; ®2 / D ,
O33 M
0 1
000
M D I  .1 C ”/C1 .®1 /C2 .®2 /E3;3 C20 .®2 /C10 .®1 /, E3;3 D @0 0 0A.
001

4
This example was worked out by P.A. Tochilin.
426 11 Verification: Hybrid Systems

Here ” 2 Œ0; 1 is the “recovery” coefficient: with ” D 1 we have the model for an
absolutely elastic impact, and with ” D 0 for a nonelastic.
Such a model may now be also formalized involving •-functions.

xP D Ax.t / C f C .C.®1 ; ®2 /  I /x•.x3 /: (11.42)

With ®2 D 0 (when plane H may rotate only around axis l1 ), then (11.42), taken in
coordinate form, is
8
ˆ
ˆ xP 1 D x4
ˆ
ˆ
ˆ
ˆ xP 2 D x5
ˆ
<
xP 3 D x6
;
ˆ
ˆ xP 4 D 0
ˆ
ˆ
ˆ
ˆ x5 D .1 C ”/.sin .®1 /x5 C sin.®1 / cos.®1 /x6 /•.x3 /
2
:̂
x6 D .1 C ”/.sin.®1 / cos.®1 /x5 C cos2 .®1 /x6 /•.x3 /  g

and with ®1 D 0 (when H may rotate only around l2 ), Eq. (11.42) in detail is:
8
ˆ
ˆ xP 1 D x4
ˆ
ˆ
ˆ
ˆ xP 2 D x5
ˆ
<
xP 3 D x6
:
ˆ xP 4
ˆ D .1 C ”/.sin2 .®1 /x4  sin.®1 / cos.®1 /x6 /•.x3 /
ˆ
ˆ
ˆ
ˆ x5 D0
:̂
x6 D .1 C ”/. sin.®1 / cos.®1 /x4 C cos2 .®1 /x6 /•.x3 /  g

At each time t the on-line controls ®1 ; ®2 are determined by the running position
x.t / of the system.
On horizontal plane H D H0 D fx W x3 D 0g consider a hole (target set)

M D fx 2 R3 W .x1  m1 /2 C .x2  m2 /2  r 2 ; x3 D 0g:

Also specify a starting position x 0 D x.t0 / D .x10 ; x20 ; x30 ; 0; 0; 0/0 , where x30 > 0.
The problem will be to direct the trajectory of the bouncing ball, by choice
of angles ®1 ; ®2 (the exogenous controls) to the target set M . Here the system
parameters and the matrix C are independent of time.
Considered are the following optimization problems.
Problem 11.4.1. Find minimal time t1  t0 , for which there exists a control
®1 ./; ®2 ./, that ensures inclusion x.t1 I t0 ; x0 /j®1 ;®2 2 M  R3 with any final
velocity.
Problem 11.4.2. Given ft0 ; x 0 g; find the minimal number k  of bounces (resets)
k 2 ZC (the set of positive integers), such that there exist a time interval Œt0 ; t1 ; and
a control ®1 ./; ®2 ./ for which the inclusion xŒt1  D x.t1 I t0 ; x 0 /j®1 ;®2 2 M  R3 is
achieved with number of bounces k  k  .
11.4 Impulse Controls in Hybrid System Models 427

Consider the following value functions


V .1/ .t1 ; x/ D min min Œ.x1 .tI t0 ; x/j®1 ;®2 m1 /2 C.x2 .tI t0 ; x/j®1 ;®2 m2 /2 r 2 C
®1 ./;®2 ./ t 2Œt0 ;t1 


CI .x3 .tI t0 ; x/j®1 ;®2 j 0/ ;


V .2/
.x; k/ D min min Œ.x1 .tI t0 ; x/j®1 ;®2 m1 /2 C.x2 .tI t0 ; x/j®1 ;®2 m2 /2 r 2 C D
t t0 ®1 ./;®2 ./


CI .x3 .tI t0 ; x/j®1 ;®2 j 0/ C ™.x.I t0 ; x/j®1 ;®2 ; t; k/ :

Here Œh.t; x/C D 0 if h.t; x/  0; and Œh.t; x/C D h.t; x/ if h.t; x/ > 0;
function ™.x.I t0 ; x/j®1 ;®2 ; t; k/ D 0 within interval £ 2 Œt0 ; t , if the number of
bounces does not exceed k, otherwise it is C1.5
To compute solutions of Problems 11.4.1 and 11.4.2, we may use the following
facts.
Lemma 11.4.1. (1) With x 0 given, the solution to Problem 11.4.1—the minimal
time t1 D t1 —is:

t1 D arg minft1  t0 W V .1/ .t1 ; x 0 /  0g;

achieved through some control sequence ®1 ./; ®2 ./.

(2) With x 0 given, the solution k D k  for which there exists a time interval Œt0 ; t1 
together with a control sequence ®1 ./; ®2 ./; that solve Problem 11.4.2, is:

k  D arg minfk 2 ZC W V .2/ .x 0 ; k/  0g:

For computing V .1/ .t1 ; x/; V .2/ .x; k/, there exist related HJB equations with appro-
priate boundary conditions (see [171]).
Exercise 11.4.2. Indicate the HJB equations with boundary conditions for comput-
ing value functions V .1/ .t1 ; x/; V .2/ .x; k/.
Remark 11.4.3. Value functions V .1/ .t1 ; x/; V .2/ .x; k/ may be also computed
through methods of nonlinear analysis and ellipsoidal techniques, along the lines of
Chaps. 2 and 3.
With V .1/ .t1 ; x/; V .2/ .x; k/ computed, one may specify the solutions to Prob-
lems 11.4.1 and 11.4.2.

5
As before I .x j D/ D 0, if x 2 D and C1 otherwise.
428 11 Verification: Hybrid Systems

Denote t1 .x/ D minft1  t0 W V .1/ .t1 :; x/  0g  0 and at collision time

t1  0 (just before the bounce) consider state ft1  0; xg; x D .x1 ; : : : ; x6 /; so that
x3 .t / D 0; x6 .t  0/ < 0.
Suppose .x1 .t1  0/  m1 /2 C .x2 .t1  0/  m2 /2  r 2 > 0; x3 D 0; but
V .t1  0; x/  0. Then, obviously t1 < t1 ; and the control solution at this point
.1/

will be of type

.®1 .x/; ®2 .x// W V .1/ .t1 .x/; C.®1 ; ®2 /x/  V .1/ .t1 .x/; x/;

until t1 D t1 ; .x1 .t1  0/  m1 /2 C .x2 .t1  0/  m2 /2  r 2 D 0; x3 D 0.

Similarly, denote k  .x/ D minfk 2 ZC W V .2/ .x; k/  0g; and suppose at
collision time t1 we have .x1 .t1  0/  m1 /2 C .x2 .t1  0/  m2 /2  r 2 > 0; x3 D 0.
Then the number of realized bounces will be k < k  and the control solution at this
point will be of type

.®1 .x/; ®2 .x// W V .2/ .C.®1 ; ®2 /x; k/  V .2/ .x; k  .x//;

until k D k   1.
Shown in Fig. 11.6 is the trajectory of the 3-d bouncing ball (a hybrid system)
which reaches the target set with minimum resets (bounces). The problem parame-
ters are: ’1 D 0:5, ’2 D 0:5, m1 D 1, m1 D 2, r 2 D 0:2, ” D 0:8, and the starting
position is x 0 D .4; 1; 4; 0; 0; 0/0 .
Shown in Fig. 11.7 is the time-optimal target-oriented trajectory of the bouncing
ball with parameters ’1 D 1, ’2 D 0:4, m1 D 1, m2 D 2, r 2 D 0:2, ” D 0:9 and
starting position x 0 D .2; 3; 4; 0; 0; 0/0 .

Fig. 11.6 Reaching target hole with minimal bounces

11.4 Impulse Controls in Hybrid System Models 429

Fig. 11.7 Reaching target hole in minimal time

References

1. Akhiezer, N.I.: The Classical Moment Problem and Some Related Questions in Analysis.
Oliver & Boyd, Edinburgh (1965)
2. Albert, A.: Regression and the Moor-Penrose Pseudoinverse. Mathematics in Science and
Engineering, vol. 94. Academic, New York (1972)
3. Artstein, Z.: Yet another proof of the Lyapunov convexity theorem. Proc. Am. Math. Soc.
108(1), 89–91 (1990)
4. Åström, K.J.: Introduction to Stochastic Control Theory. Academic, New York (1970)
5. Aubin, J.-P.: Viability Theory. SCFA. Birkhäuser, Boston (1991)
6. Aubin, J.-P., Bayen, A.M., St.Pierre, P.: Viability Theory: New Directions. Springer,
Heidelberg (2011)
7. Aubin, J.-P., Cellina, A.: Differential Inclusions. Springer, New York (1984)
8. Aubin, J.-P., Frankowska, H.: Set-Valued Analysis. SCFA 2. Birkhäuser, Boston (1990)
9. Aumann, R.J.: Integrals of set-valued functions. J. Math. Anal. Appl. 12, 1–12 (1965)
10. Bai, E.W., Cho, H., Tempo, R.: Optimization with few violated constraints for linear bounded
error parameter estimation. IEEE Trans. Automat. Control 47(7), 1067–1077 (2002)
11. Balakrishnan, A.V.: Applied Functional Analysis. Springer, New York (1976)
12. Balluchi, A., Benvenuti, L., Di Benedetto, M.D., Sangiovanni-Vincentelli, A.: The design
of dynamical observers for hybrid systems: theory and application to an automotive control
problem. Automatica 49(4), 915–925 (2013)
13. Baras, J.S., Bensoussan, A., James, M.R.: Dynamic observers and asymptotic limits of
recursive filters: special case. SIAM J. Appl. Math. 48(5), 1147–1158 (1988) 1
14. Baras, J.S., Kurzhanski, A.B.: Nonlinear filtering: the set-membership (bounding) and the
H1 techniques. In: Proceedings of the 3rd IFAC Symposium NOLCOS, pp. 409–418.
Pergamon Press, Oxford (1995)
15. Barbashin, E.A.: On the theory of generalized dynamic systems. Sci. Notes Moscow State
Univ. 135, 110–133 (1949)
16. Bardi, M., Capuzzo Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton–
Jacobi–Bellman Equations. SCFA. Birkhäuser, Boston (1997)
17. Barron, E.N., Jensen, R.: Semicontinuous viscosity solutions for Hamilton–Jacobi equations
with convex Hamiltonians. Commun. Partial Differ. Equ. 15(12), 1713–1742 (1990)
18. Başar, T., Bernhard, P.: H1 Optimal Control and Related Minimax Design Problems. SCFA,
2nd edn. Birkhäuser, Basel (1995)
19. Ba sar, T., Olsder, J.: Dynamic Noncooperative Game Theory. Academic, New York (1982)

© Springer International Publishing Switzerland 2014 431

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1
432 References

20. Basile, G., Marro, G.: Controlled and Conditioned Invariants in Linear System Theory.
Prentice-Hall, Englewood Cliffs (1992)
21. Bellman, R., Dreyfus, S.: Applied Dynamic Programming. Princeton University Press,
Princeton (1962)
22. Bellman, R.: Introduction to the Mathematical Theory of Controlled Processes, vol. 1/2.
Academic, New York (1967/1971)
23. Bensoussan, A., Lions, J.-L.: Contrôle impulsionnel et inéquations quasi-variationnelles.
Dunod, Paris (1982)
24. Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. 1/2. Athena Scientific,
Belmont (1996/2012)
25. Bertsekas, D.P., Rhodes, I.: Recursive state estimation for a set-membership description of
uncertainty. IEEE Trans. Automat. Control 16(2), 117–128 (1971)
26. Blagodatskih, V.I, Filippov, A.F.: Differential inclusions and optimal control. Proc. Steklov
Math. Inst. 169, 194–252 (1985)
27. Blanchini, F., Miani, S.: Set-Theoretic Methods in Control. Springer, Boston (2008)
28. Blaquière, A., Leitmann, G. (eds.): Dynamical Systems and Microphysics: Control Theory
and Mechanics. Academic, Orlando (1982)
29. Bliss, G.A.: Calculus of Variations. Mathematical Association of America, Chicago (1944)
30. Boltyanski, V.G.: Mathematical Methods of Optimal Control. Holt, New York (1971)
31. Boltyanski, V.G.: Tent methods in optimal control. ISNM International Series of Numerical
Mathematics, vol. 111, pp. 3–20
32. Bohner, M., Peterson, A.: Dynamic Equations on Time Scales: An Introduction with
Applications. Birkhauser, Boston (2001)
33. Bolza, O.: Lectures on Calculus of Variations. Hafer Pub. Co, New York (1946). Dover reprint
34. Bonneuil, N.: Computing the viability kernel in large state dimensions. J. Math. Anal. Appl.
323, 1444–154 (2006)
35. Boyd, S., Ghaoui, L.E., Feron, E., Balakrishnan, V.: Linear Matrix Inequalities in System and
Control Theory. Studies in Applied Mathematics, vol. 15. SIAM, Philadelphia (1994)
36. Branicky, M.S., Borkar, V.S., Mitter, S.K.: A unified framework for hybrid control: model and
optimal control theory. IEEE Trans. Automat. Control 43(1), 31–45 (1998)
37. Bressan, A., Piccoli, B.: Introduction to the Mathematical Theory of Control, vol. 2. American
Institute for Mathematical Sciences, Springfield (2007)
38. Brockett, R.W.: Finite-Dimensional Linear Systems. Wiley, New York (1970)
39. Brockett, R.W.: Hybrid models for motion control systems. In: Trentelman, H., Willems,
J. (eds.) Essays in Control: Perspectives in the Theory and Its Applications, pp. 29–53.
Birkhäuser, Boston (1993)
40. Callier, F.M., Desoer, C.A.: Linear System Theory. Springer, New York (1991)
41. Castaing, C., Valadier, M.: Convex Analysis and Measurable Multifunctions. Lecture Notes
in Mathematics, vol. 580. Springer, Berlin (1977)
42. Carter, T.E.: Optimal impulsive space trajectories based on linear equations. J. Optim. Theory
Appl. 70(2), 277–297 (1991)
43. Carter, T.E., Brient, J.: Linearized impulsive rendezvous problem. J. Optim. Theory Appl.
86(3), 553–584 (1995)
44. Chernousko, F.L.: Ellipsoidal bounds for sets of attainability and uncertainty in control.
Optim. Control Appl. Methods 3(2), 187–202 (1982)
45. Chernousko, F.L.: State Estimation for Dynamic Systems. CRC Press, Boca Raton (1994)
46. Chutinan, A., Krogh, B.H.: Verification of polyhedral-invariant hybrid automata using
polygonal flow pipe approximations. In: Hybrid Systems: Computation and Control. Lecture
Notes in Control Science, vol. 1569, pp. 76–90. Springer, Berlin (1999)
47. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)
48. Clarke, F.H., Ledyaev Yu, S., Stern, R.J., Wolenski, P.R.: Nonsmooth Analysis and Control
Theory. Graduate Texts in Mathematics, vol. 178. Springer, New York (1998)
49. Crandall, M.G., Evans, L.C., Lions, P.-L.: Some properties of solutions of Hamilton–Jacobi
equations. Trans. Am. Math. Soc. 282(2), 487–502 (1984)
References 433

50. Crandall, M.G., Lions, P.-L.: Viscosity solutions of Hamilton–Jacobi equations. Trans. Am.
Math. Soc. 277(1), 1–41 (1983)
51. Courant , R., Hilbert, D.: Methods of Mathematical Physics, vol. 1 & 2. Wiley, New York
(1989) (Wiley GmbH & Co., KGaA, 2008)
52. Dar’in, A.N., Kurzhanskii, A.B.: Nonlinear control synthesis under two types of constraints.
Differ. Equ. 37(11), 1549–1558 (2001)
53. Daryin, A.N., Kurzhanskii, A.B.: Control under indeterminacy and double constraints. Differ.
Equ. 39(11), 1554–1567 (2003)
54. Daryin, A.N., Kurzhanskii, A.B., Seleznev, A.V.: The dynamic programming method in
impulsive control synthesis. Differ. Equ. 41(11), 1566–1576 (2005)
55. Daryin, A.N., Kurzhanski, A.B., Seleznev, A.V.: A dynamic programming approach to
the impulse control synthesis problem. In: Proceedings of Joint 44th IEEE Conference
on Decision and Control and European Control Conference ECC 2005, Seville, Spain,
pp. 8215–8220 (2005)
56. Daryin, A.N., Digailova, I.A., Kurzhanski, A.B.: On the problem of impulse measurement
feedback control. Proc. Steklov Math. Inst. 268(1), 71–84 (2010)
57. Daryin, A.N., Kurzhanski, A.B.: Estimation of reachability sets for large-scale uncertain
systems: from theory to computation. In: Proceedings of CDC-2012 (2012)
58. Daryin, A.N., Kurzhanski, A.B.: Parallel algorithm for calculating the invariant sets of high-
dimensional linear systems under uncertainty. Comput. Math. Math. Phys. 53(1), 34–43
(2013)
59. Davis, M.H.A., Varaiya, P.: Information states for linear stochastic systems. J. Math. Anal.
Appl. 37, 384–402 (1972)
60. Demyanov, V.F.: Minimax: Directional Differentiation. Leningrad University Pub. Leningrad
(1974)
61. Demyanov, V.F., Malozemov, V.N.: Introduction to Minimax. Wiley, New York (1974)
62. Demyanov, V.F., Rubinov, A.M.: Quasidifferential Calculus. Optimization Software,
New York (1986)
63. Digailova, I.A., Kurzhanski, A.B.: The joint model and state estimation problem under
set-membership uncertainty. In: Proceedings of the 15-th IFAC Congress, Barcelona (2002)
64. Digailova, I.A., Kurzhanski, A.B.: Reachability analysis under control-dependent stochastic
noise. In: Proceedings of 16th IFAC World Congress, Prague (2005)
65. Digailova, I.A., Kurzhanski, A.B., Varaiya, P.: Stochastic reachability and measurement feed-
back. In: Proceedings of 17th World Congress IFAC, Seoul, 6–11 July 2008, pp. 14336–14341
66. Dontchev, A.L.: Perturbations, Approximations, and Sensitivity Analysis of Optimal Control
Systems. Lecture Notes in Control and Information Sciences, vol. 52. Springer, New York
(1983)
67. Dunford, N., Schwartz, J.T.: Linear Operators. Interscience, New York/London (1958)
68. Dykhta, V.A., Samsonyuk, O.N.: Optimal Impulse Control and Its Applications. Fizmatlit,
Moscow (2003) [in Russian]
69. Ekeland, I., Temam, R.: Analyse Convexe et Problemes Variationelles. Dunod, Paris (1973)
70. Fan, K.: Minimax theorems. Proc. Natl. Acad. Sci. USA 39(1), 42–47 (1953)
71. Fan, K.: On systems of linear inequalities. In: Kuhn, H.W., Tucker, A.W. (eds.) Linear
Inequalities and Related Systems. Annals of Mathematical Studies, vol. 38, pp. 99–156.
Princeton University Press, Princeton (1956)
72. Fan, K.: Existence theorems and extreme solutions for inequalities concerning convex
functions or linear transformations. Mathematische Zeitschrift 68(1), 205–216 (1957)
73. Fenchel, W.: On conjugate convex functions. Can. J. Math. 1, 73–77 (1949)
74. Fenchel, W.: Convex Cones, Sets and Functions. Princeton University Press, Princeton (1951)
75. Filippov, A.F.: On certain questions in the theory of optimal control. SIAM J. Control. 1,
76–84 (1962)
76. Filippov, A.F.: Differential Equations with Discontinuous Righthand Sides. Kluwer, Dor-
drecht (1988)
434 References

77. Filippova, T.F.: A note on the evolution property of the assembly of viable solutions to a
differential inclusion. Comput. Math. Appl. 25(2), 115–121 (1993)
78. Filippova, T.F., Kurzhanski, A.B., Sugimoto, K., Valýi, I.: Ellipsoidal calculus, singular
perturbations and the state estimation problems for uncertain systems. Working Paper
WP-92-051, IIASA, Laxenburg (1992)
79. Filippova, T.: Approximation techniques in impulsive control problems for the tubes of solu-
tions of uncertain differential systems. In: Anastassiou, G.A., Duman, O. (eds.) Advances in
Applied Mathematics and Approximation Theory: Contributions from AMAT, pp. 385–396.
Springer, New York (2012)
80. Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions. Springer,
New York (1993)
81. Gamkrelidze, R.V.: Principles of Optimal Control Theory. Mathematical Concepts and
Methods in Science and Engineering, vol. 7. Plenum Press, New York (1978)
82. Gantmacher, F.R.: Matrix Theory, vol. 2. Chelsea Publishing, New York (1964)
83. Gao, Y., Lygeros, L., Quincampoix, M.: On the reachability problem for uncertain hybrid
systems. IEEE Trans. Automat. Control 52(9), 1572–1586 (2007)
84. Garcia, C.E., Prett, D.M., Morari, M.: Model predictive control: theory and practice-a survey.
Automatica 25(3), 335–348 (1989)
85. Gayek, J.E.: A survey of techniques for approximating reachable and controllable sets, Proc.
of the CDC Conf. Brighton, England, 1724–1729 (1991)
86. Gelfand, I.M., Fomin, S.V.: Calculus of Variations. Academic, New York (1964)
87. Gelfand, I.M., Shilov, G.E.: Generalized Functions. Dover, New York (1991)
88. Giannessi, F., Maugeri, A. (eds.): Variational Methods and Applications. Kluwer, New York
(2004)
89. Godunov, S.K.: Guaranteed Accuracy in Numerical Linear Algebra. Springer, Dordrecht
(1993)
90. Godunov, S.K.: Ordinary Differential Equations with Constant Coefficients, vol. 169. AMS
Bookstore, Providence (1997)
91. Goebel, R., Sanfelice, R.G., Teel, A.R.: Hybrid Dynamical Systems. Princeton University
Press, Princeton (2012)
92. Golub, G.H., Van Loan, C.F.: Matrix Computations. John Hopkins Un. Press, Baltimore, MI
(1996)
93. Grant, M., Boyd, S.: Graph interpretation for nonsmooth convex programs. In: Recent
Advances in Learning and Control, pp. 95–110. Springer, Berlin (2008)
94. Greenstreet, M.R., Mitchell, I.: Reachability analysis using polygonal projections. In:
Hybrid Systems: Computation and Control. Lecture Notes in Control Science, vol. 1569,
pp. 103–116. Springer, Berlin (1999)
95. Gurman, V.I.: The Extension Principle in Problems of Control. Nauka, Moscow (1997)
96. Gusev, M.I., Kurzhanski, A.B.: Optimization of controlled systems with bounds on the
controls and state coordinates, I. Differ. Equ. 7(9), 1591–1602 (1971) [translated from
Russian]
97. Gusev, M.I., Kurzhanski, A.B.: Optimization of controlled systems with bounds on the
controls and state coordinates, II. Differ. Equ. 7(10), 1789–1800 (1971) [translated from
Russian]
98. Helton, J.W., James, M.R.: Extending H1 Control to Nonlinear Systems. SIAM, Philadel-
phia (1999)
99. Ioffe, A.D., Tikhomirov, V.M.: Theory of Extremal Problems. North-Holland Amsterdam
(1979)
100. Isidori, A., Astolfi, A.: Disturbance attenuation and H1 control via measurement feedback
in nonlinear systems. IEEE Trans. Automat. Control 37, 1283–1293 (1992)
101. Isidori, A., Marconi, L.: Asymptotic analysis and observer design in the theory of nonlinear
output. In: Nonlinear Observers and Applications. Lecture Notes in Control and Information
Sciences, vol. 363, pp. 181–210. Springer, Berlin (2007)
102. Isaacs, R.: Differential Games. Wiley, New York (1965)
References 435

103. James, M.R., Baras, J.S.: Partially observed differential games, infinite-dimensional
Hamilton–Jacobi–Isaacs equations and nonlinear H1 control . SIAM J. Control Optim 34(4),
1342–1364 (1996)
104. Johansson, M.: Piecewise Linear Control Systems. Lecture Notes in Control and Information
Sciences, vol. 284. Springer, Berlin (2003)
105. Kailath, T.: Linear Systems. Prentice Hall, Englewood Cliffs (1980)
106. Kalman, R.E.: Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana. 5,
102–119 (1959)
107. Kalman, R.E.: On the general theory of control systems. In: Proceedings of the 1-st IFAC
Congress, Butterworth, London, vol. 1 (1960)
108. Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME
82(Series D), 35–45 (1960)
109. Kalman, R.E., Ho, Y.-C., Narendra, K.S.: Controllability of linear dynamical systems.
Contrib. Theory Differ. Equ. 1(2), 189–213 (1963)
110. Kolmogorov, A.N., Fomin, S.V.: Elements of the Theory of Functions and Functional
Analysis. Courier Dover, New York (1999)
111. Komarov, V.A.: Estimates of reachability sets for differential inclusions. Math. Notes
(Matematicheskiye Zametki) 37(6), 916–925 (1985)
112. Komarov, V.A.: Equation of reachability sets for differential inclusions under state constraints.
Proc. Steklov Math. Inst. 185, 116–125 (1989)
113. Koscheev, A.S., Kurzhanski, A.B.: On adaptive estimation of multistage systems under
uncertainty. Izvestia Academy Sci. 2, 72–93 (1983)
114. Kostousova, E.K.: State estimation for dynamic systems via parallelotopes: optimization and
parallel computations. Optim. Methods Softw. 9(4), 269–306 (1998)
115. Kostousova, E.K.: Control synthesis via parallelotopes: optimization and parallel computa-
tions. Optim. Methods Softw. 14(4), 267–310 (2001)
116. Kostousova, E.K.: External polyhedral estimates for reachable sets of linear discrete-time
systems with integral bounds on controls. Int. J. Pure Appl. Math. 50(2), 187–194 (2009)
117. Kostousova, E.K.: On tight polyhedral estimates for reachable sets of linear differential
systems. In: ICNPAA 2012 World Congress: 9th International Conference on Mathematical
Problems in Engineering, Aerospace and Sciences (2012)
118. Kostousova, E.K., Kurzhanski, A.B.: Theoretical framework and approximation techniques
for parallel computation in set-membership state estimation. In: Proceedings of Symposium
on Modelling, Analysis and Simulation. CESA’96 IMACS Multiconference, CESA’96, Lille-
France, vol. 2, pp. 849–854 (1996)
119. Krasovski, N.N.: On the theory of controllability and observability of linear dynamic systems.
Appl. Math. Mech. (PMM) 28(1), 3–14 (1964)
120. Krasovski, N.N.: Motion Control Theory. Nauka, Moscow (1968) [in Russian]
121. Krasovski, N.N.: Rendezvous Game Problems. National Technical Information Service,
Springfield (1971)
122. Krasovskii, A.N., Krasovskii, N.N.: Control Under Lack of Information. Birkhäuser, Boston
(1995)
123. Krasovskii, N.N., Subbotin, A.I.: Game-Theoretical Control Problems. Springer Series in
Soviet Mathematics. Springer, New York (1988)
124. Krein, M., Smulian, V.: On regular convex sets in the space conjugate to a Banach space. Ann.
Math. 41, 556–583 (1940)
125. Krener, A.J.: Eulerian and Lagrangian observability of point vortex flows. Tellus A 60,
1089–1102 (2008)
126. Krener, A.J., Ide, K.: Measures of unobservability. In: Proceedings of the IEEE Conference
on Decision and Control, Shanghai (2009)
127. Krener, A.J., Wang, K.: Rational minmax filtering. In: Proceedings of IFAC-2011 World
Congress, Milan (2011)
128. Krener, A.J., Kang, W.: Linear time-invariant minmax filtering. Ann. Rev. Control 35,
166–171 (2011)
436 References

129. Kruzhkov, S.N.: Generalized solutions of nonlinear first-order PDE’s with many independent
variables. Math. Sb. 70((112)3), 394–415 (1966)
130. Kuhn, H.W., Tucker, A.W. (eds.): Linear Inequalities and Related Systems. Annals of
Mathematical Studies, vol. 38. Princeton University Press, Princeton (1956)
131. Kuratowski, K.: Topology, vol. 1/2. Academic, New York (1966/1968)
132. Kurzhanskiy, A.A., Varaiya, P.: Ellipsoidal Toolbox. University of California at Berkeley.
http://code.google.com/p/ellipsoids/ (2005)
133. Kurzhanskiy, A.A., Varaiya, P.: Ellipsoidal techniques for reachability analysis of discrete-
time linear systems. IEEE Trans. Automat. Control 52(1), 26–38 (2007)
134. Kurzhanskiy, A.A., Varaiya, P.: Active traffic management on road networks. Philos. Trans.
R. Soc. A 368, 4607–4626 (2010)
135. Kurzhanski, A.B.: On duality of problems of optimal control and observation. Appl. Math.
Mech. 34(3), 429–439 (1970) [translated from the Russian]
136. Kurzhanski, A.B.: Differential games of observation. Russ. Math. Dokl. (Dok. Akad. Nauk
SSSR) 207(3), 527–530 (1972)
137. Kurzhanski, A.B.: Control and Observation Under Uncertainty Conditions, 392 pp. Nauka,
Moscow (1977)
138. Kurzhanskii, A.B.: On stochastic filtering approximations of estimation problems for systems
with uncertainty. Stochastics 23(6), 109–130 (1982)
139. Kurzhanski, A.B.: Evolution equations for problems of control and estimation of uncertain
systems. In: Proceedings of the International Congress of Mathematicians, pp. 1381–1402
(1983)
140. Kurzhanski, A.B.: On the analytical description of the set of viable trajectories of a differential
system. Dokl. Akad. Nauk SSSR. 287(5), 1047–1050 (1986)
141. Kurzhanski, A.B.: Identification: a theory of guaranteed estimates. In: Willems, J.C. (ed.)
From Data to Model, pp. 135–214. Springer, Berlin (1989)
142. Kurzhanski, A.B.: The identification problem: a theory of guaranteed estimates. Autom.
Remote Control (Avtomatica i Telemekhanika) 4, 3–26 (1991)
143. Kurzhanski, A.B.: Set-Valued Analysis and Differential Inclusion. Progress in Systems and
Control Theory, vol. 16. Birkhäuser, Boston (1993)
144. Kurzhanski, A.B.: Modeling Techniques for Uncertain Systems. Progress in Systems and
Control Theory, vol. 18. Birkhäuser, Boston (1994)
145. Kurzhanski, A.B.: Set-valued calculus and dynamic programming in problems of feedback
control. Int. Ser. Numer. Math. 124, 163–174 (1998)
146. Kurzhanski, A.B.: The principle of optimality in measurement feedback control for linear
systems. In: Ranter, A., Byrnes, C.V. (eds.) Directions in Mathematical Systems Theory
and Optimization. Lecture Notes in Control Science, vol. 286, pp. 193–202. Springer, Berlin
(2003)
147. Kurzhanski, A.B.: The problem of measurement feedback control. J. Appl. Math. Mech.
(PMM) 68(4), 487–501 (2004) [translated from the Russian]
148. Kurzhanski, A.B.: The diagnostics of safety zones in motion planning. Optim. Methods Softw.
20(2/3), 231–239 (2005)
149. Kurzhanski, A.B.: Comparison principle for equations of the Hamilton–Jacobi type in control
theory. Proc. Steklov Math. Inst. 253(1), 185–195 (2006)
150. Kurzhanski, A.B.: “Selected Works of A.B. Kurzhanski”, 755 pp. Moscow State University
Pub., Moscow (2009)
151. Kurzhanski, A.B.: Hamiltonian techniques for the problem of set-membership state estima-
tion. Int. J. Adapt. Control Signal Process. 25(3), 249–263 (2010)
152. Kurzhanski, A.B.: On synthesizing impulse controls and the theory of fast controls. Proc.
Steklov Math. Inst. 268(1), 207–221 (2010)
153. Kurzhanski, A.B., Baras, J.: Nonlinear filtering: the set-membership (bounding) and the
H-infinity techniques. In: Proceedings of the IFAC Symposium NOLCOS-95 at Lake Tahoe,
USA, pp. 409–418. Pergamon Press, Oxford (1995)
References 437

154. Kurzhanski, A.B., Daryin, A.N.: Dynamic programming for impulse controls. Ann. Rev.
Control 32(2), 213–227 (2008)
155. Kurzhanski, A.B., Digailova, I.A.: Attainability problems under stochastic perturbations.
Differ. Equ. 40(4), 1573–1578 (2004)
156. Kurzhanski, A.B., Filippova, T.F.: On the set-valued calculus in problems of viability and
control for dynamic processes: the evolution equation. Annales de l’institut Henri Poincaré.
Analyse non linéaire S6, 339–363 (1989)
157. Kurzhanski, A.B., Filippova, T.F.: On the method of singular perturbations for differential
inclusions. Dokl. Akad. Nauk SSSR 321(3), 454–459 (1991)
158. Kurzhanski, A.B., Filippova, T.F.: On the theory of trajectory tubes: a mathematical formalism
for uncertain dynamics, viability and control. In: Advances in Nonlinear Dynamics and
Control. Progress in Systems and Control Theory, vol. 17, pp. 122–188. Birkhäuser, Boston
(1993)
159. Kurzhanski, A.B., Filippova, T.F., Sugimoto, K., Valyi, I.: Ellipsoidal state estimation for
uncertain dynamical systems. In: Milanese, M., Norton, J., Piet-Lahanier, H., Walter, E. (eds.)
Bounding Approaches to System Identification, pp. 213–238. Plenum Press, New York (1996)
160. Kurzhanski, A.B., Kirilin, M.N.: Ellipsoidal techniques for reachability problems under
nonellipsoidal constraints. In: Proceedings of the NOLCOS-01, St. Petersburg, pp. 768–768
(2001)
161. Kurzhanski, A.B., Mitchell, I.M., Varaiya, P.: Optimization techniques for state-constrained
control and obstacle problems. J. Optim. Theory Appl. 128(3), 499–521 (2006)
162. Kurzhanski, A.B., Mitchell, I.M., Varaiya, P.: Control synthesis for state constrained systems
and obstacle problems. In: Proceedings of the IFAC Eds. Allgower F., Zeitz M., Oxford.
NOLCOS 6-th Symposium at Suttgart, Elsevier, Amsterdam (2005–2014)
163. Kurzhanski, A.B., Nikonov, O.I.: On the control strategy synthesis problem. Evolution
equations and set-valued integration. Dokl. Akad. Nauk SSSR. 311(4), 788–793 (1990)
164. Kurzhanski, A.B., Nikonov, O.I.: Evolution equations for trajectory bundles of synthesized
control systems. Russ. Acad. Sci. Dokl. Math. 333(5), 578–581 (1993)
165. Kurzhanski, A.B., Osipov, Y.S.: The problem of control with bounded phase coordinates.
Appl. Math. Mech. (PMM) 32(2), 188–195 (1968)
166. Kurzhanski, A.B., Osipov, Y.S.: On optimal control under state constraints. Appl. Math.
Mech. (PMM) 33(4), 705–719 (1969)
167. Kurzhanski, A.B., Sivergina, I.F.: Method of guaranteed estimates and regularization prob-
lems for evolutionary systems. J. Comput. Math. Math. Phys. 32(11), 1720–1733 (1992)
168. Kurzhanski, A.B., Sugimoto, K., Vályi, I.: Guaranteed state estimation for dynamical systems:
ellipsoidal techniques. Int. J. Adapt. Control Signal Process. 8(1), 85–101 (1994)
169. Kurzhanski, A.B., Tanaka, M.: Identification: deterministic versus stochastic models. Öster-
reichische Zeitschrift für Statistik und Informatik 19(1), 30–56 (1989)
170. Kurzhanski, A.B., Tochilin, P.A.: Weakly invariant sets of hybrid systems. Differ. Equ. 44(11),
1585–1594 (2008)
171. Kurzhanski, A.B., Tochilin, P.A.: Impulse controls in models of hybrid systems. Differ. Equ.
45(5), 731–742 (2009)
172. Kurzhanski, A.B., Tochilin, P.A.: On the problem of control synthesis under uncertainty from
the outputs of finite observers. Differ. Equ. 47(11), 1–9 (2011)
173. Kurzhanski, A.B., Vályi, I.: Ellipsoidal calculus for estimation and feedback control. In:
Systems and Control in the Twenty-First Century, pp. 229–243. Birkhäuser, Boston (1997)
174. Kurzhanski, A.B., Vályi, I.: Ellipsoidal Calculus for Estimation and Control. SCFA.
Birkhäuser, Boston (1997)
175. Kurzhanski, A.B., Varaiya, P.: Ellipsoidal techniques for reachability analysis. In: Proceed-
ings of the Pittsburg Conference “Hybrid Systems-2000”. Lecture Notes in Control Science,
vol. 1790, pp. 202–214. Springer, Berlin (2000)
176. Kurzhanski, A.B., Varaiya, P.: On the reachability problem under persistent disturbances.
Dokl. Math. 61(3), 3809–3814 (2000)
438 References

177. Kurzhanski, A.B., Varaiya, P.: Ellipsoidal techniques for reachability analysis. Internal
approximation. Syst. Control Lett. 41, 201–211 (2000)
178. Kurzhanski, A.B., Varaiya, P.: Dynamic optimization for reachability problems. J. Optim.
Theory Appl. 108(2), 227–251 (2001)
179. Kurzhanski, A.B., Varaiya, P.: Reachability under state constraints - the ellipsoidal technique.
In: Proceedings of the 15-th IFAC Congress, Barcelona (2002)
180. Kurzhanski, A.B., Varaiya, P.: Reachability analysis for uncertain systems—the ellipsoidal
technique. Dyn. Continuous Dis. Impulsive Syst. Ser. B 9(3), 347–367 (2002)
181. Kurzhanski, A.B., Varaiya, P.: Ellipsoidal techniques for reachability analysis. Part I: external
approximations. Optim. Methods Softw. 17(2), 177–206 (2002)
182. Kurzhanski, A.B., Varaiya, P.: Ellipsoidal techniques for reachability analysis. Part II: internal
approximations. Box-valued constraints. Optim. Methods Softw. 17(2), 207–237 (2002)
183. Kurzhanski, A.B., Varaiya, P.: On reachability under uncertainty. SIAM J. Control Optim.
41(1), 181–216 (2002)
184. Kurzhanski, A.B., Varaiya, P.: On some nonstandard dynamic programming problems of
control theory. In: Giannessi, F., Maugeri, A. (eds.) Variational Methods and Applications,
pp. 613–627. Kluwer, New York (2004)
185. Kurzhanski, A.B., Varaiya, P.: Ellipsoidal techniques for reachability under state constraints.
SIAM J. Control Optim. 45(4), 1369–1394 (2006)
186. Kurzhanski, A.B., Varaiya, P.: The Hamilton–Jacobi equations for nonlinear target control
and their approximation. In: Analysis and Design of Nonlinear Control Systems (in Honor of
Alberto Isidori), pp. 77–90. Springer, Berlin (2007)
187. Kurzhanski, A.B., Varaiya, P.: On synthesizing team target controls under obstacles and
collision avoidance. J. Franklin Inst. 347(1), 130–145 (2010)
188. Kurzhanski, A.B., Varaiya, P.: Impulsive inputs for feedback control and hybrid system
modeling. In: Sivasundaram, S., Vasundhara Devi, J., Udwadia, F.E., Lasiecka, I. (eds.)
Advances in Dynamics and Control: Theory, Methods and Applications. Cambridge Scientific
Publications, Cottenham (2009)
189. Kurzhanski, A.B., Varaiya, P.: Output feedback control under set-membership uncertainty.
J. Optim. Theory Appl. 151(1), 11–32 (2011)
190. Kvasnica, M., Grieder, P., Baotic, M., Morari, M.: Multiparametric toolbox (MPT). In: Hybrid
Systems: Computation and Control, pp. 448–462. Springer, Berlin (2004)
191. Kwakernaak, H., Sivan, R.: Linear Optimal Control Systems. Wiley-Interscience, New York
(1972)
192. Lancaster, P.: Theory of Matrices. Academic, New York (1969)
193. Lattès, R., Lions, J.-L.: Méthode de quasi-réversibilité et applications. Travaux et recherches
mathématiques, vol. 15. Dunod, Paris (1967)
194. Laurent, P.J.: Approximation and Optimization. Hermann, Paris (1972)
195. Lee, E.B., Markus, L.: Foundations of Optimal Control Theory. Wiley, New York (1967)
196. Lefschetz, S.: Differential Equations: Geometric Theory. Dover, Mineola (1977)
197. Le Guernic, C., Girard, A.: Nonlinear Anal. Hybrid Syst. 4(2), 250–262 (2010)
198. Leitmann, G.: Optimality and reachability with feedback controls. In: Blaquière, A., Leit-
mann, G. (eds.) Dynamical Systems and Microphysics: Control Theory and Mechanics.
Academic, Orlando (1982)
199. Lempio, F., Veliov, V.: Discrete approximations of differential inclusions. Bayreuther Mathe-
matische Schriften 54, 149–232 (1998)
200. Lions, P.-L.: Viscosity solutions and optimal control. In: Proceedings of ICIAM 91,
pp. 182–195. SIAM, Washington (1991)
201. Lions, P.-L., Souganidis, P.E.: Differential games, optimal control and directional derivatives
of viscosity solutions of Bellman’s and Isaac’s equations. SIAM J. Control Optim. 23,
566–583 (1995)
202. Lyapunov, A.A: On fully additive vector functions. Izvestia USSR Acad. Sci., Ser. Math.,
4(6), 465–478 (1940)
References 439

203. Lygeros, J.: On the relation of reachability to minimum cost optimal control. In: Proceedings
of the 41st IEEE CDC, Las Vegas, pp. 1910–1915 (2002)
204. Lygeros, J., Tomlin, C., Sastry, S.: Controllers for reachability specifications for hybrid
systems. Automatica 35(3), 349–370 (1999)
205. Lygeros, J., Quincampoix, M., Rzezuchowski, T.: Impulse differential inclusions driven by
discrete measures. In: Hybrid Systems: Computation and Control. Lecture Notes in Computer
Science, vol. 4416, pp. 385–398. Springer, Berlin (2007)
206. Margellos, K., Lygeros, J.: Hamilton-Jacobi formulation for reach-avoid differential games.
IEEE Trans. Automat. Control 56(8), 1849–1861 (2011)
207. Mayne, D.Q., Rawlings, J.B., Rao, C.V., Scokaert, C.O.M.: Constrained model predictive
control: stability and optimality. Automatica 36, 789–814 (2000)
208. Mayne, D.Q., Seron, M.M., Rakovic, S.V.: Robust model predictive control of constrained
linear systems with bounded disturbances. Automatica 41, 219–224 (2005)
209. Mazurenko, S.S.: A differential equation for the gauge function of the star-shaped attainability
set of a differential inclusion. Doklady Math., Moscow 86(1), 476–479 (2012) [original
Russian Text published in Doklady Akademii Nauk 445(2), 139–142 (2012)]
210. Milanese, M., Norton, J., Piet-Lahanier, H., Walter, E. (eds.): Bounding Approach to System
Identification. Plenum Press, London (1996)
211. Milanese, M., Tempo, R.: Optimal algorithms theory for robust estimation and prediction.
IEEE Trans. Automat. Control 30(8), 730–738 (1985)
212. Miller, B.M., Rubinovich, E.Ya.: Impulsive Control in Continuous and Discrete-Continuous
Systems. Kluwer, New York (2003)
213. Mitchell, I.M.: Application of level set methods to control and reachability problems in
continuous and hybrid systems. Ph.D. thesis, Stanford University (2002)
214. Mitchell, I.M., Tomlin, C.J.: Overapproximating reachable sets by Hamilton-Jacobi projec-
tions. J. Sci. Comput. 19(1), 323–346 (2003)
215. Mitchell, I.M., Templeton, J.A.: A toolbox of Hamilton-Jacobi solvers for analysis of
nondterministic in continuous and hybrid systems. In: Hybrid Systems: Computation and
Control, pp. 480–494. Springer, Berlin (2005)
216. Morari, M., Thiele, L. (eds.): Hybrid Systems: Computation and Control. Lecture Notes in
Computer Sciences, vol. 3414. Springer, Berlin/Heidelberg (2005)
217. Moreau, J.J.: Fonctionelles Convexes. College de France, Paris (1966)
218. Motta, M., Rampazzo, F.: Dynamic programming for nonlinear system driven by ordinary
and impulsive control. SIAM J. Control Optim. 34(1), 199–225 (1996)
219. Neustadt, L.W.: Optimization, a moment problem and nonlinear programming. SIAM J.
Control Optim. 2(1), 33–53 (1964)
220. Nikolski, M.S.: On the approximation of a reachability set of a differential inclusion. Herald
of the Moscow Univ. (Vestnik MGU), Ser. Comput. Math. Cybern. 4, 31–34 (1987)
221. Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Applied Mathe-
matical Sciences, vol. 153. Springer, New York (2003)
222. Osipov, Yu.S.: On the theory of differential games in distributed parameter systems. Sov.
Math. Dokl. 223(6), 1314–1317 (1975)
223. Osipov, Yu.S., Kryazhimski, A.V.: Inverse Problems for Ordinary Differential Equations:
Dynamical Solutions. Gordon and Breach, London (1995)
224. Panasyuk, V.I., Panasyuk, V.I.: On one equation generated by a differential inclusion . Math.
Notes 27(3), 429–437 (1980)
225. Patsko, V.S., Pyatko, S.G., Fedotov, A.A.: Three-dimensional reachability set for a nonlinear
control system. J. Comput. Syst. Sci. Int. 42(3), 320–328 (2003)
226. Pontryagin, L.S., Boltyansky, V.G., Gamkrelidze, P.V., Mischenko, E.F.: The Mathematical
Theory of Optimal Processes. Wiley Interscience, New York (1962)
227. Pschenichniy, B.N.: Convex Analysis and Extremal Problems. Nauka, Moscow (1980)
228. Puri, A., Varaiya, P.: Decidability of hybrid systems with rectangular differential inclusions.
In: Proceedings of the 6th Workshop on Computer-Aided Verification. Lecture Notes in
Control Science, vol. 818, pp. 95–104. Springer, New York (1994)
440 References

229. Puri, A., Borkar, V.S., Varaiya, P.: ©-Approximation of differential inclusions. In:
Hybrid Systems. Lecture Notes in Control Science, vol. 1066, pp. 362–376. Springer,
New York (1996)
230. Rakovic, S.R., Kerrigan, E., Mayne, D., Lygeros, J.: Reachability analysis of discrete-time
systems with disturbances. IEEE Trans. Automat. Control 51, 546–561 (2006)
231. Rakovic, S.R., Kouvritakis, B., Cannon, M., Panos, Ch., Findeisen, R.: Parametrized tube
model predictive control. IEEE Trans. Automat. Control 57, 2746–2761 (2012)
232. Rakovic, S.R., Kouvritakis, B., Findeisen, R., Cannon, M.: Homothetic model predictive
control. Automatica 48, 1631–1637 (2012)
233. Rakovic, S.R., Kouvritakis, B., Cannon, M.: Equi-normalization and exact scaling dynamics
in homothetic tube control. Syst. Control Lett. 62, 209–217 (2013)
234. Riesz, F., Sz-.Nagy, B.: Leçons d’analyse fonctionnelle. Akadémiai Kiadó, Budapest (1972)
235. Rockafellar, R.T.: Level sets and continuity of conjugate convex functions. Trans. Am. Math.
Soc. 123, 46–63 (1966)
236. Rockafellar, R.T.: State constraints in convex problems of Bolza. SIAM J. Control Optim
10(4), 691–715 (1972)
237. Rockafellar, R.T.: Convex Analysis, 2nd edn. Princeton University Press, Princeton (1999)
238. Rockafellar, R.T., Wets, R.J.: Variational Analysis. Springer, Berlin (2005)
239. Roxin, E.: On generalized dynamical systems defined by contingent equations. J. Differ. Equ.
1(2), 188–205 (1965)
240. Saint-Pierre, P.: Approximation of the viability kernel. Appl. Math. Optim. 29(2), 187–209
(1994)
241. Schlaepfer, F.M., Schweppe, F.C.: Continuous-time state estimation under disturbances
bounded by convex sets. IEEE Trans. Automat. Control 17(2), 197–205 (1972)
242. Schwartz, L.: Théorie des Distributions. Hermann, Paris (1950)
243. Schweppe, F.C.: Uncertain Dynamic Systems. Prentice Hall, Englewood Cliffs (1973)
244. Sethian, J.A.: Level Set Methods and Fast Marching Methods. Cambridge University Press,
Cambridge (1999)
245. Skelton, R.E.: Dynamic Systems and Control. Wiley, New York (1988)
246. Sorensen, A.J.: A survey of dynamic positioning control systems. Ann. Rev. Control 35,
123–136 (2011)
247. Subbotin, A.I.: Generalized Solutions of First-Order PDE’s. The Dynamic Optimization
Perspective. SCFA. Birkhäuser, Boston (1995)
248. Subbotina, N.N., Kolpakova, E.A., Tokmantsev, T.B., Shagalova, L.G.: Method of Character-
istics for the Hamilton-Jacobi-Bellman Equation. Ural Scientific Center/Russian Academy of
Sciences, Yekaterinburg (2013)
249. Summers, S., Kamgarpour, M., Tomlin, C.J., Lygeros, J.: A stochastic reach-avoid problem
with random obstacles. In: Belta, C., Ivancic, F. (eds.) Hybrid Systems: Computation and
Control. ACM, New York (2013)
250. Summers, S., Kamgarpour, M., Tomlin, C.J., Lygeros, J.: Stochastic hybrid system con-
troller synthesis for reachability specifications encoded by random sets. Automatica 49(9),
2906–2910 (2013)
251. Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-Posed Prolems. Winston, New York (1975)
252. Tikhonov, A.N., Goncharsky, A.V.: Ill-Posed Problems in the Natural Sciences. Oxford
University Press, Oxford (1987)
253. Tolstonogov, A.A.: Differential Inclusions ina Banach Space. Kluwer Academic, Boston
(2000)
254. Tomlin, C.J., Lygeros, J., Sastry, S.: A game theoretic approach to controller design for hybrid
systems. Proceedings of IEEE 88, 949–969 (2000)
255. Ushakov, V.N.: Construction of solutions in differential games of pursuit-evasion. Lecture
Notes in Nonlinear Analysis, vol. 2(1), pp. 269–281. Polish Academic Publishers, Warsaw
(1997)
256. Usoro, P.B., Schweppe, F., Wormley, D., Gould, L.: Ellipsoidal set-theoretic control synthesis.
J. Dyn. Syst. Meas. Control 104, 331–336 (1982)
References 441

257. Ustyuzhanin, A.M.: On the problem of matrix parameter identification. Probl. Control Inf.
Theory 15(4), 265–273 (1986)
258. Van der Schaft, A., Schumacher, H.: An Introduction to Hybrid Dynamical Systems. Lecture
Notes in Control and Information Sciences, vol. 25. Springer, Berlin (2000)
259. Varaiya, P.: Reach set computation using optimal control. In: Proceedings of the KIT
Workshop on Verification of Hybrid Systems, Grenoble, October 1998
260. Varaiya, P., Lin, J.: Existence of saddle points in differential games. SIAM J. Control Optim.
7(1), 142–157 (1969)
261. Vazhentsev, A.Yu.: On internal ellipsoidal approximations for problems of feedback control
under state constraints. Proc. Russ. Acad. Sci. Theory Syst. Control 3, 70–77 (2000)
262. Veliov, V.: Second order discrete approximations to strongly convex differential inclusions.
Syst. Control Lett. 13(3), 263–269 (1989)
263. Veliov,V.: Stability-like properties of differential inclusions. Set Valued Anal. 5(1), 73–88
(1997)
264. Vidal, R., Schaffert, S., Lygeros, J., Sastry, S.: Controlled invariance of discrete time systems.
In: Hybrid Systems: Computation and Control. Lecture Notes in Computer Science, vol. 1790,
pp. 437–450. Springer, Berlin (2000)
265. Vinter, R.B.: A characterization of the reachable set for nonlinear control systems. SIAM J.
Control Optim. 18(6), 599–610 (1980)
266. Vinter, R.B., Wolenski, P.: Hamilton–Jacobi theory for optimal control problems with data
measurable in time. SIAM J. Control Optim. 28(6), 1404–1419 (1990)
267. Vinter, R.B.: Optimal Control. Birkhäuser, Boston (2010)
268. Vostrikov, I.V., Dar’in, A.N., Kurzhanskii, A.B.: On the damping of a ladder-type vibration
system subjected to uncertain perturbations. Differ. Equ. 42(11), 1524–1535 (2006)
269. Whittaker, E.T.: Analytical Dynamics. Dover, New York (1944)
270. Willemsm, J.C. (ed.): From Data to Model. Springer, New York (1989)
271. Witsenhausen, H.: Set of possible states of linear systems given perturbed observations. IEEE
Trans. Automat. Control 13(1), 5–21 (1968)
272. Wolenski, P.: The exponential formula for the reachable set of a lipschitz differential
inclusion. SIAM J. Control Optim. 28(5), 1148–1161 (1990)
273. Wonham, W.M.: Linear Multivariable Control: A Geometrical Approach. Springer, Berlin
(1985)
274. Zadeh, l.A., Desoer, C.A.: Linear System Theory: The State Space Approach. Dover,
New York (2008)
275. Zakroischikov, V.N.: Reachability sets of hybrid systems under sequential switchings. Herald
MSU 2, 37–44 (2009)
Index

B state constrained, 289–309

Backward reachability, 47, 61–64, 82, 83, target, 75–78, 128, 131, 167, 170, 182, 311,
128–131, 144, 158–167, 170, 371
183, 197, 199, 326–328, 338, time-optimal, 18–24, 83–86
383, 385 Controllability, 10–18, 32, 33, 72, 90, 101,
169, 237, 256, 293

C
Closed loop target control, 75–78, 167, 371 D
Colliding tubes, 70–75, 330 Differential inclusion, 10, 19, 23, 60, 61, 63,
Comparison principle 76, 88, 89, 115, 132, 268, 270, 280,
convex reach sets, 198 290, 312, 313, 315–317, 322, 324,
nonconvex reach sets, 208–213 326, 331, 339, 344, 365, 367, 369,
under state constraints, 281–284 370, 375, 378, 385
Constraints (bounds) Disturbance, 2, 9, 25, 63, 196–198, 341–343,
ellipsoidal, quadratic, 128, 369, 388 345, 352, 358, 364, 371–373,
joint (double), 7 392–394
magnitude, integral, 5–8 Dynamic programming, 46–86, 253, 260, 371,
Control 375, 377, 383, 424
closed-loop, 1, 5–10, 46, 52, 53, 54, 58, 63, Dynamic programming equation, 48–54
75, 78, 167, 200, 255, 322, 326–331,
338–339, 373
feedback, 9, 47, 52, 128, 161, 170, 184,
195–196, 260, 269, 270, 272, 275, E
284, 311–339, 371–394, 420, Ellipsoidal approximations
423 box-valued constraints, 125, 229–231,
feedforward, model predictive, 46 235–241, 285
minimal magnitude, 12–16 ellipsoidal constraints, 88–91, 202, 388,
norm-minimal, minimal energy, 10 398
open-loop, 1, 8–10, 24–40, 52, 63, 253, external, tight, 96, 98
261, 402, 403 internal, tight, 120, 131, 161, 246
optimal, linear, 1–46 zonotope constraints, 241–251, 388

© Springer International Publishing Switzerland 2014 443

A.B. Kurzhanski, P. Varaiya, Dynamics and Control of Trajectory
Tubes, Systems & Control: Foundations & Applications 85,
DOI 10.1007/978-3-319-10277-1
444 Index

Equation open loop, value function, 255–260

HJB (Hamilton–Jacobi–Bellman), 47, 51, realistic approximation, controls, 266–269
53, 54, 56, 58–66, 68, 69, 75, 84, Information space, 381–386
197–208, 218, 224, 253, 267, 341, Information tubes, calculation
342, 344–348, 374, 383, 427 linear-quadratic approximation, 363–367
HJB (state constrained), 277–279, 281, singular perturbations, discontinuous
282, 284, 327, 336 measurements, 370
HJBI (Hamilton–Jacobi–Bellman–Isaacs), stochastic filtering approximation, 367–370
197, 205, 383, 385 Integral
integral funnel, 102, 280, 313, 367, 377 set-valued, 20, 115, 144, 231, 288
Riccati, 56, 214, 367 Stieltjes, Lebesgue–Stieltjes, 254, 266, 268,
286, 288, 293

F
Function L
conjugate (Fenchel), 28, 68, 261, 268 Legendre transformation, 29
convex, 29, 38, 39, 66, 68, 81, 216, 254, Lipschitz condition, 312, 378
396
second conjugate (Fenchel), 81
strictly convex, 38, 39 M
support, 19, 20, 22–24, 39, 47, 64, 65, Matching conditions, 196, 393
69, 80, 89, 90, 111, 119, 120, Maximum condition, 14, 24, 28–31, 34, 36, 44,
129, 144, 150, 157, 158, 185, 188, 287, 294
189, 191, 202, 241, 242, 280, 285, Maximum principle, 1, 14, 24–40, 44, 45, 72,
287, 289–292, 315, 319, 320, 330, 74, 90, 99, 108, 109, 245, 256, 287,
363–365, 410 289–294, 308, 325
ellipsoidal, 107–110, 251
Measurement equation, 312, 341, 345, 346,
G 348, 373
Generalized state, 372–375, 377, 378 Metric space, 281, 315, 381, 383
Gram matrix, 11, 15, 16 Minimum condition, 28, 35, 44
Guaranteed state estimation, 341–370, 372, Moment problem, 10, 12, 17–18, 255
377–381

O
H Obstacle problem
Hausdorff distance, 61, 103, 105, 281, 319 closed-loop control, 338–339
Hausdorff semidistance, 61, 103, 280, 281, 319 complementary convex constraint, 336
Hybrid system reach-evasion set, 335–336
branching trajectory tubes, 395 Optimization problem
ellipsoidal methods, 398–401, 418–419 linear-quadratic, 46, 54–57, 363, 367, 368
enabling zones, resets, 402, 414–415 primal, dual, 40–46
impulsive feedback control, 420, 423 Output feedback control
reachability, 395, 396, 401–416 ellipsoidal approximations, 388–392
value functions, 396, 397, 399, 407–416, linear-convex systems, 372, 379, 381,
418, 427 386–388
overall solution, 323, 372, 375–377
I
Impulse control P
approximating motion, 269–273 Parallel computation, 127, 195–196
closed loop, 260–266 Polyhedral techniques, output feedback
delta function, number of impulses, 7, 269, control, 373
421 Principle of optimality, 53, 59, 62, 263, 277,
HJB variational inequality, 260–266 278, 327
Index 445

R ellipsoidal approximation, 350–355,

Reachability, 18–24, 27, 32, 47, 58–70, 388–391
78–84, 87–145, 151, 158–167, 170, HJB equation, 341, 342, 344–346
182–184, 186, 189–193, 197–199, information set, information tube, 341–344,
201, 202, 208–215, 218, 222–229, 347, 349, 352, 368, 375, 377–381
235–239, 242, 243, 245–251, information state, 341, 344, 377–381
275–289, 300, 311, 326–328, 338, linear system, 342, 345, 348–350, 354–355,
341, 344, 352–356, 365, 369, 383, 358, 366, 368, 379
385, 388, 395, 396, 401–413, 415, Subdifferential, 28, 60, 279
416 Switching system, 403, 420
System
controlled, 1–5, 87, 141, 177, 179, 182,
S 185, 276, 371, 401, 403
Saddle point, 26, 36, 210 linear, 4, 18, 24, 54, 60, 66, 68, 88–91, 202,
Separation principle for estimation and control, 203, 213, 253, 281, 284, 285, 289,
391–392 316–317, 322, 325, 327, 339, 342,
Set 345, 348–350, 354–355, 358, 366,
box-valued, 198 368, 377, 379, 386–391, 393
convex, 5, 8, 38, 39, 187, 359, 416 linear-convex, 24–40, 47, 64–70, 183, 229,
ellipsoidal, 115 285–289
invariant, strong, 63, 64, 79 quasilinear, 5
invariant, weak, 61, 62, 202, 382–384
nonconvex, 200, 211, 220, 418
unknown but bounded, 8, 341, 358 T
zonotope, 198, 242, 246 Trajectory tube
Set-valued analysis, set-membership evolution equation, 311–319
uncertainty, 371 funnel equations, bilinear system, 315–318
Set-valued observer, 378, 379, 392 viable tube
Solvability set calculation, 313
extremal aiming, 78, 132, 195, 384 linear-convex system, 313
Krasovski bridge, 384 parametrization, 313
State constraints Transition matrix, 2, 148, 291, 292, 301, 391,
closed loop (feedback) control, 5–10, 322, 409
326–331
ellipsoidal approximation, 275, 294, 301,
304 V
linear-convex system, 285–289 Value function, 47, 48, 50–52, 54, 56, 58–60,
maximum principle (modified), 289–294, 62–72, 80, 81, 85, 201, 209, 217,
325 219, 221, 224, 255–260, 262, 264,
maximum principle (standard), 287, 294 266, 267, 276–278, 281, 283, 286,
reachability set, value function, 341, 356 326, 327, 333–335, 337, 341, 344,
State estimation, bounding approach 349, 372, 374, 375, 379, 381–384,
comparison principle, 342, 346–348, 350 386, 396, 397, 399, 407–416, 418,
continuous dynamics, discrete observation, 427
355–358 Verification problem
discrete dynamics, discrete observation, ellipsoidal methods, 398–401, 418–419
358–363 solution scheme, 396–398