Abstract Dynamic Programming Second Edition

Dimitri P. Bertsekas download

https://textbookfull.com/product/abstract-dynamic-programming-
second-edition-dimitri-p-bertsekas/

Download more ebook from https://textbookfull.com

We believe these products will be a great fit for you. Click
the link to download now, or visit textbookfull.com
to discover even more!

Introduction To Probability 2nd Edition Dimitri P

Bertsekas John N Tsitsiklis

https://textbookfull.com/product/introduction-to-probability-2nd-
edition-dimitri-p-bertsekas-john-n-tsitsiklis/

Dynamic Innovation in Outsourcing Leslie P. Willcocks

https://textbookfull.com/product/dynamic-innovation-in-
outsourcing-leslie-p-willcocks/

Abstract Algebra An Interactive Approach Second Edition

William Paulsen

https://textbookfull.com/product/abstract-algebra-an-interactive-
approach-second-edition-william-paulsen/

Brief Dynamic Therapy Second Edition Levenson

https://textbookfull.com/product/brief-dynamic-therapy-second-
edition-levenson/
Robust Adaptive Dynamic Programming 1st Edition Hao Yu

https://textbookfull.com/product/robust-adaptive-dynamic-
programming-1st-edition-hao-yu/

Programming Interview Problems: Dynamic Programming

(with solutions in Python) 1st Edition Leonardo Rossi

https://textbookfull.com/product/programming-interview-problems-
dynamic-programming-with-solutions-in-python-1st-edition-
leonardo-rossi/

Advanced High Dynamic Range Imaging, Second Edition

Francesco Banterle

https://textbookfull.com/product/advanced-high-dynamic-range-
imaging-second-edition-francesco-banterle/

Adaptive Dynamic Programming with Applications in

Optimal Control 1st Edition Derong Liu

https://textbookfull.com/product/adaptive-dynamic-programming-
with-applications-in-optimal-control-1st-edition-derong-liu/

Algorithms Illuminated Part 3 Greedy Algorithms and

Dynamic Programming Tim Roughgarden

https://textbookfull.com/product/algorithms-illuminated-
part-3-greedy-algorithms-and-dynamic-programming-tim-roughgarden/
Abstract Dynamic Programming
SECOND EDITION

Dimitri P. Bertsekas
Massachusetts Institute of Technology

WWW site for book information and orders

http://www.athenasc.com

Athena Scientific, Belmont, Massachusetts

Athena Scientific
Post Office Box 805
Nashua, NH 03061-0805
U.S.A.

Email: [email protected]
WWW: http://www.athenasc.com

Cover design and photography: Dimitri Bertsekas

Cover Image from Simmons Hall, MIT (Steven Holl, architect)

c 2018 Dimitri P. Bertsekas

All rights reserved. No part of this book may be reproduced in any form
by any electronic or mechanical means (including photocopying, recording,
or information storage and retrieval) without permission in writing from
the publisher.

Publisher’s Cataloging-in-Publication Data

Bertsekas, Dimitri P.
Abstract Dynamic Programming: Second Edition
Includes bibliographical references and index
1. Mathematical Optimization. 2. Dynamic Programming. I. Title.
QA402.5 .B465 2018 519.703 01-75941

ISBN-10: 1-886529-46-9, ISBN-13: 978-1-886529-46-5

 ABOUT THE AUTHOR

Dimitri Bertsekas studied Mechanical and Electrical Engineering at the

National Technical University of Athens, Greece, and obtained his Ph.D.
in system science from the Massachusetts Institute of Technology. He has
held faculty positions with the Engineering-Economic Systems Department,
Stanford University, and the Electrical Engineering Department of the Uni-
versity of Illinois, Urbana. Since 1979 he has been teaching at the Electrical
Engineering and Computer Science Department of the Massachusetts In-
stitute of Technology (M.I.T.), where he is currently the McAfee Professor
of Engineering.
His teaching and research spans several fields, including determinis-
tic optimization, dynamic programming and stochastic control, large-scale
and distributed computation, and data communication networks. He has
authored or coauthored numerous research papers and sixteen books, sev-
eral of which are currently used as textbooks in MIT classes, including
“Dynamic Programming and Optimal Control,” “Data Networks,” “Intro-
duction to Probability,” “Convex Optimization Theory,” “Convex Opti-
mization Algorithms,” and “Nonlinear Programming.”
Professor Bertsekas was awarded the INFORMS 1997 Prize for Re-
search Excellence in the Interface Between Operations Research and Com-
puter Science for his book “Neuro-Dynamic Programming” (co-authored
with John Tsitsiklis), the 2001 AACC John R. Ragazzini Education Award,
the 2009 INFORMS Expository Writing Award, the 2014 AACC Richard
Bellman Heritage Award, the 2014 Khachiyan Prize for Life-Time Accom-
plishments in Optimization, and the MOS/SIAM 2015 George B. Dantzig
Prize. In 2001, he was elected to the United States National Academy of
Engineering for “pioneering contributions to fundamental research, practice
and education of optimization/control theory, and especially its application
to data communication networks.”

iii
 ATHENA SCIENTIFIC
OPTIMIZATION AND COMPUTATION SERIES

1. Abstract Dynamic Programming, 2nd Edition, by Dimitri P.

Bertsekas, 2018, ISBN 978-1-886529-46-5, 360 pages
2. Dynamic Programming and Optimal Control, Two-Volume Set,
by Dimitri P. Bertsekas, 2017, ISBN 1-886529-08-6, 1270 pages
3. Nonlinear Programming, 3rd Edition, by Dimitri P. Bertsekas,
2016, ISBN 1-886529-05-1, 880 pages
4. Convex Optimization Algorithms, by Dimitri P. Bertsekas, 2015,
ISBN 978-1-886529-28-1, 576 pages
5. Convex Optimization Theory, by Dimitri P. Bertsekas, 2009,
ISBN 978-1-886529-31-1, 256 pages
6. Introduction to Probability, 2nd Edition, by Dimitri P. Bertsekas
and John N. Tsitsiklis, 2008, ISBN 978-1-886529-23-6, 544 pages
7. Convex Analysis and Optimization, by Dimitri P. Bertsekas, An-
gelia Nedić, and Asuman E. Ozdaglar, 2003, ISBN 1-886529-45-0,
560 pages
8. Network Optimization: Continuous and Discrete Models, by Dim-
itri P. Bertsekas, 1998, ISBN 1-886529-02-7, 608 pages
9. Network Flows and Monotropic Optimization, by R. Tyrrell Rock-
afellar, 1998, ISBN 1-886529-06-X, 634 pages
10. Introduction to Linear Optimization, by Dimitris Bertsimas and
John N. Tsitsiklis, 1997, ISBN 1-886529-19-1, 608 pages
11. Parallel and Distributed Computation: Numerical Methods, by
Dimitri P. Bertsekas and John N. Tsitsiklis, 1997, ISBN 1-886529-
01-9, 718 pages
12. Neuro-Dynamic Programming, by Dimitri P. Bertsekas and John
N. Tsitsiklis, 1996, ISBN 1-886529-10-8, 512 pages
13. Constrained Optimization and Lagrange Multiplier Methods, by
Dimitri P. Bertsekas, 1996, ISBN 1-886529-04-3, 410 pages
14. Stochastic Optimal Control: The Discrete-Time Case, by Dimitri
P. Bertsekas and Steven E. Shreve, 1996, ISBN 1-886529-03-5,
330 pages

iv
Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . p. 1
1.1. Structure of Dynamic Programming Problems . . . . . . . p. 2
1.2. Abstract Dynamic Programming Models . . . . . . . . . . p. 5
1.2.1. Problem Formulation . . . . . . . . . . . . . . . . p. 5
1.2.2. Monotonicity and Contraction Properties . . . . . . . p. 7
1.2.3. Some Examples . . . . . . . . . . . . . . . . . . p. 10
1.2.4. Approximation Models - Projected and Aggregation . . . .
Bellman Equations . . . . . . . . . . . . . . . . p. 24
1.2.5. Multistep Models - Temporal Difference and . . . . . . .
Proximal Algorithms . . . . . . . . . . . . . . . . p. 26
1.3. Organization of the Book . . . . . . . . . . . . . . . . p. 29
1.4. Notes, Sources, and Exercises . . . . . . . . . . . . . . . p. 31

2. Contractive Models . . . . . . . . . . . . . . . . . p. 39
2.1. Bellman’s Equation and Optimality Conditions . . . . . . . p. 40
2.2. Limited Lookahead Policies . . . . . . . . . . . . . . . p. 47
2.3. Value Iteration . . . . . . . . . . . . . . . . . . . . . p. 52
2.3.1. Approximate Value Iteration . . . . . . . . . . . . . p. 53
2.4. Policy Iteration . . . . . . . . . . . . . . . . . . . . . p. 56
2.4.1. Approximate Policy Iteration . . . . . . . . . . . . p. 59
2.4.2. Approximate Policy Iteration Where Policies . . . . . . .
Converge . . . . . . . . . . . . . . . . . . . . . p. 61
2.5. Optimistic Policy Iteration and λ-Policy Iteration . . . . . . p. 63
2.5.1. Convergence of Optimistic Policy Iteration . . . . . . p. 65
2.5.2. Approximate Optimistic Policy Iteration . . . . . . . p. 70
2.5.3. Randomized Optimistic Policy Iteration . . . . . . . . p. 73
2.6. Asynchronous Algorithms . . . . . . . . . . . . . . . . p. 77
2.6.1. Asynchronous Value Iteration . . . . . . . . . . . . p. 77
2.6.2. Asynchronous Policy Iteration . . . . . . . . . . . . p. 84
2.6.3. Optimistic Asynchronous Policy Iteration with a . . . . . .
Uniform Fixed Point . . . . . . . . . . . . . . . . p. 89
2.7. Notes, Sources, and Exercises . . . . . . . . . . . . . . . p. 96

v
 vi Contents

3. Semicontractive Models . . . . . . . . . . . . . . p. 105

3.1. Pathologies of Noncontractive DP Models . . . . . . . . p. 107
3.1.1. Deterministic Shortest Path Problems . . . . . . . p. 111
3.1.2. Stochastic Shortest Path Problems . . . . . . . . . p. 113
3.1.3. The Blackmailer’s Dilemma . . . . . . . . . . . . p. 115
3.1.4. Linear-Quadratic Problems . . . . . . . . . . . . p. 118
3.1.5. An Intuitive View of Semicontractive Analysis . . . . p. 123
3.2. Semicontractive Models and Regular Policies . . . . . . . p. 125
3.2.1. S-Regular Policies . . . . . . . . . . . . . . . . p. 128
3.2.2. Restricted Optimization over S-Regular Policies . . . p. 130
3.2.3. Policy Iteration Analysis of Bellman’s Equation . . . p. 136
3.2.4. Optimistic Policy Iteration and λ-Policy Iteration . . p. 144
3.2.5. A Mathematical Programming Approach . . . . . . p. 148
3.3. Irregular Policies/Infinite Cost Case . . . . . . . . . . p. 149
3.4. Irregular Policies/Finite Cost Case - A Perturbation . . . . . .
Approach . . . . . . . . . . . . . . . . . . . . . . p. 155
3.5. Applications in Shortest Path and Other Contexts . . . . p. 161
3.5.1. Stochastic Shortest Path Problems . . . . . . . . . p. 162
3.5.2. Affine Monotonic Problems . . . . . . . . . . . . p. 170
3.5.3. Robust Shortest Path Planning . . . . . . . . . . p. 179
3.5.4. Linear-Quadratic Optimal Control . . . . . . . . . p. 189
3.5.5. Continuous-State Deterministic Optimal Control . . . p. 191
3.6. Algorithms . . . . . . . . . . . . . . . . . . . . . . p. 195
3.6.1. Asynchronous Value Iteration . . . . . . . . . . . p. 195
3.6.2. Asynchronous Policy Iteration . . . . . . . . . . . p. 196
3.7. Notes, Sources, and Exercises . . . . . . . . . . . . . . p. 203

4. Noncontractive Models . . . . . . . . . . . . . . p. 215

4.1. Noncontractive Models - Problem Formulation . . . . . . p. 217
4.2. Finite Horizon Problems . . . . . . . . . . . . . . . . p. 219
4.3. Infinite Horizon Problems . . . . . . . . . . . . . . . p. 225
4.3.1. Fixed Point Properties and Optimality Conditions . . p. 228
4.3.2. Value Iteration . . . . . . . . . . . . . . . . . . p. 240
4.3.3. Exact and Optimistic Policy Iteration - . . . . . . . . . .
λ-Policy Iteration . . . . . . . . . . . . . . . . p. 244
4.4. Regularity and Nonstationary Policies . . . . . . . . . . p. 249
4.4.1. Regularity and Monotone Increasing Models . . . . . p. 255
4.4.2. Nonnegative Cost Stochastic Optimal Control . . . . p. 257
4.4.3. Discounted Stochastic Optimal Control . . . . . . . p. 260
4.4.4. Convergent Models . . . . . . . . . . . . . . . . p. 262
4.5. Stable Policies for Deterministic Optimal Control . . . . . p. 266
4.5.1. Forcing Functions and p-Stable Policies . . . . . . . p. 270
4.5.2. Restricted Optimization over Stable Policies . . . . . p. 273
4.5.3. Policy Iteration Methods . . . . . . . . . . . . . p. 285
 Contents vii

4.6. Infinite-Spaces Stochastic Shortest Path Problems . . . . . p. 291

4.6.1. The Multiplicity of Solutions of Bellman’s Equation . p. 299
4.6.2. The Case of Bounded Cost per Stage . . . . . . . . p. 301
4.7. Notes, Sources, and Exercises . . . . . . . . . . . . . . p. 304

Appendix A: Notation and Mathematical Conventions . . p. 321

A.1. Set Notation and Conventions . . . . . . . . . . . . . p. 321
A.2. Functions . . . . . . . . . . . . . . . . . . . . . . p. 323

Appendix B: Contraction Mappings . . . . . . . . . . p. 325

B.1. Contraction Mapping Fixed Point Theorems . . . . . . . p. 325
B.2. Weighted Sup-Norm Contractions . . . . . . . . . . . p. 329

References . . . . . . . . . . . . . . . . . . . . . p. 335

Index . . . . . . . . . . . . . . . . . . . . . . . p. 343
Preface of the First Edition

This book aims at a unified and economical development of the core the-
ory and algorithms of total cost sequential decision problems, based on
the strong connections of the subject with fixed point theory. The analy-
sis focuses on the abstract mapping that underlies dynamic programming
(DP for short) and defines the mathematical character of the associated
problem. Our discussion centers on two fundamental properties that this
mapping may have: monotonicity and (weighted sup-norm) contraction. It
turns out that the nature of the analytical and algorithmic DP theory is
determined primarily by the presence or absence of these two properties,
and the rest of the problem’s structure is largely inconsequential.
In this book, with some minor exceptions, we will assume that mono-
tonicity holds. Consequently, we organize our treatment around the con-
traction property, and we focus on four main classes of models:
(a) Contractive models, discussed in Chapter 2, which have the richest
and strongest theory, and are the benchmark against which the the-
ory of other models is compared. Prominent among these models are
discounted stochastic optimal control problems. The development of
these models is quite thorough and includes the analysis of recent ap-
proximation algorithms for large-scale problems (neuro-dynamic pro-
gramming, reinforcement learning).
(b) Semicontractive models, discussed in Chapter 3 and parts of Chap-
ter 4. The term “semicontractive” is used qualitatively here, to refer
to a variety of models where some policies have a regularity/contrac-
tion-like property but others do not. A prominent example is stochas-
tic shortest path problems, where one aims to drive the state of
a Markov chain to a termination state at minimum expected cost.
These models also have a strong theory under certain conditions, of-
ten nearly as strong as those of the contractive models.
(c) Noncontractive models, discussed in Chapter 4, which rely on just
monotonicity. These models are more complex than the preceding
ones and much of the theory of the contractive models generalizes in
weaker form, if at all. For example, in general the associated Bell-
man equation need not have a unique solution, the value iteration
method may work starting with some functions but not with others,
and the policy iteration method may not work at all. Infinite hori-
zon examples of these models are the classical positive and negative
DP problems, first analyzed by Dubins and Savage, Blackwell, and

ix
x Preface

Strauch, which are discussed in various sources. Some new semicon-

tractive models are also discussed in this chapter, further bridging
the gap between contractive and noncontractive models.
(d) Restricted policies and Borel space models, which are discussed
in Chapter 5. These models are motivated in part by the complex
measurability questions that arise in mathematically rigorous theories
of stochastic optimal control involving continuous probability spaces.
Within this context, the admissible policies and DP mapping are
restricted to have certain measurability properties, and the analysis
of the preceding chapters requires modifications. Restricted policy
models are also useful when there is a special class of policies with
favorable structure, which is “closed” with respect to the standard DP
operations, in the sense that analysis and algorithms can be confined
within this class.
We do not consider average cost DP problems, whose character bears
a much closer connection to stochastic processes than to total cost prob-
lems. We also do not address specific stochastic characteristics underlying
the problem, such as for example a Markovian structure. Thus our re-
sults apply equally well to Markovian decision problems and to sequential
minimax problems. While this makes our development general and a con-
venient starting point for the further analysis of a variety of different types
of problems, it also ignores some of the interesting characteristics of special
types of DP problems that require an intricate probabilistic analysis.
Let us describe the research content of the book in summary, de-
ferring a more detailed discussion to the end-of-chapter notes. A large
portion of our analysis has been known for a long time, but in a somewhat
fragmentary form. In particular, the contractive theory, first developed by
Denardo [Den67], has been known for the case of the unweighted sup-norm,
but does not cover the important special case of stochastic shortest path
problems where all policies are proper. Chapter 2 transcribes this theory
to the weighted sup-norm contraction case. Moreover, Chapter 2 develops
extensions of the theory to approximate DP, and includes material on asyn-
chronous value iteration (based on the author’s work [Ber82], [Ber83]), and
asynchronous policy iteration algorithms (based on the author’s joint work
with Huizhen (Janey) Yu [BeY10a], [BeY10b], [YuB11a]). Most of this
material is relatively new, having been presented in the author’s recent
book [Ber12a] and survey paper [Ber12b], with detailed references given
there. The analysis of infinite horizon noncontractive models in Chapter 4
was first given in the author’s paper [Ber77], and was also presented in the
book by Bertsekas and Shreve [BeS78], which in addition contains much
of the material on finite horizon problems, restricted policies models, and
Borel space models. These were the starting point and main sources for
our development.
The new research presented in this book is primarily on the semi-
Preface xi

contractive models of Chapter 3 and parts of Chapter 4. Traditionally,

the theory of total cost infinite horizon DP has been bordered by two ex-
tremes: discounted models, which have a contractive nature, and positive
and negative models, which do not have a contractive nature, but rely
on an enhanced monotonicity structure (monotone increase and monotone
decrease models, or in classical DP terms, positive and negative models).
Between these two extremes lies a gray area of problems that are not con-
tractive, and either do not fit into the categories of positive and negative
models, or possess additional structure that is not exploited by the theory
of these models. Included are stochastic shortest path problems, search
problems, linear-quadratic problems, a host of queueing problems, multi-
plicative and exponential cost models, and others. Together these problems
represent an important part of the infinite horizon total cost DP landscape.
They possess important theoretical characteristics, not generally available
for positive and negative models, such as the uniqueness of solution of Bell-
man’s equation within a subset of interest, and the validity of useful forms
of value and policy iteration algorithms.
Our semicontractive models aim to provide a unifying abstract DP
structure for problems in this gray area between contractive and noncon-
tractive models. The analysis is motivated in part by stochastic shortest
path problems, where there are two types of policies: proper , which are
the ones that lead to the termination state with probability one from all
starting states, and improper , which are the ones that are not proper.
Proper and improper policies can also be characterized through their Bell-
man equation mapping: for the former this mapping is a contraction, while
for the latter it is not. In our more general semicontractive models, policies
are also characterized in terms of their Bellman equation mapping, through
a notion of regularity, which generalizes the notion of a proper policy and
is related to classical notions of asymptotic stability from control theory.
In our development a policy is regular within a certain set if its cost
function is the unique asymptotically stable equilibrium (fixed point) of
the associated DP mapping within that set. We assume that some policies
are regular while others are not , and impose various assumptions to ensure
that attention can be focused on the regular policies. From an analytical
point of view, this brings to bear the theory of fixed points of monotone
mappings. From the practical point of view, this allows application to a
diverse collection of interesting problems, ranging from stochastic short-
est path problems of various kinds, where the regular policies include the
proper policies, to linear-quadratic problems, where the regular policies
include the stabilizing linear feedback controllers.
The definition of regularity is introduced in Chapter 3, and its theoret-
ical ramifications are explored through extensions of the classical stochastic
shortest path and search problems. In Chapter 4, semicontractive models
are discussed in the presence of additional monotonicity structure, which
brings to bear the properties of positive and negative DP models. With the
xii Preface

aid of this structure, the theory of semicontractive models can be strength-

ened and can be applied to several additional problems, including risk-
sensitive/exponential cost problems.
The book has a theoretical research monograph character, but re-
quires a modest mathematical background for all chapters except the last
one, essentially a first course in analysis. Of course, prior exposure to DP
will definitely be very helpful to provide orientation and context. A few
exercises have been included, either to illustrate the theory with exam-
ples and counterexamples, or to provide applications and extensions of the
theory. Solutions of all the exercises can be found in Appendix D, at the
book’s internet site
http://www.athenasc.com/abstractdp.html
and at the author’s web site
http://web.mit.edu/dimitrib/www/home.html
Additional exercises and other related material may be added to these sites
over time.
I would like to express my appreciation to a few colleagues for inter-
actions, recent and old, which have helped shape the form of the book. My
collaboration with Steven Shreve on our 1978 book provided the motivation
and the background for the material on models with restricted policies and
associated measurability questions. My collaboration with John Tsitsiklis
on stochastic shortest path problems provided inspiration for the work on
semicontractive models. My collaboration with Janey (Huizhen) Yu played
an important role in the book’s development, and is reflected in our joint
work on asynchronous policy iteration, on perturbation models, and on
risk-sensitive models. Moreover Janey contributed significantly to the ma-
terial on semicontractive models with many insightful suggestions. Finally,
I am thankful to Mengdi Wang, who went through portions of the book
with care, and gave several helpful comments.

Dimitri P. Bertsekas
Spring 2013
 Preface xiii

Preface to the Second Edition

The second edition aims primarily to amplify the presentation of the semi-
contractive models of Chapter 3 and Chapter 4, and to supplement it with
a broad spectrum of research results that I obtained and published in jour-
nals and reports since the first edition was written. As a result, the size
of this material more than doubled, and the size of the book increased by
about 40%.
In particular, I have thoroughly rewritten Chapter 3, which deals with
semicontractive models where stationary regular policies are sufficient. I
expanded and streamlined the theoretical framework, and I provided new
analyses of a number of shortest path-type applications (deterministic,
stochastic, affine monotonic, exponential cost, and robust/minimax), as
well as several types of optimal control problems with continuous state
space (including linear-quadratic, regulation, and planning problems).
In Chapter 4, I have extended the notion of regularity to nonstation-
ary policies (Section 4.4), aiming to explore the structure of the solution set
of Bellman’s equation, and the connection of optimality with other struc-
tural properties of optimal control problems. As an application, I have
discussed in Section 4.5 the relation of optimality with classical notions
of stability and controllability in continuous-spaces deterministic optimal
control. In Section 4.6, I have similarly extended the notion of a proper
policy to continuous-spaces stochastic shortest path problems.
I have also revised Chapter 1 a little (mainly with the addition of
Section 1.2.5 on the relation between proximal algorithms and temporal
difference methods), added to Chapter 2 some analysis relating to λ-policy
iteration and randomized policy iteration algorithms (Section 2.5.3), and I
have also added several new exercises (with complete solutions) to Chapters
1-4. Additional material relating to various applications can be found in
some of my journal papers, reports, and video lectures on semicontractive
models, which are posted at my web site.
In addition to the changes in Chapters 1-4, I have also eliminated from
the second edition the analysis that deals with restricted policies (Chap-
ter 5 and Appendix C of the first edition). This analysis is motivated in
part by the complex measurability questions that arise in mathematically
rigorous theories of stochastic optimal control with Borel state and control
spaces. This material is covered in Chapter 6 of the monograph by Bert-
sekas and Shreve [BeS78], and followup research on the subject has been
limited. Thus, I decided to just post Chapter 5 and Appendix C of the first
xiv Preface

edition at the book’s web site (40 pages), and omit them from the second
edition. As a result of this choice, the entire book now requires only a
modest mathematical background, essentially a first course in analysis and
in elementary probability.
The range of applications of dynamic programming has grown enor-
mously in the last 25 years, thanks to the use of approximate simulation-
based methods for large and challenging problems. Because approximations
are often tied to special characteristics of specific models, their coverage in
this book is limited to general discussions in Chapter 1 and to error bounds
given in Chapter 2. However, much of the work on approximation methods
so far has focused on finite-state discounted, and relatively simple deter-
ministic and stochastic shortest path problems, for which there is solid and
robust analytical and algorithmic theory (part of Chapters 2 and 3 in this
monograph). As the range of applications becomes broader, I expect that
the level of mathematical understanding projected in this book will become
essential for the development of effective and reliable solution methods. In
particular, much of the new material in this edition deals with infinite-state
and/or complex shortest path type-problems, whose approximate solution
will require new methodologies that transcend the current state of the art.

Dimitri P. Bertsekas
January 2018
1

Introduction

Contents

1.1. Structure of Dynamic Programming Problems . . . . . p. 2

1.2. Abstract Dynamic Programming Models . . . . . . . . p. 5
1.2.1. Problem Formulation . . . . . . . . . . . . . . p. 5
1.2.2. Monotonicity and Contraction Properties . . . . . p. 7
1.2.3. Some Examples . . . . . . . . . . . . . . . . p. 10
1.2.4. Approximation Models - Projected and Aggregation . .
Bellman Equations . . . . . . . . . . . . . . p. 24
1.2.5. Multistep Models - Temporal Difference and . . . . .
Proximal Algorithms . . . . . . . . . . . . . . p. 26
1.3. Organization of the Book . . . . . . . . . . . . . . p. 29
1.4. Notes, Sources, and Exercises . . . . . . . . . . . . . p. 31

1
 2 Introduction Chap. 1

1.1 STRUCTURE OF DYNAMIC PROGRAMMING PROBLEMS

Dynamic programming (DP for short) is the principal method for analysis
of a large and diverse class of sequential decision problems. Examples are
deterministic and stochastic optimal control problems with a continuous
state space, Markov and semi-Markov decision problems with a discrete
state space, minimax problems, and sequential zero-sum games. While the
nature of these problems may vary widely, their underlying structures turn
out to be very similar. In all cases there is an underlying mapping that de-
pends on an associated controlled dynamic system and corresponding cost
per stage. This mapping, the DP operator, provides a compact “mathemat-
ical signature” of the problem. It defines the cost function of policies and
the optimal cost function, and it provides a convenient shorthand notation
for algorithmic description and analysis.
More importantly, the structure of the DP operator defines the math-
ematical character of the associated problem. The purpose of this book is to
provide an analysis of this structure, centering on two fundamental prop-
erties: monotonicity and (weighted sup-norm) contraction. It turns out
that the nature of the analytical and algorithmic DP theory is determined
primarily by the presence or absence of one or both of these two properties,
and the rest of the problem’s structure is largely inconsequential.

A Deterministic Optimal Control Example

To illustrate our viewpoint, let us consider a discrete-time deterministic

optimal control problem described by a system equation

xk+1 = f (xk , uk ), k = 0, 1, . . . . (1.1)

Here xk is the state of the system taking values in a set X (the state space),
and uk is the control taking values in a set U (the control space). † At stage
k, there is a cost
αk g(xk , uk )
incurred when uk is applied at state xk , where α is a scalar in (0, 1] that has
the interpretation of a discount factor when α < 1. The controls are chosen
as a function of the current state, subject to a constraint that depends on
that state. In particular, at state x the control is constrained to take values
in a given set U (x) ⊂ U . Thus we are interested in optimization over the
set of (nonstationary) policies

Π = {µ0 , µ1 , . . .} | µk ∈ M, k = 0, 1, . . . ,

† Our discussion of this section is somewhat informal, without strict adher-

ence to mathematical notation and rigor. We will introduce a rigorous mathe-
matical framework later.
Sec. 1.1 Structure of Dynamic Programming Problems 3

where M is the set of functions µ : X 7→ U defined by


M = µ | µ(x) ∈ U (x), ∀ x ∈ X .

The total cost of a policy π = {µ0 , µ1 , . . .} over an infinite number of

stages (an infinite horizon) and starting at an initial state x0 is the limit
superior of the N -step costs
N
X −1


Jπ (x0 ) = lim sup αk g xk , µk (xk ) , (1.2)
N →∞
k=0

where the state sequence {xk } is generated by the deterministic system

(1.1) under the policy π:


xk+1 = f xk , µk (xk ) , k = 0, 1, . . . .

(We use limit superior rather than limit to cover the case where the limit
does not exist.) The optimal cost function is

J * (x) = inf Jπ (x), x ∈ X.

π∈Π

For any policy π = {µ0 , µ1 , . . .}, consider the policy π1 = {µ1 , µ2 , . . .}

and write by using Eq. (1.2),



Jπ (x) = g x, µ0 (x) + αJπ1 f (x, µ0 (x)) .

We have for all x ∈ X

n

o
J * (x) = inf g x, µ0 (x) + αJπ1 f (x, µ0 (x))
π={µ0 ,π1 }∈Π
n

o
= inf g x, µ0 (x) + α inf Jπ1 f (x, µ0 (x))
µ0 ∈M π1 ∈Π
n

o
= inf g x, µ0 (x) + αJ * f (x, µ0 (x)) .
µ0 ∈M

The minimization over µ0 ∈ M can be written as minimization over all

u ∈ U (x), so we can write the preceding equation as
n
o
J * (x) = inf g(x, u) + αJ * f (x, u) , ∀ x ∈ X. (1.3)
u∈U(x)

This equation is an example of Bellman’s equation, which plays a

central role in DP analysis and algorithms. If it can be solved for J * ,
an optimal stationary policy {µ∗ , µ∗ , . . .} may typically be obtained by
minimization of the right-hand side for each x, i.e.,
n
o
µ∗ (x) ∈ arg min g(x, u) + αJ * f (x, u) , ∀ x ∈ X. (1.4)
u∈U(x)
4 Introduction Chap. 1

We now note that both Eqs. (1.3) and (1.4) can be stated in terms of
the expression


H(x, u, J) = g(x, u) + αJ f (x, u) , x ∈ X, u ∈ U (x).
Defining

(Tµ J)(x) = H x, µ(x), J , x ∈ X,
and
(T J)(x) = inf H(x, u, J) = inf (Tµ J)(x), x ∈ X,
u∈U(x) µ∈M

we see that Bellman’s equation (1.3) can be written compactly as

J * = T J *,
i.e., J * is the fixed point of T , viewed as a mapping from the set of functions
on X into itself. Moreover, it can be similarly seen that Jµ , the cost function
of the stationary policy {µ, µ, . . .}, is a fixed point of Tµ . In addition, the
optimality condition (1.4) can be stated compactly as
Tµ∗ J * = T J * .
We will see later that additional properties, as well as a variety of algorithms
for finding J * can be stated and analyzed using the mappings T and Tµ .
The mappings Tµ can also be used in the context of DP problems
with a finite number of stages (a finite horizon). In particular, for a given
¯ N ) for the state xN at
policy π = {µ0 , µ1 , . . .} and a terminal cost αN J(x
the end of N stages, consider the N -stage cost function
N −1
¯ N) +
X

Jπ,N (x0 ) = αN J(x αk g xk , µk (xk ) . (1.5)
k=0

Then it can be verified by induction that for all initial states x0 , we have
¯ 0 ).
Jπ,N (x0 ) = (Tµ0 Tµ1 · · · TµN−1 J)(x (1.6)
Here Tµ0 Tµ1 · · · TµN−1 is the composition of the mappings Tµ0 , Tµ1 , . . . TµN−1 ,
i.e., for all J,


(Tµ0 Tµ1 J)(x) = Tµ0 (Tµ1 J) (x), x ∈ X,
and more generally


(Tµ0 Tµ1 · · · TµN−1 J)(x) = Tµ0 (Tµ1 (· · · (TµN−1 J))) (x), x ∈ X,
(our notational conventions are summarized in Appendix A). Thus the
finite horizon cost functions Jπ,N of π can be defined in terms of the map-
pings Tµ [cf. Eq. (1.6)], and so can the infinite horizon cost function Jπ :
¯
Jπ (x) = lim sup(Tµ0 Tµ1 · · · TµN−1 J)(x), x ∈ X, (1.7)
N →∞

where J¯ is the zero function, J(x)

¯ = 0 for all x ∈ X.
 Sec. 1.2 Abstract Dynamic Programming Models 5

Connection with Fixed Point Methodology

The Bellman equation (1.3) and the optimality condition (1.4), stated in
terms of the mappings Tµ and T , highlight a central theme of this book,
which is that DP theory is intimately connected with the theory of abstract
mappings and their fixed points. Analogs of the Bellman equation, J * =
T J * , optimality conditions, and other results and computational methods
hold for a great variety of DP models, and can be stated compactly as
described above in terms of the corresponding mappings Tµ and T . The
gain from this abstraction is greater generality and mathematical insight,
as well as a more unified, economical, and streamlined analysis.

1.2 ABSTRACT DYNAMIC PROGRAMMING MODELS

In this section we formally introduce and illustrate with examples an ab-

stract DP model, which embodies the ideas just discussed.

1.2.1 Problem Formulation

Let X and U be two sets, which we loosely refer to as a set of “states”

and a set of “controls,” respectively. For each x ∈ X, let U (x) ⊂ U be a
nonempty subset of controls that are feasible at state x. We denote by M
the set of all functions µ : X 7→ U with µ(x) ∈ U (x), for all x ∈ X.
In analogy with DP, we refer to sequences π = {µ0 , µ1 , . . .}, with
µk ∈ M for all k, as “nonstationary policies,” and we refer to a sequence
{µ, µ, . . .}, with µ ∈ M, as a “stationary policy.” In our development,
stationary policies will play a dominant role, and with slight abuse of ter-
minology, we will also refer to any µ ∈ M as a “policy” when confusion
cannot arise.
Let R(X) be the set of real-valued functions J : X 7→ ℜ, and let
H : X × U × R(X) 7→ ℜ be a given mapping. † For each policy µ ∈ M, we
consider the mapping Tµ : R(X) 7→ R(X) defined by


(Tµ J)(x) = H x, µ(x), J , ∀ x ∈ X, J ∈ R(X),

and we also consider the mapping T defined by ‡

(T J)(x) = inf H(x, u, J), ∀ x ∈ X, J ∈ R(X).

u∈U(x)

† Our notation and mathematical conventions are outlined in Appendix A.

In particular, we denote by ℜ the set of real numbers, and by ℜn the space of
n-dimensional vectors with real components.
‡ We assume that H, Tµ J, and T J are real-valued for J ∈ R(X) in the
present chapter and in Chapter 2. In Chapters 3 and 4 we will allow H(x, u, J),
and hence also (Tµ J)(x) and (T J)(x), to take the values ∞ and −∞.
6 Introduction Chap. 1

We will generally refer to T and Tµ as the (abstract) DP mappings or DP

operators or Bellman operators (the latter name is common in the artificial
intelligence and reinforcement learning literature).
Similar to the deterministic optimal control problem of the preceding
section, the mappings Tµ and T serve to define a multistage optimization
problem and a DP-like methodology for its solution. In particular, for some
function J¯ ∈ R(X), and nonstationary policy π = {µ0 , µ1 , . . .}, we define
for each integer N ≥ 1 the functions
¯
Jπ,N (x) = (Tµ0 Tµ1 · · · TµN−1 J)(x), x ∈ X,
where Tµ0 Tµ1 · · · TµN−1 denotes the composition of the mappings Tµ0 , Tµ1 ,
. . . , TµN−1 , i.e.,


Tµ0 Tµ1 · · · TµN−1 J = Tµ0 (Tµ1 (· · · (TµN−2 (TµN−1 J))) · · ·) , J ∈ R(X).
We view Jπ,N as the “N -stage cost function” of π [cf. Eq. (1.5)]. Consider
also the function
¯
Jπ (x) = lim sup Jπ,N (x) = lim sup(Tµ0 Tµ1 · · · TµN−1 J)(x), x ∈ X,
N →∞ N →∞

which we view as the “infinite horizon cost function” of π [cf. Eq. (1.7); we
use lim sup for generality, since we are not assured that the limit exists].
We want to minimize Jπ over π, i.e., to find
J * (x) = inf Jπ (x), x ∈ X,
π

and a policy π ∗ that attains the infimum, if one exists.

The key connection with fixed point methodology is that J * “typi-
cally” (under mild assumptions) can be shown to satisfy
J * (x) = inf H(x, u, J * ), ∀ x ∈ X,
u∈U(x)

i.e., it is a fixed point of T . We refer to this as Bellman’s equation [cf. Eq.

(1.3)]. Another fact is that if an optimal policy π ∗ exists, it “typically” can
be selected to be stationary, π ∗ = {µ∗ , µ∗ , . . .}, with µ∗ ∈ M satisfying an
optimality condition, such as for example
(Tµ∗ J * )(x) = (T J * )(x), x ∈ X,
[cf. Eq. (1.4)]. Several other results of an analytical or algorithmic nature
also hold under appropriate conditions, which will be discussed in detail
later.
However, Bellman’s equation and other related results may not hold
without Tµ and T having some special structural properties. Prominent
among these are a monotonicity assumption that typically holds in DP
problems, and a contraction assumption that holds for some important
classes of problems. We describe these assumptions next.
Sec. 1.2 Abstract Dynamic Programming Models 7

1.2.2 Monotonicity and Contraction Properties

Let us now formalize the monotonicity and contraction assumptions. We

will require that both of these assumptions hold for most of the next chap-
ter, and we will gradually relax the contraction assumption in Chapters 3
and 4. Recall also our assumption that Tµ and T map R(X) (the space
of real-valued functions over X) into R(X). In Chapters 3 and 4 we will
relax this assumption as well.

Assumption 1.2.1: (Monotonicity) If J, J ′ ∈ R(X) and J ≤ J ′ ,

then
H(x, u, J) ≤ H(x, u, J ′ ), ∀ x ∈ X, u ∈ U (x).

Note that by taking infimum over u ∈ U (x), we have

J(x) ≤ J ′ (x), ∀ x ∈ X ⇒ inf H(x, u, J) ≤ inf H(x, u, J ′ ), ∀ x ∈ X,
u∈U (x) u∈U (x)

or equivalently, †
J ≤ J′ ⇒ T J ≤ T J ′.
Another way to arrive at this relation, is to note that the monotonicity
assumption is equivalent to
J ≤ J′ ⇒ Tµ J ≤ Tµ J ′ , ∀ µ ∈ M,
and to use the simple but important fact
inf H(x, u, J) = inf (Tµ J)(x), ∀ x ∈ X, J ∈ R(X),
u∈U(x) µ∈M

i.e., for a fixed x ∈ X, infimum over u is equivalent

 to infimum over µ,
which holds because of the definition M = µ | µ(x) ∈ U (x), ∀ x ∈ X ,
so that M can be viewed as the Cartesian product Πx∈X U (x). We will be
writing this relation as T J = inf µ∈M Tµ J.
For the contraction assumption, we introduce a function v : X 7→ ℜ
with
v(x) > 0, ∀ x ∈ X.
Let us denote by B(X) the space of real-valued functions J on X such
that J(x)/v(x) is bounded as x ranges over X, and consider the weighted
sup-norm
J(x)
kJk = sup
x∈X v(x)

† Unless otherwise stated, in this book, inequalities involving functions, min-

ima and infima of a collection of functions, and limits of function sequences are
meant to be pointwise; see Appendix A for our notational conventions.
8 Introduction Chap. 1

= 0 Tµ J = 0 Tµ J

=0 J TJ =0 ) Jµ J TJ

Figure 1.2.1. Illustration of the monotonicity and the contraction assumptions in

one dimension. The mapping Tµ on the left is monotone but is not a contraction.
The mapping Tµ on the right is both monotone and a contraction. It has a unique
fixed point at Jµ .

on B(X). The properties of B(X) and some of the associated fixed point
theory are discussed in Appendix B. In particular, as shown there, B(X)
is a complete normed space, so any mapping from B(X) to B(X) that is a
contraction or an m-stage contraction for some integer m > 1, with respect
to k · k, has a unique fixed point (cf. Props. B.1 and B.2).

Assumption 1.2.2: (Contraction) For all J ∈ B(X) and µ ∈ M,

the functions Tµ J and T J belong to B(X). Furthermore, for some
α ∈ (0, 1), we have

kTµ J − Tµ J ′ k ≤ αkJ − J ′ k, ∀ J, J ′ ∈ B(X), µ ∈ M. (1.8)

Figure 1.2.1 illustrates the monotonicity and the contraction assump-

tions. It can be shown that the contraction condition (1.8) implies that

kT J − T J ′ k ≤ αkJ − J ′ k, ∀ J, J ′ ∈ B(X), (1.9)

so that T is also a contraction with modulus α. To see this we use Eq.

(1.8) to write

(Tµ J)(x) ≤ (Tµ J ′ )(x) + αkJ − J ′ k v(x), ∀ x ∈ X,

from which, by taking infimum of both sides over µ ∈ M, we have

(T J)(x) − (T J ′ )(x)
≤ αkJ − J ′ k, ∀ x ∈ X.
v(x)
Sec. 1.2 Abstract Dynamic Programming Models 9

Reversing the roles of J and J ′ , we also have

(T J ′ )(x) − (T J)(x)
≤ αkJ − J ′ k, ∀ x ∈ X,
v(x)
and combining the preceding two relations, and taking the supremum of
the left side over x ∈ X, we obtain Eq. (1.9).
Nearly all mappings related to DP satisfy the monotonicity assump-
tion, and many important ones satisfy the weighted sup-norm contraction
assumption as well. When both assumptions hold, the most powerful an-
alytical and computational results can be obtained, as we will show in
Chapter 2. These are:
(a) Bellman’s equation has a unique solution, i.e., T and Tµ have unique
fixed points, which are the optimal cost function J * and the cost
functions Jµ of the stationary policies {µ, µ, . . .}, respectively [cf. Eq.
(1.3)].
(b) A stationary policy {µ∗ , µ∗ , . . .} is optimal if and only if

Tµ∗ J * = T J * ,

[cf. Eq. (1.4)].

(c) J * and Jµ can be computed by the value iteration method,

J * = lim T k J, Jµ = lim Tµk J,

k→∞ k→∞

starting with any J ∈ B(X).

(d) J * can be computed by the policy iteration method, whereby we gen-
erate a sequence of stationary policies via

Tµk+1 Jµk = T Jµk ,

starting from some initial policy µ0 [here Jµk is obtained as the fixed
point of Tµk by several possible methods, including value iteration as
in (c) above].
These are the most favorable types of results one can hope for in
the DP context, and they are supplemented by a host of other results,
involving approximate and/or asynchronous implementations of the value
and policy iteration methods, and other related methods that combine
features of both. As the contraction property is relaxed and is replaced
by various weaker assumptions, some of the preceding results may hold
in weaker form. For example J * turns out to be a solution of Bellman’s
equation in most of the models to be discussed, but it may not be the
unique solution. The interplay between the monotonicity and contraction-
like properties, and the associated results of the form (a)-(d) described
above is a recurring analytical theme in this book.
10 Introduction Chap. 1

1.2.3 Some Examples

In what follows in this section, we describe a few special cases, which indi-
cate the connections of appropriate forms of the mapping H with the most
popular total cost DP models. In all these models the monotonicity As-
sumption 1.2.1 (or some closely related version) holds, but the contraction
Assumption 1.2.2 may not hold, as we will indicate later. Our descriptions
are by necessity brief, and the reader is referred to the relevant textbook
literature for more detailed discussion.

Example 1.2.1 (Stochastic Optimal Control - Markovian

Decision Problems)

Consider the stationary discrete-time dynamic system

xk+1 = f (xk , uk , wk ), k = 0, 1, . . . , (1.10)

where for all k, the state xk is an element of a space X, the control uk is

an element of a space U , and wk is a random “disturbance,” an element of a
space W . We consider problems with infinite state and control spaces, as well
as problems with discrete (finite or countable) state space (in which case the
underlying system is a Markov chain). However, for technical reasons that
relate to measure-theoretic issues, we assume that W is a countable set.
The control uk is constrained to take values in a given nonempty subset
U (xk ) of U , which depends on the current state xk [uk ∈ U (xk ), for all
xk ∈ X]. The random disturbances wk , k = 0, 1, . . ., are characterized by
probability distributions P (· | xk , uk ) that are identical for all k, where P (wk |
xk , uk ) is the probability of occurrence of wk , when the current state and
control are xk and uk , respectively. Thus the probability of wk may depend
explicitly on xk and uk , but not on values of prior disturbances wk−1 , . . . , w0 .
Given an initial state x0 , we want to find a policy π = {µ0 , µ1 , . . .},
where µk : X 7→ U , µk (xk ) ∈ U (xk ), for all xk ∈ X, k = 0, 1, . . ., that
minimizes the cost function
(N−1 )
X
αk g xk , µk (xk ), wk


Jπ (x0 ) = lim sup E , (1.11)
N→∞ wk
k=0,1,... k=0

subject to the system equation constraint

xk+1 = f xk , µk (xk ), wk , k = 0, 1, . . . .

This is a classical problem, which is discussed extensively in various sources,

including the author’s text [Ber12a]. It is usually referred to as the stochastic
optimal control problem or the Markovian Decision Problem (MDP for short).
Note that the expected value of the N -stage cost of π,
(N−1 )
X k


E α g xk , µk (xk ), wk ,
wk
k=0,1,... k=0
Sec. 1.2 Abstract Dynamic Programming Models 11

is defined as a (possibly countably infinite) sum, since the disturbances wk ,

k = 0, 1, . . ., take values in a countable set. Indeed, the reader may verify
that all the subsequent mathematical expressions that involve an expected
value can be written as summations over a finite or a countable set, so they
make sense without resort to measure-theoretic integration concepts. †
In what follows we will often impose appropriate assumptions on the
cost per stage g and the scalar α, which guarantee that the infinite horizon
cost Jπ (x0 ) is defined as a limit (rather than as a lim sup):
(N−1 )
X k


Jπ (x0 ) = lim E α g xk , µk (xk ), wk .
N→∞ wk
k=0,1,... k=0

In particular, it can be shown that the limit exists if α < 1 and the expected
value of |g| is uniformly bounded, i.e., for some B > 0,

E g(x, u, w) ≤ B, ∀ x ∈ X, u ∈ U (x). (1.12)

In this case, we obtain the classical discounted infinite horizon DP prob-

lem, which generally has the most favorable structure of all infinite horizon
stochastic DP models (see [Ber12a], Chapters 1 and 2).
To make the connection with abstract DP, let us define


H(x, u, J) = E g(x, u, w) + αJ f (x, u, w) ,

so that



(Tµ J)(x) = E g x, µ(x), w + αJ f (x, µ(x), w) ,

and 

(T J)(x) = inf E g(x, u, w) + αJ f (x, u, w) .
u∈U (x)

Similar to the deterministic optimal control problem of Section 1.1, the N -

stage cost of π, can be expressed in terms of Tµ :
(N−1 )
X
¯ 0) = αk g xk , µk (xk ), wk


(Tµ0 · · · TµN−1 J)(x E ,
wk
k=0,1,... k=0

† As noted in Appendix A, the formula for the expected value of a random

variable w defined over a space Ω is

E{w} = E{w+ } + E{w− },

where w+ and w− are the positive and negative parts of w,

w+ (ω) = max 0, w(ω) , w− (ω) = min 0, w(ω) ,

 
∀ ω ∈ Ω.

In this way, taking also into account the rule ∞−∞ = ∞ (see Appendix A), E{w}
is well-defined as an extended real number if Ω is finite or countably infinite.
12 Introduction Chap. 1

where J¯ is the zero function, J(x)

¯ = 0 for all x ∈ X. The same is true for
the infinite-stage cost [cf. Eq. (1.11)]:

Jπ (x0 ) = lim sup (Tµ0 · · · TµN−1 J¯)(x0 ).

N→∞

It can be seen that the mappings Tµ and T are monotone, and it is

well-known that if α < 1 and the boundedness condition (1.12) holds, they
are contractive as well (under the unweighted sup-norm); see e.g., [Ber12a],
Chapter 1. In this case, the model has the powerful analytical and algorith-
mic properties (a)-(d) mentioned at the end of the preceding subsection. In
particular, the optimal cost function J ∗ [i.e., J ∗ (x) = inf π Jπ (x) for all x ∈ X]
can be shown to be the unique solution of the fixed point equation J ∗ = T J ∗ ,
also known as Bellman’s equation, which has the form

J ∗ (x) = E g(x, u, w) + αJ ∗ f (x, u, w)



inf , x ∈ X,
u∈U (x)

and parallels the one given for deterministic optimal control problems [cf. Eq.
(1.3)].
These properties can be expressed and analyzed in an abstract setting
by using just the mappings Tµ and T , both when Tµ and T are contractive
(see Chapter 2), and when they are only monotone and not contractive while
either g ≥ 0 or g ≤ 0 (see Chapter 4). Moreover, under some conditions, it is
possible to analyze these properties in cases where Tµ is contractive for some
but not all µ (see Chapter 3, and Section 4.4).

Example 1.2.2 (Finite-State Discounted Markovian Decision

Problems)

In the special case of the preceding example where the number of states is
finite, the system equation (1.10) may be defined in terms of the transition
probabilities


pxy (u) = Prob y = f (x, u, w) | x , x, y ∈ X, u ∈ U (x),

so H takes the form

X

H(x, u, J) = pxy (u) g(x, u, y) + αJ(y) .
y∈X

When α < 1 and the boundedness condition

g(x, u, y) ≤ B, ∀ x, y ∈ X, u ∈ U (x),

[cf. Eq. (1.12)] holds (or more simply, when U is a finite set), the mappings Tµ
and T are contraction mappings with respect to the standard (unweighted)
sup-norm. This is a classical model, referred to as discounted finite-state
MDP , which has a favorable theory and has found extensive applications (cf.
[Ber12a], Chapters 1 and 2). The model is additionally important, because it
is often used for computational solution of continuous state space problems
via discretization.
Sec. 1.2 Abstract Dynamic Programming Models 13

Example 1.2.3 (Discounted Semi-Markov Problems)

With x, y, and u as in Example 1.2.2, consider a mapping of the form

X
H(x, u, J) = G(x, u) + mxy (u)J(y),
y∈X

where G is some function representing expected cost per stage, and mxy (u)
are nonnegative scalars with
X
mxy (u) < 1, ∀ x ∈ X, u ∈ U (x).
y∈X

The equation J ∗ = T J ∗ is Bellman’s equation for a finite-state continuous-

time semi-Markov decision problem, after it is converted into an equivalent
discrete-time problem (cf. [Ber12a], Section 1.4). Again, the mappings Tµ and
T are monotone and can be shown to be contraction mappings with respect
to the unweighted sup-norm.

Example 1.2.4 (Discounted Zero-Sum Dynamic Games)

Let us consider a zero-sum game analog of the finite-state MDP Example

1.2.2. Here there are two players that choose actions at each stage: the
first (called the minimizer ) may choose a move i out of n moves and the
second (called the maximizer ) may choose a move j out of m moves. Then
the minimizer gives a specified amount aij to the maximizer, called a payoff .
The minimizer wishes to minimize aij , and the maximizer wishes to maximize
aij .
The players use mixed strategies, whereby the minimizer selects a prob-
ability distribution u = (u1 , . . . , un ) over his n possible moves and the max-
imizer selects a probability distribution v = (v1 , . . . , vm ) over his m possible
moves. Thus the probability of selecting i and j is ui vj , and the expected
a u v or u′ Av, where A is the n × m matrix
P
payoff for this stage is i,j ij i j
with components aij .
In a single-stage version of the game, the minimizer must minimize
maxv∈V u′ Av and the maximizer must maximize minu∈U u′ Av, where U and
V are the sets of probability distributions over {1, . . . , n} and {1, . . . , m},
respectively. A fundamental result (which will not be proved here) is that
these two values are equal:

min max u′ Av = max min u′ Av. (1.13)

u∈U v∈V v∈V u∈U

Let us consider the situation where a separate game of the type just
described is played at each stage. The game played at a given stage is repre-
sented by a “state” x that takes values in a finite set X. The state evolves
according to transition probabilities qxy (i, j) where i and j are the moves
selected by the minimizer and the maximizer, respectively (here y represents
14 Introduction Chap. 1

the next game to be played after moves i and j are chosen at the game rep-
resented by x). When the state is x, under u ∈ U and v ∈ V , the one-stage
expected payoff is u′ A(x)v, where A(x) is the n × m payoff matrix, and the
state transition probabilities are
n m
X X
pxy (u, v) = ui vj qxy (i, j) = u′ Qxy v,
i=1 j=1

where Qxy is the n × m matrix that has components qxy (i, j). Payoffs are
discounted by α ∈ (0, 1), and the objectives of the minimizer and maximizer,
roughly speaking, are to minimize and to maximize the total discounted ex-
pected payoff. This requires selections of u and v to strike a balance between
obtaining favorable current stage payoffs and playing favorable games in fu-
ture stages.
We now introduce an abstract DP framework related to the sequential
move selection process just described. We consider the mapping G given by
X
G(x, u, v, J) = u′ A(x)v + α pxy (u, v)J(y)
y∈X

X
! (1.14)
′
=u A(x) + α Qxy J(y) v,
y∈X

where α ∈ (0, 1) is discount factor, and the mapping H given by

H(x, u, J) = max G(x, u, v, J).

v∈V

The corresponding mappings Tµ and T are

(Tµ J)(x) = max G x, µ(x), v, J , x ∈ X,
v∈V

and
(T J)(x) = min max G(x, u, v, J).
u∈U v∈V

It can be shown that Tµ and T are monotone and (unweighted) sup-norm

contractions. Moreover, the unique fixed point J ∗ of T satisfies

J ∗ (x) = min max G(x, u, v, J ∗ ), ∀ x ∈ X,

u∈U v∈V

(see [Ber12a], Section 1.6.2).

We now note that since
X
A(x) + α Qxy J(y)
y∈X

[cf. Eq. (1.14)] is a matrix that is independent of u and v, we may view J ∗ (x)
as the value of a static game (which depends on the state x). In particular,
from the fundamental minimax equality (1.13), we have

min max G(x, u, v, J ∗ ) = max min G(x, u, v, J ∗ ), ∀ x ∈ X.

u∈U v∈V v∈V u∈U
Sec. 1.2 Abstract Dynamic Programming Models 15

This implies that J ∗ is also the unique fixed point of the mapping

(T J)(x) = max H(x, v, J),

v∈V

where
H(x, v, J) = min G(x, u, v, J),
u∈U

i.e., J ∗ is the fixed point regardless of the order in which minimizer and
maximizer select mixed strategies at each stage.
In the preceding development, we have introduced J ∗ as the unique
fixed point of the mappings T and T . However, J ∗ also has an interpretation
in game theoretic terms. In particular, it can be shown that J ∗ (x) is the value
of a dynamic game, whereby at state x the two opponents choose multistage
(possibly nonstationary) policies that consist of functions of the current state,
and continue to select moves using these policies over an infinite horizon. For
further discussion of this interpretation, we refer to [Ber12a] and to books on
dynamic games such as [FiV96]; see also [PaB99] and [Yu11] for an analysis
of the undiscounted case (α = 1) where there is a termination state, as in the
stochastic shortest path problems of the subsequent Example 1.2.6.

Example 1.2.5 (Minimax Problems)

Consider a minimax version of Example 1.2.1, where w is not random but is

rather chosen by an antagonistic player from a set W (x, u). Let
h
i
H(x, u, J) = sup g(x, u, w) + αJ f (x, u, w) .
w∈W (x,u)

Then the equation J ∗ = T J ∗ is Bellman’s equation for an infinite horizon

minimax DP problem. A special case of this mapping arises in zero-sum
dynamic games (cf. Example 1.2.4).

Example 1.2.6 (Stochastic Shortest Path Problems)

The stochastic shortest path (SSP for short) problem is the special case of
the stochastic optimal control Example 1.2.1 where:
(a) There is no discounting (α = 1).
(b) The state space is X = {t, 1, . . . , n} and we are given transition proba-
bilities, denoted by

pxy (u) = P (xk+1 = y | xk = x, uk = u), x, y ∈ X, u ∈ U (x).

(c) The control constraint set U (x) is finite for all x ∈ X.

(d) A cost g(x, u) is incurred when control u ∈ U (x) is selected at state x.
Discovering Diverse Content Through
Random Scribd Documents
content to be an administrator or president elected by free suffrage,
and above all an ever-ready captain in time of war; on taking his
office he swore solemnly to respect the laws, customs, and privileges
of the republic; if he committed a perjury, the assembly convened in
the public square at the clang of an ancient bell, and the prince,
having been declared a traitor, was stripped, expelled, and cast into
the mud, according to the forcible popular expression. This
industrious republic reached the acme of its prosperity in the
thirteenth and fourteenth centuries, after which the rising principality
of Moscow, now sure of its future, came and took down the bells of
Novgorod the Great, and so silenced their voices of bronze and the
voice of Russian liberties, though not without a bloody battle, as
witnesseth the whirlpool—which is still pointed out to the curious
traveller—under the bridge of the ancient republican city, whose
inhabitants were drowned there by Ivan the Terrible. Upon their
dead bodies he founded the unity of the empire. Nor are the free
towns the only tradition of autonomy which disturbed the growing
autocratic power. The Cossacks for a long time formed an
independent and warlike aristocracy, proud and indomitable; and to
subdue and incorporate these bellicose tribes with the rest of the
nation it was necessary to employ both skill and force.
We may say without vanity that although the Spaniards exalted
monarchical loyalty into a cult, they never depreciated human
dignity. Amongst us the king is he who makes right (face derecho),
and if he makes it not, we consider him a tyrant, a usurper of the
royal prerogative; in acknowledging him lord of life and property, we
protest (by the mouth of Calderon's honest rustic) against the idea
that he can arrogate to himself also the dominion over conscience
and soul; and the smallest subject in Spain would not endure at the
king's hand the blows administered by Peter the Great for the
correction of his nobles, themselves descendants of Rurik. In Russia,
where the inequalities and extremes of climate seem to have been
communicated to its institutions, there was nothing between the
independent republics and the autocracy. In Spain, the slightest
territorial disaffection, the fruit of partial conquests or insignificant
victories, was an excuse for some upstart princeling, our instinctive
tendencies being always monarchical and anything like absolute
authority and Cæsarism, so odious that we never allowed it even in
our most excellent kings; a dream of imperial power would almost
have cost them the throne. In Russia, absolutism is in the air,—one
sole master, one lord omnipotent, the image of God himself.
Read the Muscovite code. The Czar is named therein the autocrat
whose power is unlimited. See the catechism which is taught in the
schools of Poland; it says that the subject owes to the Czar, not love
or loyalty, but adoration. Hear the Russian hymn; amid its harmonies
the same idea resounds. In all the common forms of salutation to
the Czar we shall find something that excites in us a feeling of
rebellion, something that represents us as unworthy to stand before
him as one mortal before another. Paul I. said to a distinguished
foreigner, "You must know that in Russia there is no person more
important than the person to whom I speak and while I speak." A
Czar who directs by means of ukases not only the dress but even the
words of the language which his subjects must use, and changes the
track of a railroad by a stroke of his pen, frightens one even more
than when he signs a sentence of proscription; for he reaches the
high-water mark of authority when he interferes in these simple and
unimportant matters, and demonstrates what one may call the
micrography of despotism. If anything can excuse or even commend
to our eyes this obedience carried to an absurdity, it is its paternal
character. There are no offences between fathers and sons, and the
Czar never can insult a subject. The serf calls him thou and Father,
and on seeing him pass he takes off his cap though the snow falls,
crossing his hands over his breast with religious veneration. For him
the Czar possesses every virtue, and is moved only by the highest
purposes; he thinks him impeccable, sacred, almost immortal. If we
abide by the judgment of those who see a symbol of the Russian
character in the call of Rurik and the voluntary placing of the power
in his hands, the autocracy will not seem a secular abuse or a violent
tyranny, but rather an organic product of a soil and a race; and it will
inspire the respect drawn forth by any spontaneous and genuine
production.
There exists in Russia a small school of thinkers on public affairs,
important by reason of the weight they have had and still have upon
public opinion. They are called Sclavophiles,—people enamoured of
their ancient land, who affirm that the essence of Russian nationality
is to be found in the customs and institutions of the laboring classes
who are not contaminated by the artificial civilization imported from
the corrupt West; who make a point of appearing on occasions in
the national dress,—the red silk blouse and velvet jacket, the long
beard and the clumsy boots. According to them, the only
independent forces on which Russia can count are the people and
the Czar,—the immense herd of peasants, and, at the top, the
autocrat. And in fact the Russian empire, in spite of official
hierarchies, is a rural state in which the sentiment of democratic
equality predominates so entirely that the people, not content with
having but yesterday taken the Czar's part against the rich and
mighty Boyars, sustains him to-day against the revolution, loves him,
and cannot conceive of intermediaries between him and his subjects,
between lord and vassal, or, to put it still more truly, between father
and son. And having once reduced the nobles, with the consent of
the people, to the condition of inoffensive hangers-on of the court,
many thinkers believe that the Czar need only lean upon the rude
hand of the peasant to quell whatever political disaffection may
arise. So illimitable is the imperial power, that it becomes impotent
against itself if it would reduce itself by relegating any of its
influence to a class, such as, for instance, the aristocracy. If
turbulent magnates or sullen conspirators manage to get rid of the
person of the Czar, the principle still remains inviolate.

VI.
 The Agrarian Communes.
At the right hand of the imperial power stands the second Russian
national institution, the municipal commune known as the mir, which
is arresting the attention of European statesmen and sociologists,
since they have learned of its existence (thanks to the work of Baron
Haxsthausen on the internal life of Russia). Who is not astonished at
finding realized in the land of the despots a large number of the
communist theories which are the terror of the middle classes in
liberal countries, and various problems, of the kind we call
formidable, there practically solved? And why should not a nation
often called barbarous swell with pride at finding itself, suddenly and
without noise or effort, safely beyond what in others threatens the
extremity of social revolution? Therefore it happens that since the
discovery of the mir, the Russians have one argument more, and not
a weak one, against the corrupt civilization of the Occident. The
European nations, they say, are running wildly toward anarchy, and
in some, as England, the concentration of property in a few hands
creates a proletariat a thousand times more unhappy than the
Russian serf ever was, a hungry horde hostile to the State and to the
wealthy classes. Russia evades this danger by means of the mir. In
the Russian village the land belongs to the municipality, amongst
whose members it is distributed periodically; each able-bodied
individual receives what he needs, and is spared hunger and
disgrace.
Foreigners have not been slow to examine into the advantages of
such an arrangement. Mackenzie Wallace has pronounced it to be
truly constitutional, as the phrase is understood in his country; not
meaning a sterile and delusive law, written upon much paper and
enwrapped in formulas, but a traditional concept which came forth
at the bidding of real and positive necessities. What an eloquent
lesson for those who think they have improved upon the plan of the
ages! History, scouting our thirst for progress, offers us again in the
mir the picture of the serpent biting his own tail. This institution, so
much lauded by the astonished traveller and the meditative
philosopher, is really a sociological fossil, remains of prehistoric
times, preserved in Russia by reason of the suspension or slow
development of the history of the race. Students of law have told me
that in the ancient forms of Castilian realty, those of Santander, for
example, there have been discovered traces of conditions analogous
to the Russian mir. And when I have seen the peasants of my own
province assembled in the church-porch after Mass, I have imagined
I could see the remains of this Saturnian and patriarchal type of
communist partition. Common possession of the land is a primitive
idea as remote as the prehistoric ages; it belongs to the
paleontology of social science, and in those countries where
civilization early flourished, gave way before individual interest and
the modern idea of property. "Happy age and blessed times were
those," exclaimed Don Quixote, looking at a handful of acorns,
"which the ancients called golden, and not because gold which in
our iron age has such a value set on it, not because gold could be
got without any trouble, but because those who lived in it were
ignorant of those two words, mine and thine! In that blessed age
everything was in common; nobody needed to take any more
trouble for his necessities than to stretch forth his hand and take
from the great oak-trees the sweet and savory fruit so liberally
offered!" Gone long ago for us is the time deplored by the ingenious
knight, but it has reappeared there in the North, where, according to
our information, it is still recent; for it is thought that the mir was
established about the sixteenth century.
The character of the mir is entirely democratic; the oldest peasant
represents the executive power in the municipal assembly, but the
authority resides in the assembly itself, which consists of all the
heads of families, and convenes Sundays in the open air, in the
public square or the church-porch. The assembly wields a sacred
power which no one disputes. Next to the Czar the Russian peasant
loves his mir, among whose members the land is in common, as also
the lake, the mills, the canals, the flocks, the granary, the forest. It
is all re-divided from time to time, in order to avoid exclusive
appropriation. Half the cultivable land in the empire is subject to this
system, and no capitalist or land-owner can disturb it by acquiring
even an inch of municipal territory; the laborer is born invested with
the right of possession as certainly as we are all entitled to a grave.
In spite of a feeling of distrust and antipathy against communism,
and of my own ignorance in these matters which precludes my
judgment of them, I must confess to a certain agreement with the
ardent apologists of the Russian agrarian municipality. Tikomirov
says that in Russia individual and collective property-rights still
quarrel, but that the latter has the upper hand; this seems strange,
since the modern tendency is decidedly toward individualism, and it
is hard to conceive of a return to patriarchal forms; but there is no
reason to doubt the vitality of the mir and its generation and growth
in the heart of the fatherland, and this is certainly worthy of note,
especially in a country like Russia, so much given to the imitation of
foreign models. Mere existence and permanence is no raison d'être
for any institution, for many exist which are pernicious and
abominable; but when an institution is found to be in harmony with
the spirit of the people, it must have a true merit and value. It is
said that the tendency to aggregate, either in agrarian municipalities
or in trades guilds and corporations, is born in the blood and bred in
the bone of the Sclavs, and that they carry out these associations
wherever they go, by instinct, as the bee makes its cells always the
same; and it is certainly true that as an ethnic force the communistic
principle claims a right to develop itself in Russia. It is certain that
the mir fosters in the poor Russian village habits of autonomous
administration and municipal liberty, and that in the shadow of this
humble and primitive institution men have found a common home
within the fatherland, no matter how scattered over its vast plains.
"The heavens are very high, and the Czar is far off," says the
Russian peasant sadly, when he is the victim of any injustice; his
only refuge is the mir, which is always close at hand. The mir acts
also as a counterbalance to a centralized administration, which is an
inevitable consequence of the conformation of Russian territory; and
it creates an advantageous solidarity among the farmers, who are
equal owners of the same heritages and subject to the same taxes.
Since 1861 the rural governments, released from all seignorial
obligations, elect their officers from among themselves, and the
smaller municipal groups, still preserving each its own autonomy,
meet together in one larger municipal body called volost, which
corresponds to the better-known term canton. No institution could
be more democratic: here the laboring man discusses his affairs en
famille, without interference from other social classes; the mir boasts
of it, as also of the fact that it has never in its corporate existence
known head or chief, even when its members were all serfs. In fine,
the mir holds its sessions without any presiding officer; rooted in the
communist and equal-rights idea, it acknowledges no law of
superiority; it votes by unanimous acclamation; the minority yields
always to the general opinion, to oppose which would be thought
base obstinacy. "Only God shall judge the mir" says the proverb; the
word mir, say the etymological students and admirers of the
institution, means, "world," "universe," "complete and perfect
microcosm," which is sufficient unto itself and is governed by its own
powers.
To what does the mir owe its vitality? To the fact that it did not
originate in the mind of the Utopian or the ideologist, but was
produced naturally by derivation from the family, from which type
the whole Russian state organization springs. It should be
understood, however, that the peasant family in Russia differs from
our conception of the institution, recalling as it does, like all purely
Russian institutions, the most ancient or prehistoric forms. The
family, or to express it in the language of the best writers on the
subject, the great Russian family, is an association of members
submitted to the absolute authority of the eldest, generally the
grandfather,—a fact personally interesting to me because of the
surprising resemblance it discloses between Russia and the province
of Gallicia, where I perceive traces of this family power in the
petrucios, or elders. In this association everything is in common, and
each individual works for all the others. To the head of the house is
given a name which may be translated as administrator, major-
domo, or director of works, but conveys no idea of relationship. The
laws of inheritance and succession are understood in the same spirit,
and very differently from our custom. When a house or an estate is
to be settled, the degree of relationship among the heirs is not
considered; the whole property is divided equally between the male
adults, including natural or adopted sons if they have served in the
family the same as legitimate sons, while the married daughter is
considered as belonging to the family of her husband, and she and
the son who has separated himself from the parent house are
excluded from the succession, or rather from the final liquidation or
settlement between the associates. Although there is a law of
inheritance written in the Russian Code, it is a dead letter to a
people opposed to the idea of individual property.
Intimately connected with this communist manner of interpreting the
rights of inheritance and succession are certain facts in Russian
history. For a long time the sovereign authority was divided among
the sons of the ruler; and as the Russian nobility rebelled against the
establishment of differences founded upon priority in birth, entail
and primogeniture took root with difficulty, in spite of the efforts
made by the emperors to import Occidental forms of law. Their idea
of succession is so characteristic that, like the Goths, they
sometimes prefer the collateral to the immediate branch, and the
brother instead of the son will mount the steps of the throne. It is
important to note these radical differences, because a race which
follows an original method in the matter of its laws has a great
advantage in setting out upon genuine literary creations.
But while the family, understood as a group or an association, offers
many advantages from the agrarian point of view, its disadvantages
are serious and considerable because it annuls individual liberty. It
facilitates agricultural labors, it puts a certain portion of land at the
service of each adult member, as well as tools, implements, fuel, and
cattle; helps each to a maintenance; precludes hunger; avoids legal
exactions (for the associated family cannot be taxed, just as the mir
cannot be deprived of its lands); but on the other hand it puts the
individual, or rather the true family, the human pair, under an
intolerable domestic tyranny. According to traditional usage, the
authority of the head of the family was omnipotent: he ordered his
house, as says an old proverb, like a Khan of the Crimea; his gray
hairs were sacred, and he wielded the power of a tribal chieftain
rather than of a head of a house. In our part of the world marriage
emancipates; in Russia, it was the first link in a galling chain. The
oppression lay heaviest upon the woman: popular songs recount the
sorrows of the daughters-in-law subjected to the maltreatment of
mothers-in-law and sisters-in-law, or the victims of the vicious
appetites of the chief, who in a literally Biblical spirit thought himself
lord of all that dwelt beneath his roof. Truly those institutions which
sometimes elicit our admiration for their patriarchal simplicity hide
untold iniquities, and develop a tendency to the abuse of power
which seems inherent in the human species.
At first sight nothing could be more attractive than the great Russian
family, nothing more useful than the rural communes; and
nowadays, when we are applying the laws and technicism of
physiology to the study of society, this primordial association would
seem the cell from which the true organism of the State may be
born; the family is a sort of lesser municipality, the municipality is a
larger family, and the whole Russian people is an immense
agglomeration, a great ant-hill whose head is the emperor. In the
popular songs we see the Oriental idea of the nation expressed as
the family, when the peasant calls the Czar father. But this primitive
machinery can never prevail against the notion of individualism
entertained among civilized peoples. Our way of understanding
property, which the admirers of the Russian commune consider
fundamentally vicious, is the only way compatible with the
independence and dignity of work and the development of industries
and arts. The Russian mir may prevent the growth of the proletariat,
but it is by putting mankind in bonds. It may be said that agrarian
communism only differs from servitude in that the latter provides
one master and the former many; and that though the laboring man
theoretically considers himself a member of a co-operative
agricultural society, he is in reality a slave, subject to collective
responsibilities and obligations, by virtue of which he is tied to the
soil the same as the vassals of our feudal epochs. Perhaps the new
social conditions which are the fruit of the emancipation of the serfs,
which struck at and violated the great associated family, will at last
undermine the mir, unless the mir learns some way to adapt itself to
any political mutations. What is most important to the study of the
historical development and the social ideas as shown in modern
Russian literature, is to understand how by means of the great
family and the agrarian municipality, communism and socialism run
in the veins of the people of Russia, so that Leroy-Beaulieu could say
with good reason, that if they are to be preserved from the
pernicious effects of the Occidental proletariat it must be by
inoculation, as vaccination exempts from small-pox.
The socialist leaven may be fairly said to lie in the most important
class in the Russian State,—important not alone by reason of
numerical superiority, but because it is the depositary of the liveliest
national energies and the custodian of the future: I mean the
peasants. There are some who think that this mitjik, this little man
or black man, tiller of still blacker soil, holds the future destinies of
Europe in his hands; and that when this great new Horde becomes
conscious some day of its strength and homogeneity, it will rise, and
in its concentrated might fall upon some portion of the globe, and
there will be no defence or resistance possible. In the rest of Europe
it is the cities, the urban element, which regulates the march of
political events. Certainly Spain is not ignorant of this fact, since she
has a vivid remembrance of civil wars in which the rustic element,
representing tradition, was vanquished. In Russia, the cities have no
proportionate influence, and that which demands the special
attention of the governor or the revolutionist is the existence, needs,
and thoughts of the innumerable peasant communities, who are the
foundation and material of an empire justly termed rural. From this
is derived a sort of cult, an apotheosis which is among the most
curious to be found in Russian modern literature. Of the peasant,
wrapped in badly cured sheepskins, and smelling like a beast; the
humble and submissive peasant, yesterday laden with the chains of
servitude; the dirty, cabbage-eating peasant, drunk with wodka, who
beats his wife and trembles with fright at ghosts, at the Devil, and at
thunder,—of this peasant, the charity of his friends and the poetic
imagination of Russian writers has made a demi-god, an ideal. So
great is the power of genius, that without detriment to the claims of
truth, picturing him with accurate and even brutal realism (which we
shall find native to the Russian novel), Russian authors have distilled
from this peasant a poetic essence which we inhale involuntarily
until we, aristocratic by instinct, disdainful of the rustic, given to
ridicule the garlic-smelling herd, yield to its power. And not content
with seeing in this peasant a brother, a neighbor, whom, according
to the word of Christ, we ought to love and succor, Russian literature
discovers in him a certain indefinable sublimity, a mysterious
illumination which other social classes have not. Not merely because
of the introduction of the picturesque element in the description of
popular customs has it been said that Russian contemporary
literature smells of the peasant, but far rather because it raises the
peasant to the heights of human moral grandeur, marks in him every
virtue, and presupposes him possessed of powers which he never
puts forth. From Turguenief, fine poet as he is, to Chtchédrine, the
biting satirist, all paint the peasant with loving touch, always find a
ready excuse for his defects, and lend him rare qualities, without
ever failing to show faithfully his true physiognomy. Corruption,
effeminacy, and vice characterize the upper classes, particularly the
employees of government, or any persons charged with public
trusts; and to make these the more odious, they are attributed with
a detestable hypocrisy made more hateful by apparent kindliness
and culture.
There is a humorous little novel by Chtchédrine (an author who
merits especial mention) entitled "The Generals[1] and the Mujik,"
which represents two generals of the most ostentatious sort,
transported to a desert island, unable either to get food or to get
away, until they meet with a mujik, who performs all sorts of
services for them, even to making broth in the hollow of his hand,
and then, after making a raft, conveys them safely to St. Petersburg;
whereupon these knavish generals, after recovering back pay, send
to their deliverer a glass of whiskey and a sum amounting to about
three cents. But this bitter allegory is a mild one compared with the
mystical apotheosis of the mujik as conceived by Tolstoï. In one of
his works, "War and Peace," the hero, after seeking vainly by every
imaginable means to understand all human wisdom and divine
revelation, finds at last the sum of it in a common soldier,
imperturbable and dull of soul, and poor in spirit, a prisoner of the
French, who endures with calm resignation ill treatment and death
without once entertaining the idea of taking the life of his foreign
captors. This poor fellow, who, owing to his rude, uncouth mode of
life, suffers persecution by other importunate lesser enemies which I
forbear to name, is the one to teach Pierre Besukof the alpha and
omega of all philosophy, wherein he is wise by intuition, and, in
virtue of his condition as the peasant, fatalistic and docile.
I have had the good fortune to see with my own eyes this idol of
Russian literature, and to satisfy a part of my curiosity concerning
some features of Holy Russia. Twenty or thirty peasants from
Smolensk who had been bitten by a rabid wolf were sent to Paris to
be treated by M. Pasteur. In company with some Russian friends I
went to a small hotel, mounted to the fourth floor, and entered a
narrow sleeping apartment. The air being breathed by ten or twelve
human beings was scarcely endurable, and the fumes of carbolic
acid failed to purify it; but while my companions were talking with
their compatriots, and a Russian young-lady medical student dressed
their wounds, I studied to my heart's content these men from a
distant land. I frankly confess that they made a profound impression
upon me which I can only describe by saying that they seemed to
me like Biblical personages. It gave me a certain pleasure to see in
them the marks of an ancient people, rude and rough in outward
appearance, but with something majestic and monumental about
them, and yet with a suggestion of latent juvenility, the grave and
religious air of dreamer or seer, different from really Oriental
peoples. Their features, as well as their limbs (which bearing the
marks of the wild beast's teeth they held out to be washed and
dressed with tranquil resignation), were large and mighty like a tree.
One old man took my attention particularly, because he presented a
type of the patriarchs of old, and might have served the painter as a
model for Abraham or Job,—a wide skull bald at the top, fringed
about with yellowish white hair like a halo; a long beard streaked
with white also; well-cut features, frontal development very
prominent, his eyes half hidden beneath bushy eyebrows. The arm
which he uncovered was like an old tree-trunk, rough and knotty, the
thick sinuous network of veins reminding one of the roots; his
enormous hands, wrinkled and horny, bespoke a life of toil, of
incessant activity, of daily strife with the energies of Mother Nature.
I heard with delight, though without understanding a word, their
guttural speech, musical and harmonious withal, and I needed not to
heat my imagination overmuch to see in those poor peasants the
realization of the great novelists' descriptions, and an expression of
patience and sadness which raised them above vulgarity and
coarseness. The sadness may have been the result of their unhappy
situation; nevertheless it seemed sweet and poetic.
The attraction which the people exercises upon refined and
cultivated minds is not surprising. Who has not sometimes
experienced with terrible keenness what may be called the æsthetic
effect of collectivity? A regiment forming, the crew of a ship about to
weigh anchor, a procession, an angry mob,—these have something
about them that is epic and sublime; so any peasant, if we see in
him an epitome of race or class, with his historic consequence and
his unconscious majesty, may and ought to interest us. The payo of
Avila who passes me indifferently in the street; the beggar in Burgos
who asks an alms with courteous dignity, wrapped in his tattered
clothes as though they were garments of costly cloth; the Gallician
lad who guides his yoke of oxen and creaking cart,—these not only
stir in my soul a sentiment of patriotism, but they have for me an
æsthetic charm which I never feel in the presence of a dress-coat
and a stiff hat. Perhaps this effect depends rather on the spectator,
and it may be our fancy that produces it; for, as regards the Russian
peasant, those who know him well say that he is by nature practical
and positive, and not at all inclined to the romantic and sentimental.
The Sclav race is a rich poetic wellspring, but it depends upon what
one means by poetry. For example, in love matters, the Russian
peasant is docile and prosaic to the last degree. The hardy rustic is
supposed to need two indispensable accessories for his work,—a
woman and a horse; the latter is procured for him by the head or old
man of the house, the former by the old woman; the wedding is
nothing more than the matriculation of the farmer; the pair is
incorporated with the great family, the agricultural commune, and
that is the end of the idyl. Amorous and gallant conduct among
peasants would be little fitting, given the low estimation in which
women are held. Although the Russian peasant considers the
woman independent, subject neither to father nor husband, invested
with equal rights with men; and although the widow or the
unmarried woman who is head of the house takes part in the
deliberations of the mir and may even exercise in it the powers of a
mayor (and in order to preserve this independence many peasant-
women remain unmarried), this consideration is purely a social one,
and individually the woman has no rights whatever. A song of the
people says that seven women together have not so much as one
soul, rather none at all, for their soul is smoke. The theory of
marriage relations is that the husband ought to love his wife as he
does his own soul, to measure and treasure her as he does his
sheepskin coat: the rod sanctions the contract. In some provinces of
Finnish or Tartar origin the bride is still bought and sold like a head
of cattle; it is sometimes the custom still to steal her, or to feign a
rape, symbolizing indeed the idea of woman as a slave and the
booty of war. So rigorous is the matrimonial yoke, that parricides are
numerous, and the jury, allowing attenuating circumstances,
generally pardons them.
Tikomirov, who, though a radical, is a wise and sensible man, says,
that far from considering the masses of the people as models worthy
of imitation, he finds them steeped in absolute ignorance, the
victims of every abuse and of administrative immorality; deprived for
many centuries of intercourse with civilized nations, they have not
outgrown the infantile period, they are superstitious, idolatrous, and
pagan, as shown by their legends and popular songs. They believe
blindly in witchcraft, to the extent that to discredit a political party
with them one has only to insinuate that it is given to the use of
sorcery and the black arts. The peasant has also an unconquerable
propensity to stealing, lying, servility, and drunkenness. Wherefore,
then, is he judged superior to the other classes of society?
In spite of the puerile humility to which the Russian peasant is
predisposed by long years of subjection, he yet obeys a democratic
impulse toward equality, which servitude has not obliterated; the
Russian does not understand the English peasant's respect for the
gentleman, nor the French reverence for the chevalier well-dressed
and decorated. When the government of Poland ordered certain
Cossack executions of the nobility, these children of the steppes
asked one another, "Brother, has the shadow of my body increased?"
Taught to govern himself, thanks to the municipal regimen, the
Russian peasant manifests in a high degree the sentiment of human
equality, an idea both Christian and democratic, rather more deeply
rooted in those countries governed by absolute monarchy and
municipal liberty, than in those of parliamentary institutions. The
Spaniard says, "None lower than the King;" the Russian says the
same with respect to the Czar. Primitive and credulous, a philosopher
in his way, the dweller on the Russian steppes wields a dynamic
force displayed in history by collectivities, be the moral value of the
individual what it may. In nations like Russia, in which the upper
classes are educated abroad, and are, like water, reflectors and
nothing more, the originality, the poetry, the epic element, is always
with the masses of the people, which comes out strong and beautiful
in supreme moments, a faithful custodian of the national life, as for
example when the butcher Minine saved his country from the yoke
of Sweden, or when, before the French invasion of 1812, they
organized bands of guerillas, or set fire to Moscow.
Hence in Russia, as in France prior to the Revolution, many thinkers
endeavor to revive the antiquated theory of the Genevan
philosopher, and proclaim the superiority of the natural man, by
contact with whom society, infected with Occidental senility, must be
regenerated. Discouraged by the incompatibility between the
imported European progress and the national tradition, unable to still
the political strife of a country where pessimist solutions are most
natural and weighty, their patriotism now uplifts, now shatters their
hopes, even in the case of those who disclaim and condemn
individual patriotism, such as Count Tolstoï; and then ensues the
apotheosis of the past, the veneration of national heroes and of the
people. "The people is great," says Turguenief in his novel "Smoke;"
"we are mere ragamuffins." And so the people, which still bears
traces of the marks of servitude, has been converted into a
mysterious divinity, the inspiration of enthusiastic canticles.
[1] Voguié explains this title of "General" to be both in the civil
and military order with the qualification of "Excellency." Without
living in Russia one can hardly understand the prestige attached
to this title, or the facilities it gives everywhere for everything. To
attain this dignity is the supreme ambition of all the servants of
the State. The common salutation by way of pleasantry among
friends is this line from the comedy of Griboiëdof, which has
become a proverb: "I wish you health and the tchin of a
General."—TR.

VII.

Social Classes in Russia.

Properly speaking, there are no social classes in Russia, a
phenomenon which explains to some extent the political life and
internal constitution; there is no co-ordinate proportion between the
rural and the urban element, and at first sight one sees in this vast
empire only the innumerable mass of peasants, just as on the map
one sees only a wide and monotonous plain. Although it is true that
a rural and commercial aristocracy did arise and flourish in old
Moscow in the twelfth and thirteenth centuries, the era of invasions,
yet the passions of the wars that followed gave it the death-blow.
The middle classes in the rich and independent republics lost their
wealth and influence, and the people, being unable of themselves to
reorganize the State, sustained the princes, who soon became
autocrats, ready at the first chance to subdue the nobles and unite
the disintegrated and war-worn nation. With the sub-division into
independent principalities and the institution of democratic
municipalities the importance of the cities decreased, and the
privileged classes were at an end. The middle class is the least
important. In the same districts where formerly it was most powerful
it has been dissolved by the continuous infusion of the peasant
element, owing to the curious custom of emigration, which is
spontaneous with this nomadic and colonizing people. Many farmers,
although enrolled in the rural villages, spend a large part of the year
in the city, filling some office, and forming a hybrid class between
the rural and artisan classes, thus sterilizing the natural instincts of
the laboring proletariat by the enervation of city life. The emperors
were not blind to the disproportion between the civic and rural
elements, and have endeavored to remedy it. The industrial and
commercial population fled from the cities to escape the taxes;
therefore they promulgated laws prohibiting emigration and the
renunciation of civic rights, under severe penalties. Yet with all these
the cities have taken but a second place in Russian history. Western
annals are full of sieges, defences, and mutinies of cities; in Russia
we hear only of the insurrection of wandering tribes or hordes of
peasants. Russian cities exist and live only at the mandate or
protection of the emperor. Every one knows what extraordinary
means were taken by Peter the Great to build St. Petersburg upon
the swamps along the Neva; in twenty-three years that remarkable
woman called the Semiramis of the North founded no less than two
hundred and sixteen cities, determined to create a mesocratic
element, to the lack of which she attributed the ignorance and
misery of her empire. Whenever we see any rapid advancement in
Russia we may be sure it is the work of autocracy, a beneficence of
despotism (that word so shocking to our ears). It was despotism
which created the modern capital opposite the old Byzantine,
legendary, retrogressive town,—the new so different from the old, so
full of the revolutionary spirit, its streets undermined by
conspirators, its pavements red with the blood of a murdered Czar.
These cities, colleges, schools, universities, theatres, founded by
imperial and autocratic hands, were the cradle of the political unrest
that rebels against their power; were there no cities, there would be
no revolutions in Russia. Although they do not harbor crowds of
famishing authors like those of London and Paris, who lie in wait for
the day of sack and ruin, yet they are full of a strange element
composed of people of divers extraction and condition, and of small
intellect, but who call themselves emphatically the intelligence of
Russia.
I have felt compelled to render justice to the good will of the
autocrats; and to be equally just I must say that whatever has
advanced culture in Russia has proceeded from the nobility, and this
without detriment to the fact that the larger energies lie with the
masses of the people. The enlightenment and thirst for progress
manifested by the nobility is everywhere apparent in Russian history.
They are descended from the retinues of the early Muscovite Czars,
to whom were given wealth and lands on condition of military
service, and they are therefore in their origin unlike any other
European nobility; they have known nothing of feudalism, nor the
Germanic symbolism of blazons, arms, titles, and privileges, pride of
race and notions of caste: these have had no influence over them.
The Boyars, who are the remnants of the ancient territorial
aristocracy, on losing their sovereign rights, rallied round the Czar in
the quality of court councillors, and received gold and treasure in
abundance, but never the social importance of the Spanish grandee
or the French baron. Hence the Russian aristocracy was an
instrument of power, but without class interests, replenished
continually by the infusion of elements from other social classes, for
no barrier prevented the peasant from becoming a merchant and the
merchant from becoming a noble, if the fates were kind. There are
legally two classes of aristocracy in Russia,—the transmissible, or
hereditary, and the personal, which is not hereditary. If the latter
surprise us for a moment, it soon strikes us with favor, since we all
acknowledge to an occasional or frequent protest against the idea of
hereditary nobility, as when we lament that men of glorious renown
are represented by unworthy or insignificant descendants. In Russia,
Krilof, the Æsop of Moscow, as he is called, put this protest into
words in the fable of the peasant who was leading a flock of geese
to the city to sell. The geese complained of the unkindness with
which they were treated, adding that they were entitled to respect
as being the descendants of the famous birds that saved the Capitol,
and to whom Rome had dedicated a feast. "And what great thing
have you done?" asked the peasant. "We? Oh, nothing." "Then to
the oven!" he replied.
The only title of purely national origin in Russia is that of prince;[1]
all others are of recent importation from Europe; in the family of the
prince, as in that of the humblest mujik, the sons are equals in rights
and honors, and the fortune of the father, as well as his title,
descends equally to all. Feudalism, the basis of nobility as a class,
never existed in Russia: according to Sclavophiles, because Russia
never suffered conquest in those ancient times; according to
positivist historians, by reason of geographical structure which did
not favor seignorial castles and bounded domains, or any other of
those appurtenances of feudalism dear to romance and poetry, and
really necessary to its existence,—the moated wall, the mole
overhanging some rocky precipice washed by an angry torrent, and
below at its foot, like a hen-roost beneath a vulture's nest, the
clustered huts of the vassals. But we have seen that the Russian
nobility acknowledges no law of superiority; like the people, they
hold the idea of divisible and common property. Hence this
aristocracy, less haughty than that of Europe, ruled by imperial
power, subject until the time of Peter III. to insulting punishment by
whip or rod, and which, at the caprice of the Czar, might at any time
be degraded to the quality of buffoons for any neglect of a code of
honor imposed by the traditions of their race,—never drew apart
from the life of the nation, and, on the contrary, was always
foremost in intellectual matters. Russian literature proves this, for it
is the work of the Russian nobility mainly, and the ardent sympathy
for the people displayed in it is another confirmation. Tolstoï, a
noble, feels an irrepressible tenderness, a physical attraction toward
the peasant; Turguenief, a noble and a rich man, in his early years
consecrated himself by a sort of vow to the abolition of servitude.
The same lack of class prejudices has made the Russian nobility a
quick soil for the repeated ingrafting of foreign culture according to
the fancy of the emperors. Catherine II. found little difficulty in
modelling her court after that of Versailles; but the same aristocracy
that powdered and perfumed itself at her behest adopted more
important reforms to a degree that caused Count Rostopchine to
exclaim, "I can understand the French citizen's lending a hand in the
revolution to acquire his rights, but I cannot understand the
Russian's doing the same to lose his." They are so accustomed to
holding the first place in intellectual matters, that no privilege seems
comparable to that of standing in the vanguard of advanced
thought. They had been urged to frequent the lyceums and debating
societies, to take up serious studies and scientific education by the
word of rulers who were enlightened, and friends to progress (as
were many of them), when all at once sciences and studies, books
and the press, began to be suspected, the censorship was
established, and the conspiracy of December was the signal for the
rupture between authority and the liberal thought of the country. But
the nobles who had tasted of the fruit of the knowledge of good and
evil did not resign themselves easily to the limited horizon offered by
the School of Pages or the antechamber of the palace; their hand
was upon the helm, and rather than let it go they generously
immolated their material interests and social importance. The
aristocracy is everywhere else the support of the throne, but in
Russia it is a destroying element; and while the people remains
attached to the autocrat, the nobles learn in the very schools
founded by the emperors to pass judgment upon the supreme
authority and to criticise the sovereign. Nicholas I. did not fail to
realize that these establishments of learning were focuses of
revolutionary ardor, and he systematically reduced the number of
students and put limits to scientific education.
It follows that the most reactionary class, or the most unstable class
in Russia, the class painted in darkest colors by the novelists and
used as a target for their shafts by the satirists, is not the noble but
the bureaucratic, the office-holders, the members of the tchin (an
institution Asiatic in form, comparable perhaps to a Chinese
mandarinate). Peter the Great, in his zeal to set everything in order,
drew up the famous categories wherein the Russian official
microcosm is divided into a double series of fourteen grades each,
from ecclesiastical dignitaries to the military. This Asiatic sort of
machinery (though conceived by the great imitator of the West)
became generally abhorred, and excited a national antipathy, less
perhaps for its hollow formalism than on account of the proverbial
immorality of the officers catalogued in it. Mercenariness, pride,
routine, and indolence are the capital sins of the Russian office-
holder, and the first has so strong a hold upon him that the people
say, "To make yourself understood by him you must talk of rubles;"
adding that in Russia everybody robs but Christ, who cannot
because his hands are nailed down. Corruption is general; it mounts
upward like a turbid wave from the humblest clerk to the archduke,
generalissimo, or admiral. It is a tremendous ulcer, that can only be
cured by a cautery of literary satire, the avenging muse of Gogol,
and the dictatorial initiative of the Czars. In a country governed by
parliamentary institutions it would be still more difficult to apply a
remedy.
The contrast is notable between the odium inspired by the
bureaucracy and the sympathy that greets the municipal institutions,
—not only those of a patriarchal character such as the mir, but those
too of a more modern origin. Among the latter may be mentioned
the zemstvo, or territorial assembly, analogous to our provincial
deputations, but of more liberal stripe, and entirely decentralized. In
this all classes are represented, and not, as in the mir, the peasants
merely. The form of this local parliament is extremely democratic;
the cities, the peasants, and the property-holders elect separate
representatives, and the assembly devotes itself to the consideration
of plain but interesting practical questions of hygiene, salubrity,
safety, and public instruction. This offers another opportunity to the
nobility, for this body engages itself particularly with the well-being
and progress of the poorer classes, in providing physicians for the
villages in place of the ignorant herb-doctors, in having the mujiks
taught to read, and in guarding their poor wooden houses from fire.
While the Russian nobility has never slept, the Russian clergy, on the
contrary, has been permanently wrapped in lethargy. The rôle
accorded to the Greek Church is dull and depressing, a petrified
image, fixed and archaic as the icons, or sacred pictures, which still
copy the coloring and design of the Byzantine epoch. Ever since it
was rent by schism from the parent trunk of Catholicism, life has
died in its roots and the sap has frozen in its veins. Since Peter the
Great abolished the Patriarchy, the ecclesiastical authority resides in
a Synod composed of prelates elected by the government. According
to the ecclesiastical statutes, the emperor is Head of the church,
supreme spiritual chief; and though there has been promulgated no
dogma of his infallibility, it amounts to the same in effect, for he may
bind and loose at will. At the Czar's command the church
anathematizes, as when for example to-day the popes are ordered
to preach against the growing desire for partition of land, against
socialism, and against the political enemies of the government; the
priest is given a model sermon after which he must pattern his own;
and such is his humiliation that sometimes he is obliged by order of
the Synod to send information, obtained through his office as
confessor, to the police, thus revealing the secrets of confiding souls.
What a loss of self-respect must follow such a proceeding! Is it a
marvel that some independent schismatics called raskolniks,
revivalists and followers of ancient rites and truths, should thrive
upon the decadence of the official clergy, who are subjected to such
insulting servitude and must give to Cæsar what belongs to God?
In view of these facts it is in vain to boast of spiritual independence
and say that the Greek church knows no head but Christ. The
government makes use of the clergy as of one arm more, which,
however, is now almost powerless through corruption. The Oriental
church has no conception of the noble devotion which has honored
Catholicism in the lives of Saint Thomas of Canterbury and Cardinal
Cisneros.
The Russian clergy is divided into black and white, or regular and
secular; the former, powerful and rich, rule in ecclesiastical
administration; the latter vegetate in the small villages, ill paid and
needy, using their wits to live at the expense of their parishioners,
and to wheedle them out of a dozen eggs or a handful of meal. Is it
strange that the parishioner respects them but little? Is it strange
that the pope lives in gross pride or scandalous immorality, and that
we read of his stealing money from under the pillow of a dying man,
of one who baptized a dog, of another who was ducked in a frozen
pond by his barino, or landlord, for the amusement of his guests? It
is true that a few occasional facts prove nothing against a class, and
that malice will produce from any source hurtful anecdotes and more
or less profane details touching sacred things; but to my mind, that
which tells most strongly against the Russian clergy is its inanity, its
early intellectual death, which shut it out completely from scientific
reflection, controversy, and apology, and therefore from all
philosophy,—realms in which the Catholic clergy has excelled. Like a
stripped and lifeless trunk the Oriental church produces no
theologians, thinkers, or savants. There are none to elaborate,
define, and ramify her dogmas; the human mind in her sounds no
depths of mystery. If there are no conflicts between religion and
science in Russia, it is because the Muscovite church weighs not a
shadow with the free-thinkers.
Certainly the adherents and members of the earlier church bear
away the palm for culture and spiritual independence. At the close of
the seventeenth century, after the struggles with Sweden and
Poland, the schismatic church aroused the national conscience, and
satisfied, to a certain extent, the moral needs of a race naturally
religious by temperament It began to discuss liturgical minutiæ, and
persecuted delinquents so fiercely that it infused all dissenters with a
spirit of protest against an authority which was disposed to treat
them like bandits or wild beasts. Such persecution demonstrates the
fact that not only ecclesiastical but secular power is irritated by
heterodoxy. In Russia, whose slumbering church is unmoved even by
a thunder-bolt, an instinct of orderliness led the less devout of the
emperors against the schismatics. To-day there are from twelve to
fifteen millions of schismatics and sects; and many among them are
given to the coarsest superstitions, practise obscene and cruel rites,
worship the Devil, and mutilate themselves in their insane fervors.
Probably Russia is the only country in the civilized world to-day
where superstition, quietism, and mysticism, without law or limit,
grow like poisonous trees; and in my work on Saint Francis of Assisi
I have remarked how the communist heresies of the Middle Ages
have survived there in the North. Some authors affirm that the
clergy shut their eyes and open their hands to receive hush-money
for their tolerance of heterodoxy. But let us not be too ready always
to believe the worst. Only lately there fell into my hands an article
written by that much respected author, Melchior de Voguié, who
assures us that he has observed signs of regeneration in many
Russian parishes.
From this review of social classes in Russia it may be deduced that
the peasant masses are the repository of national energies, while
the nobility has until now displayed the most apparent activity. The
proof of this is to be found in the consideration of a memorable
historical event,—the greatest perhaps that the present century has
known,—the emancipation of the serfs.
[1] "The term translated 'prince' perhaps needs some explanation.
A Russian prince may be a bootblack or a ferryman. The word
kniaz denotes a descendant of any of the hundreds of petty
rulers, who before the time of the unification of Russia held the
land. They all claim descent from the semi-mythical Rurik; and as
every son of a kniaz bears the title, it may be easily imagined how
numerous they are. The term 'prince,' therefore, is really a too
high-sounding title to represent it."—Nathan Haskell Dole.
 VIII.

Russian Serfdom.
Russia boasts of never having known that black stain upon ancient
civilizations, slavery; but the pretension, notwithstanding many
allegations thereto in her own chronicles, is refuted by Herodotus,
who speaks of the inhuman treatment inflicted by the Scythians on
their slaves, even putting out their eyes that they might better
perform certain tasks; and the same historian refers to the treachery
of the slaves to their masters in raping the women while they were
at war with the Medes, and to the insurrection of these slaves which
was put down by the Scythians by means of the whip alone,—the
whip being in truth a characteristic weapon of a country accustomed
to servitude. Herodotus does say in another place that "among the
Scythians the king's servants are free youths well-born, for it is not
the custom in Scythia to buy slaves;" from which it may be inferred
that the slaves were prisoners of war. Howbeit, Russian authors
insist that in their country serfs were never slaves, and serfdom was
rather an abuse of the power of the nobility and the government
than an historic natural result.
To my mind this is not so; and I must say that I think servitude had
an actual beginning, and that there was a cause for it. The
Muscovite empire was but sparsely populated, and the population
was by temperament adventurous, nomadic, restless, and expansive.
We have observed that the limitless plains of Russia offer no climatic
antagonisms, for the reason that there are no climatic boundaries;
but it was not merely the love of native province that was lacking in
the Russian, but the attachment to the paternal roof and to the
home village. It is said that the origin of this sentiment is embedded
in rock; where dwellings are built of wood and burn every seven
years on an average, there is no such thing as the paternal roof,
there is no such thing as home. With his hatchet in his belt the
Russian peasant will build another house wherever a new horizon
allures him. But if the scanty rural population scatters itself over the
steppes, it will be lost in it as the sand drinks in the rain, and the
earth will remain unploughed and waste; there will be nothing to
tax, and nobody to do military service. Therefore, about the end of
the sixteenth century, when all the rest of Europe was beginning to
feel the stirrings of political liberty and the breath of the
Renaissance, the Regent, Boris Godonof, riveted the chains of
slavery upon the wrists of many millions of human beings in Russia.
It is very true that Russian servitude does not mean the subjection
of man to man, but to the soil; for the decree of Godonof converted
the peasant into a slave merely by abrogating the traditional right of
the "black man" to change his living-place on Saint George's day.
The peasant perceived no other change in his condition than that of
finding himself fastened, chained, bound to the soil. The Russian
word which we translate "serf" means "consolidated," "adherent."
It is easy to see the historical transition from the free state to that of
servitude. The military and political organization of the Russian State
in the twelfth and thirteenth centuries hedged in the peasant's
liberty of action, and his situation began to resemble that of the
Roman colonus, or husbandman, who was neither "bond nor free."
When the nation was constituted upon firmer bases, it seemed
indispensable to fix every man's limitation, to range the population in
classes, and to lay upon them obligations consistent with the needs
of the empire. These bonds were imposed just as the other peoples
of Europe were breaking away from theirs.
Servitude, or serfdom, did not succeed throughout the empire,
however. Siberia and the independent Cossacks of the South
rejected it; only passive consent could sanction a condition that was
not the fruit of conquest nor had as an excuse the right of the
strongest. Even in the rest of Russia the peasant never was entirely
submissive, never willingly bent his neck to the yoke, and the
seventeenth and eighteenth centuries witnessed bitter and
sanguinary uprisings of the serfs, who were prompt to follow the
first impostor who pronounced words of promise; and, strange to
say, what was most galling was his entail upon the land rather than
the deprivation of his own liberty. He imagined that the lord of the
whole earth was the Czar, that by his favor it was temporarily in
possession of the nobles, but that in truth and justice it belonged to
him who tilled it. Pugatchef, the pretender to the title of Peter III., in
order to rally to his standard an innumerable host of peasants, called
himself the rural emperor, and declared that no sooner should he
gain the throne of his ancestors than he would shower treasure
upon the nobles and restore the land to the tillers of it.
Those who forged the fetters of serfdom had little faith in the
stability of it, however. And although the abuses arising out of it
were screened and tacitly consented to,—and never more so than
during the reign of the humane philosopher, friend, and
correspondent of Voltaire, the Empress Catherine II.,—yet law and
custom forever refused to sanction them. Russian serfdom assumed
rather a patriarchal character, and this softened its harshness. It was
considered iniquitous to alienate the serfs, and it was only lawful in
case of parting with the land whereon those serfs labored; in this
way was preserved the thin line of demarcation between agrarian
servitude and slavery.
There were, however, serfs in worse condition, true helots, namely,
the domestic servants, who were at the mercy of the master's
caprice, like the fowls in his poultry-yard. Each proprietor maintained
a numerous household below stairs, useless and idle as a rule,
whose children he brought up and had instructed in certain ways in
order to hire them out or sell them by and by. The players in the
theatres were generally recruited from this class, and until Alexander
I. prohibited such shameless traffic, it was not uncommon to see
announced in the papers the sale of a coachman beside that of a
Holstein cow. But like every other institution which violates and
offends human conscience, Russian serfdom could not exist forever,
in spite of some political and social advantages to the empire.
Certain Russian writers affirm that the assassination of masters and
proprietors was of frequent occurrence in the days of serfdom, and
that even now the peasant is disposed to quarrels and acts of
violence against the nobles. Yet, on the whole, I gather from my
reading on the subject that the relations in general between the serf
and the master were, on the one side, humble, reverent, and filial;
on the other, kind, gentle, and protecting. The important question
for the peasant is that of the practical ownership of the land. It is
not his freedom but his agrarian rights that have been restored to
him; and this must be borne in mind in order to understand why the
recent emancipation has not succeeded in pacifying the public mind
and bringing about a new and happy Russia.
Given the same problem to the peasant and the man of mind, it will
be safe to say that they will solve it in very different ways, if not in
ways diametrically opposed. The peasant will be guided by the
positive and concrete aspect of the matter; the man of mind by the
speculative and ideal. The peasant calculates the influence of
atmospheric phenomena upon his crops, while the other observes
the beauty of the sunset or the tranquillity of the night. In social
questions the peasant demands immediate utility, no matter how
small it may be, while the other demands the application of
principles and the triumph of ideas. Under the care of a master the
Russian serf enjoyed a certain material welfare, and if he fell to the
lot of a good master—and Russian masters have the reputation of
being in general excellent—his situation was not only tolerable but
advantageous. On the other hand, the intelligent could not put up
with the monstrous and iniquitous fact of human liberty being
submitted to the arbitrary rule of a master who could apply the lash
at will, sell men like cattle, and dispose as he would of bodies and
souls. Where this exists, since Christ came into the world, either
there is no knowledge, or the ignominy must be stamped out.
We all know that celebrated story of "Uncle Tom's Cabin," the
famous Abolitionist novel by Mrs. Harriet Beecher Stowe. There were
also novelists in Russia who set themselves to plead for the
emancipation of the serfs. But there is a difference between them
and the North American authoress, in that the Russians, in order to
achieve their object, had no need to exaggerate the reality, to paint
sensitive slaves and children that die of pity, but, with an artistic
instinct, they appealed to æsthetic truth to obtain human justice.
"Dead Souls," by Gogol, or one of the poetical and earnest brochures
of Turguenief, awakens a more stirring and permanent indignation
than the sentimental allegory of Mrs. Stowe; and neither Gogol nor
Turguenief misrepresented the serf or defamed the master, but
rather they present to us both as they were in life, scorning recourse
to bad taste for the sake of capturing tender hearts. The noblest
sentiments of the soul, divine compassion, equity, righteous
vengeance, the generous pity that moves to sacrifice, rise to the
inspired voice of great writers; we see the abuse, we feel it, it hurts
us, it oppresses us, and by a spontaneous impulse we desire the
good and abhor the evil. This enviable privilege has been granted to
the Russian novelists; had they no greater glory, this would suffice to
save them from oblivion.
The Abolitionist propaganda subtly and surely spread through the
intelligent classes, created an opinion, communicated itself naturally
to the press in as far as the censor permitted, and little by little the
murmur grew in volume, like that raised against the administrative
corruption after the Crimean War. And it is but just to add that the
Czars were never behind in this national movement. Had it not been
for their omnipotent initiative, who knows if even now slavery would
not stain the face of Europe? There is reason to believe it when one
sees the obstacles that hinder other reforms in Russia in which the
autocrat takes no part. Doubtless the mind of the emperor was
influenced by the words of Alexander II., in 1856, to the Muscovite
nobles: "It is better to abolish serfdom by decrees from above than
to wait for it to be destroyed by an impulse from below." A purely
human motive; yet in every generous act there may be a little
egotistical leaven. Let us not judge the unfortunate Emancipator too
severely.
The Crimean War and its grave internal consequences aided to
undermine the infamous institution of serfdom, at the same time
that it disclosed the hidden cancer of the administration, the
misgovernment and ruin of the nation. With the ill success of the
campaign, Russia clearly saw the need for self-examination and
reorganization. Among the many and pressing questions presented
to her, the most urgent was that of the serfs, and the impossibility of
re-forming a prosperous State, modern and healthy, while this taint
existed within her. Alexander II., whose variability and weakness are
no bar to his claim of the honored title of the Liberator, exhorted the
aristocracy to consummate this great work, and (a self-abnegation
worthy of all praise, and which only a blind political passion can deny
them) the nobles coincided and co-operated with him with perfect
good faith, and even with the electrical enthusiasm characteristic of
the Sclavic race. One cannot cease to extol this noble act, which,
taken as a whole, is sublime, although, being the work of large
numbers, it may be overloaded with details and incidents in which
the interest flags. It may be easy to preach a reform whose aims do
not hurt our pride, shatter our fortunes, alter our way of living, or
conflict with the ideas inculcated upon us in childhood by our
parents; but to do this to one's own detriment deserves especial
recognition. The nobility on this occasion only put into practice
certain theories which had stirred in their hearts of old. The first
great Russian poet, Prince Kantemire, wrote in 1738, in his satires,
that Adam did not beget nobles, nor did Noah save in the ark any
but his equals,—humble husbandmen, famous only for their virtues.
To my mind the best praise to the Russian nobility is for having
offered less hindrance to the emancipation of the serfs than the
North American democracy to the liberation of the slaves; and I
solicit especial applause for this self-sacrificing, redeeming
aristocracy.
The fruits of the emancipation were not what desire promised. The
peasants, from their positivist point of view, set little value on liberty
itself, and scarcely understood it. "We are yours," they were
accustomed to say to their masters; "but the soil is ours." When it
became known that they must go on paying even for the goods of
the community, they rebelled; they declared that emancipation was
a farce, a lie, and that true emancipation ought to abolish rent and
distribute the land in equal parts. Did not the proclamation of the
Czar read that they were free? Well, freedom, in their language,
meant emancipation from labor, and the possession of the land. One
mir even sent a deputation to the governor, announcing that as he
had been a good master he would still be allowed the use and profit
of his house and farm. The peasant believed himself free from all
obligation, and even refused to work until the government forced
him to do so; and the result was that the lash and the rod were
never so frequently laid across Russian shoulders as in the first three
years of emancipation and liberty.
What cared they—"the little black men"—for the dignity of the
freeman or the rights of citizenship? That which laid strongest hold
of their primitive imagination was the desire to possess the whole
land,—the old dream of what they called the black partition, the
national Utopia. One Russian revolutionary journal adopted the
name of "Land and Liberty," a magic motto to a peasant country,
giving the former the first place, or at least making the two
synonymous. The Russian people ask no political rights, but rather
the land which is watered by the sweat of their brow; and if some
day the anarchists—the agitators who go from village to village
propagating their sanguinary doctrines—succeed in awakening and
stirring this Colossus to action, it will be by touching this tender spot
and alluring by the promise of this traditional dream. The old serf
lives in hopes of a Messiah, be he emperor or conspirator, who shall
deliver the earth into his hands; and at times the vehemence of this
insatiable desire brings forth popular prophets, who announce that
the millennium is at hand, and that by the will of Heaven the land is
to be divided among the cultivators thereof. From his great love to
the autocrat the peasant believes that he also desires this
distribution, but being hampered by his counsellors and menaced by
his courtiers, he cannot authorize it yet. "For," says the peasant, "the
land never belonged to the lords, but first to the sovereign and then
to the mir." The idea of individual proprietorship is so repugnant to
this people that they say that even death is beautiful shared in
common.
All the schismatic sects in Russia preach community of possessions.
Some among them live better than the orthodox Greeks; some are
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com