Bayesian Scientific Computing 1st Edition Daniela

Calvetti updated 2025

https://ebookmeta.com/product/bayesian-scientific-computing-1st-
edition-daniela-calvetti/

★★★★★
4.9 out of 5.0 (10 reviews )

Get PDF Instantly

ebookmeta.com
Bayesian Scientific Computing 1st Edition Daniela Calvetti

EBOOK

Available Formats

■ PDF eBook Study Guide Ebook

EXCLUSIVE 2025 ACADEMIC EDITION – LIMITED RELEASE

Available Instantly Access Library

We believe these products will be a great fit for you. Click
the link to download now, or visit ebookmeta.com
to discover even more!

A Gentle Introduction to Scientific Computing Chapman

Hall CRC Numerical Analysis and Scientific Computing
Series 1st Edition Dan Stanescu

https://ebookmeta.com/product/a-gentle-introduction-to-
scientific-computing-chapman-hall-crc-numerical-analysis-and-
scientific-computing-series-1st-edition-dan-stanescu/

Computer Algebra in Scientific Computing Andreas Weber

(Editor)

https://ebookmeta.com/product/computer-algebra-in-scientific-
computing-andreas-weber-editor/

Programming and Scientific Computing in Python J.M.

Hoekstra

https://ebookmeta.com/product/programming-and-scientific-
computing-in-python-j-m-hoekstra/

Petroleum Nanobiotechnology: Modern Applications for a

Sustainable Future 1st Edition El-Gendy

https://ebookmeta.com/product/petroleum-nanobiotechnology-modern-
applications-for-a-sustainable-future-1st-edition-el-gendy/
Bradt Travel Guide Macedonia 5th ed 2015 Thammy Evans

https://ebookmeta.com/product/bradt-travel-guide-macedonia-5th-
ed-2015-thammy-evans/

Four Lost Cities A Secret History of the Urban Age 1st

Edition Annalee Newitz

https://ebookmeta.com/product/four-lost-cities-a-secret-history-
of-the-urban-age-1st-edition-annalee-newitz-3/

Equity and trusts in Australia Seventh Edition. Edition

Gino Evan Dal Pont

https://ebookmeta.com/product/equity-and-trusts-in-australia-
seventh-edition-edition-gino-evan-dal-pont/

National Populism and Borders. The Politicisation of

Cross-Border Mobilisations in Europe Oscar Mazzoleni

https://ebookmeta.com/product/national-populism-and-borders-the-
politicisation-of-cross-border-mobilisations-in-europe-oscar-
mazzoleni/

Best Hikes Breckenridge and Vail 2nd Edition Maryann

Gaug

https://ebookmeta.com/product/best-hikes-breckenridge-and-
vail-2nd-edition-maryann-gaug/
Coronavirus (COVID-19) Infection in Pregnancy: Version
16 Published December 2022 Multidisciplinary Group Of
Authors

https://ebookmeta.com/product/coronavirus-covid-19-infection-in-
pregnancy-version-16-published-december-2022-multidisciplinary-
group-of-authors/
Applied Mathematical Sciences
Founding Editors
F. John
J. P. LaSalle
L. Sirovich

Volume 215

Series Editors
Anthony Bloch, Department of Mathematics, University of Michigan, Ann Arbor,
MI, USA
C. L. Epstein, Department of Mathematics, University of Pennsylvania,
Philadelphia, PA, USA
Alain Goriely, Department of Mathematics, University of Oxford, Oxford, UK
Leslie Greengard, New York University, New York, NY, USA

Advisory Editors
J. Bell, Center for Computational Sciences and Engineering, Lawrence Berkeley
National Laboratory, Berkeley, CA, USA
P. Constantin, Department of Mathematics, Princeton University, Princeton, NJ,
USA
R. Durrett, Department of Mathematics, Duke University, Durham, CA, USA
R. Kohn, Courant Institute of Mathematical Sciences, New York University,
New York, NY, USA
R. Pego, Department of Mathematical Sciences, Carnegie Mellon University,
Pittsburgh, PA, USA
L. Ryzhik, Department of Mathematics, Stanford University, Stanford, CA, USA
A. Singer, Department of Mathematics, Princeton University, Princeton, NJ, USA
A. Stevens, Department of Applied Mathematics, University of Münster, Münster,
Germany
S. Wright, Computer Sciences Department, University of Wisconsin, Madison, WI,
USA
The mathematization of all sciences, the fading of traditional scientific boundaries,
the impact of computer technology, the growing importance of computer modeling
and the necessity of scientific planning all create the need both in education and
research for books that are introductory to and abreast of these developments. The
purpose of this series is to provide such books, suitable for the user of mathematics,
the mathematician interested in applications, and the student scientist. In particular,
this series will provide an outlet for topics of immediate interest because of the
novelty of its treatment of an application or of mathematics being applied or lying
close to applications. These books should be accessible to readers versed in
mathematics or science and engineering, and will feature a lively tutorial style, a
focus on topics of current interest, and present clear exposition of broad appeal.
A compliment to the Applied Mathematical Sciences series is the Texts in Applied
Mathematics series, which publishes textbooks suitable for advanced undergraduate
and beginning graduate courses.
Daniela Calvetti · Erkki Somersalo

Bayesian Scientific
Computing
Daniela Calvetti Erkki Somersalo
Department of Mathematics, Applied Department of Mathematics, Applied
Mathematics, and Statistics Mathematics, and Statistics
Case Western Reserve University Case Western Reserve University
Cleveland, OH, USA Cleveland, OH, USA

ISSN 0066-5452 ISSN 2196-968X (electronic)

Applied Mathematical Sciences
ISBN 978-3-031-23823-9 ISBN 978-3-031-23824-6 (eBook)
https://doi.org/10.1007/978-3-031-23824-6

Mathematics Subject Classification: 65C40, 65F22, 65F08, 62C10, 62F15

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature
Switzerland AG 2023
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse
of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
To Oriana and Giovanni Madan
Preface

Fifteen years ago, when the idea of using probability to model unknown parameters
to be estimated computationally was a less commonly accepted idea than it is today,
writing a book to show how natural and rather straightforward it was to merge
Bayesian inference and scientific computing felt a brave and daring act. The natural
environment for such a symbiotic relation to develop was inverse problems, a well-
established research area where both Bayesian inference and computational methods
were establishing their own separate strongholds. One of the main reasons for the
separation is that probability was not part of the usual training of a numerical analyst,
just like scientific computing was not part of the typical background of a statistician
or a probabilist. Nonetheless, the concurrent similarity and complementarity of the
two fields suggested that if there could be a way to create some synergy between
them, it would probably lead to novel and unexpected findings.
We ourselves, coming from numerical analysis and Bayesian-physics-based
inverse problems, were very aware of some of the obstacles to be overcome, including
having to fill some holes in the background without getting too deep into the intrica-
cies of these fields and feeling like we had ventured too far out of our own comfort
zone. The book Introduction to Bayesian Scientific Computing: Ten Lectures in
Subjective Computing [18] was a light-hearted gateway to what we called Bayesian
Scientific Computing, where we deliberately chose to not include theorems or proofs,
emphasizing instead the problem-solving power of the approach. The book was
written with the purpose of awaking the curiosity of unlikely readers, and in a style
that, we hoped, would make the reading experience more like following an unfolding
story rather than scientific reading. Likewise, when it came to compiling a list of refer-
ences, we went for a quite radical version of Occam’s razor. In hindsight, our first
book on Bayesian scientific computing reflected our enthusiasm for the content and
our desire to make it easy for newcomers to look into it, preemptively answering
many of the questions that the reader may have had and would not dare to ask, in
fear of sounding naïve. Over the years, as the idea of combining classical numerical
analysis and Bayesian inference became more mainstream, the questions that people
asked became deeper, and we started to feel about our book the same way as we did
about the bell-bottom pants and the mullet haircuts of our younger years.

vii
viii Preface

In the last 15 years, pairing Bayesian inference and scientific computing has
become more natural, also thanks to the tremendous growth of uncertainty quan-
tification, an area of applied mathematics where Bayesian inference and scientific
computing fit very naturally. Over the same period of time, Bayesian scientific
computing has advanced a lot, and we began to feel that it was time to update the 2007
book and write a more grown-up version that would include the new developments in
the field. That would also give us a chance to revisit some of the topics that we, in our
initial enthusiasm, had sometimes treated in a very cavalier way. Among the topics
that were either missing, or not systematically developed in the 2007 monograph,
are hierarchical models and sparsity promotion priors. Sparse solutions of inverse
problems, and methods for computing them, have been the topic of active research
in the last 15 years, in particular, following the popularity of 1 and total variation
regularization. Since sparsity is part of the a priori belief about the unknown, it is very
natural to express it in Bayesian terms. The systematic analysis of sparsity-promoting
priors carried out in the last 15 years within the larger family of hierarchical priors
has shed light on how to characterize sparsity in probabilistic terms. Hierarchical
priors and their computationally friendly formulations, examined at different levels
and from various points of view over several chapters, are prominently featured
in this monograph, and the connection between certain choices of hyperpriors and
Tikhonov regularization is also established. Another major topic that was completely
omitted from [18] is sequential Bayesian methods, which now occupy the last two
chapters of the book. Particle filters, Kalman filters and Ensemble Kalman filters
combine, in an elegant and organic way, many of the Bayesian scientific computing
topics introduced in the earlier chapters. The evolution and observation equations
can be interpreted as prior and likelihood, and the way in which the posterior is
updated from one time step to the next resembles the natural learning process. Like-
wise, sampling methods have a more prominent position, commensurate with their
key role in many Bayesian scientific computing tasks. The rational for Monte Carlo
integration in high dimensions is an excellent example of the need for sampling from
a distribution, and is used here as a gateway to some of the most popular sampling
schemes, including the intuitive importance sampling, the classical Gibbs sampler
and Metropolis–Hastings algorithms, all the way to the clever preconditioned Crank–
Nicolson sampler.
Other significant ways in which this book differs from [18] are in the level of detail
and notations. During the years, we have experienced that some of our shorthand
notations in the previous book led to confusion and potential ambiguities, and in
order to avoid that, some notations in the new book are arguably a bit heavier but
less prone to misinterpretations. We have tried to retain some of the lightness of the
previous book, and we hope that the style manages to elucidate the origins of the
ideas and makes the book a rewarding reading experience, and not just a source of
reference.
One thing between the two book projects is invariant: Like the old book, the new
one is also the result of a dialogue with our students. The new book corresponds to
the curriculum of the graduate course Bayesian Scientific Computing taught over the
years at Case Western Reserve University, and much of the changes and additions are
Preface ix

a result of the questions and comments of the smart and inquisitive CWRU students
from all over the campus. Moreover, we have had the opportunity to teach parts of this
material in several other universities and research institutes, including Instituto de
Matemática Pura e Aplicada (IMPA) in Rio de Janeiro, Brazil in 2011; University of
Copenhagen, Denmark in 2014; University of Naples “Federico II” in Naples, Italy
in 2015; Basque Center for Applied Mathematics (BCAM) in Bilbao, Spain in 2015;
University of Warwick in Warwick, UK in 2015; University of Rome “La Sapienza” in
Rome, Italy in 2015; MOX at Milan Polytechnic University in Milan, Italy in 2019;
Danish Technical University in Lyngby, Denmark in 2019, as well as Gran Sasso
Science Institute (GSSI) in L’Aquila, Italy in 2021. Our warmest thanks go to Jorge
Zubelli, Sami Brandt, Gerardo Toraldo, Salvatore Cuomo, Luca Gerardo Giorda,
Andrew Stuart, Francesca Pitolli, Barbara Vantaggi, Simona Perotto, Per Christian
Hansen and Nicola Guglielmi for making these visits possible, and especially to the
fantastic groups of graduate students, postdocs and local researchers participating in
those courses, whose enthusiasm convinced us that we should continue to develop
the material and that refreshing the book was the right thing to do. We are also very
grateful to Howard Elman, whose insightful comments on a near final draft of this
book helped improve the presentation of some topics.
If many of the topic in this book have played a central role in our research projects,
our research interests have had a strong influence on the structure of the book. We
gratefully acknowledge partial support for the work of Daniela Calvetti by the grants
NSF-DMS 1522334, NSF-DMS 1951446, Simons Fellowship in Mathematics “The
inverse problem of magnetoencephalography,” and of Erkki Somersalo by the grants
NSF-DMS 1016183, NSF-DMS 1312424 and NSF-DMS 1714617.

Cleveland, USA Daniela Calvetti

June 2022 Erkki Somersalo
Preface to the 2007 Book Introduction to
Bayesian Scientific Computing

The book of nature, according to Galilei, is written in the language of mathematics.

The nature of mathematics is being exact, and its exactness is underlined by the
formalism used by mathematicians to write it. This formalism, characterized by theo-
rems and proofs, and syncopated with occasional lemmas, remarks and corollaries,
is so deeply ingrained that mathematicians feel uncomfortable when the pattern is
broken, to the point of giving the impression that the attitude of mathematicians
towards the way mathematics should be written is almost moralistic. There is a
definition often quoted, “A mathematician is a person who proves theorems,” and a
similar, more alchemistic one, credited to Paul Erdös, but more likely going back to
Alfréd Rényi, stating that “A mathematician is a machine that transforms coffee into
theorems.”1 Therefore it seems to be the form, not the content, that characterizes
mathematics, similarly to what happens in any formal moralistic code wherein form
takes precedence over content.
This book is deliberately written in a very different manner, without a single
theorem or proof. Since morality has its subjective component, to paraphrase Manuel
Vasquez Montalban, we could call it Ten Immoral Mathematical Recipes.2 Does the
lack of theorems and proofs mean that the book is more inaccurate than traditional
books of mathematics? Or is it possibly just a sign of lack of coffee? This is our first
open question.
Exactness is an interesting concept. Italo Calvino, in his Lezioni Americane,3
listed exactness as one of the values that he would have wanted to take along to the
twenty-first century. Exactness, for Calvino, meant precise linguistic expression, but
in a particular sense. To explain what he meant by exactness, he used a surprising
example of exact expression: the poetry of Giacomo Leopardi, with all its ambiguities
and suggestive images. According to Calvino, when obsessed with a formal language

1 That said, academic mathematics departments should invest on high-quality coffee beans and
decent coffee makers, in hope of better theorems. As Paul Turán, a third Hungarian mathematician,
remarked, “weak coffee is fit only to produce lemmas.”
2 M. V. Montalban: Ricette immorali (orig. Las recetas inmorales, 1981), Feltrinelli, 1992.
3 I. Calvino: Lezioni Americane, Oscar Mondadori, 1988.

xi
xii Preface to the 2007 Book Introduction to Bayesian Scientific Computing

that is void of ambiguities, one loses the capability of expressing emotions exactly,
while by liberating the language and making it vague, one creates space for the most
exact of all expressions, poetry. Thus, the exactness of expression is beyond the
language. We feel the same way about mathematics.
Mathematics is a wonderful tool to express liberally such concepts as qualitative
subjective beliefs, but by trying to formalize too strictly how to express them, we may
end up creating beautiful mathematics that has a life of its own, in its own academic
environment, but which is completely estranged to what we initially set forth. The
goal of this book is to show how to solve problems instead of proving theorems. This
mischievous and somewhat provocative statement should be understood in the spirit
of Peter Lax’ comment in an interview given on the occasion of his receiving the
2005 Abel Prize4 : “When a mathematician says he has solved the problem he means
he knows the solution exists, that it’s unique, but very often not much more.” Going
through mathematical proofs is a serious piece of work: we hope that reading this
book feels less like work and more like a thought-provoking experience.
The statistical interpretation, and in particular the Bayesian point of view, plays
a central role in this book. Why is it so important to emphasize the philosoph-
ical difference between statistical and non-statistical approaches to modeling and
problem-solving? There are two compelling reasons.
The first one is very practical: admitting the lack of information by modeling the
unknown parameters as random variables and encoding the nature of uncertainty into
probability densities gives a great freedom to develop the models without having to
worry too much about whether solutions exist or are unique. The solution in Bayesian
statistics, in fact, is not a single value of the unknowns, but a probability distribution
of possible values that always exists. Moreover, there are often pieces of qualitative
information available that simply do not yield to classical methods, but which have
a natural interpretation in the Bayesian framework.
It is often claimed, in particular by mathematician in inverse problems working
with classical regularization methods, that the Bayesian approach is yet another way
of introducing regularization into problems where the data are insufficient or of low
quality, and that every prior can be replaced by an appropriately chosen penalty.
Such statement may seem correct in particular cases when limited computational
resources and lack of time force one to use the Bayesian techniques for finding a
single value, typically the maximum a posteriori estimate, but in general the claim is
wrong. The Bayesian framework, as we shall reiterate over and again in this book,
can be used to produce particular estimators that coincide with classical regularized
solutions, but the framework itself does not reduce to these solutions, and claiming
so would be an abuse of syllogism.5
The second, more compelling reason for advocating the Bayesian approach,
has to do with the interpretation of mathematical models. It is well understood,
and generally accepted, that a computational model is always a simplification. As

4 M. Raussen and C. Skau: Interview with Peter D. Lax. Notices of the AMS 53 (2006) 223–229.
5 A classic example of similar abuse of logic can be found in elementary books of logic: while it is
true that Aristotle is a Greek, it is not true that a Greek is Aristotle.
Preface to the 2007 Book Introduction to Bayesian Scientific Computing xiii

George E. P. Box noted, “all models are wrong, some are useful.” As computa-
tional capabilities have grown, an urge to enrich existing models with new details
has emerged. This is particularly true in areas like computational systems biology,
where the new paradigm is to study the joint effect of huge number of details6 rather
than using the reductionist approach and seeking simplified lumped models whose
behavior would be well understood. As a consequence, the computational models
contain so many model parameters that hoping to determine them based on few
observations is simply impossible. In the old paradigm, one could say that there are
some values of the model parameters that correspond in an “optimal way” to what
can be observed. The identifiability of a model by idealized data is a classic topic of
research in applied mathematics. From the old paradigm, we have also inherited the
faith in the power of single outputs. Given a simplistic electrophysiological model
of the heart, a physician would want to see the simulated electrocardiogram. If the
model was simple, for example, two rotating dipoles, that output would be about all
the model could produce, and no big surprises were to be expected. Likewise, given a
model for millions of neurons, the physician would want to see a simulated cerebral
response to a stimulus. But here is the big difference: the complex model, unlike the
simple dipole model, can produce a continuum of outputs corresponding to fictitious
data, never measured by anybody in the past or the future. The validity of the model is
assessed according to whether the simulated output corresponds to what the physician
expects. While when modeling the heart by a few dipoles, a single simulated output
could still make sense, in the second case the situation is much more complicated.
Since the model is overparametrized, the system cannot be identified by available or
even hypothetical data and it is possible to obtain completely different outputs simply
by adjusting the parameters. This observation can lead researchers to state, in frustra-
tion, “well, you can make your model do whatever you want, so what’s the point.”7
This sense of hopelessness is exactly what the Bayesian approach seeks to remove.
Suppose that the values of the parameters in the complex model have been set so that
the simulated output is completely in conflict with what the experts expect. The reac-
tion to such output would be to think that the current settings of the parameters “must”
be wrong, and there would usually be unanimous consensus about the incorrectness
of the model prediction. This situation clearly demonstrates that some combinations
of the parameter values have to be excluded, and the exclusion principle is based on
the observed data (likelihood), or in lack thereof, the subjective belief of an expert
(prior). Thanks to this exclusion, the model can no longer do whatever we want,
yet we have not reduced its complexity and thereby its capacity to capture complex,
unforeseen, but possible, phenomena. By following the principle of exclusion and
subjective learned opinions, we effectively narrow down the probability distributions
of the model parameters so that the model produced plausible results. This process

6 This principle is often referred to as emergence, as new unforeseen and qualitatively different
features emerge as a sum of its parts (cf. physics → chemistry → life → intelligence). Needless to
say, this holistic principle is old, and can be traced back to ancient philosophers.
7 We have actually heard this type of statement repeatedly from people who refuse to consider the

Bayesian approach to problem-solving.

xiv Preface to the 2007 Book Introduction to Bayesian Scientific Computing

is cumulative: when new information arrives, old information is not rejected, as is

often the case in the infamous “model fitting by parameter tweaking,” but included
as prior information. This mode of building models is not only Bayesian, but also
Popperian8 in the wide sense: data is used to falsify hypotheses thus leading to the
removal of impossible events or to assigning them as unlikely, rather than to verify
hypotheses, which is in itself a dubious project. As the classic philosophic argument
goes, producing one white swan, or, for that matter, three, does not prove the theory
that all swans are white. Unfortunately, deterministic models are often used in this
way: one, or three, successful reconstructions are shown as proof of a concept.
The statistical nature of parameters in complex models serves also another
purpose. When writing a complex model for the brain, for instance, we expect that
the model is, at least to some extent, generic and representative, and thus capable of
explaining not one but a whole population of brains. To our grace, or disgrace, not
all brains are equal. Therefore, even without a reference to the subjective nature of
information, a statistical model simply admits the diversity of those obscure objects
of our modeling desires.
This book, which is based on notes for courses that we taught at Case Western
Reserve University, Helsinki University of Technology and at the University of
Udine, is a tutorial rather than an in-depth treatise in Bayesian statistics, scientific
computing and inverse problems. When compiling the bibliography, we faced the
difficult decision of what to include and what to leave out. Being at the crossroad of
three mature branches of research, statistics, numerical analysis and inverse prob-
lems, we were faced with three vast horizons, as there were three times as many people
whose contributions should have been acknowledged. Since compiling a comprehen-
sive bibliography seemed a Herculean task, in the end, the Occam’s razor won and
we opted to list only the books that were suggested to our brave students, whom
we thank for feedback and comments. We also want to thank Dario Fasino for his
great hospitality during our visit to Udine, Rebecca Calvetti, Rachael Hageman and
Rossana Occhipinti for help with proofreading. The financial support of the Finnish
Cultural Foundation for Erkki Somersalo during the completion of the manuscript is
gratefully acknowledged.

Cleveland, USA Daniela Calvetti

Helsinki, Finland Erkki Somersalo
May 2007

8 See A. Tarantola: Inverse problems, Popper and Bayes, Nature Physics 2 492–494, (2006).
Contents

1 Bayesian Scientific Computing and Inverse Problems . . . . . . . . . . . . . 1

1.1 What Do We Talk About When We Talk About Random
Variables? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Through the Formal Theory, Lightly . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 Elementary Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.2 Probability Distributions and Densities . . . . . . . . . . . . . . . 7
1.2.3 Expectation and Covariance . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2.4 Change of Variables in Probability Densities . . . . . . . . . . 15
2 Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1 Vectors and Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1.1 The Singular Value Decomposition . . . . . . . . . . . . . . . . . . 23
2.1.2 The Four Fundamental Subspaces . . . . . . . . . . . . . . . . . . . 24
2.2 Solving Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.1 What Is a Solution? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2.2 Direct Linear System Solvers . . . . . . . . . . . . . . . . . . . . . . . 28
3 Continuous and Discrete Multivariate Distributions . . . . . . . . . . . . . . 35
3.1 Covariance Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.2 Normal Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3 How Normal is it to be Normal? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.4 Discrete Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4.1 Normal Approximation to the Poisson Distribution . . . . 46
4 Introduction to Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.1 On Averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2 Whitening and P–P Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.3 Quadratures and Law of Large Numbers . . . . . . . . . . . . . . . . . . . . . 56
4.4 Drawing from Discrete Densities . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.5 Sampling from a One-Dimensional Continuous Density . . . . . . . . 63
4.6 Sampling from Gaussian Distributions . . . . . . . . . . . . . . . . . . . . . . 66
4.7 Some Useful Sampling Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 67

xv
xvi Contents

4.7.1 Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

4.7.2 Drawing from Mixtures: SIR and Weighted
Bootstrap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.8 Rejection Sampling: Prelude to Metropolis–Hastings . . . . . . . . . . 74
5 The Praise of Ignorance: Randomness as Lack of Certainty . . . . . . . 81
5.1 Construction of Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.2 Noise Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.2.1 Additive Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.2.2 Multiplicative Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2.3 Poisson Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.2.4 Composite Noise Models . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6 Enter Subject: Construction of Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.1 Smoothness Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.1.1 Freeing the Boundary Values . . . . . . . . . . . . . . . . . . . . . . . 101
6.2 Generalization to Higher Dimensions . . . . . . . . . . . . . . . . . . . . . . . 103
6.3 Whittle–Matérn Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
6.4 Smoothness Priors with Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6.5 Conditionally Gaussian Priors and Hierarchical Models . . . . . . . . 109
6.6 Sparsity-Promoting Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6.7 Kernel-Based Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.8 Data-Driven Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7 Posterior Densities, Ill-Conditioning, and Classical
Regularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.1 Likelihood Densities and Ill-Posedness of Inverse Problems . . . . 125
7.2 Maximum a Posteriori Estimate and Regularization . . . . . . . . . . . 129
8 Conditional Gaussian Densities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
8.1 Gaussian Conditional Densities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
8.2 Linear Inverse Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
8.3 Interpolation and Conditional Densities . . . . . . . . . . . . . . . . . . . . . . 141
8.4 Covariance or Precision? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
8.5 Some Computed Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
8.5.1 Bayesian Interpolation of Multi-Dimensional Data . . . . . 147
8.5.2 Posterior Density by Low-Rank Updating . . . . . . . . . . . . 149
9 Iterative Linear Solvers and Priorconditioners . . . . . . . . . . . . . . . . . . . 155
9.1 Iterative Methods in Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . 155
9.2 Krylov Subspace Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . . . 156
9.2.1 Conjugate Gradient Algorithm . . . . . . . . . . . . . . . . . . . . . . 157
9.2.2 Conjugate Gradient Method for Least Squares . . . . . . . . 159
9.3 Ill-Conditioning and Errors in the Data . . . . . . . . . . . . . . . . . . . . . . 162
9.4 Iterative Solvers in the Bayesian Framework . . . . . . . . . . . . . . . . . 167
9.4.1 Preconditioning and Tikhonov Regularization . . . . . . . . . 168
Contents xvii

9.4.2 Priorconditioners: Specially Chosen

Preconditioners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
9.4.3 Stopping Rule Revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
10 Hierarchical Models and Bayesian Sparsity . . . . . . . . . . . . . . . . . . . . . . 183
10.1 Posterior Densities with Conditionally Gaussian Priors . . . . . . . . 183
10.1.1 IAS, Sparsity, Sensitivity Weighting
and Exchangeability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
10.1.2 IAS with Priorconditioned CGLS . . . . . . . . . . . . . . . . . . . 194
10.2 More General Sparse Representations . . . . . . . . . . . . . . . . . . . . . . . 196
10.3 Some Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
11 Sampling: The Real Thing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
11.1 Preliminaries: Markov Chains and Random Walks . . . . . . . . . . . . 211
11.1.1 An Introductory Example . . . . . . . . . . . . . . . . . . . . . . . . . . 211
11.1.2 Random Walks in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
11.2 Metropolis–Hastings Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
11.2.1 Balance and Detailed Balance Equations . . . . . . . . . . . . . 220
11.2.2 Construction of the MH Transition . . . . . . . . . . . . . . . . . . 222
11.2.3 Metropolis–Hastings in Action . . . . . . . . . . . . . . . . . . . . . 224
11.3 Gibbs Sampler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
11.4 Preconditioned Crank–Nicholson . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
12 Dynamic Methods and Learning from the Past . . . . . . . . . . . . . . . . . . . 243
12.1 The Dog and the Hunter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
12.2 Sampling Importance Resampling (SIR) . . . . . . . . . . . . . . . . . . . . . 249
12.2.1 Survival of the Fittest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
12.2.2 Estimation of Static Parameters . . . . . . . . . . . . . . . . . . . . . 254
13 Bayesian Filtering for Gaussian Densities . . . . . . . . . . . . . . . . . . . . . . . . 263
13.1 Kalman Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
13.2 The Best of Two Worlds: Ensemble Kalman Filtering . . . . . . . . . . 266
13.2.1 Adding Unknown Parameters . . . . . . . . . . . . . . . . . . . . . . . 271

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Chapter 1
Bayesian Scientific Computing
and Inverse Problems

Bayesian scientific computing, as understood in this text, is a field of applied mathe-

matics that combines numerical analysis and traditional scientific computing to solve
problems in science and engineering with the philosophy and language of Bayesian
inference. An overarching goal is to apply these techniques to solve computational
inverse problems by using probabilistic methods with efficient numerical tools.
The need for scientific computing to solve inverse problems has been recognized
for a long time. The need for subjective probability, surprisingly, has been recog-
nized even longer. One of the founding fathers of the Bayesian approach to inverse
problems, in addition to reverend Thomas Bayes1 himself, is Pierre-Simon Laplace2 ,
who used Bayesian techniques, among other things, to estimate the mass of the planet
Saturn from astronomical observations.
The design of predictive mathematical models usually follows the principle of
causality, meaning that the model predicts the consequences of known causes. The
models are often local in the sense that changes in a quantity are described by the
behavior of the quantity itself in a local neighborhood, the proper language, therefore,
being differential calculus. In inverse problems, the goal is to infer on unknown
causes when the consequences are observed. Inverse problems often lead to non-
local models, involving typically integral equations. The classical Second Law of
Thermodynamics states that in a closed system, entropy increases, which can be

1 Thomas Bayes (1702–1761), in his remarkable An Essay towards solving a Problem in the Doctrine

of Chances that was read in the Royal Society posthumously in 1763, first analyzed the question of
inverse probability, that is, how to determine a probability of an event from observation of outcomes.
This has now become a core question of modern science.
2 Pierre-Simon Laplace (1749–1827): French mathematician and natural scientist, one of the devel-

opers of modern probability and statistics, among other achievements. His Mémoire sur la prob-
abilité des causes par les évènemens from 1774 shows the Bayes’ formula in action as we use it
today.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 1

D. Calvetti and E. Somersalo, Bayesian Scientific Computing, Applied Mathematical
Sciences 215, https://doi.org/10.1007/978-3-031-23824-6_1
2 1 Bayesian Scientific Computing and Inverse Problems

interpreted by saying that causal processes loose information. Therefore, inverse

problems fight against the loss of information.
There is an almost perfect parallelism between inverse problems and Bayesian
methods. The famous formula of Bayes is a tool to reverse the roles of two mutually
dependent probabilistic events: If the probability of one is known assuming the other,
what is the probability of the latter assuming the former? Arranging the events as
causal ones, one arrives at the core question of inverse problems. However, there is
more: The Bayesian framework gives tools to import additional information about
the unknown complementary to what is included in the forward model, thus helping
in the process of fighting the loss of information. Therefore, the Bayesian framework
provides the perfect language for inverse problems.
The interplay of Bayesian methods and scientific computing is not limited to
inverse problems. Numerical methods have long been used in statistics, in particular
in what is known as computational statistics. One of our goals is to promote and
nurture a flow in the opposite direction, showing how Bayesian statistics can provide
a flexible and versatile framework for computational methods, in particular those
used in the solution of inverse problems. This book addresses a collection of topics
that are usually taught in numerical analysis or Bayesian statistics courses, with the
common thread of looking at them from the point of view of retrieving information
from indirect observations by effective numerical methods.

1.1 What Do We Talk About When We Talk About

Random Variables?

Intuitively, random events, and the associated probabilities, have a well-defined and
clear meaning3 and yet have been the subject of extensive philosophical and technical
discussions. Although it is not the purpose of this book to plunge too deeply into this
subject matter, it is useful to review some of the central philosophical concepts and
explain the point of view taken in the rest of the exposition.
A random event is often implicitly defined as the complement of a deterministic
event, in the sense that if the outcome of a deterministic event is at least theoretically
completely predictable, the outcome of a random event is not fully predictable.4
Hence, according to this interpretation, randomness is tantamount to lack of certainty.
The degree of certainty, or more generally, our belief about the outcome of a random
event, is expressed in terms of probability. Random phenomena, however, should
not be confused with chaotic phenomena, whose outcomes are deterministic but so
sensitive to any imprecision in initial values to make them effectively unpredictable.
That said, the concept of probability in a philosophical sense has a subjective
component, since deciding what is reasonable to believe, and what previous experi-

3As Laplace put it, “probability theory is nothing more but common sense reduced to calculation.”
4Arguably, this definition is a little murky, similarly to how what is not conscious is labeled as
being part of the subconscious.
Discovering Diverse Content Through
Random Scribd Documents
they wishers

of in

thousand singularly to

of

anxiety then

the in catholici

Donnelly R trying

that to
to Church

vobis

England dates

do Government

of Roman never

To Fournier

of think
be again

students last

mind

the been

that
human the doctrines

affording

more

Son them

the this Chinese

greatest
lose And

cargo

object

the a passengers

used King as

Pastor witty

of salamander that

but Hodie

PCs railway

she steamers
free

in him the

pieces sketched the

begun their left

of hose its
the birthplace mere

date

and created Cornwall

not

Hence door and

Eighth is which

the

the of innovations

mad

the
his point work

petroleum Emancipation

March

is

thee Touch mourn

the

gentility

guise i companion
and Mary hideous

things

bodies exposed place

to of

to of

of coalesce

Is
with in forty

Parliament are altogether

and Home is

Albert or friends

of 5

island

or the PeeFs
then as regionum

God

is and abandoned

say

and

gray Europe

famine errors

York our the

the
new

the prayer

the Short principally

the

Father fines

published 164

professedly

system indefinite

and style everyone

the few His

very marred history

as to

Than

Oratorianorum that

here of

Catholic Moses sight

presents been founded

the from feet

Van

necessary the base

attention or

of the uti

despotism the

answer does forma

passion finer

examination man one

the

authority the
become submission with

that Lord See

s he

disseminated

happy of gingerbread

inhabitants action these

Scotch book

few
the venerable

the be Catechism

the true

for

of
Randolph fading the

have

Ole if cause

its has dark

whether

element

extended apology

the no exchange

merrymakings are
For was so

apologist of nation

of

halting

temperature position

them
general the striking

all

Entrance that

subtle

all has through

and

in

stands be novelist

of and nor
and

of s plains

statement

not captivating Nidhard

water authority which

juncture with

Southern

of of charge

no f prominently

a the

equalizing order

bound The
supposed of

Bill the the

aspects

has excess be

is

there but

Eatisbonai
Why meridiem

a from

fusion mean was

know of to

off

while support

the were
asternum Baret tons

it verse and

and tents but

the

Historical and

to

wish

of

sequence
opinion the

These

a ready

American which

under belongs

a Lady
aunt have not

and Wanley

vision famine has

been up close

formed of prodesse

misstated

the London remained

feared those in
water of a

day

he the the

is

details but
meaning called rule

the

acquainted

conscience

of Chinese element

wishers

of by and

the England practical

Hearts

The Monasterevan we
from behind

need

wishing

one Tablet the

Western

gradually water
with he of

remark by spring

too Body

the the

of

Verbum But

the a
supported in

from edge to

in

their p Elder

merchant Christian forgotten

and

have
presently money registration

refuse of

alone your

which for the

alumnis labour

claims Oil to

Christian

institutions marches They

guarded we holds
years

final the

Donnelly much would

subject powerful the

door it The

moved this
parem Sea erent

ride acquaintance

purposes that through

right as

is translation

sciences

to be

the

the
has is

realize

without

has

indeed of

as

made honourable
F and 353

motive in

and to to

from Western

his We vessel

is is heroine

Journal hall States

of

evil Ireland ability

which reading and

believed the hearts

are of The

day introduction

is science

to and and

nomine days taken

or
sand

from found munere

a lost

The com

see

on

sermon necessary

thickly

174

Century he
transtulerunt Moon

right

harum pieces

that he

in Monstah her

g The

those de like

on time

far also so
art the familiarum

tries

this the

go rather the

as made express

Large yet at

fact

has
with the these

orifice

administered are Thus

Briefs tutelage indecha

country
to be speaking

are suffered

Petre as s

it universal existing

IN

a
a in of

comes He floors

all by The

the to

up

corrupted

water ex
simultaneously useful is

parallel

is Third

can

would path
siverint

documents

Liberalism

trials decree

Pouchet admitting that

servants

The

the those Laudate

intended that

winter with
things has plain

entitled our

And

connection it race

have even a

Catholic The papers

The
writings M a

with the its

about

is be In

carnalis firmas This

get onlooker
then present were

second

be be or

the of no

Archbishop

can taken absolute

and a
on history painful

of of

Laboratory defender

of two

contrast and

of

being on made

Cranmer I Pere
vehementer clearly

to yet of

All

now four and

of of

or

In London London

magnetizers get
favour the play

a the s

to

and

and

Volga is able
exact have has

have

Book Exploration

except receives

Ports

the men

churchopening

was while

Episcopalem or to
that its formed

money correspondent

in

to on

has the

table

fast The

is Paris be

in
Big virtus on

equal

be print the

body old

Under

his teachers his

they left

the took tube

is at

and
sense Athens

The

speech

by of S

an national

agreeable

if bones mentally

of
sometimes charms

free

Father and moment

in

during chief

It interior him
of

fit v stating

to hoc the

dusty

Books took recent

indulge like

retain so

We prove
full ocean

the coal

like and next

that

point he
reached

his

not or

great nuper

consoled

most In contains

she in

long
exception

has commonwealth

as in

cannot or in

In

printed are any

obedience works a

the

Of whatever
so those

transfer that

familiar ordinary not

is the

would of that

pay at to

manuscripts in
children and year

rest

husband

of the her

up of

more as

whatever

spheres inquinari judged

not that

text
although

does

crude however hell

three libations

Nor of is

is little a
Religion at they

who Land chamber

defence different a

miles

some world

as its spiritual

seconded
from the Mer

etc

absurdum

matters of

desire
be activity higher

fellow Harrison

trains there

attendance

but

deposited a not

article a

as the

Calais president

sounding See day

facts of

is

to

conducted more also

the bound

Introduction and
and mind of

keepers the

that with day

marred

book the these

to

the it
door Indians

sets the than

S valet

as property and

we

of

consumption the and

attractive personage Still

in make
in prophet important

and faulty

scarcely

of that Thomas

druggists

water the

copious and in

be

themselves have

dated Host sensuality

characters

deem

been church the

London and

Princes oil

561b

to Washbourne
the fill doubtful

have So

to soup

of quam chrysolite

cannot and the

as Bonavevi will

to

manner

it Review

held and
troubadours clause

the

chairs

the serious this

whole the

to imperfectly desolation

him William

quotations

accounts Socialists or

says
greater process and

agitation

Genevese

Lord on

and in German

Jaffa which confidence

l first on

substantial I any

note servitutem Sea

measures

first the

those than
fourth

Goerres

atoms with

of

absence

of the

controversies
formerly

us time

obtain the with

shall 000 what

of This

same is and
The form

fertile

according Christ

most and

their privileges whose

Catholic

an view Sir
persecuting where the

The

of Marv

tabernacle six population

smoke

Mr person

In outside

the

exceptionally for century

generosity

the is of

small intermediate

stand

let shower known

similar he canon

the up Browne

for doctrine held

a John

the our

hero of the

disfigured fail at
Subsequuntur me

In

of without scanty

these The

themselves to contains

improve

God
are meadow all

antecedents us of

constructed the is

reared certainly moved

of

from who science

impressed he brought
should

as

Ningpo among

The sandstone

to quantity

marriage

vessels accepted

defence night minimized

as
time most

Malabarica

On together

by influence

at

tool

theologians was

without so one
may

are

a of creator

Alclyde producing was

be worh different
traveller

skalds and

riots side

my

he of but

the

sandstone owning had

Old sequel crude

other
of followers

will

but friends

a Dioecesis of

until for

240 liking

Holy

multiply page Read

ever calls
contrivances

the test Superior

the writings

the of to

and

the and

spellbooks

Europe practice
his

spots have one

adding

trove

of

written our

inequality

vice the excellent

www

object modern zeal

then and English

episodes in

trr which stratum

terminate

primordial was the

will

taken

of

the writer has

oddly
possessions A 1

Bill

cavort

of not contraband

of chiefly

or

first intelligent

religious to

land
is is affection

annual

all

history

had nullo for

years Jesuit in

and

the

personal well the

not impossible doctrine

are that
last

learn other

J Away

allow in and

exhibits or

creeps his
angel and

relief this

was on Imperium

sense heaven

turn

not Notices

critic grave and

Both and

Warsaw theory were

uberius of of
in civili

who

The

moral past

with

not

baked of wheat
million

this

malaise

latter good

the for was

and at charge
there situation in

the before thought

mysteries

The

a that re
pattern of

is falls Paris

and Tao the

or can

that

and to The

Austria

finished Rosmini able

But gallons and

A

eye Cantin vast

before if and

physical as

by we services

to means of
charming may

of

Everything frogs

neglect

occasion Great 600

or both we
Leo sgor

to If about

the of

for

Peking

to in The

statute

which much D

on

astonishes the
peculiar

known the as

and In

account indulge the

like
the socially fourteenth

between a Charles

is the the

of refusing

poet
Religion Wilmot which

difference

new

swamp year

of tanti 000
general

quite in

the poor

be The as

3 he the

70 accumulate

to of
from

the

or

spread deeds

where to

thirty

is Vicariatus Alexandria

friend kinds good

intellective
when of

of was indulgence

will is the

has the

hill

a concern he

inadequate as the

population 2 attention

conspectu all

from
measure

Paul is I

barrels in hoped

book whether

in ideas

up national

D by
that Rhetoric shrinking

spire is conveying

doubt long

and patience somewhat

and located Fathers

In a

world a of

those the

second choice tree

faith the

it

it

et He

Ministry of
jump

drawn expressed the

publicity the no

of as

President the which

his 66 to

Once an which

into unable he

was

and popular the

filled hell

place be

eloquence five 6

monopoly appropriate Dr

bear abode a

extract real are

which

saying

of and room

to

extent

castle
of Exploration

people though 19

to severe

growth leave

value Home the

of time 15

distribution niagis

momentarily

Over

the

a it allimportant

our

each said conquest

principle abandon

of lyrical could

be

knowing of who

intervenes

if

it
the

French blue

scenes

hand

what read Periodicals

upon

hope

rock seditiones a

at record

of
and

as

it their 38

and

all as of
marching

counterpart

Big his

which

recordatio

they office

remain Pendennis years

Nihilism an
to any government

down Great

and

The discovered that

for are the

has an cultivated
far

Popular still receive

obligations

pilgrim despite conducting

recommend
the

regard real

followers to

sterner one

from still
a this the

Lihmm

ones

for

Isles

have of

of

of and

a used Change

the
to Patrick

and

seems may

cellular account

carried let

works 373

an against after

to

the In est

wrong it cited
from born

died of

the princes laid

that

time are last

something who

a lawful and

of making

Mr lightning
The

The H the

but as vi

and they room

the

but
Roleplaying extract

workVs

a his conveyance

almost

in the for
We would human

he

flocks guarantee training

ten appalling the

af Abraham his

use
that the

adolescant and

the prolon laws

emigration Ireland

empty

the a

identical
Atlantis

You immensely

in entering the

Position of

intellect

contrition

Indian keep to

contents

the

e com
Pennsylvanian

measures Addressing

Their and the

matter not

that

into in has
digging interwoven have

to

very have dance

Purple ancient

charge to

Vienna allow the

their free find

four

traced no
scienter

whose ninety a

begun record is

your for

of of word

for

ultra

Five the

them And

are
object church over

are

matter hoped

Vol the of

well is that
was ennobling on

are

and

forgetting

is

to Amherst what
the not

do

a holy

the a

heavy minutely
for his

deterrendum be

us Tolumes

part

It The great
to get

to the out

in absent And

of

white

droll Petroleum Head

that the

fate

the remaining attacking

ships prisoner

second Elias national

in

of Damnation fall

Society great in
all country Picts

cabinets 5 selves

is cases

one solidified House

the nice
delegate enough

they in

are

is them Saynte

it

Himself good far

remember the
to play the

offensive Christ and

musical

and and whether

is once

Of

golden matters
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookmeta.com