Download the full version of the textbook now at textbookfull.

com

Introduction to Machine Learning with

Applications in Information Security 1st
Edition Mark Stamp

https://textbookfull.com/product/introduction-to-
machine-learning-with-applications-in-information-
security-1st-edition-mark-stamp/

Explore and download more textbook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Introduction to machine learning with R rigorous

mathematical analysis First Edition Burger

https://textbookfull.com/product/introduction-to-machine-learning-
with-r-rigorous-mathematical-analysis-first-edition-burger/

textbookfull.com

An Introduction To Statistical Learning With Applications

In R Gareth James

https://textbookfull.com/product/an-introduction-to-statistical-
learning-with-applications-in-r-gareth-james/

textbookfull.com

Fundamentals of optimization theory with applications to

machine learning Gallier J.

https://textbookfull.com/product/fundamentals-of-optimization-theory-
with-applications-to-machine-learning-gallier-j/

textbookfull.com

Models of Quantum matter : a first course on integrability

and the Bethe Ansatz First Edition. Edition Eckle

https://textbookfull.com/product/models-of-quantum-matter-a-first-
course-on-integrability-and-the-bethe-ansatz-first-edition-edition-
eckle/
textbookfull.com
Routledge Handbook of the Study of the Commons 1st Edition
Blake Hudson (Editor)

https://textbookfull.com/product/routledge-handbook-of-the-study-of-
the-commons-1st-edition-blake-hudson-editor/

textbookfull.com

The Cambridge Handbook of Social Problems Volume 2 1st

Edition A Javier Treviño

https://textbookfull.com/product/the-cambridge-handbook-of-social-
problems-volume-2-1st-edition-a-javier-trevino/

textbookfull.com

Design User Experience and Usability Understanding Users

and Contexts 6th International Conference DUXU 2017 Held
as Part of HCI International 2017 Vancouver BC Canada July
9 14 2017 Proceedings Part III 1st Edition Aaron Marcus
https://textbookfull.com/product/design-user-experience-and-usability-
understanding-users-and-contexts-6th-international-conference-
duxu-2017-held-as-part-of-hci-international-2017-vancouver-bc-canada-
july-9-14-2017-proceedings-part/
textbookfull.com

Evolving Narratives of Hazard and Risk: The Gorkha

Earthquake, Nepal, 2015 1st Edition Louise Bracken

https://textbookfull.com/product/evolving-narratives-of-hazard-and-
risk-the-gorkha-earthquake-nepal-2015-1st-edition-louise-bracken/

textbookfull.com

From Big Data to Big Profits Success with Data and

Analytics 1st Edition Russell Walker

https://textbookfull.com/product/from-big-data-to-big-profits-success-
with-data-and-analytics-1st-edition-russell-walker/

textbookfull.com
Aesthetics The Classic Readings 2nd Edition David E.
Cooper

https://textbookfull.com/product/aesthetics-the-classic-readings-2nd-
edition-david-e-cooper/

textbookfull.com
 INTRODUCTION TO

MACHINE
LEARNING with
APPLICATIONS
in INFORMATION
SECURITY
 Chapman & Hall/CRC
Machine Learning & Pattern Recognition Series

SERIES EDITORS

Ralf Herbrich Thore Graepel

Amazon Development Center Microsoft Research Ltd.
Berlin, Germany Cambridge, UK

AIMS AND SCOPE

This series reflects the latest advances and applications in machine learning and pattern rec-
ognition through the publication of a broad range of reference works, textbooks, and hand-
books. The inclusion of concrete examples, applications, and methods is highly encouraged.
The scope of the series includes, but is not limited to, titles in the areas of machine learning,
pattern recognition, computational intelligence, robotics, computational/statistical learning
theory, natural language processing, computer vision, game AI, game theory, neural networks,
computational neuroscience, and other relevant topics, such as machine learning applied to
bioinformatics or cognitive science, which might be proposed by potential contributors.

PUBLISHED TITLES

BAYESIAN PROGRAMMING
Pierre Bessière, Emmanuel Mazer, Juan-Manuel Ahuactzin, and Kamel Mekhnacha
UTILITY-BASED LEARNING FROM DATA
Craig Friedman and Sven Sandow
HANDBOOK OF NATURAL LANGUAGE PROCESSING, SECOND EDITION
Nitin Indurkhya and Fred J. Damerau
COST-SENSITIVE MACHINE LEARNING
Balaji Krishnapuram, Shipeng Yu, and Bharat Rao
COMPUTATIONAL TRUST MODELS AND MACHINE LEARNING
Xin Liu, Anwitaman Datta, and Ee-Peng Lim
MULTILINEAR SUBSPACE LEARNING: DIMENSIONALITY REDUCTION OF
MULTIDIMENSIONAL DATA
Haiping Lu, Konstantinos N. Plataniotis, and Anastasios N. Venetsanopoulos
MACHINE LEARNING: An Algorithmic Perspective, Second Edition
Stephen Marsland
SPARSE MODELING: THEORY, ALGORITHMS, AND APPLICATIONS
Irina Rish and Genady Ya. Grabarnik
A FIRST COURSE IN MACHINE LEARNING, SECOND EDITION
Simon Rogers and Mark Girolami
INTRODUCTION TO MACHINE LEARNING WITH APPLICATIONS IN
INFORMATION SECURITY
Mark Stamp
 Chapman & Hall/CRC
Machine Learning & Pattern Recognition Series

INTRODUCTION TO

MACHINE
LEARNING with
APPLICATIONS
in INFORMATION
SECURITY

Mark Stamp
San Jose State University
California
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742

© 2018 by Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed on acid-free paper

International Standard Book Number-13: 978-1-138-62678-2 (Hardback)

This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize
to copyright holders if permission to publish in this form has not been obtained. If any copyright material
has not been acknowledged please write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information storage
or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access
www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc.
(CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization
that provides licenses and registration for a variety of users. For organizations that have been granted a
photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation without intent to infringe.

Visit the Taylor & Francis Web site at

http://www.taylorandfrancis.com

and the CRC Press Web site at

http://www.crcpress.com
To Melody, Austin, and Miles.
Contents

Preface xiii

About the Author xv

Acknowledgments xvii

1 Introduction 1
1.1 What Is Machine Learning? . . . . . . . . . . . . . . . . . . . 1
1.2 About This Book . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Necessary Background . . . . . . . . . . . . . . . . . . . . . . 4
1.4 A Few Too Many Notes . . . . . . . . . . . . . . . . . . . . . 4

I Tools of the Trade 5

2 A Revealing Introduction to Hidden Markov Models 7

2.1 Introduction and Background . . . . . . . . . . . . . . . . . . 7
2.2 A Simple Example . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 The Three Problems . . . . . . . . . . . . . . . . . . . . . . . 14
2.4.1 HMM Problem 1 . . . . . . . . . . . . . . . . . . . . . 14
2.4.2 HMM Problem 2 . . . . . . . . . . . . . . . . . . . . . 14
2.4.3 HMM Problem 3 . . . . . . . . . . . . . . . . . . . . . 14
2.4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5 The Three Solutions . . . . . . . . . . . . . . . . . . . . . . . 15
2.5.1 Solution to HMM Problem 1 . . . . . . . . . . . . . . 15
2.5.2 Solution to HMM Problem 2 . . . . . . . . . . . . . . 16
2.5.3 Solution to HMM Problem 3 . . . . . . . . . . . . . . 17
2.6 Dynamic Programming . . . . . . . . . . . . . . . . . . . . . 20
2.7 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.8 All Together Now . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.9 The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.10 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

vii
viii CONTENTS

3 A Full Frontal View of Profile Hidden Markov Models 37

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.2 Overview and Notation . . . . . . . . . . . . . . . . . . . . . 39
3.3 Pairwise Alignment . . . . . . . . . . . . . . . . . . . . . . . . 42
3.4 Multiple Sequence Alignment . . . . . . . . . . . . . . . . . . 46
3.5 PHMM from MSA . . . . . . . . . . . . . . . . . . . . . . . . 50
3.6 Scoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.7 The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4 Principal Components of Principal Component Analysis 63

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.2.1 A Brief Review of Linear Algebra . . . . . . . . . . . . 64
4.2.2 Geometric View of Eigenvectors . . . . . . . . . . . . 68
4.2.3 Covariance Matrix . . . . . . . . . . . . . . . . . . . . 70
4.3 Principal Component Analysis . . . . . . . . . . . . . . . . . 73
4.4 SVD Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.5 All Together Now . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.5.1 Training Phase . . . . . . . . . . . . . . . . . . . . . . 80
4.5.2 Scoring Phase . . . . . . . . . . . . . . . . . . . . . . . 82
4.6 A Numerical Example . . . . . . . . . . . . . . . . . . . . . . 83
4.7 The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

5 A Reassuring Introduction to Support Vector Machines 95

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.2 Constrained Optimization . . . . . . . . . . . . . . . . . . . . 102
5.2.1 Lagrange Multipliers . . . . . . . . . . . . . . . . . . . 104
5.2.2 Lagrangian Duality . . . . . . . . . . . . . . . . . . . . 108
5.3 A Closer Look at SVM . . . . . . . . . . . . . . . . . . . . . . 110
5.3.1 Training and Scoring . . . . . . . . . . . . . . . . . . . 112
5.3.2 Scoring Revisited . . . . . . . . . . . . . . . . . . . . . 114
5.3.3 Support Vectors . . . . . . . . . . . . . . . . . . . . . 115
5.3.4 Training and Scoring Re-revisited . . . . . . . . . . . . 116
5.3.5 The Kernel Trick . . . . . . . . . . . . . . . . . . . . . 117
5.4 All Together Now . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.5 A Note on Quadratic Programming . . . . . . . . . . . . . . . 121
5.6 The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
CONTENTS ix

6 A Comprehensible Collection of Clustering Concepts 133

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.2 Overview and Background . . . . . . . . . . . . . . . . . . . . 133
6.3 �-Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
6.4 Measuring Cluster Quality . . . . . . . . . . . . . . . . . . . . 141
6.4.1 Internal Validation . . . . . . . . . . . . . . . . . . . . 143
6.4.2 External Validation . . . . . . . . . . . . . . . . . . . 148
6.4.3 Visualizing Clusters . . . . . . . . . . . . . . . . . . . 150
6.5 EM Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . 151
6.5.1 Maximum Likelihood Estimator . . . . . . . . . . . . 154
6.5.2 An Easy EM Example . . . . . . . . . . . . . . . . . . 155
6.5.3 EM Algorithm . . . . . . . . . . . . . . . . . . . . . . 159
6.5.4 Gaussian Mixture Example . . . . . . . . . . . . . . . 163
6.6 The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . 170
6.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

7 Many Mini Topics 177

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
7.2 �-Nearest Neighbors . . . . . . . . . . . . . . . . . . . . . . . 177
7.3 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . 179
7.4 Boosting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
7.4.1 Football Analogy . . . . . . . . . . . . . . . . . . . . . 182
7.4.2 AdaBoost . . . . . . . . . . . . . . . . . . . . . . . . . 183
7.5 Random Forest . . . . . . . . . . . . . . . . . . . . . . . . . . 186
7.6 Linear Discriminant Analysis . . . . . . . . . . . . . . . . . . 192
7.7 Vector Quantization . . . . . . . . . . . . . . . . . . . . . . . 202
7.8 Naı̈ve Bayes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
7.9 Regression Analysis . . . . . . . . . . . . . . . . . . . . . . . 205
7.10 Conditional Random Fields . . . . . . . . . . . . . . . . . . . 208
7.10.1 Linear Chain CRF . . . . . . . . . . . . . . . . . . . . 209
7.10.2 Generative vs Discriminative Models . . . . . . . . . . 210
7.10.3 The Bottom Line on CRFs . . . . . . . . . . . . . . . 213
7.11 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

8 Data Analysis 219

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
8.2 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . 220
8.3 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
8.4 ROC Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
8.5 Imbalance Problem . . . . . . . . . . . . . . . . . . . . . . . . 228
8.6 PR Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
8.7 The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . 231
8.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
x CONTENTS

II Applications 235

9 HMM Applications 237

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
9.2 English Text Analysis . . . . . . . . . . . . . . . . . . . . . . 237
9.3 Detecting Undetectable Malware . . . . . . . . . . . . . . . . 240
9.3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . 240
9.3.2 Signature-Proof Metamorphic Generator . . . . . . . . 242
9.3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 243
9.4 Classic Cryptanalysis . . . . . . . . . . . . . . . . . . . . . . . 245
9.4.1 Jakobsen’s Algorithm . . . . . . . . . . . . . . . . . . 245
9.4.2 HMM with Random Restarts . . . . . . . . . . . . . . 251

10 PHMM Applications 261

10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
10.2 Masquerade Detection . . . . . . . . . . . . . . . . . . . . . . 261
10.2.1 Experiments with Schonlau Dataset . . . . . . . . . . 262
10.2.2 Simulated Data with Positional Information . . . . . . 265
10.3 Malware Detection . . . . . . . . . . . . . . . . . . . . . . . . 269
10.3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . 270
10.3.2 Datasets and Results . . . . . . . . . . . . . . . . . . . 271

11 PCA Applications 277

11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
11.2 Eigenfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
11.3 Eigenviruses . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
11.3.1 Malware Detection Results . . . . . . . . . . . . . . . 280
11.3.2 Compiler Experiments . . . . . . . . . . . . . . . . . . 282
11.4 Eigenspam . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
11.4.1 PCA for Image Spam Detection . . . . . . . . . . . . . 285
11.4.2 Detection Results . . . . . . . . . . . . . . . . . . . . . 285

12 SVM Applications 289

12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
12.2 Malware Detection . . . . . . . . . . . . . . . . . . . . . . . . 289
12.2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . 290
12.2.2 Experimental Results . . . . . . . . . . . . . . . . . . 293
12.3 Image Spam Revisited . . . . . . . . . . . . . . . . . . . . . . 296
12.3.1 SVM for Image Spam Detection . . . . . . . . . . . . 298
12.3.2 SVM Experiments . . . . . . . . . . . . . . . . . . . . 300
12.3.3 Improved Dataset . . . . . . . . . . . . . . . . . . . . 304
CONTENTS xi

13 Clustering Applications 307

13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
13.2 �-Means for Malware Classification . . . . . . . . . . . . . . 307
13.2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . 308
13.2.2 Experiments and Results . . . . . . . . . . . . . . . . 309
13.2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 313
13.3 EM vs �-Means for Malware Analysis . . . . . . . . . . . . . 314
13.3.1 Experiments and Results . . . . . . . . . . . . . . . . 314
13.3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 317

Annotated Bibliography 319

Index 338
Preface

“Perhaps it hasn’t one,” Alice ventured to remark.

“Tut, tut, child!” said the Duchess.
“Everything’s got a moral, if only you can find it.”
— Lewis Carroll, Alice in Wonderland

For the past several years, I’ve been teaching a class on “Topics in Information
Security.” Each time I taught this course, I’d sneak in a few more machine
learning topics. For the past couple of years, the class has been turned on
its head, with machine learning being the focus, and information security
only making its appearance in the applications. Unable to find a suitable
textbook, I wrote a manuscript, which slowly evolved into this book.
In my machine learning class, we spend about two weeks on each of the
major topics in this book (HMM, PHMM, PCA, SVM, and clustering). For
each of these topics, about one week is devoted to the technical details in
Part I, and another lecture or two is spent on the corresponding applica-
tions in Part II. The material in Part I is not easy—by including relevant
applications, the material is reinforced, and the pace is more reasonable.
I also spend a week covering the data analysis topics in Chapter 8 and
several of the mini topics in Chapter 7 are covered, based on time constraints
and student interest.1
Machine learning is an ideal subject for substantive projects. In topics
classes, I always require projects, which are usually completed by pairs of stu-
dents, although individual projects are allowed. At least one week is allocated
to student presentations of their project results.
A suggested syllabus is given in Table 1. This syllabus should leave time
for tests, project presentations, and selected special topics. Note that the
applications material in Part II is intermixed with the material in Part I.
Also note that the data analysis chapter is covered early, since it’s relevant
to all of the applications in Part II.
1
Who am I kidding? Topics are selected based on my interests, not student interest.

xiii
xiv PREFACE

Table 1: Suggested syllabus

Chapter Hours Coverage

1. Introduction 1 All
2. Hidden Markov Models 3 All
9. HMM Applications 2 All
8. Data Analysis 3 All
3. Profile Hidden Markov Models 3 All
10. PHMM Applications 2 All
4. Principal Component Analysis 3 All
11. PCA Applications 2 All
5. Support Vector Machines 3 All
12. SVM Applications 3 All
6. Clustering 3 All
13. Clustering Applications 2 All
7. Mini-topics 6 LDA and selected topics
Total 36

My machine learning class is taught at the beginning graduate level. For

an undergraduate class, it might be advisable to slow the pace slightly. Re-
gardless of the level, labs would likely be helpful. However, it’s important to
treat labs as supplemental to—as opposed to a substitute for—lectures.
Learning challenging technical material requires studying it multiple times
in multiple different ways, and I’d say that the magic number is three. It’s no
accident that students who read the book, attend the lectures, and conscien-
tiously work on homework problems learn this material well. If you are trying
to learn this subject on your own, the author has posted his lecture videos
online, and these might serve as a (very poor) substitute for live lectures.2
I’m also a big believer in learning by programming—the more code that you
write, the better you will learn machine learning.

Mark Stamp
Los Gatos, California
April, 2017

2
In my experience, in-person lectures are infinitely more valuable than any recorded or
online format. Something happens in live classes that will never be fully duplicated in any
dead (or even semi-dead) format.
About the Author

My work experience includes more than seven years at the National Security
Agency (NSA), which was followed by two years at a small Silicon Valley
startup company. Since 2002, I have been a card-carrying member of the
Computer Science faculty at San Jose State University (SJSU).
My love affair with machine learning began during the early 1990s, when
I was working at the NSA. In my current job at SJSU, I’ve supervised vast
numbers of master’s student projects, most of which involve some combination
of information security and machine learning. In recent years, students have
become even more eager to work on machine learning projects, which I would
like to ascribe to the quality of the book that you have before you and my
magnetic personality, but instead, it’s almost certainly a reflection of trends
in the job market.
I do have a life outside of work.3 Recently, kayak fishing and sailing my
Hobie kayak in the Monterey Bay have occupied most of my free time. I also
ride my mountain bike through the local hills and forests whenever possible.
In case you are a masochist, a more complete autobiography can be found at

http://www.sjsu.edu/people/mark.stamp/

If you have any comments or questions about this book (or anything else)
you can contact me via email at [email protected]. And if you happen
to be local, don’t hesitate to stop by my office to chat.

3
Of course, here I am assuming that what I do for a living could reasonably be classified
as work. My wife (among others) has been known to dispute that assumption.

xv
Acknowledgments

The first draft of this book was written while I was on sabbatical during the
spring 2014 semester. I first taught most of this material in the fall semester
of 2014, then again in fall 2015, and yet again in fall 2016. After the third
iteration, I was finally satisfied that the manuscript had the potential to be
book-worthy.
All of the students in these three classes deserve credit for helping to
improve the book to the point where it can now be displayed in public without
excessive fear of ridicule. Here, I’d like to single out the following students
for their contributions to the applications in Part II.

Topic Students
HMM Sujan Venkatachalam, Rohit Vobbilisetty
PHMM Lin Huang, Swapna Vemparala
PCA Ranjith Jidigam, Sayali Deshpande, Annapurna Annadatha
SVM Tanuvir Singh, Annapurna Annadatha
Clustering Chinmayee Annachhatre, Swathi Pai, Usha Narra

Extra special thanks go to Annapurna Annadatha and Fabio Di Troia.

In addition to her major contributions to two of the applications chapters,
Annapurna helped to improve the end-of-chapter exercises. Fabio assisted
with most of my recent students’ projects and he is a co-author on almost
all of my recent papers. I also want to thank Eric Filiol, who suggested
broadening the range of applications. This was excellent advice that greatly
improved the book.
Finally, I want to thank Randi Cohen and Veronica Rodriguez at the
Taylor & Francis Group. Without their help, encouragement, and patience,
this book would never have been published.
A textbook is like a large software project, in that it must contain bugs.
All errors in this book are solely the responsibility of your humble scribe.
Please send me any errors that you find, and I will keep an updated errata
list on the textbook website.

xvii
Chapter 1

Introduction

I took a speed reading course and read War and Peace in twenty minutes.
It involves Russia.
— Woody Allen

1.1 What Is Machine Learning?

For our purposes, we’ll view machine learning as a form of statistical discrim-
ination, where the “machine” does the heavy lifting. That is, the computer
“learns” important information, saving us humans from the hard work of
trying to extract useful information from seemingly inscrutable data.
For the applications considered in this book, we typically train a model,
then use the resulting model to score samples. If the score is sufficiently high,
we classify the sample as being of the same type as was used to train the
model. And thanks to the miracle of machine learning, we don’t have to
work too hard to perform such classification. Since the model parameters are
(more-or-less) automatically extracted from training data, machine learning
algorithms are sometimes said to be data driven.
Machine learning techniques can be successfully applied to a wide range
of important problems, including speech recognition, natural language pro-
cessing, bioinformatics, stock market analysis, information security, and the
homework problems in this book. Additional useful applications of machine
learning seem to be found on a daily basis—the set of potential applications
is virtually unlimited.
It’s possible to treat any machine learning algorithm as a black box and, in
fact, this is a major selling points of the field. Many successful machine learn-
ers simply feed data into their favorite machine learning black box, which,
surprisingly often, spits out useful results. While such an approach can work,

1
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
2 INTRODUCTION

the primary goal of this book is to provide the reader with a deeper un-
derstanding of what is actually happening inside those mysterious machine
learning black boxes.
Why should anyone care about the inner workings of machine learning al-
gorithms when a simple black box approach can—and often does—suffice? If
you are like your curious author, you hate black boxes, and you want to know
how and why things work as they do. But there are also practical reasons
for exploring the inner sanctum of machine learning. As with any technical
field, the cookbook approach to machine learning is inherently limited. When
applying machine learning to new and novel problems, it is often essential to
have an understanding of what is actually happening “under the covers.” In
addition to being the most interesting cases, such applications are also likely
to be the most lucrative.
By way of analogy, consider a medical doctor (MD) in comparison to a
nurse practitioner (NP).1 It is often claimed that an NP can do about 80%
to 90% of the work that an MD typically does. And the NP requires less
training, so when possible, it is cheaper to have NPs treat people. But, for
challenging or unusual or non-standard cases, the higher level of training of
an MD may be essential. So, the MD deals with the most challenging and
interesting cases, and earns significantly more for doing so. The aim of this
book is to enable the reader to earn the equivalent of an MD in machine
learning.
The bottom line is that the reader who masters the material in this book
will be well positioned to apply machine learning techniques to challenging
and cutting-edge applications. Most such applications would likely be beyond
the reach of anyone with a mere black box level of understanding.

1.2 About This Book

The focus of this book is on providing a reasonable level of detail for a reason-
ably wide variety of machine learning algorithms, while constantly reinforcing
the material with realistic applications. But, what constitutes a reasonable
level of detail? I’m glad you asked.
While the goal here is for the reader to obtain a deep understanding of
the inner workings of the algorithms, there are limits.2 This is not a math
book, so we don’t prove theorems or otherwise dwell on mathematical theory.
Although much of the underlying math is elegant and interesting, we don’t
spend any more time on the math than is absolutely necessary. And, we’ll
1
A physician assistant (PA) is another medical professional that is roughly comparable
to a nurse practitioner.
2
However, these limits are definitely not of the kind that one typically finds in a calculus
book.
1.2 ABOUT THIS BOOK 3

sometimes skip a few details, and on occasion, we might even be a little bit
sloppy with respect to mathematical niceties. The goal here is to present
topics at a fairly intuitive level, with (hopefully) just enough detail to clarify
the underlying concepts, but not so much detail as to become overwhelming
and bog down the presentation.3
In this book, the following machine learning topics are covered in chapter-
length detail.

Topic Where
Hidden Markov Models (HMM) Chapter 2
Profile Hidden Markov Models (PHMM) Chapter 3
Principal Component Analysis (PCA) Chapter 4
Support Vector Machines (SVM) Chapter 5
Clustering (�-Means and EM) Chapter 6

Several additional topics are discussed in a more abbreviated (section-length)

format. These mini-topics include the following.

Topic Where
�-Nearest Neighbors (�-NN) Section 7.2
Neural Networks Section 7.3
Boosting and AdaBoost Section 7.4
Random Forest Section 7.5
Linear Discriminant Analysis (LDA) Section 7.6
Vector Quantization (VQ) Section 7.7
Naı̈ve Bayes Section 7.8
Regression Analysis Section 7.9
Conditional Random Fields (CRF) Section 7.10

Data analysis is critically important when evaluating machine learning ap-

plications, yet this topic is often relegated to an afterthought. But that’s
not the case here, as we have an entire chapter devoted to data analysis and
related issues.
To access the textbook website, point your browser to

http://www.cs.sjsu.edu/~stamp/ML/

where you’ll find links to PowerPoint slides, lecture videos, and other relevant
material. An updated errata list is also available. And for the reader’s benefit,
all of the figures in this book are available in electronic form, and in color.
3
Admittedly, this is a delicate balance, and your unbalanced author is sure that he didn’t
always achieve an ideal compromise. But you can rest assured that it was not for lack of
trying.
4 INTRODUCTION

In addition, extensive malware and image spam datasets can be found on

the textbook website. These or similar datasets were used in many of the
applications discussed in Part II of this book.

1.3 Necessary Background

Given the title of this weighty tome, it should be no surprise that most of
the examples are drawn from the field of information security. For a solid
introduction to information security, your humble author is partial to the
book [137]. Many of the machine learning applications in this book are
specifically focused on malware. For a thorough—and thoroughly enjoyable—
introduction to malware, Aycock’s book [12] is the clear choice. However,
enough background is provided so that no outside resources should be neces-
sary to understand the applications considered here.
Many of the exercises in this book require some programming, and basic
computing concepts are assumed in a few of the application sections. But
anyone with a modest amount of programming experience should have no
trouble with this aspect of the book.
Most machine learning techniques do ultimately rest on some fancy math.
For example, hidden Markov models (HMM) build on a foundation of dis-
crete probability, principal component analysis (PCA) is based on sophisti-
cated linear algebra, Lagrange multipliers (and calculus) are used to show
how and why a support vector machine (SVM) really works, and statistical
concepts abound. We’ll review the necessary linear algebra, and generally
cover relevant math and statistics topics as needed. However, we do assume
some knowledge of differential calculus—specifically, finding the maximum
and minimum of “nice” functions.

1.4 A Few Too Many Notes

Note that the applications presented in this book are largely drawn from your
author’s industrious students’ research projects. Note also that the applica-
tions considered here were selected because they illustrate various machine
learning techniques in relatively straightforward scenarios. In particular, it is
important to note that applications were not selected because they necessarily
represent the greatest academic research in the history of academic research.
It’s a noteworthy (and unfortunate) fact of life that the primary function of
much academic research is to impress the researcher’s (few) friends with his
or her extreme cleverness, while eschewing practicality, utility, and clarity.
In contrast, the applications presented here are supposed to help demystify
machine learning techniques.
 Part I

Tools of the Trade

5
Random documents with unrelated
content Scribd suggests to you:
are not generally supposed to have composed the principal or most
numerous part of the invaders, yet it so happened, that the whole of
the conquered country and also the language of its new inhabitants
took their names from them. They took possession of the ancient
country, or province of the Iceni, and there founded the kingdom of
East Anglia, or of East Angles, comprehending the present counties
of Norfolk, Suffolk, Cambridge, and Huntingdon, which made some
figure among its sister kingdoms in the time of the Heptarchy. The
kingdom of Mercia and that of Northumberland also, it seems, were
inhabited by the same people.
Of Lynn, during that dark and disastrous period, no account has
been preserved. It was probably destroyed by those merciless
invaders, during their long and bloody contest with the ill fated
natives, along with many other towns, all over the country, which
certainly met the same fate. [236] At what time it revived, or rose
again into existence, is no where recorded. But from the
convenience and advantage of its situation it may be supposed to
have done so pretty soon after the government of the East-Angles
had assumed a settled form, and acquired a competent or tolerable
share of stability. That it existed under the East-Anglian kings,
seems a very natural and credible supposition; but whether it stood
then on the western side only, or on both sides of the river, cannot
now be ascertained. Under the Saxon princes that succeeded the
dissolution of the Heptarchy, it is well known to have extended to
the eastern shore of the river; and it is then, most probably, that we
are to date the origin of the present town or borough of Lynn. In
the time of Edward, called the confessor, we find it a place of trade
and considerable note; a plain proof that it must have been in being,
and growing into consequence a good while before that period. It
belonged then to Ailmar, bishop of Elmham, and his brother Stigand,
archbishop of Canterbury, when blind superstition and ecclesiastical
servility may naturally be supposed to have been among the
principal or most distinguished characteristics of its inhabitants. It
continued afterward under episcopal domination and ghostly
discipline till the memorable reign of Henry VIII. who thought proper
to take into his own hands that power or supremacy which was
before vested in the bishops. In consequence of which, it has ever
since been called King’s Lynn, instead of Bishop’s Lynn, which was its
former appellation: an appellation, by the bye, which will serve
further to corroborate the idea, that it was formerly the deleterious
abode of priest-ridden credulity and ecclesiastical thraldom. Indeed
it may be said to have been long distinguished for illiberality,
intolerance, and a persecuting spirit: and it must appear somewhat
remarkable, that the very first person taken up and burnt, in
England, under that diabolical law, De hæretico comburendo, was a
Lynn man, as was also the last, or one of the very last that
underwent persecution for nonconformity under the infamous
conventicle Act: The former was one of the preachers belonging to
St. Margaret’s Church, in the reign of Henry IV. and the latter a
licenced dissenting minister in that of William III. Of each of them a
more particular account shall be given in its proper place.
Not only Lynn, but most, if not all, of the adjacent towns and villages
appear to have been in being long before the conquest. They are
noticed in the celebrated old record, called Domesday, as places
then in existence, and seemingly of long standing and remote
origin. They had, in all probability, been erected and inhabited many
ages before that period, though it seems not likely that many of
their present names, or those given them in the Domesday book, are
to be traced to a British origin, as Parkin and others pretend. [238a]
That Lynn had become a place of considerable trade in the Saxon
times, or before the Norman invasion, is evident from
unquestionable existing documents. It had then a toll-booth, and
enjoyed certain duties and customs, payable on the arrival of any
goods or merchandise, of which the bishop was in full possession of
a moiety. This episcopal privilege is supposed to have been as early
as the conversion of the East Angles, and establishment of
Christianity among them. The town continued daily to flourish and
acquire increasing importance; and at an early period after the
conquest, one of the writers of that time calls it, “a noble city,” on
account of its trading and commercial magnificence. [238b] This was
at a period when Hull did not exist, and when Liverpool, if it did
exist, was but a very obscure and insignificant place.

Section III.

Of the Saltworks formerly at and about Lynn—paucity of

appropriate materials—apology.
The vicinity of Lynn in the Saxon times, and long after, appears to
have been remarkable for its numerous Saltworks. [239a] At Gaywood
alone, in the Confessor’s time, there were no less than thirty Salt
pans, or pits. [239b] The Salt springs of Droitwich, Nantwich,
Northwich, &c. were then, it seems, not so much attended to as to
afford a supply to the distant parts of the kingdom. The people of
these parts were therefore obliged to manufacture their own salt.
To what extent the work was carried on, or what quantity was
generally, or annually produced, cannot now be ascertained. Nor
are we informed of the particular mode, or process adopted and
pursued in carrying on this ancient manufactory. It was, probably,
pretty simple and not very unlike that used in latter times in the salt-
works of South-town, by Yarmouth, and at other places. By the
great number of Salt-works then at Lynn, or in its neighbourhood, it
seems probable, that a considerable part of the adjacent country,
and the interior districts, were supplied from thence with that
necessary article: which might easily, even at that early period, be
conveyed thither, by means of the inland navigation, which always
gave to Lynn the vast advantage of a free and easy intercourse with
all those places, however distant, that are situated near the banks,
or in the neighbourhood of its numerous rivers. The Salt
manufactured here was made, it seems, from the sea water which
the tides brought up to the town, and which must have been, of
course, much less salt, and less fit for the purpose than the water
found below in the roads, or at sea: it appears therefore rather odd,
that those salt-works should be placed so far up the country, or so
distant from the sea: and yet so it was; every village and hamlet,
almost, had then its Salt-work, or the moiety of one. [240] Here it
may be proper to observe, that, at the periods of which we have
been speaking, salt was not an article of revenue, and must
therefore have been a pretty cheap commodity compared to what it
is now, when the duty laid upon it by government is said to be
above ten times its prime, or original cost.
In attempting to give an account of the state of things at Lynn
during the period which we are now contemplating, almost all our
light must be borrowed from the general history of the kingdom in
the mean while, as the paucity of materials, relating particularly to
this town, leaves us, for the most part, no other clew for our
guidance. The reader must not therefore be displeased with the
method here generally pursued, in exhibiting the state or history of
Lynn under its East-Anglian and Anglo-Saxon sovereigns.

CHAP. III.
Of the religious profession of the first Anglian inhabitants of Lynn
—their renouncing heathenism, and assuming the christian name
—account of their conversion, and character of their Christianity.

Section I.

Heathenism the religion of the first inhabitants of this town after

its revival, or restoration, under the East-Anglian government—
they, and the rest of the East Angles, together with the other
branches of the Heptarchy, become professors of Christianity—
account of their conversion.
The inhabitants of Lynn, after it had been rebuilt and repeopled by
its Anglian masters, appear to have been blind heathens, and gross
idolaters; for when the Angles, or Anglo-Saxons seized upon this
country, and founded the East-Anglian kingdom, they were a nation
of pagans, worshippers of Thor and Woden, and the rest of the
miserable objects of northern, or Scandinavian adoration; and so
continued till the seventh century. At that period, one of their
princes, named Sigebert, having lived sometime in exile among the
Francs, was there converted to Christianity. At his restoration to his
kingdom, he brought over with him one Felix, a Burgundian priest,
who was employed in recommending to the people the religion of
their sovereign, in which he appears not to have been unsuccessful.
He was consequently appointed the first bishop of the East Angles,
and had his see fixed at Soham, [242] in Cambridgeshire, and
afterwards at Domnoc, or Dunwich, in Suffolk. He is said to have
taken no small pains in promoting the conversion of the inhabitants;
and the parts about Lynn seem to have engaged a considerable
share of his attention. In these very parts he is reported to have
commenced his labours, which issued in the conversion of the whole
country. Tradition gives to Babingley, by Lynn, the honour of being
the place where he first landed, and where was erected the very first
christian chapel, or place of worship among the East Angles. The
second edifice of the same description is said to have been erected
at Sharnborn, in the same neighbourhood. At what time the first
place of that sort was built at Lynn, cannot now be determined; but
it seems very probable that it must have been as early, at least, as
the middle part of the seventh century. It cannot, however be said,
that the Christianity then introduced was of any great value. The
national character was not much, if at all, mended by it; and the
people still remained grossly ignorant, profligate, and savage. What
they wanted in rational piety and real Christianity, they made up in
stupid credulity, blind zeal, and miserable superstition; and it had
been well if their descendants had always carefully avoided the
imitation of their wretched and pernicious example.
It is somewhat remarkable that Christianity, as it was called, was not
received among the East-Angles till it had made considerable
progress in most of the other kingdoms of the Heptarchy. In Kent it
had been received about the year 526, or soon after, by the ministry
of Austin the monk: and even before that time, several years, some
of the Kentish people had been brought to think favourably of that
religion, by the means of Luidhart, a French bishop, who had
accompanied the princess Birtha, daughter of Cherebert, king of
Paris, upon her marriage with Ethelbert the Kentish king. The
conversion of the East Saxons took place about the beginning of the
seventh Century under the ministry of Mellitus, their first bishop: and
soon after, that of the Northumbrians, where Paulinus appeared as a
very active and successful labourer.
Felix did not begin his labours among the East Angles till about the
year 630, when that religion had made some progress in all the
other kingdoms, perhaps, except that of Mercia, which seems to
have been the last of the seven to adhere to the profession of
paganism. The Mercians, however, were afterwards converted, and
their country, at one time, formed into an archiepiscopal province,
whose seat or metropolis was Litchfield. Thus the different branches
of the hierarchy were all, by degrees, nominally christianized. Of the
nature, character, and value of that Christianity, a just and proper
idea may be formed from the following representations.

Section. II.

Effects of the conversion of the East-Angles, and the other sister-

kingdoms—character of their Christianity.
No sooner were the good people of this country converted from
paganism than monkery began to be in great request among them.
Many monasteries were accordingly founded in all parts of England,
which were quickly crowded with inhabitants. A fondness for the
monastic life is said to have been here very much increased by an
impious doctrine which began to be broached and believed toward
the close of the seventh century, “That as soon as any person put on
the habit of a monk, all the sins of his former life were forgiven.”
This is said to have engaged many princes and great men (who are
sometimes as great sinners as their inferiors) to put on the monkish
habit, and end their days in monasteries; which, whatever it might
be to themselves, was, probably, no mighty loss to their subjects
and vassals, or to the world.
Another remarkable feature in the character of the English christians
of the seventh and following centuries, was an extravagant
veneration for relics; in which the Romish priests drove then a very
gainful trade, as few good christians thought themselves safe from
the perils or disasters of life, and the machinations of the devil,
unless they carried about them the relics of some paint: nor could
any church be dedicated without a decent or certain quantity of the
same sacred and precious ware. [244]
Stories of dreams, visions, and miracles were also propagated
without a blush, by the clergy, and believed without a doubt, by the
laity. Extraordinary watchings, fastings, and other arts of tormenting
the body in order to save the soul, became frequent and
fashionable; and it began to be believed in the seventh century, that
a journey to Rome was a most meritorious undertaking, and even, of
all others, the most direct road to heaven.—Such was the christianity
of the good people of Lynn and the rest of their countrymen in those
days.
In the eighth century the humour of making pilgrimages to Rome,
and of retiring into monasteries, still increasing, Coinred king of
Mercia, as Henry informs us, laid down his sceptre, and took up the
pilgrim’s staff, in 709, and travelled to Rome, accompanied by Offa, a
young prince of the royal family of the East-Saxons, where they both
became monks. Not long after, Ina, the warlike and victorious king
of the West Saxons imitated their example, and ended his days in a
cloister at Rome, where he founded a house for the entertainment of
English pilgrims, and the education of English youth.
Great numbers of nuns also, and other English women, were among
those devout pilgrims who then travelled to Rome: but we have the
testimony of Winfred, alias Boniface, archbishop of Mentz, who lived
at that period, that they were generally debauched before they
returned, and even that many of them became common prostitutes
in the cities of France and Italy: he therefore exhorts and charges
his friend Cuthbert, then archbishop of Canterbury, to put a speedy
stop to these scandalous female pilgrimages.
The religious prayers and songs, which constituted the church
service, were then all in Latin, and so not understood by the
common people, who were therefore directed by the 27th. canon of
the first council of Cloveshoos, or Cliff, in Kent, [246] held in 747, to
affix any meaning they pleased to the words in their own minds, and
to pray in their hearts for any or every thing they wanted, no matter
how foreign to the real sense of the public prayers. A curious salvo,
says Henry, for the absurd practice of praying in an unknown
tongue. The same canon contains also the following short form of
prayer, for the dead; “Lord, according to the greatness of thy mercy,
grant rest to his soul, and for thy infinite pity, vouchsafe to him the
joys of eternal light with thy saints.”
Some of the nobility, or great men of that time, not very fond, it
seems, of going themselves through all the fastings and prayers
enjoined them by their confessors, would fain be allowed to have the
service performed by proxy: and it appears that they actually hired
and employed poor people to fast and pray in their stead. It was,
certainly, a very notable as well as convenient device, and became, it
seems, pretty fashionable, though it is said not to have the good
fortune of obtaining the approbation of the said council of
Cloveshoos. [247a]
A late historian, [247b] alluding to this period, observes, “that long
fasting was then ordered frequently; but as the wealthy might
abstain by proxy, a seven years fast might be performed in three
days, if the principal could prevail with 840 persons each to take his
share. This concise plan of atonement for crimes, (he adds) was
condemned solemnly at the council of Cloveshoos; but the decree
was disregarded.” The practice therefore seems to have gone on
unchecked, notwithstanding the decree of that council. It was not to
be expected that so convenient a custom could be very easily
abolished. It may be worth while to inquire, whether it had in it,
after all, any greater absurdity, than there is in the present practice
of the infants at the font, making a confession of their faith by
proxy.
Towards the latter part of the above period, a law was enacted,
enjoining every priest to learn some handycraft, or manual
occupation: which might be very useful; at least, it could do no
harm. Another law enjoined, that they should all be capable of
repeating the Creed, and the Lord’s prayer: which also might be very
right and proper; but it indicates that learning was then at a very
low ebb among the English clergy. We learn, however, that in the
reign of Ethelred, styled the unready, a mission was sent from hence
to Norway, at the request of the king of that country, to convert the
Norwegians and Swedes to the christian religion, and that the
archbishop of York, and other divines, actually went over on that
occasion, and met with great success; though some of them
afterwards are said to have suffered martyrdom; which seems rather
odd, if the king, as above suggested, was their patron. Whatever
their learning might be, their zeal must have been highly
commendable and exemplary.

Section III.

Christianity of the ancient inhabitants of Lynn, and of this

country, further characterized—whether very materially improved
during the reign of Alfred—remarks on that reign—papal
instructions to the first missionaries.
Ignorance and superstition, instead of diminishing, appear to have
increased in England, during the eighth century. Pilgrimages to
Rome became far more frequent, and were attended with worse
effects than formerly; the rage of retiring to monasteries became
more violent in persons of all ranks, to the ruin of every useful art;
the clergy became more knavish and rapacious, and the laity more
abject and stupid, than at any former period: of which the trade of
relics, then at its height, and which can never be carried on, but
between knaves and fools, is a sufficient evidence.
During the memorable reign of the celebrated Alfred, the state of
religion has been supposed to have improved; but how far we are
warranted to admit, or carry that idea, does not seem very clear.
Alfred was, doubtless, a most excellent prince, as may fairly and
justly be inferred from that notable clause in his Will, “that the
English had an undoubted right to be free as their own thoughts,”
and particularly from his so greatly magnifying, and acting upon that
never to be forgotten precept of Christ, “to do unto others as we
would have others do unto us;” not to mention the many other
notable and commendable deeds ascribed to him: [249] but that he
was instrumental in very materially reforming, or improving the
religion of his country, appears rather doubtful, if not improbable.
His altering the Ten Commandments, leaving out the second, and
adding another, to humour the worshippers of images, make very
considerably against the notion of his having much advanced the
work of religious reformation. The commandment which he added,
was expressed in these words, “Make not thou gods of gold, or of
silver,” a precept, as it has been observed, which few of his subjects
could afford to transgress. [250]
From Alfred’s days to the conquest, the religion of England
experienced no amendment; nor ever after till the Era of the
reformation, or the 16th. century, except what took place under the
influence of Wickliff, and the Lollards; but they were soon crushed
under the heavy and strong hand of priestly and royal persecution:
the very first victim, as was before observed, was a Lynn man.
From the above representation of the original christianity of the
English nation, and of the Lynn people among the rest, one cannot
be very much biassed in its favour. But we shall cease to wonder at
its being no better, when we consider whence it proceeded, and
under what sort of rules or maxims it was introduced. It was first
brought hither and promulgated by Austin, a monk of the convent of
St. Andrew’s at Rome, accompanied by forty other Romish monks, all
sent by the then Pope, whose name was Gregory, commonly called
Gregory the first, and Saint Gregory, who was advanced to the papal
chair in 590. Austin and his companions arrived here in 596.
Among the instruction which pope Gregory gave Austin for the
regulation of his conduct and ministry, the following are not the least
remarkable.—

“He was not to destroy the heathen temples of the English, but
only to remove the images of their gods, to wash the wails with
holy water, to erect altars, and deposit relics in them, and so
convert them into christian churches; not only to save the
expence of building new ones, but that the people might more
easily be prevailed upon to frequent those places of worship, to
which they had been accustomed. He directs him further, to
accommodate the ceremonies of the christian worship as much
as possible to those of the heathen, that the people might not
be much startled at the change; and in particular he advises
him to allow the christian converts, on certain festivals, to kill
and eat a great number of oxen, to the glory of God, as they
had done formerly to the honour of the devil.” [252]

These admonitions, (says Dr. Henry) which were but too well
observed, introduced the grossest corruption into the christian
worship, and shew how much the apostles of the sixth and seventh
centuries had departed from the simplicity and sincerity of those of
the first.
 CHAP. IV.
Miscellaneous observations, on the social distinctions, and the
general state of the community among the Anglo-Saxons.

Section I.

State of society at Lynn, and in this country, before the

Conquest.
Of the state of Society in this country, as to the different ranks
among the inhabitants, the following is thought to be a pretty fair
and true representation.—The next rank below that of the royal
family was held by the Thanes, which were, it seems, of different
degrees, and we are told that the highest order among them went
by the name of king’s thanes. [253] These also are said to have been
of two different sorts, Eoldermen, and Eorles; the former supposed
to be supreme in the administration of justice; the latter
comprehended military as well as civil authority, but was not
hereditary till the close of the Saxon dynasty. The Ceorles (Churles,
or Carles) were next below the thanes. They were free, descended
from yeomen, and were chiefly engaged in husbandry. To them the
gate of nobility was open, and they might become thanes by five
different methods. [254a] Another order, or description of men, in
those times, was called Huscarles; (i.e. house-carles:) they were
retainers, or domestic dependents of the thanes, and reputed
freemen. All the rest of the community, it seems, were slaves, of
different descriptions. Of them however, it would sometimes
happen, though but very rarely, that some obtained manumission,
and they formed a particular class, denominated Freed-men; but we
are told that they were few, and little regarded. They could obtain,
it seems, no rank in the state; and applying, for the most part, to
mechanical employments, seem hardly to have been distinguished
from the race which they had quitted.—Slaves were never suffered
to carry arms, and the very gift of a weapon conferred freedom. Of
the other orders no man went abroad without his spear; and laws
were enacted to guard against damages occasioned by the careless
bearer. In battle the ceorles who formed the infantry, beside a
broad sword, and sometimes a club, bore only a round shield with
an offensive pointed weapon in the centre. The cavalry being
composed of thanes, huscarles, and the richer ceorles, who could
afford to keep horses, was better provided with defensive armour.
[254b]

The enslaved part of the inhabitants, and which is said to have

constituted by far the most numerous class of the community, went,
like the privileged orders, by different names. Of them the chief and
most remarkably were the Villani and the Servi. In regard to the
former, we are told that Villenage was of two kinds: 1st Pure
Villenage, to which some were subject from their birth, from whom
uncertain and indeterminate service was due to the lord. The
successors of these are our copy-holders; who, though time has
dealt favourably with them in other respects, still retain one mark of
their original vassalage, or servitude; for as of old the former were
not reckoned as members of the commonwealth, but merely as part
and parcel of the owner’s substance, no way entitled to the
privileges of freemen, so do their successors still continue without
any right to vote at elections by virtue of their copy-holds. 2nd
Villenage by Tenure, which bound the tenant to perform certain
services agreed upon between him and his lord; such as ploughing
his ground, reaping his corn, &c.
The lowest, as well as most numerous class of slaves among the
Anglo-Saxons were the Servi, who, (as well as all the rest of the
unfreemen,) were probably the descendants of the aboriginal
inhabitants, who had escaped the general massacre, or whose lives
had been spared at the reduction and conquest of the country.
These were protected by neither law nor religion, for a very long
period; and they consequently suffered the most unfeeling and cruel
treatment. Christianity is said to have ameliorated their condition.
It certainly ought to have done so; but from such a christianity no
material amelioration could well have been expected. Even our own
boasted protestantism, how feeble has generally been its influence
in such cases! The long and bloody contest between the rival and
barbarous houses of York and Lancaster did more, it seems, for the
relief and emancipation of those poor English slaves than any thing
else; for the contending parties, in order to recruit and reinforce
their armies, found it convenient and necessary to liberate great
numbers of them: at length they were all manumitted, and Britain
now contains no people of that description. It would be well if the
same could be said of every other, part of the British dominions.
These Servi are often mentioned by Bloomfield and Parkin, and
appear to have been very numerous in the parts about Lynn before
the conquest, and even long after that period.
Beside the Villani and Servi, we meet with other descriptions of
bondmen, whose condition seem to have been less abject; at least,
less so than the latter. Of those one sort was called Bordarii,
Bordars, or Borderers: they were such as held a cottage, or some
small parcel of land, on condition of supplying the lord with poultry,
eggs, and other small provisions for his board and entertainment.
Such small estates were formerly called Bord-lands, now demesnes.
—Coliberti was the name of another description of bondmen among
the Anglo-Saxons; and they were, it seems, a middle sort of tenants,
between servile and free; they had their patrons, to whom they paid
rent, and were manumised as servants used to be, but were not
absolutely free. They were such of the Servi as were enfranchised,
or liberated in a certain degree, but still paid some duties to the
superior lord. They appear to have been held in scarcely any higher
estimation than the class to which they had originally belonged.—
The Burgaris, Burgenses, or Burgesses, was another order of
bondmen among the Anglo-Saxons. They were tradesmen in great
towns, and had their patrons, under whose protection they traded,
and to whom they paid an acknowledgement; but some of them
were in a more servile condition, and altogether under the power of
the king, or other lords. [257a] Some of the above descriptions of
Anglo-Saxon bondmen, or British slaves, and particularly the Villani,
the Servi, and the Bordarii, were very numerous in the parts about
Lynn, in the times of which we are now speaking, and long after.
[257b]
England did then so abound in slaves, and was so much a land
of slaves, as to be able to carry on a trade in that commodity with
other nations, and especially the Irish, whom English merchants, for
a long time, abundantly supplied with that favourite article, out of
their home stock, or native produce, with as little shame or remorse
as they have in modern times supplied the West Indies and North
America with the poor defenceless natives of ill-fated Africa. [257c]
And yet the English was then, as well as now, a christian nation,
priding herself in the fond idea of the purity and pre-eminence of her
goodness, faith, and piety. Alas, for poor christianity! How often
hath her venerable name been profaned and postituted on the vilest
occasions, and for the basest of purposes!

Section II.

Of the Wittenagemote and other courts, maxims of jurisprudence,

institution of tythings, nuptial and funeral rites, sacerdotal,
domestic, and other customs, among the Anglo-Saxons.
The Wittenagemote, or assembly of wise men, was the highest court
among the Anglo-Saxons, and from which our parliament seems to
have originated. Bishops, judges, and thanes composed it, and it
does not appear that the lower orders, or bulk of the people were
there any how represented. The business of this assembly was
prepared and opened by the king.—Another high court, but inferior
to the former, was the Shiregemote, in which much business was
transacted in the way of a modern assize. The Eolderman, or the
Eorle was the president, and the domesmen, or judges, with certain
lawmen, as they were called, formed the bench. It was held only
twice in the year.—The Hundred Court came next, over which the
Hundredary presided. Sales of estates, registering of Wills,
manumission of slaves, &c. were here transacted. It was also called
Wapontake, from the custom of always attending well armed. It was
the repository of deeds and records, was held monthly, and had the
jurisdiction of ten tythings.
Compensation to the injured party or his family, rather than the
annoyance of the criminal, being the principle of the Anglo-Saxon
jurisprudence, capital punishments were unfrequent. The chastity of
maidens was protected by very severe laws; the ravisher of a nun
was fined as an assassin, and the violator of a child incurred the
penalty of a severe mutilation. Murder, as well as manslaughter,
might be atoned for, at a stated price: every wound had its exact
value; robbery was venial, and when committed on a bordering
country, (although in peace) was almost deemed laudable. [259]
It was in the institution of Tythings, or neighbourships, that the
wisdom of the Anglo-Saxons appeared most conspicuous and
admirable. Every ten families were connected together, as fellows in
arms and in civil society. Each answered for the others’ good
behaviour to the magistrate, and each joined in paying the penalty
which any one member might casually incur. A man who was not
inrolled in these tythings was avoided by all, as a vagabond and
person of bad character; nor could he hope to be admitted to a
tything unless his probity was generally acknowledged. To Alfred
this excellent institution is said to owe its perfection; and its effects
on society must have been very great and salutary. [260]
Of the customs of the ancient inhabitants of Lynn, and of the Anglo-
Saxons in general, relating to matrimony, the following appear to
have been some of the most remarkable and striking.—Every
unmarried woman was supposed to have a guardian, or owner: the
virgin belonged to her father, brother, uncle, or nearest male
relation; the widow claimed the same protection from her husband’s
male relatives; the lover was obliged to buy his mistress, of her
guardian, by a gift, the amount of which was settled by a law, that
set a higher price on the maid, by one half, than on the relict. If
unadvisedly the wooer wedded the lady without the guardian’s
consent, her property and goods were still the property of that
guardian, and an injury offered to her was to be atoned for to him,
and not to the spouse. At the wedding, the guardian delivered up
his ward to the husband, a friend of whom had previously avowed
himself the guarantee of a proper provision for the bride in case of
his death. At the feast which followed, the usual and large presents
of gold, silver, arms, cloths, household stuff, &c. made by the invited
friends and relations, formed the portion of the bride, who had
beside, from custom immemorial, a right to ask of her mate, on the
next sun-rising after her nuptials, a gift, to serve her as pin-money.
As to what related to divorces, among these people, we meet with
no particular account. In the education of their children, they only
sought to render them dauntless, and apt for the two most
important occupations of their future lives, war and the chase. It
was a usual trial of a child’s courage to place him on the sloping roof
of a building, and if, without screaming or apparent terror, he held
fast, it was deemed a favourable omen, and he was pronounced a
brave boy.
The burial ceremony is said to have been much more joyous among
them than that of marriage; which seems to imply something very
unnatural, as well as barbarous. The house in which the body lay till
its burial, was a perpetual scene of feasting, singing, dancing, and
every species of riot. This, of course, was very expensive to the
family of the deceased; and it was in some places carried so far, that
the corpse was forcibly kept unburied by the visiting friends, till they
were certain that they had consumed, in games and frantic festivity,
all the wealth the deceased had left behind him.—Nothing can well
exceed the barbarism and brutishness of such a custom: and yet it
seems to have long continued in some parts of this island after the
introduction of christianity, and even of protestantism. Nay, some
remains of it are known to have existed in some places within the
memory of some of the present inhabitants. It is surprising how
tenacious mankind often are of their ancient customs, be they ever
so vile, unseemly, or heathenish. Heathenish, certainly, or of pagan
origin, must this most odious practice have been: but it is not the
only English custom that comes under that description. The Waites,
that usually go about before christmas, may be considered as of the
self-same origin, and belonged, in all likelihood, to a certain pagan,
riotous, and lawless feast, celebrated at that time of the year: the
precursor and prototype of our principal christian festival. The
ushering in of May with the blowing of horns, a custom now almost,
if not altogether peculiar to Lynn, seems evidently to be of the same
class. It is still most tenaciously kept up in this town, by the boys
and children, though nobody pretends to know either its meaning or
its origin. But as May-Day is known to have been one of the highest
and most notable days of the year among our heathen ancestors,
the said custom may very safely be concluded to have originated
with them; especially as that day does not appear to have ever been
very much thought of by the papists.
A notable custom among the Anglo-Saxon christians of the eighth
century, and from which Lynn cannot be supposed exempted, was
that of the Clergy usually celebrating Mass, or administering the
Sacrament without Shoes and Stockings, and with chalices made of
horn: which seems to shew, that they had not then arrived at that
sacerdotal pride and pomp, at least in regard to their dress, which
became so prevalent among those of their order in latter times,
when they thought proper to assume a consequence so far above
the other orders of the community.
In private life the Anglo-Saxons are said to have been devout to the
extreme of credulity, and hospitable to drunken extravagance. Their
manners were rough, but social; their tables were plainly, but
plentifully served. Large joints of roasted meat seem to have had
the preference; salted victuals also were much in use. At table, the
rank of the guests was strictly observed; and, by the laws of Canute,
a person sitting above his proper station was to be pelted out of his
place with bones, at the discretion of the company, without the
privilege of taking offence. The lady, (or, as the Saxons named her,
leaf-dien, the bread giver) sat, as now, at the upper end of the
table, and distributed the provisions to her guests. The liquors used
at genteel tables were wine, ale, and spiced ale, pigment (a
composition of wine, spice, and honey,) morat, (honey diluted with
mulberry juice) and mead. [263] Such, as may reasonably be
concluded, was the state of things with regard to these matters, in
the best of the families of Lynn at those times.

Section III.

State of learning, and of the medical profession, among the

Anglo-Saxons.
Learning during the time now alluded to was at a very low ebb in
this Country. “Among the various discouragements, (says Andrews)
which literature was obliged to encounter in this ill-fated period may
be reckoned the extreme scarcity of materials for writing.” A strong
proof of which (he adds) “is that many of the MSS. of the 10th and
11th centuries are written on parchment, from which older works
(perhaps the Decades of Livy) have been erased.” It was for want of
parchment to draw the deeds upon, (as he supposes,) that estates,
were then frequently conveyed from one family to another by the
ceremony of a turf and a stone, delivered before witnesses, without
any written agreement. However that was, England even in those
dark times, exhibited some rays of intellectual light, and produced
some literary characters that would have done honour to more
enlightened ages. Bede, in particular, styled the Venerable, who
flourished in the 8th century, and has been called, the wise Saxon, is
believed to have comprised, in eight folio volumes, the whole body
of knowledge that his age afforded. To him may be added Egbert,
arch-bishop of York, and his pupil Alcuin, both distinguished in their
day for extraordinary literary attainments. Alfred and his learned
associates appeared in the 9th century, and were the ornaments of
that dark age; but the light which they exhibited was not lasting,
and they left no successors that were any way worthy of them. In
the 10th (and most part of the 11th) century, scarce any man of
literature appeared among the English. Elfric is said to have been by
far the most remarkable and eminent. He was styled the
grammarian, from his having written a Latin grammar. Two volumes
of homilies, in MS. translated by him from the Latin into the Saxon
language, are said to be still extant. Very few beside have in any
degree contributed to illuminate the gloom of that dismal period.
Gerbert, however, who, from a low origin, was advanced to the
papal chair in 999, under the name of Silvester II. deserves to be
respectfully noticed, as it is to his experience, gained by travel, and
a long residence among foreign nations, that our arithmetic is said to
owe the use of the Saracen numerals.—But as none of these persons
appear to have sprung from Lynn, or its vicinity, no further notice
can properly be taken of them in this work; and what has been
already said of them and other extraneous matters, was chiefly
intended for the purpose of pointing out the probable state of things
at Lynn in the meantime, for want of more suitable and appropriate
materials.
Physic and Surgery, during those early ages, were in a most wretched
state in this country, and, of course, among the inhabitants of Lynn.
Old women were then the chief professors of the medical art; and as
they mingled charms and spells with their prescriptions, the patient’s
fancy sometimes effected, or, at least, assisted in effecting the cure.
As Christianity gained ground, the clergy, having much time on their
hands, applied themselves to the study of medicine, but made so
little progress, that for a long time, Holy Water seems to have been
the prescription to which they chiefly trusted. If holy water were still
in use, as a popular, fashionable, or favourite medical prescription,
instead of the innumerable patent medicines, and other vile
quackeries that now disgrace this ill-fated country, it had been
better, no doubt, for the health and constitutions of myriads of our
unwary and credulous fellow-subjects. There seems, however, but
little prospect of an end to this great and growing evil, while
quackery continues to be so convenient and gainful to the state, or
to contribute so largely, as it now does, to the revenue of the
kingdom. But it is not the only public evil, the prospect of whose
extinction appears very distant and hopeless.

Section IV.
 Expressive and remarkable names of the months—state of the
coinage, or currency—general value of different commodities in
this country before the conquest—slavery—comparison with the
present course of things.
The inhabitants of Lynn and the rest of their countrymen, in the
Anglo-Saxon ages, could give more satisfactory reasons, it seems,
for the names of their months, than we can for those of ours.
December, which with them stood first, was called Midwinter-
monath, the midwinter month. January, was denominated Aefter-
yula, that is, after Christmas, or rather, after the feast called Yula, a
pagan, riotous, lawless festival, observed at that time of the year,
and to which our Christmas succeeded, with no small resemblance.
February, they called Sol-monath, the sun month, from the returning
of the sun at that season. March, they named Rhede, or Reth-
monath, the rough, or rugged month. April’s name was Easter-
monath, from a favourite Saxon goddess, whose festival was kept at
that time, and may be said to be still kept by us, under the idea of
the christian passover, which we seem to have dedicated to that
same pagan goddess, by our continuing to preserve her precious
memory, and celebrating the feast still in her name. May was called
Trimilchi, from the cows being then milked three times in the day.
June’s name was Seremonath, the dry month, July was called Mœd-
monath, the mead month, from the meads being then in their bloom
and beauty, or the people being there employed in hay making.
August had the name of Weod-monath, the weed month, from the
luxuriance, or abundance of weeds at that time. September was
named Hærfest-monath, or the harvest month. October bore the
name of Winter-fyllith, or winterfall, from winter approaching with
the full moon of that month. November, their last month, they called
Blot-monath, blood month, from the blood of the cattle then slain
and stored for winter provision.
The Anglo-Saxons are said to have made use of coins as early as the
reign of Ethelbright, or Ethelbert, who governed Kent from 561 to
616; as the fines ordered in his laws are all estimated by shillings,
which was even then a denomination of money. The money-pound
of the Anglo-Saxons, is thought to have been the same with the
Tower-pound long in use at the mint, and to have weighed less than
the Troy-pound by ¾ of a Troy-ounce. Its value was about 2l. 16s.
3d. of modern money. [268a] The Mark, like the Pound an imaginary
coin, weighed eight ounces, or two thirds of the Pound. The
merchant reckoned 12 ounces to the mark. Its value was 1l. 17s.
9d. The Mancus, a real coin, was valued at the 8th. of a mark, or
4s. and 8d. The Shilling, a real coin, was worth about eleven pence
farthing of our money. The Anglo-Saxon penny, (pening, or sceata,)
was a silver coin, and weighed near three-pence of our money. This
little piece would do more in those times for its owner, than some
shillings would do now. Halflings and Feorthlings, were the half, and
the fourth, or quarter of the Anglo-Saxon penny, and were of silver.
To these may be added a small brass coin called Styca. Beside these
coins, it was usual with the Anglo-Saxons to complete the sum
destined for any particular purpose, by adding what they called live
money, such as oxen, sheep, horses, or slaves; [268b] which last
species of traffick was carried to an almost incredible height of
brutality.
The value or price of cattle, land, and other commodities, in the
times of which we are now speaking, amounted to but a very small
portion of what they now fetch.

“By the laws of Athelstan, (says Dr. Mavor) a sheep was valued
at a shilling, or fifteen-pence of our money: an ox was
computed at six times the value of a sheep, and a cow at four.
A horse was valued at thirty shillings of our money, and a mare
at twenty-four. Between the years 900 and 1000, a hide of land
was purchased for about one hundred and eighteen shillings,
which was little more than a shilling per acre. [269] On the
whole, (he adds) when we combine the alteration in the weight
of the pound, and the modern value of the precious metals from
their greater plenty, we may conceive every sum of money
mentioned by historians, during the Anglo-Saxon, and even the
 Norman times, as if it were multiplied more than a hundred-fold
above a sum of the same denomination at present.”

Section V.

Probability that Lynn was formerly concerned in the exportation

of slaves—comparison between the ancient and modern English
slave-dealers—slaves and horses the chief exports of this country
in those days—corn not then exported, though it had been
formerly—imports—commerce—miscellaneous hints and
observations.
Considering how very fruitful in slaves England appears to have been
under the Anglo-Saxons, and how commonly they bought and sold
their slaves, (and even their own kindred,) and that they were
actually a principal article of their exports to other countries, [270] it is
more than probable that Lynn and other Norfolk ports were then
deeply concerned in that traffic.—Slaves are known to have then
abounded in the parts about this town; and no other commodity, or
produce of the country, was more marketable, or saleable, both at
home and abroad; we may therefore be sure that the merchants and
opulent people of Lynn were not inattentive to so fashionable and
profitable a branch of commerce. Some indeed, even then,
disapproved of it, and a bishop, of the name of Wolfstan, is said to
have firmly set his face against it at Bristol, and to have made the
people somewhat ashamed of their proceedings; but it does not
appear that they relinquished it, for Bristol continued to be the chief
English mart for slaves, long after his time. His conduct, however,
was highly laudable, memorable, and exemplary; but where among
our modern prelates, can we find one that has virtue or fortitude
enough to imitate the noble example of Wolfstan!
How vile and mercenary must the character of those ancient English
dealers in human flesh appear, when we contemplate them as selling
their own countrymen and neighbours, and even their kindred! It
reminds us of what has often been said of the modern commercial,
or mercantile character: that a merchant would sell his own father, if
he could do it safely and gainfully.—Between our ancient and
modern English slave dealers, there is some dissimilarity, though
they both acted from the same principle, and the conduct of each
appears thoroughly unjustifiable and atrocious.—The latter dealt only
in strangers, at a great distance, and of another colour; but the
former trafficked, as was before observed, in their own countrymen,
and near neighbours, brought up among them, and, occasionally,
even in their own near relations. Of the two, therefore, the conduct
of the ancients appears, at first sight, as far the most unnatural and
stocking; but that will cease to be the case, upon further
consideration, and when times and circumstances are duly attended
to. Those ancients lived in rude and barbarous ages, when the
natural rights of man were not understood, and when darkness
visible was every where predominant; which must, in some measure,
extenuate their misdeeds. But our modern slave-dealers have
carried on their operations in the open day, and in the very face of
the sun—they have adhered to this most barbarous and savage
traffick in the most enlightened age of the world—they have
persisted in it, in spite of the frequent and solemn remonstrances of
the most virtuous and enlightened of their countrymen, and in
defiance of the clearest demonstrations of the flagiciousness of their
conduct. They have, therefore, no cloak for their sin, no excuse or
palliating plea for their atrocities. To them belongs the pre-
eminence of turpitude and infamy, and they may be said to stand at
the head of those monsters who have been a disgrace to christianity,
to humanity, and to their country.
Slaves and horses appear to have been the principal, if not the only
articles exported from this country during the Anglo-Saxon ages.
Corn constituted then no article of our exports, though it had done
so formerly, in a considerable degree, while Britain formed a part of
the Roman Empire. Agriculture must therefore have miserably
declined here since the arrival of the Saxons, and the country had no
reason to congratulate itself on its change of masters. After the
introduction of Christianity the monks are said to have been, by
much the best husbandmen, and also the best, if not the only
gardeners in the country. They were certainly the most enlightened
class of the community, and the little knowledge and learning which
the country then possessed were chiefly, if not entirely, confined
within the solemn precincts of the monasteries.
Of our imports in those days, books, relics, pictures, and images of
saints, clerical vestments, and church ornaments, are said to have
been the chief articles; which gives but a very miserable idea of the
state of the country, and its commerce, in the mean time.—They
were however, not the only articles, for it appears that wines also
were imported from France and Spain, cloths from Germany and
Flanders, furs, deer-skins, (and probably, bear-skins,) ropes, whale
oil, &c. from Scandinavian and even a portion of all the different
commodities then known in any part of Europe is supposed to have
been at that period imported to this country. Yet the balance of
trade is said to have been much in our favour—that is, we got much
more by the sale of our Slaves and horses, in which our exports
consisted, than what we lost or laid out in the purchase of all the
various articles which we imported, many of which, at the same
time, must have been pretty expensive. This seems to imply, that
those Slaves and horses, with which foreign markets were supplied
from hence, must have been very numerous, as well as very
beneficial and lucrative to our English merchants.
During some part of this period the shipping of England seems to
have been pretty numerous; but what portion of it belonged to the
port of Lynn does not appear. The royal navy too, was at times on a
respectable footing, particularly in the reigns of Alfred and Canute,
as well as in those of Edgar, surnamed the peaceable, and Ethelred:
the latter is said to have possessed near 800 sail of men of war, [274]
but they were all what would now be called small vessels. The
military force of the kingdom consisted generally of about 50,000,
though on extraordinary occasions it considerably exceeded that
number.
For a long time, markets were usually kept on Sundays, in or near
some church, but that being found somewhat inconvenient, as
interfering with the religious service of the day, they were afterwards
changed to Saturdays. The fairs of those times were also generally
kept within the precincts of some great churches, or monasteries, on
some notable day, generally the anniversary of the patron saint, and
it was customary to oblige every comer to the fair, at the gate,
before he entered, to swear that he would neither lie, steal, nor
cheat: which might be very useful, if the people had then a proper
sense of the sacredness of oaths, otherwise it would be of but little
avail, as it is to be feared it would also be in the present day, when,
from the multiplicity or commonness of oaths, a disposition to trifle
with, or make light of them is notoriously prevalent. For holding
these fairs, bishops and abbots obtained charters from the crown,
with a view to increase their own revenues by the tolls which such
charters would authorise them to levy on such occasions. Thus
every thing contributed to the aggrandizement of the church. Before
the end of this period, the clergy had possession of more than a
third part of all the land in the kingdom, with the tithes of all the
rest.
Much attention was then bestowed on the decoration of churches
and religious houses. Organs and bells were introduced toward the
latter part of this period. The famous Saint Dunstan gave a fine
organ, in the reign of Edgar, to the abbey of Glastonbury. Bells
became very common about the 10th century, and were hung in the
towers of churches, which were then all of wood: only the altars
were, it seems, built of stone. The first set of bells in this kingdom,
that we hear of, was at Croyland, in Lincolnshire, in the reign of
Athelstan, a gift of the abbot Turketul to that celebrated monastery.
[275]
There had, however, been single bells in England long before
that period, and even as early as the 7th century, as is attested by
Bede. In the time of M. Paris, bells were not allowed to ring at
funerals, as inspiring too gay and unsuitable ideas. Clocks also
began to be introduced here toward the close of this period. About
the same time, the English began to be expert and noted
manufacturers of woollen cloth; the value of a sheep’s fleece, of
course, was then well understood, and rated at two fifths of the
animal’s whole price. Silk, though now beginning to be imported,
was not woven here until some centuries afterwards: linen, in the
mean time, was extremely scarce. It is very remarkable, but
seemingly an unquestionable fact, that highly finished works in gold
and silver, were the production even of our darkest ages. The
monks, in those times, were the best artists, and the famous St.
Dunstan inferior to none of them. Yet the means of supplying life
with necessaries, appear to have been but imperfectly known and
cultivated. The pagans of Sussex, in the 7th. century, though
starving for want of food, knew not how to catch any fish, except
eels, until bishop Wilfred, who came thither in 678, instructed them
in the use of nets. He took 300 at a draught; and by thus supplying
their bodily wants, rendered their minds tractable to his doctrines,
and easily succeeded in their conversion. [276]—Our modern
missionaries to the south sea isles, and other foreign parts, would do
well to imitate his example, and not confine their attention or
labours solely to religious instruction. A goodly pattern of the same
kind has also been very lately set before them among the North
American Indians by the Quaker missionaries. But it is to be feared
that they and their employers are too wise in their own eyes to profit
by such examples.

Section VI.

Population of Lynn, and the country in general, before the

conquest—condition of the bulk of the inhabitants in the mean
time—sufferings of the inhabitants of Lynn and the adjacent
country from the Danes—intrepid and ferocious character of that
people—instruments of vengeance on the Anglo-Saxons—their
disposition and character not much changed by their conversion
to christianity—remarkable instances of imposition, superstition,
and credulity.
Of the population of Lynn, at any time during that long period, from
the establishment of the Saxons to that of the Normans, no estimate
can now be formed; but it is pretty certain that the major part of its
inhabitants, as well as those of the adjacent country, and of all the
rest of England, were Slaves, during the whole of that time, and long
after. Those unfortunate people, for the most part at least, appear
to have been the descendants of the original inhabitants, who were
reduced to that condition, at the subjugation, of conquest of the
country, and whose lives had been then spared, on condition of their
becoming the property, or goods and chattels of the conquerors. So
did the Saxons treat those of the natives whose lives they
condescended to spare; all the rest they butchered without mercy,
except such of them as were fortunate enough to escape to the
unsubdued parts of the island. Of these cruel and horrid deeds,
they never appear to have repented, even after they assumed the
name of christians, for the bondage still continued; but, in time, a
severe retaliation took place, and the Saxons, in their turn, were
treated much after the same manner as they had formerly treated
the Britons. Long peace had destroyed their martial spirit: from a
very warlike people, they became gradually a most dastardly race,
and so fell an easy prey to the ferocious Danes. The difference at
that time between these two nations in point of military prowess, is
said to be so great that the Saxons, alias the English, frequently fled
before inferior numbers of the Danes, and could hardly be prevailed
upon to meet them in the field of battle on any terms. “Oh the
misery and worldly shame in which England is involved through the
wrath of God! (said an English bishop in the reign of Ethelred the
unready) How often doth two or three troops of Danes drive the
whole English army before them from sea to sea, to our eternal
infamy, if we were capable of feeling shame! But, alas! so abject are
we become, that we worship those who trample upon us, and load
us with indignities.” Such was then the abject submission of the
English to the insolence of the Danes, “that when an Englishman
met a Dane on a bridge, or in a narrow path, where he could not
avoid him, he was obliged to stand still, with his head uncovered,
and in a bowing posture, till he was out of sight.” Nay, we are
assured that English submission and Danish insolence and brutality
were sometimes carried still further, and even to degrees that are
almost incredible, as well as too indelicate to relate. [278] These
Danes, who now became the instruments of retaliation and
vengeance upon the Anglo-Saxons, were remarkable for their
extraordinary military skill and intrepidity; and they were as
unfeeling and ferocious as the latter appear to have been at the time
when they invaded and conquered this country: they were therefore
probably the fitter for the execution of the work in which they were
employed. Much has been said of the cruelties committed by the
Danes in this country: they were certainly very enormous and
shocking; but there is no reason to conclude that they exceeded, or,
perhaps, even equalled those which the Saxons had before exercised
upon the former inhabitants. Of all the perpetrators of Danish
enormities, in this island, Guthrum seems to be the foremost, or
most conspicuous, in the pages of our ancient historians. Of him
one of them speaks thus—“The cruel Guthrum arrived in England A.D.
878, at the head of an army of Pagan Danes, no less cruel than
himself; who, like inhuman savages, destroyed all before them with
fire and sword, involving cities, towns, and villages, with their
inhabitants, in devouring flames; and cutting those in pieces with
their battle-axes who attempted to escape from their burning
houses. The tears, cries, and lamentations of men, women, and
children, made no impressions upon their unrelenting hearts; even
the most tempting bribes, and the humblest offers of becoming their
slaves, had no effect. All the towns through which they passed
exhibited the most deplorable scenes of misery and desolation; as,
venerable old men lying with their throats cut before their own
doors; the streets covered with the bodies of young men and
children, without heads, legs, or arms; and of matrons and virgins,
who had been first publicly dishonoured, and then put to death.”
[279]
This is very shocking, and looks like providential retaliation.
The annals of history exhibit many instances of the same kind. The
Danish warriors were always prodigal of life, and not only did not
fear, but even courted a violent death. A natural death they
dreaded, as a most ignoble and disgraceful end, and which they
always appeared very anxious to avoid. No wonder that they
became the terror of every nation against which they happened to
wage war. No greater evil could well befall any people than to have
them for their enemies and invaders.
To no part of this island did the Danes prove a greater, or more
terrible scourge than to the province of the East Angles, which
became one of their principal settlements, and where they
committed the most shocking barbarities. Hence we may very safely
conclude that the sufferings which the inhabitants of Lynn
experienced from them must have been exceedingly grievous and
deplorable. But as those sufferings have not been recorded they
cannot now be described or particularized.
The Danes, as well as the Anglo-Saxons, when they invaded this
country were pagans. Both of them afterward took up the
profession of christianity; but it was only its profession, or bare
name that they did take up. Their former ferocity still remained.
They continued grossly ignorant, superstitious, and heathenish, and
exhibited scarcely a spark of the real spirit of the religion of Christ,
except perhaps in the latter part of the reign of Alfred. Their
ghostly, or religious instructors were miserable and blind guides, or
knavish and artful impostors, who taught them that the most
meritorious actions consisted in erecting and endowing monasteries,
performing pilgrimages, and reverencing the priesthood. From such
pretended or pseudo-christianity, what good effect could be
expected? Grapes cannot be gathered of thorns, or figs of thistles.—
When Earl Alwine, who was the greatest and richest man in England,
in the reign of Edgar the peaceable, consulted St. Oswald, bishop of
York, what he should do to obtain the pardon of his sins, the sainted
prelate made him the following eloquent harangue: “I beseech your
excellency to believe that those holy men who have retired from the
world, and spend their days in poverty and prayer, are the greatest
favourites of heaven, and the greatest blessings to the world. It is
by their merits that the divine judgements are averted and changed;
that plagues and famines are removed; that healthful seasons and
plentiful harvests are procured; that states and kingdoms are