Read Anytime Anywhere Easy Ebook Downloads at ebookmeta.

com

Intelligent Software Defect Prediction 1st Edition

Xiao-Yuan Jing

https://ebookmeta.com/product/intelligent-software-defect-
prediction-1st-edition-xiao-yuan-jing/

OR CLICK HERE

DOWLOAD EBOOK

Visit and Get More Ebook Downloads Instantly at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Intelligent Modeling, Prediction, and Diagnosis from

Epidemiological Data: COVID-19 and Beyond 1st Edition
Siddhartha Bhattacharyya
https://ebookmeta.com/product/intelligent-modeling-prediction-and-
diagnosis-from-epidemiological-data-covid-19-and-beyond-1st-edition-
siddhartha-bhattacharyya/
ebookmeta.com

Challenges of Software Verification Intelligent Systems

Reference Library 238 Vincenzo Arceri (Editor)

https://ebookmeta.com/product/challenges-of-software-verification-
intelligent-systems-reference-library-238-vincenzo-arceri-editor/

ebookmeta.com

Software Project Estimation: Intelligent Forecasting,

Project Control, and Client Relationship Management 1st
Edition Dimitre Dimitrov
https://ebookmeta.com/product/software-project-estimation-intelligent-
forecasting-project-control-and-client-relationship-management-1st-
edition-dimitre-dimitrov/
ebookmeta.com

The Ethics of Governance Moral Limits of Policy Decisions

1st Edition Shashi Motilal

https://ebookmeta.com/product/the-ethics-of-governance-moral-limits-
of-policy-decisions-1st-edition-shashi-motilal/

ebookmeta.com
Routine Blood Results Explained, 4th Edition Andrew Blann

https://ebookmeta.com/product/routine-blood-results-explained-4th-
edition-andrew-blann/

ebookmeta.com

Amidst Secrets and Shadows 1st Edition Stacy Jones Harper

Wylde

https://ebookmeta.com/product/amidst-secrets-and-shadows-1st-edition-
stacy-jones-harper-wylde/

ebookmeta.com

the WELLNESS ZONE your guide for optimal wellness 3rd

Edition Dominique Livkamal

https://ebookmeta.com/product/the-wellness-zone-your-guide-for-
optimal-wellness-3rd-edition-dominique-livkamal/

ebookmeta.com

Easy French Reader, Premium Fourth Edition R. De Roussy De

Sales

https://ebookmeta.com/product/easy-french-reader-premium-fourth-
edition-r-de-roussy-de-sales/

ebookmeta.com

The Serpent Lord Wren Murphy

https://ebookmeta.com/product/the-serpent-lord-wren-murphy/

ebookmeta.com
101 Things I Learned in Culinary School 2nd Edition Louis
Eguaras

https://ebookmeta.com/product/101-things-i-learned-in-culinary-
school-2nd-edition-louis-eguaras/

ebookmeta.com
Xiao-Yuan Jing · Haowen Chen
Baowen Xu

Intelligent
Software Defect
Prediction
Intelligent Software Defect Prediction
Xiao-Yuan Jing • Haowen Chen • Baowen Xu

Intelligent Software Defect

Prediction
Xiao-Yuan Jing Haowen Chen
School of Computer Science School of Computer Science
Wuhan University Wuhan University
Wuhan, Hubei, China Wuhan, Hubei, China

Baowen Xu
Computer Science & Technology
Nanjing University
Nanjing, Jiangsu, China

ISBN 978-981-99-2841-5 ISBN 978-981-99-2842-2 (eBook)

https://doi.org/10.1007/978-981-99-2842-2

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore
Pte Ltd. 2023
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse
of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore

Paper in this product is recyclable.

Preface

With the increase of complexity and dependency of software, the software product
may suffer from low quality, high cost, hard-to-maintain, and even the occurrence
of defects. Software defect usually produces incorrect or unexpected results and
behaviors in unintended ways. Software defect prediction (SDP) is one of the most
active research fields in software engineering and plays an important role in software
quality assurance. According to the feedback of SDP, developers can subsequently
conduct defect location and repair under reasonable resource allocation, which is
helpful in reducing the maintenance cost.
The early task of SDP is performed within a single project. Developers can make
use of the well-labeled historical data of the currently maintained project to build the
model and predict the defect-proneness of the remaining instances. This process is
called within-project defect prediction (WPDP). However, the annotation for defect
data (i.e., defective or defective-free) is time-consuming and high-cost, which is a
hard task for practitioners in the development or maintenance cycle. To solve this
problem, researchers consider introducing other projects with sufficient historical
data to conduct the cross-project defect prediction (CPDP) which has received
extensive attention in recent years. As the special case of CPDP, heterogeneous
defect prediction (HDP) refers to the scenario that training and test data have
different metrics, which can relax the restriction on source and target projects’
metrics. Besides, there also exist other research questions of SDP to be further
studied, such as cross-version defect prediction, just-in-time (JIT) defect prediction,
and effort-aware JIT defect prediction.
In the past few decades, more and more researchers pay attention to SDP and
a lot of intelligent SDP techniques have been presented. In order to obtain the
high-quality representations of defect data, a lot of machine learning techniques
such as dictionary learning, semi-supervised learning, multi-view learning, and deep
learning are applied to solve SDP problems. Besides, transfer learning techniques
are also used to eliminate the divergence between different project data in CPDP
scenario. Therefore, the combination with machine learning techniques is conducive
to improving the prediction efficiency and accuracy, which can promote the research
of intelligent SDP to make significant progress.

v
vi Preface

We propose to draft this book to provide a comprehensive picture of the

current state of SDP researches instead of improving and comparing existing SDP
approaches. More specifically, this book introduces a range of machine learning-
based SDP approaches proposed for different scenarios (i.e., WPDP, CPDP, and
HDP). Besides, this book also provides deep insight into current SDP approaches’
performance and learned lessons for further SDP researches.
This book is mainly applicable to graduate students, researchers who work in
or have interests in the areas of SDP, and the developers who are responsible for
software maintenance.

Wuhan, China Xiao-Yuan Jing

December, 2022 Haowen Chen
Acknowledgments

We thank Li Zhiqiang, Wu Fei, Wang Tiejian, Zhang Zhiwu, and Sun Ying
from Wuhan University for their contributions to this research. We would like to
express our heartfelt gratitude to Professor Baowen Xu and his team from Nanjing
University for their selfless technical assistance in the compilation of this book. We
are so thankful for the invaluable help and support provided by Professor Xiaoyuan
Xie from Wuhan University, whose valuable advice and guidance was crucial to the
successful completion of this book. We wanted to express our sincere appreciation
for the unwavering support provided by Nanjing University, Wuhan University,
and Nanjing University of Posts and Telecommunications, as well as the editing
suggestions provided by Kamesh and Wei Zhu from Springer Publishing House.
We just wanted to thank you from the bottom of our hearts for your unwavering
support and guidance throughout the compilation of this book. Finally, we would
like to express our heartfelt appreciation to two master students, Hanwei and Xiuting
Huang, who participated in the editing process and made indelible contributions to
the compilation of this book.

vii
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Software Quality Assurance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Software Defect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Research Directions of SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.1 Within-Project Defect Prediction (WPDP) . . . . . . . . . . . . . . . . . . . . 3
1.3.2 Cross-Project Defect Prediction (CPDP) . . . . . . . . . . . . . . . . . . . . . . 4
1.3.3 Heterogeneous Defect Prediction (HDP) . . . . . . . . . . . . . . . . . . . . . . 4
1.3.4 Other Research Questions of SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Notations and Corresponding Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Structure of This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Machine Learning Techniques for Intelligent SDP. . . . . . . . . . . . . . . . . . . . . . . 13
2.1 Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Deep Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Other Techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.1 Dictionary Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.2 Semi-Supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.3 Multi-View Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3 Within-Project Defect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1 Basic WPDP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1.1 Dictionary Learning Based Software Defect Prediction . . . . . . 19
3.1.2 Collaborative Representation Classification Based
Software Defect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.2 Semi-supervised WPDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.1 Sample-Based Software Defect Prediction with
Active and Semi-supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . 28
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

ix
x Contents

4 Cross-Project Defect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.1 Basic CPDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1.1 Manifold Embedded Distribution Adaptation . . . . . . . . . . . . . . . . . 36
4.2 Class Imbalance Problem in CPDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.2.1 An Improved SDA Based Defect Prediction Framework . . . . . 46
4.3 Semi-Supervised CPDP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.3.1 Cost-Sensitive Kernelized Semi-supervised
Dictionary Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5 Heterogeneous Defect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.1 Basic HDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.1.1 Unified Metric Representation and CCA-Based
Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.2 Class Imbalance Problem in HDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.2.1 Cost-Sensitive Transfer Kernel Canonical
Correlation Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.2.2 Other Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.3 Multiple Sources and Privacy Preservation Problems in HDP . . . . . . . . 104
5.3.1 Multi-Source Selection Based Manifold
Discriminant Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.3.2 Sparse Representation Based Double Obfuscation
Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6 An Empirical Study on HDP Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.1 Goal Question Metric (GQM) Based Research Methodology . . . . . . . . 139
6.1.1 Major Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.1.2 Review of Research Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
6.1.3 Analysis on Research Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.1.4 Research Goal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.1.5 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
6.1.6 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
6.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.2.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.2.2 SDP Approaches for Comparisons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.2.3 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.2.4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
6.3 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7 Other Research Questions of SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
7.1 Cross-Version Defect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
7.1.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
7.1.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
7.1.3 Discussions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
7.2 Just-in-Time Defect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
Contents xi

7.2.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

7.2.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
7.2.3 Discussions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
7.3 Effort-Aware Just-in-Time Defect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . 188
7.3.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
7.3.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
7.3.3 Discussions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
8.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
Chapter 1
Introduction

1.1 Software Quality Assurance

With the increasing pressures of expediting software projects that is always

increasing in size and complexity to meet rapidly changing business needs, quality
assurance activities such as fault prediction models have thus become extremely
important. The main purpose of a fault prediction model is the effective allocation
or prioritization of quality assurance effort (test effort and code inspection effort).
Construction of these prediction models are mostly dependent on historical or
previous software project data referred to as a dataset.
However, a prevalent problem in data mining is the skewness of a dataset. Fault
prediction datasets are not excluded from this phenomenon. Most datasets have the
majority of the instances being either clean or not faulty and conventional learning
methods are primarily designed for balanced datasets. Common classifiers such
as Neural Networks (NN), Support Vector Machines (SVM), and decision trees
work best toward optimizing their objective functions, which lead to the maximum
overall accuracy—the ratio of correctly predicted instances to the total number of
instances. The use of imbalanced datasets for training a classifier will most likely
generate a classifier that tends to over-predict the presence of the majority class
but a lower probability of predicting the minority or faulty modules. When the
model predicts the minority class, it often has a higher error rate compared to
predictions for the majority class. This impacts the performance of classifiers in
machine learning and is known as learning from imbalanced datasets. This affects
the prediction performance of classifiers, and in machine learning, this issue is
known as learning from imbalanced datasets. Several methods have been proposed
in machine learning for dealing with the class imbalanced issue such as random
over and under sampling creating synthetic data application of cleaning techniques
for data sampling and cluster-based sampling. With a significant amount of literature
in machine learning for imbalanced datasets, very few studies have tackled it in the
area of fault prediction. The first of such studies by Kamei et al. [1] showed that

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 1
X.-Y. Jing et al., Intelligent Software Defect Prediction,
https://doi.org/10.1007/978-981-99-2842-2_1
2 1 Introduction

sampling techniques improved the prediction performance of linear and logistics

models, whilst other two models (neural network and classification tree) did not
have a better performance upon application of the sampling techniques.
Interestingly, sampling techniques applied to datasets during fault prediction
are mostly evaluated in terms of Accuracy, AUC, F1-measure, Geometric Mean
Accuracy just to name a few; however, these measures ignore the effort needed to
fix faults, that is, they do not distinguish between a predicted fault in a small module
and a predicted fault in a large module. Nickerson et al. [2] conclude that to evaluate
the performance of classifiers on imbalanced datasets, accuracy or its inverse error
rate should never be used. Chawla et al. [3] also allude to the conclusion that simple
predictive accuracy might not be appropriate for an imbalanced dataset. The goal
of this research is to improve the prediction performance of fault-prone module
prediction models, applying over and under sampling approaches to rebalance
number of fault-prone modules and non-fault-prone modules in the training dataset
and to find the appropriate distribution or proportion of faulty and non-faulty
modules that results in the best performance. The experiment focuses on the use
of Norm(Popt), which is an effort-aware measure proposed by Kamei et al. [4] to
evaluate the effect of over/under sampling on prediction models to find out if the
over/under sampling is still effective in a more realistic setting.

1.2 Software Defect Prediction

The defect is a flaw in the component or system which can cause it to fail to perform
its desired function, that is, an incorrect statement or data definition. A defect, if
encountered during execution, may cause a failure of the system or a component.
Defect prediction helps in identifying the vulnerabilities in the project plan in
terms of lack of resources, improperly defined timelines, predictable defects, etc.
It can help organizations to fetch huge profits without getting delayed on schedules
planned or overrun on estimates of budget. It helps in modifying the parameters
in order to meet the schedule variations. The methods to estimate the software
defects are regression, genetic programming, clustering, neural network, statistical
technique of discriminate analysis, dictionary learning approach, hybrid attribute
selection approach, classification, attribute selection and instance filtering, Bayesian
belief networks, K-means clustering, and association rule mining. In the domain of
software defect prediction, people have developed many software defect prediction
models. These models are mostly described in two classes: one class is in the later
period of the software life cycle (testing phase), having gotten defect data, predicts
how many defects still in the software with these data. Models in this class include:
capture-recapture method based model, neural network based model, and measure
method based on scale and complexity of source code. The other class, which occurs
before the software development phase, aims to predict the number of defects that
will arise during the software development process by analyzing defect data from
previous projects. Presently, published models in this class include: phase based
1.3 Research Directions of SDP 3

model proposed by Gaffney and Davis, Ada programming defect prediction model
proposed by Agresti and Evanco, early prediction model proposed by USA ROME
lab, software development early prediction method proposed by Carol Smidts in
Maryyland University, and early fuzzy neural network based model. However,
there are a number of serious theoretical and practical problems in these methods.
Software development is an extremely complicated process. Defects relate to many
factors. If you want to measure exactly, you would consider as many correlative
factors as possible, but it would make the model more complicated. If considering
the solvability, you would have to simplify the model. However, it would not make
out the convinced answer. Neural network based prediction model, for instance, has
lots of problems in training and verifying the sample collection. Software test in
many organizations is still in the original phase, so lots of software hardly gives
the defects number requested, which would bring certain difficulties to sample
collection. Early models consider inadequately on the uncertain factors in software
develop process; the dependence to data factors is great besides. Therefore, many
methods have difficulty in application.

1.3 Research Directions of SDP

1.3.1 Within-Project Defect Prediction (WPDP)

Some defect data in the same project are used as the training set to build the
prediction model, and the remaining small number of data are used as test set to
test the performance. At present, some researchers mainly use the machine learning
algorithm to construct the defect prediction model on the within-project defect
prediction. In addition, how to optimize the data structure and extract effective
feature are also the focus of current research. Some important research works
will be summarized below. Elish et al. [5] use support vector machine (SVM)
to conduct defect prediction and compare its predictive performance with eight
statistical and machine learning models on four NASA datasets. Lu et al. [6]
leverage active learning to predict defect, and they also use feature compression
techniques to make feature reduction on defect data. Li et al. [7] propose a novel
semi-supervised learning method—ACoForest—which can sample the prediction
modules that are most helpful for learning. Rodriguez et al. [8] compare different
methods for different data preprocessing problems, such as sampling method, cost
sensitive method, integration method, and hybrid method. The final experimental
results show that the above different methods can effectively improve the accuracy
of defect prediction after performing the class imbalance. Seiffert et al. [9] analyze
11 different algorithms and seven different data sampling techniques and find that
class imbalance and data noise would have the negative impact on prediction
performance.
4 1 Introduction

1.3.2 Cross-Project Defect Prediction (CPDP)

When data are insufficient or non-existent for building quality defect predictors,
software engineers can use data from other organizations or projects. This is called
cross-project defect prediction (CPDP). Acquiring data from other sources is a
non-trivial task when data owners are concerned about confidentiality. In practice,
extracting project data from organizations is often difficult due to the business
sensitivity associated with the data. For example, at a keynote address at ESEM’11,
Elaine Weyuker doubted that she will ever be able to release the AT&T data she used
to build defect predictors [10]. Due to similar privacy concerns, we were only able to
add seven records from two years of work to our NASA-wide software cost metrics
repository [11]. In a personal communication, Barry Boehm stated that he was able
to publish less than 200 cost estimation records even after 30 years of COCOMO
effort. To enable sharing, we must assure confidentiality. In our view, confidentiality
is the next grand challenge for CPDP in software engineering. In previous work, we
allowed data owners to generate minimized and obfuscated versions of their original
data. Our MORPH algorithm [12] reflects on the boundary between an instance
and its nearest instance of another class, and MORPH’s restricted mutation policy
never pushes an instance across that boundary. MORPH can be usefully combined
with the CLIFF data minimization algorithm [13]. CLIFF is an instance selector
that returns a subset of instances that best predict for the target class. Previously
we reported that this combination of CLIFF and MORPH resulted in 7/10 defect
datasets studied retaining high privacy scores, while remaining useful for CPDP
[13]. This is a startling result since research by Grechanik et al. [14] and Brickell
et al. [15] showed that standard privacy methods increase privacy while decreasing
data mining efficacy. While useful CLIFF and MORPH only considered a single-
party scenario where each data owner privatized their data individually without
considering privatized data from others. This resulted in privatized data that were
directly proportional in size (number of instances) to the original data. Therefore, in
a case where the size of the original data is small enough, any minimization might
be meaningless, but if the size of the original data is large, minimization may not be
enough to matter in practice.

1.3.3 Heterogeneous Defect Prediction (HDP)

Existing CPDP approaches are based on the underlying assumption that both
source and target project data should exhibit the same data distribution or are
drawn from the same feature space (i.e., the same software metrics). When the
distribution of the data changes, or when the metrics features for source and target
projects are different, one cannot expect the resulting prediction performance to be
satisfactory. We consider these scenarios as Heterogeneous Cross-Project Defect
Prediction (HCPDP). Mostly, the software defect datasets are imbalanced, which
1.3 Research Directions of SDP 5

means the number of the defective modules is usually much smaller than that of the
defective-free modules. The imbalanced nature of data can cause poor prediction
performance. That is, the probability of defect prediction can be low, while the
overall performance is high. Without taking this issue into account, the effectiveness
of software defect prediction in many real-world tasks would be greatly reduced.
Recently, some researchers have noticed the importance of these problems in
software defect prediction. For example, Nam et al. [16] used the metrics selection
and metrics matching to select similar metrics for building a prediction model with
heterogeneous metrics set. They discarded dissimilar metrics, which may contain
useful information for training. Jing et al. [17] introduced Canonical Correlation
Analysis (CCA) into HCPDP, by constructing the common correlation space to
associate cross-project data. Then, one can simply project the source and target
project data into this space for defect prediction. Like previous CPDP methods,
the class imbalance problem of software defect datasets was not taken into account.
Ryu et al. [18] designed the Value-Cognitive Boosting with Support Vector Machine
(VCB-SVM) algorithm which exploited sampling techniques to solve the class
imbalance issue for cross-project environments. Nevertheless, sampling strategy
alters the distribution of the original data, where it may discard some potentially
useful samples that could be important for prediction process. Therefore, these
methods are not good solutions for addressing the class imbalance issue under
heterogeneous cross-project environments.

1.3.4 Other Research Questions of SDP

1.3.4.1 Cross-Version Defect Prediction

Cross Version Defect Prediction (CVDP) is a practical scenario by training the

classification model on the historical data of the prior version and then predicting
the defect labels of modules of the current version. Bennin et al. [19] evaluated
the defect prediction performance of 11 basic classification models in IVDP and
CVDP scenarios with an effort-aware indicator. They conducted experiments on 25
projects (each one has two versions with process metrics) and found that the optimal
models for the two defect prediction scenarios are not identical due to different data
as the training set. However, the performance differences of the 11 models are not
significant in both scenarios. Premraj et al. [20] investigated the impacts of code and
network metrics on the defect prediction performance of six classification models.
They considered three scenarios, including IVDP, CVDP, and CPDP. CPDP uses
the defect data of another project as the training set. Experiments on three projects
(each with two versions) suggested that the network metrics are better than the code
metrics in most cases. Holschuh et al. [21] explored the performance of CVDP
on a large software system by collecting four types of metrics. The experiments
on six projects (each with three versions) showed that the overall performance
is unsatisfactory. Monden et al. [22] evaluated the cost effectiveness of defect
6 1 Introduction

prediction on three classification models by comparing seven test effort allocation

strategies. The results on one project with five versions revealed that the reduction of
test effort relied on the appropriate test strategy. Khoshgoftaar et al. [23] studied the
performance of six classification models on one project with four versions and found
that CART model with lease absolute deviation performed the best. Zhao et al. [24]
investigated the relationship between the context-based cohesion metrics and the
defect-proneness in IVDP and CVDP scenarios. They conducted CVDP study on
four projects with total 19 versions and found that context-based cohesion metrics
had negative impacts on defect prediction performance but can be complementary to
non-context-based metrics. Yang et al. [25] surveyed the impacts of code, process,
and slice-based cohesion metrics on defect prediction performance in IVDP, CVDP,
and CPDP scenarios. They conducted CVDP study on one project with seven
versions and found that slice-based cohesion metrics had adverse impacts on defect
prediction performance but can be complementary to the commonly used metrics.
Wang et al. [26] explored the performance of their proposed semantic metrics on
defect prediction in CVDP and CPDP scenarios. The experiments on ten projects
with 26 versions showed the superiority of the semantic metrics compared with
traditional CK metrics and AST metrics.

1.3.4.2 Just-in-Time Defect Prediction

Just-in-time defect prediction aims to predict if a particular file involved in a commit

(i.e., a change) is buggy or not. Traditional just-in-time defect prediction techniques
typically follow the following steps:
Training Data Extraction. For each change, label it as buggy or clean by mining
a project’s revision history and issue tracking system. Buggy change means the
change contains bugs (one or more), while clean change means the change has no
bug.
Feature Extraction. Extract the values of various features from each change.
Many different features have been used in past change classification studies. Model
Learning. Build a model by using a classification algorithm based on the labeled
changes and their corresponding features.
Model Application. For a new change, extract the values of various features.
Input these values to the learned model to predict whether the change is buggy or
clean.
The studies by Kamei et al. [32] are great source of inspiration for our work. They
proposed a just-in-time quality assurance technique that predicts defects at commit-
level trying to reduce the effort of a reviewer. Later on, they also evaluated how
just-in-time models perform in the context of cross-project defect prediction [19].
Findings report good accuracy for the models not only in terms of both precision
and recall but also in terms of saved inspection effort. Our work is complementary
to these papers. In particular, we start from their basis of detecting defective commits
and complement this model with the attributes necessary to filter only those files that
are defect-prone and should be more thoroughly reviewed. Yang et al. [25] proposed
1.4 Notations and Corresponding Descriptions 7

the usage of alternative techniques for just-in-time quality assurance, such as cached
history, deep learning, and textual analysis, reporting promising results. We did not
investigate these further in the current chapter, but studies can be designed and
carried out to determine if and how these techniques can be used within the model
we present in this chapter to further increase its accuracy.

1.3.4.3 Effort-Aware Defect Prediction

Traditional SDP models based on some binary classification algorithms are not
sufficient for software testing in practice, since they do not distinguish between a
module with many defects or high defect density (i.e., number of defects/lines of
source codes) and a module with a small number of defects or low defect density.
Clearly, both modules require a different amount of effort to inspect and fix, yet they
are considered equal and allocated the same testing resources. Therefore, Mende et
al. [27] proposed effort-aware defect prediction (EADP) models to rank software
modules based on the possibility of these modules being defective, their predicted
number of defects, or defect density. Generally, EADP models are constructed by
using learning to rank techniques [28]. These techniques can be grouped into three
categories, that is, the pointwise approach, the pairwise approach, and the listwise
approach [29–31]. There exists a vast variety of learning to rank algorithms in
literature. It is thus important to empirically and statistically compare the impact
and effectiveness of different learning to rank algorithms for EADP. To the best
of our knowledge, few prior studies [32–36] evaluated and compared the existing
learning to rank algorithms for EADP. Most of these studies, however, conducted
their study with few learning to rank algorithms across a small number of datasets.
Previous studies [34–36] conducted their study with as many as five EADP models
and few datasets. For example, Jiang et al. [34] investigated the performance of only
five classification-based pointwise algorithms for EADP on two NASA datasets.
Nguyen et al. [36] investigated three regression based pointwise algorithm and two
pairwise algorithms for EADP on five Eclipse CVS datasets.

1.4 Notations and Corresponding Descriptions

We will briefly introduce some of the symbols and abbreviations that appear in this
book, as listed in the Table 1.1: Some parts are listed in the table, and the parts that
are not listed will be made in the corresponding text: Detailed description.
8 1 Introduction

Table 1.1 Symbols and corresponding descriptions

Symbol/Abbreviation Description
SDP Software Defect Prediction
WPDP Within-Project Defect Prediction
HCCDP Heterogeneous cross-company defect predecton
CPDP Cross-Project Defect Prediction
HDP Heterogeneous Defect Prediction
CCA Canonical correlation analysis
TKCCA Transfer kernel canonical correlation analysis
CTKCCA Cost-sensitive transfer kernel canonical correlation analysis
GQM Goal Question Metric
ROC Receiver operating characteristic
MDA A manifold embedded distribution adaptation
SDA Subclass discriminant analysis
.⇒ The source company data and the right side of “.⇒” represents the
target company data
.a = [a1 , a2 , . . . an ] .a is a vector, and .ai is the .ith component
.a The length of a vector
.∈ An element belongs to a set
.tr(·) The trace of a matrix

1.5 Structure of This Book

In the second chapter of this book, several common learning algorithms and their
applications in software defect prediction are briefly introduced, including deep
learning, transfer learning, dictionary learning, semi-supervised learning, and multi-
view learning.
In Chap. 3, we discussed mainly about within-project defect prediction but first
introduced basic WPDP including dictionary learning based software defect pre-
diction, collaborative representation classification based software defect prediction
and then introduced the sample-based software defect prediction with active and
semi-supervised learning belonging to the semi-supervised WPDP.
In Chap. 4, we expounded some methodologies on cross-project defect pre-
diction, including basic CPDP, among which we introduced manifold embedded
distribution adaptation; for class imbalance problem in CPDP, we proposed an
improved SDA based defect prediction framework; finally, in semi-supervised
CPDP, we introduced cost-sensitive kernelized semi-supervised dictionary learning.
In Chap. 5, we introduce Heterogeneous Defect Prediction (HDP), first explain-
ing unified metric representation and CCA-based transfer learning in basic HDP;
then in class imbalance problem in HDP, we introduce cost-sensitive transfer kernel
canonical correlation analysis. Finally, regarding multiple sources and privacy
preservation problems in HDP, we have introduced multi-source selection based
References 9

manifold discriminant alignment and sparse representation based double obfusca-

tion algorithm.
In Chap. 6, an empirical study on HDP approaches is introduced, including
heterogeneous defect prediction and Goal Question Metric (GQM) based research
methodology.
Finally, in Chap. 7 of this book, we discuss other research questions of SDP,
mainly including the following aspects: cross-version defect prediction, just-in-time
defect prediction and effort-aware just-in-time defect prediction.

References

1. Kamei Y, Monden A, Matsumoto S, Kakimoto T, Matsumoto KI (2007) The Effects of

Over and Under Sampling on Fault-prone Module Detection. In: Proceedings of the First
International Symposium on Empirical Software Engineering and Measurement, pp 196–204.
https://doi.org/10.1109/ESEM.2007.28
2. Nickerson A, Japkowicz N, Milios EE (2001) Using Unsupervised Learning to Guide
Resampling in Imbalanced Data Sets. In: Proceedings of the Eighth International Workshop
on Artificial Intelligence and Statistics. http://www.gatsby.ucl.ac.uk/aistats/aistats2001/files/
nickerson155.ps
3. Chawla NV (2010) Data Mining for Imbalanced Datasets: An Overview. In: Proceedings of
the Data Mining and Knowledge Discovery Handbook, pp 875–886. https://doi.org/10.1007/
978-0-387-09823-4_45
4. Kamei Y, Matsumoto S, Monden A, Matsumoto K, Adams B, Hassan AE (2010) Revisiting
common bug prediction findings using effort-aware models. In: Proceedings of the 26th IEEE
International Conference on Software Maintenance, pp 1–10. https://doi.org/10.1109/ICSM.
2010.5609530
5. Elish KO, Elish MO (2008) Predicting defect-prone software modules using support vector
machines. J Syst Softw 81(5):649–660. https://doi.org/10.1016/j.jss.2007.07.040
6. Lu H, Kocaguneli E, Cukic B (2014) Defect Prediction between Software Versions with
Active Learning and Dimensionality Reduction. In: Proceedings of the 25th IEEE International
Symposium on Software Reliability Engineering, pp 312–322. https://doi.org/10.1109/ISSRE.
2014.35
7. Li M, Zhang H, Wu R, Zhou Z (2012) Sample-based software defect prediction with active and
semi-supervised learning. Autom Softw Eng 19(2):201–230. https://doi.org/10.1007/s10515-
011-0092-1
8. Rodríguez D, Herraiz I, Harrison R, Dolado JJ, Riquelme JC (2014) Preliminary comparison
of techniques for dealing with imbalance in software defect prediction. In: Proceedings of
the 18th International Conference on Evaluation and Assessment in Software Engineering, pp
43:1–43:10. https://doi.org/10.1145/2601248.2601294
9. Seiffert C, Khoshgoftaar TM, Hulse JV, Folleco A (2007) An Empirical Study of the
Classification Performance of Learners on Imbalanced and Noisy Software Quality Data. In:
Proceedings of the IEEE International Conference on Information Reuse and Integration, pp
651–658. https://doi.org/10.1109/IRI.2007.4296694
10. Weyuker EJ, Ostrand TJ, Bell RM (2008) Do too many cooks spoil the broth? Using the number
of developers to enhance defect prediction models. Empir Softw Eng 13(5):539–559. https://
doi.org/10.1007/s10664-008-9082-8
11. Menzies T, El-Rawas O, Hihn J, Feather MS, Madachy RJ, Boehm BW (2007) The business
case for automated software engineering. In: Proceedings of the 22nd IEEE/ACM International
Conference on Automated Software Engineering ASE 2007, pp 303–312. https://doi.org/10.
1145/1321631.1321676
10 1 Introduction

12. Peters F, Menzies T (2012) Privacy and utility for defect prediction: Experiments with
MORPH. In: Proceedings of the 34th International Conference on Software Engineering, pp
189–199. https://doi.org/10.1109/ICSE.2012.6227194
13. Peters F, Menzies T, Gong L, Zhang H (2013) Balancing Privacy and Utility in Cross-Company
Defect Prediction. IEEE Trans Software Eng 39(8):1054–1068. https://doi.org/10.1109/TSE.
2013.6
14. Grechanik M, Csallner C, Fu C, Xie Q (2010) Is Data Privacy Always Good for Software
Testing?. In: Proceedings of the IEEE 21st International Symposium on Software Reliability
Engineering, pp 368–377. https://doi.org/10.1109/ISSRE.2010.13
15. Brickell J, Shmatikov V (2008) The cost of privacy: destruction of data-mining utility
in anonymized data publishing. In: Proceedings of the 14th ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining, pp 70–78. https://doi.org/10.1145/
1401890.1401904
16. Nam J, Kim S (2015) Heterogeneous defect prediction. In: Proceedings of the 10th Joint Meet-
ing on Foundations of Software Engineering, pp 508–519. https://doi.org/10.1145/2786805.
2786814
17. Jing X, Wu F, Dong X, Qi F, Xu B (2015) Heterogeneous cross-company defect prediction
by unified metric representation and CCA-based transfer learning. In: Proceedings of the 2015
10th Joint Meeting on Foundations of Software Engineering, pp 496–507. https://doi.org/10.
1145/2786805.2786813
18. Ryu D, Choi O, Baik J (2016) Value-cognitive boosting with a support vector machine
for cross-project defect prediction. Empir Softw Eng 21(1):43–71. https://doi.org/10.1007/
s10664-014-9346-4
19. Bennin KE, Toda K, Kamei Y, Keung J, Monden A, Ubayashi N (2016) Empirical Evaluation
of Cross-Release Effort-Aware Defect Prediction Models. In: Proceedings of the 2016 IEEE
International Conference on Software Quality, pp 214–221. https://doi.org/10.1109/QRS.2016.
33
20. Premraj R, Herzig K (2011) Network Versus Code Metrics to Predict Defects: A Replication
Study. In: Proceedings of the 5th International Symposium on Empirical Software Engineering
and Measurement, pp 215–224. https://doi.org/10.1109/ESEM.2011.30
21. Holschuh T, Pauser M, Herzig K, Zimmermann T, Premraj R, Zeller A (2009) Predicting
defects in SAP Java code: An experience report. In: Proceedings of the 31st International Con-
ference on Software Engineering, pp 172–181. https://doi.org/10.1109/ICSE-COMPANION.
2009.5070975
22. Monden A, Hayashi T, Shinoda S, Shirai K, Yoshida J, Barker M, Matsumoto K (2013)
Assessing the Cost Effectiveness of Fault Prediction in Acceptance Testing. IEEE Trans Softw
Eng 39(10):1345–1357. https://doi.org/10.1109/TSE.2013.21
23. Khoshgoftaar TM, Seliya N (2003) Fault Prediction Modeling for Software Quality Estimation:
Comparing Commonly Used Techniques Empir. Softw Eng 8(3):255–283. https://doi.org/10.
1023/A:1024424811345
24. Zhao Y, Yang Y, Lu H, Liu J, Leung H, Wu Y, Zhou Y, Xu B (2017) Understanding the value
of considering client usage context in package cohesion for fault-proneness prediction Autom.
Softw Eng 24(2):393–453. https://doi.org/10.1007/s10515-016-0198-6
25. Yang Y, Zhou Y, Lu H, Chen L, Chen Z, Xu B, Leung HKN, Zhang Z (2015) Are Slice-Based
Cohesion Metrics Actually Useful in Effort-Aware Post-Release Fault-Proneness Prediction?
An Empirical Study IEEE Trans. Softw Eng 41(4):331–357. https://doi.org/10.1109/TSE.2014.
2370048
26. Wang S, Liu T, Tan L (2016) Automatically learning semantic features for defect prediction.
In: Proceedings of the 38th International Conference on Software Engineering, pp 297–308.
https://doi.org/10.1145/2884781.2884804
27. Mende T, Koschke R (2010) Effort-Aware Defect Prediction Models. In: Proceedings of the
14th European Conference on Software Maintenance and Reengineering, pp 107–116. https://
doi.org/10.1109/CSMR.2010.18
References 11

28. Wang F, Huang J, Ma Y (2018) A Top-k Learning to Rank Approach to Cross-Project Software
Defect Prediction. In: Proceedings of the 25th Asia-Pacific Software Engineering Conference,
pp 335–344. https://doi.org/10.1109/APSEC.2018.00048
29. Shi Z, Keung J, Bennin KE, Zhang X (2018) Comparing learning to rank techniques in hybrid
bug localization. Appl Soft Comput 62636-648. https://doi.org/10.1016/j.asoc.2017.10.048
30. Liu T (2010) Learning to rank for information retrieval. In: Proceedings of the Proceeding of
the 33rd International ACM SIGIR Conference on Research and Development in Information
Retrieval, pp 904. https://doi.org/10.1145/1835449.1835676
31. Yu X, Li Q, Liu J (2019) Scalable and parallel sequential pattern mining using spark. World
Wide Web 22(1):295–324. https://doi.org/10.1007/s11280-018-0566-1
32. Bennin KE, Toda K, Kamei Y, Keung J, Monden A, Ubayashi N (2016) Empirical Evaluation
of Cross-Release Effort-Aware Defect Prediction Models. In: Proceedings of the 2016 IEEE
International Conference on Software Quality, pp 214–221. https://doi.org/10.1109/QRS.2016.
33
33. Yang X, Wen W (2018) Ridge and Lasso Regression Models for Cross-Version Defect
Prediction. IEEE Trans Reliab 67(3):885–896. https://doi.org/10.1109/TR.2018.2847353
34. Jiang Y, Cukic B, Ma Y (2008) Techniques for evaluating fault prediction models. Empir Softw
Eng 13(5):561–595. https://doi.org/10.1007/s10664-008-9079-3
35. Mende T, Koschke R (2009) Revisiting the evaluation of defect prediction models. In:
Proceedings of the 5th International Workshop on Predictive Models in Software Engineering,
pp 7. https://doi.org/10.1145/1540438.1540448
36. Nguyen TT, An TQ, Hai VT, Phuong TM (2014) Similarity-based and rank-based defect
prediction. In: Proceedings of the 2014 International Conference on Advanced Technologies
for Communications (ATC 2014), pp 321–325.
Chapter 2
Machine Learning Techniques
for Intelligent SDP

Abstract In this chapter, several common learning algorithms and their applica-
tions in software defect prediction are briefly introduced, including deep learning,
transfer learning, dictionary learning, semi-supervised learning, and multi-view
learning.

2.1 Transfer Learning

In many real world applications, it is expensive or impossible to recollect the needed

training data and rebuild the models. It would be nice to reduce the need and
effort to recollect the training data. In such cases, transfer learning (TL) between
task domains would be desirable. Transfer learning exploits the knowledge gained
from a previous task to improve generalization on another related task. Transfer
learning can be useful when there is not enough labeled data for the new problem
or when the computational cost of training a model from scratch is too high.
Traditional data mining and machine learning algorithms make predictions on the
future data using statistical models that are trained on previously collected labeled
or unlabeled training data. Most of them assume that the distributions of the labeled
and unlabeled data are the same. Transfer learning (TL), in contrast, allows the
domains, tasks, and distributions used in training and testing to be different. It is
used to improve a learner from one domain by transferring information from a
related domain. Research on transfer learning has attracted more and more attention
since 1995. Today, transfer learning methods appear in several top venues, most
notably in data mining and applications of machine learning and data mining
Due to their strong ability of domain adaptation, researchers introduce TL
techniques to cross-project or heterogeneous defect prediction in recent years.
The application of TL in cross-project defect prediction (CPDP) aims to reduce
the distribution difference between source and target data. For example, Nam
et al. [1] proposed a new CPDP method called TCA+, which extends transfer
component analysis (TCA) by introducing a set of rules for selecting an appropriate
normalization method to obtain better CPDP performance. Krishna and Menzies [2]
introduced a baseline method named Bellwether for cross-project defect prediction

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 13
X.-Y. Jing et al., Intelligent Software Defect Prediction,
https://doi.org/10.1007/978-981-99-2842-2_2
14 2 Machine Learning Techniques for Intelligent SDP

based on existing CPDP methods. For heterogeneous defect prediction (HDP),

TL techniques are applied not only to reduce the distribution difference between
source and target data but also to eliminate the heterogeneity of metrics between
source and target projects. Jing et al. [3] proposed an HDP method named CCA+,
which uses the canonical correlation analysis (CCA) technique and the unified
metric representation (UMR) to find the latent common feature space between the
source and target projects. Specifically, the UMR is made of three kinds of metrics,
including the common metrics of the source and target data, source-specific metrics,
and target-specific metrics. Based on UMR, the transfer learning method based on
CCA is introduced to find common metrics by maximizing the canonical correlation
coefficient between source and target data.

2.2 Deep Learning

Deep learning (DL) is an extension of prior work on neural networks where the
“deep” refers to the use of multiple layers in the network. In the 1960s and 1970s,
it was found that very simple neural nets can be poor classifiers unless they are
extended with (a) extra layers between inputs and outputs and (b) a nonlinear
activation function controlling links from inputs to a hidden layer (which can be
very wide) to an output layer. Essentially, deep learning is a modern variation
on the above which is concerned with a potentially unbounded number of layers
of bounded size. In the last century, most neural networks used the “sigmoid”
activation function .f (x) = 1+e1 −x , which was subpar to other learners in several
tasks. It was only when the ReLU activation function .f (x) = max(0, x) was
introduced by Nair and Hinton [4] that their performance increased dramatically,
and they became popular.
With its strong representation learning ability, deep learning technology has
quickly gained favor in the field of software engineering. In software defect
prediction (SDP), researchers began to use DL techniques to extract deep features
of defect data. Wang et al. [6] first introduced the Deep Belief Network (DBN)
[5] that learns semantic features and then uses classical learners to perform defect
prediction. In this approach, for each file in the source code, they extract tokens,
disregarding ones that do not affect the semantics of the code, such as variable
names. These tokens are vectorized and given unique numbers, forming a vector
of integers for each source file. Wen et al. [7] utilized Recurrent Neural Network
(RNN) to encode features from sequence data automatically. They propose a novel
approach called FENCES, which extracts six types of change sequences covering
different aspects of software changes via fine-grained change analysis. It approaches
defect prediction by mapping it to a sequence labeling problem solvable by RNN.
2.3 Other Techniques 15

2.3 Other Techniques

2.3.1 Dictionary Learning

Both sparse representation and dictionary learning have been successfully applied
to many application fields, including image clustering, compressed sensing as
well as image classification tasks. In sparse representation based classification,
the dictionary for sparse coding could be predefined. For example, Wright et al.
[8] directly used the training samples of all classes as the dictionary to code the
query face image and classified the query face image by evaluating which class
leads to the minimal reconstruction error. However, the dictionary in his method
may not be effective enough to represent the query images due to the uncertain
and noisy information in the original training images. In addition, the number of
atoms of dictionary that is made up of image samples can also be very large, which
increases the coding complexity. Dictionary learning (DL) aims to learn from the
training samples’ space where the given signal could be well represented or coded
for processing. Most DL methods attempt to learn a common dictionary shared by
all classes as well as a classifier of coefficients for classification.
Usually, the dictionary can be constructed by directly using the original training
samples, whereas the original samples have much redundancy and noise, which are
adverse to prediction. For the purpose of further improving the classification ability,
DL techniques have been adopted in SDP tasks recently to represent project modules
well. For example, Jing et al. [14] are the first to apply the DL technology to the field
of software defect prediction and proposed a cost-sensitive discriminative dictionary
learning (CDDL) approach. Specifically, CDDL introduces misclassification costs
and builds the over-complete dictionary for software project modules.

2.3.2 Semi-Supervised Learning

Due to the lack of labeled data, Semi-Supervised Learning (SSL) has always been
a hot topic in machine learning. A myriad of SSL methods have been proposed.
For example, co-training is a well-known disagreement-based SSL method, which
trains different learners to exploit unlabeled data. Pseudo-label style methods
label unlabeled data with pseudo labels. Graph-based methods aim to construct a
similarity graph, through which label information propagates to unlabeled nodes.
Local smoothness regularization-based methods represent another widely recog-
nized category of semi-supervised learning (SSL) techniques, which leverage the
inherent structure of the data to improve learning accuracy. Different methods apply
different regularizers, such as Laplacian regularization, manifold regularization, and
virtual adversarial regularization. For example, Miyato et al. [11] proposed a smooth
regularization method called virtual adversarial training, which enables the model
16 2 Machine Learning Techniques for Intelligent SDP

to output a smooth label distribution for local perturbations of a given input. There
are other popular methods, for example, Ladder Network.
Since large unlabeled data exist in software projects, many SSL techniques have
been considered in SDP tasks. Wang et al. [9] proposed a non-negative sparse-based
semiboost learning approach for software defect prediction. Benefit from the idea
of semi-supervised learning, this approach is capable of exploiting both labeled and
unlabeled data and is formulated in a boosting framework. Besides, Zhang et al. [10]
used graph-based semi-supervised learning technique to predict software defect.
This approach utilizes not only few labeled data but also abundant unlabeled ones
to improve the generalization capability.

2.3.3 Multi-View Learning

Representation learning is a prerequisite step in many multi-view learning tasks.

In recent years, a variety of classical multi-view representation learning methods
have been proposed. These methods follow the previously presented taxonomy,
that is, joint representation, alignment representation, as well as shared and specific
representation. For example, based on Markov network, Chen et al. [12] presented a
large-margin predictive multi-view subspace learning method, which joints features
learned from multiple views. Jing et al. [13] proposed an intra-view and inter-view
supervised correlation analysis method for image classification, in which CCA was
applied to align multi-view features.
Deep multi-view representation learning works also follow the joint repre-
sentation, alignment representation, as well as shared and specific representation
classification paradigm. For example, Kan et al. [14] proposed a multi-view deep
network for cross-view classification. This network first extracts view-specific
features with a sub-network and then concatenates and feeds these features into
a common network, which is designed to project them into one uniform space.
Harwath et al. [15] presented an unsupervised audiovisual matchmap neural net-
work, which applies similarity metric and pairwise ranking criterion to align visual
objects and spoken words. Hu et al. [16] introduced a sharable and individual multi-
view deep metric learning method. It leverages view-specific networks to extract
individual features from each view and employs a common network to extract
shared features from all views.

References

1. Nam, Jaechang and Pan, Sinno Jialin and Kim, Sunghun. Transfer defect learning. 35th
international conference on software engineering (ICSE), 382–391, 2013.
2. Krishna, Rahul and Menzies, Tim. Bellwethers: A baseline method for transfer learning. IEEE
Transactions on Software Engineering, 45(11):1081–1105, 2018.
References 17

3. Jing, Xiaoyuan and Wu, Fei and Dong, Xiwei and Qi, Fumin and Xu, Baowen. Heterogeneous
cross-company defect prediction by unified metric representation and CCA-based transfer
learning. In Proceedings of the 2015 10th joint meeting on foundations of software engineering,
pages 496–507, 2015.
4. Nair, Vinod and Hinton, Geoffrey E. Rectified linear units improve restricted Boltzmann
machines. In Icml’10, 2010.
5. Hinton, Geoffrey E. Deep belief networks. Scholarpedia, 4(5):5947, 2009.
6. Wang, Song and Liu, Taiyue and Tan, Lin. Automatically learning semantic features for defect
prediction. In IEEE/ACM 38th International Conference on Software Engineering (ICSE),
pages 297–308, 2016.
7. Wen, Ming and Wu, Rongxin and Cheung, Shing-Chi. How well do change sequences
predict defects? sequence learning from software changes. IEEE Transactions on Software
Engineering, 46(11):1155–1175, 2018.
8. Wright, John and Yang, Allen Y and Ganesh, Arvind and Sastry, S Shankar and Ma, Yi. Robust
face recognition via sparse representation. IEEE transactions on pattern analysis and machine
intelligence, 31(2):210–227, 2008.
9. Wang, Tiejian and Zhang, Zhiwu and Jing, Xiaoyuan and Liu, Yanli. Non-negative sparse-
based SemiBoost for software defect prediction. Software Testing, Verification and Reliability,
26(7):498–515, 2016.
10. Zhang, Zhi-Wu and Jing, Xiao-Yuan and Wang, Tie-Jian. Label propagation based semi-
supervised learning for software defect prediction. Automated Software Engineering,
24(7):47–69, 2017.
11. Miyato, Takeru and Maeda, Shin-ichi and Koyama, Masanori and Ishii, Shin. Virtual
adversarial training: a regularization method for supervised and semi-supervised learning.
IEEE transactions on pattern analysis and machine intelligence, 41(8):1979–1993, 2018.
12. Chen, Ning and Zhu, Jun and Sun, Fuchun and Xing, Eric Poe. Large-margin predictive latent
subspace learning for multiview data analysis. IEEE transactions on pattern analysis and
machine intelligence, 34(12):2365–2378, 2012.
13. Jing, Xiao-Yuan and Hu, Rui-Min and Zhu, Yang-Ping and Wu, Shan-Shan and Liang, Chao
and Yang, Jing-Yu. Intra-view and inter-view supervised correlation analysis for multi-view
feature learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1882–
1889, 2014.
14. Kan, Meina and Shan, Shiguang and Chen, Xilin. Multi-view deep network for cross-view
classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pages 4847–4855, 2016.
15. Harwath, David and Torralba, Antonio and Glass, James. Unsupervised learning of spoken
language with visual context. In Advances in Neural Information Processing Systems, pages
1858–1866, 2016.
16. Hu, Junlin and Lu, Jiwen and Tan, Yap-Peng. Sharable and individual multi-view metric
learning. IEEE transactions on pattern analysis and machine intelligence, 40(9):2281–2288,
2017.
Chapter 3
Within-Project Defect Prediction

Abstract In order to improve the quality of a software system, software defect

prediction aims to automatically identify defective software modules for efficient
software test. To predict software defect, those classification methods with static
code attributes have attracted a great deal of attention. In recent years, machine
learning techniques have been applied to defect prediction. Due to the fact that
there exists the similarity among different software modules, one software module
can be approximately represented by a small proportion of other modules. And
the representation coefficients over the pre-defined dictionary, which consists of
historical software module data, are generally sparse. We propose a cost-sensitive
discriminative dictionary learning (CDDL) approach for software defect classifica-
tion and prediction. The widely used datasets from NASA projects are employed
as test data to evaluate the performance of all compared methods. Experimental
results show that CDDL outperforms several representative state-of-the-art defect
prediction methods.

3.1 Basic WPDP

3.1.1 Dictionary Learning Based Software Defect Prediction

3.1.1.1 Methodology

To fully exploit the discriminative information of training samples for improving the
performance of classification, we design a supervised dictionary learning approach,
which learns a dictionary that can represent the given software module more
effectively. Moreover, the supervised dictionary learning can also reduce both
the number of dictionary atoms and the sparse coding complexity. Instead of
learning a shared dictionary for all classes, we learn a structured dictionary .D =
[D1 , . . . , Di , . . . , Dc ], where .Di is the class-specified sub-dictionary associated
with class i, and c is the total number of classes. We use the reconstruction error
to do classification with such a dictionary D, as the SRC method does.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 19
X.-Y. Jing et al., Intelligent Software Defect Prediction,
https://doi.org/10.1007/978-981-99-2842-2_3
20 3 Within-Project Defect Prediction

Suppose that .A = [A1 , . . . , Ai , . . . , Ac ] is the set of training samples (labeled

software modules), .Ai is the subset of the training samples from class i, .X =
[X1 , . . . , Xi , . . . , Xc ] is the coding coefficient matrix of A over D, that is, .A ≈ DX,
where .Xi is the sub-matrix containing the coding coefficients of .Ai over D. We
require that D should have not only powerful reconstruction capability of A but
also powerful discriminative capability of classes in A. Thus, we propose the cost-
sensitive discriminative dictionary learning (CDDL) model as follows:

J(D,X) = arg min {r(A, D, X) + ‖X‖1 }

. (3.1)
(D,X)

where .r(A, D, X) is the discriminative fidelity term; .‖X‖1 is the sparsity constraint;
λ is a balance factor.
.
j
Let .Xi = [Xi1 , Xi2 , Xic ], where .Xi is the coding coefficient matrix of .Ai over
the sub-dictionary .Dj . Denote the representation of .Dk to .Ai as .Rk = Dk Xik . First
of all, the dictionary D should be able to well represent .Ai , and, therefore, .Ai ≈
DXi = D1 Xi1 + · · · + Di Xii + · · · + Dc Xic . Secondly, since .Di is associated with the
ith class, it is expected that .Ai should be well represented by .Di (not by .Dj , j /= i),
 2  
 j 2
which means both .Ai − Di Xii F and .Dj Xi  should be minimized. Thus the
F
discriminative fidelity term is


c
r(A, D, X) = r (Ai , D, Xi )
i=1
.
⎛ ⎞

c  2 c  2
⎜  i  j ⎟
= ⎝‖Ai − DXi ‖F + Ai − Di Xi  +
2
Dj Xi  ⎠
F F
i=1 j =1
j /=i
(3.2)

An intuitive explanation of three terms in .r(Ai , D, Xi ) is shown in Fig. 3.1. In

software defect prediction, there are two kinds of modules: the defective modules
and the defective-free modules. Figure 3.1a shows that if we only minimize

Fig. 3.1 Illustration of the

discriminative fidelity term
3.1 Basic WPDP 21

the .‖Ai − DXi ‖2F on the total dictionary D, .Ri may deviate much from .Ai so
that sub-dictionary .Di could not well represent .Ai . In order to achieve better
powerful reconstruction capability and powerful discriminative capability, we add
 2
another two parts .Ai − Di Xii F (which minimizes the reconstruction error on sub-
 
 j 2
dictionary of its own class) and .Dj Xi  (which minimizes the reconstruction
F
term on sub-dictionary of the other class); both of them should also be minimized.
Figure 3.1b shows that the proposed discriminative fidelity term could overcome the
problem in Fig. 3.1a.
As previously stated, misclassifying defective-free modules leads to increasing
the development cost, and misclassifying defective ones is related with risk cost.
Cost-sensitive learning can incorporate the different misclassification costs into the
classification process. In this section, we emphasize the risk cost such that we add
the penalty factor .cost (i, j ) to increase the punishment when a defective software
module is predicted as a defective-free software module. As a result, cost-sensitive
dictionary learning makes the prediction incline to classify a module as a defective
one and generates a dictionary for classification with minimum misclassification
cost. The discriminative fidelity term with penalty factors is

c
r(A, D, X) = r (Ai , D, Xi )
i=1
. ⎡ ⎤

c  2 c  2
 
⎣‖Ai − DXi ‖2F + Ai − Di Xii  +  
cost(i, j ) Dj Xi  ⎦
j
=
F F
i=1 j =1
(3.3)

Since there are only two classes in software defect prediction (the defective class and
the defective-free class), that is, .c = 2, the model of cost-sensitive discriminative
dictionary learning is
 2 
  2
 
J(D,X) = arg min ‖Ai − DXi ‖2F + Ai − Di Xii 
(D,X) F
i=1
. ⎤ ⎫ (3.4)

2   ⎬
 j 2 ⎦
+ cost(i, j ) Dj Xi  + λ‖X‖1
F ⎭
j =1

where the cost matrix is shown in Table 3.1.

Table 3.1 Cost matrix for CDDL

Predicts defective one Predicts defective-free one
Actually defective 0 .cost(1, 2)

Actually defective-free .cost(2, 1) 0

22 3 Within-Project Defect Prediction

The CDDL objective function in Formula 3.4 can be divided into two sub-
problems: updating X by fixing D and updating D by fixing X. The optimization
procedure is iteratively implemented for the desired discriminative dictionary D
and corresponding coefficient matrix X. At first, suppose that D is fixed, the
objective function in formula is reduced to a sparse coding problem to compute
.X = [X1 , X2 ]. Here .X1 and .X2 are calculated one by one. We calculate .X1 with

fixed .X2 and then compute .X2 with fixed .X1 . Thus, formula is rewritten as
  2
 
J(Xi ) = arg min ‖Ai − DXi ‖2F + Ai − Di Xii 
(Xi ) F
⎫ (3.5)
  ⎬
.

2
 j 2
+ cost(i, j ) Dj Xi  + λ ‖Xi ‖1
F ⎭
j =1

Formula 3.5 can be solved by using the IPM algorithm in [1]. When X is fixed,
we in turn update .D1 and .D2 . When we calculate .D1 , .D2 is fixed, then we compute
.D2 , .D1 is fixed. Thus Formula 3.4 is rewritten as

⎧ 2
⎪  
⎨  2 
 
J(Di ) = arg min −Di Xi − Dj X j 
⎪
(Di ) ⎩ 
j =1 
j /=i
. F (3.6)
⎫
 2 
2   ⎬
   j 2
+ Ai − Di Xii  + cos(i, j ) Dj Xi 
F F⎭
j =1

where .Xi is the coding coefficient matrix of A over .Di . Formula 3.6 is a quadratic
programming problem, and we can solve it by using the algorithm in [2].
By utilizing the PCA technique, we are able to initialize the sub-dictionary for
each class. Given the low data dimension of software defect prediction, PCA can
create a fully initialized sub-dictionary for every class. This means that all sub-
dictionaries have an equal number of atoms, which is generally equivalent to the
data dimension.
The algorithm of CDDL converges since its two alternative optimizations are
both convex. Figure 3.2 illustrates the convergence of the algorithm.

3.1.1.2 Experiments

To evaluate our CDDL approach, we conduct some experiments. For all selected
datasets, we use the 1:1 random division to obtain the training and testing sets for
all compared methods. The random division treatment may affect the prediction
performance. Therefore, we use the random division, perform prediction 20 times,
and report the average prediction results in the following discussions.
3.1 Basic WPDP 23

a b
2 10
Total objective function value

Total objective function value

8
1.5
6
1
4
0.5
2

0 0
1 5 10 15 20 25 30 1 5 10 15 20 25 30
Iteration number Iteration number

c d
2 5
Total objective function value

Total objective function value 4

1.5
3
1
2
0.5
1

0 0
1 5 10 15 20 25 30 1 5 10 15 20 25 30
Iteration number Iteration number

Fig. 3.2 Convergence of the realization algorithm of CDDL on four NASA benchmark datasets.
(a) CM1 dataset. (b) KC1 dataset. (c) MW1 dataset. (d) PC1 dataset

In our approach, in order to emphasize the risk cost, the parameters cost (1,2) and
cost (2,1) are set as 1:5. For various projects, users can select a different cost ratio,
such as cost(1,2) to cost(2,1) [3]. And the parameter is determined by searching a
wide range of values and choosing the one that yields the best F-measure value.
We compare the proposed CDDL approach with several representative methods,
particularly presented in the last five years, including support vector machine (SVM)
[4], Compressed C4.5 decision tree (CC4.5) [5], weighted Naïve Bayes (NB) [6],
coding based ensemble learning (CEL) [7], and cost-sensitive boosting neural
network (CBNN) [8]. In this section, we present the detailed experimental results of
our CDDL approach and other compared methods.

3.1.1.3 Discussions

Table 3.2 shows the Pd and Pf values of our approach and other compared methods
on 10 NASA datasets. For each dataset, Pd and Pf values of all methods are the
mean values calculated from the results of 20 runs. The results of Pf values suggest
that in spite of not acquiring the best Pf values on most datasets, CDDL can achieve
24 3 Within-Project Defect Prediction

Table 3.2 Experimental results: Pd and Pf comparisons on NASA’s ten datasets

Dataset M SVM CC4.5 NB CEL CBNN CDDL
CM1 Pd 0.15 0.26 0.44 0.43 0.59 0.74
Pf 0.04 0.11 0.18 0.15 0.29 0.37
JM1 Pd 0.53 0.37 0.14 0.32 0.54 0.68
Pf 0.45 0.17 0.32 0.14 0.29 0.35
KC1 Pd 0.19 0.40 0.31 0.37 0.69 0.81
Pf 0.02 0.12 0.06 0.13 0.30 0.37
KC3 Pd 0.33 0.41 0.46 0.29 0.51 0.71
Pf 0.08 0.16 0.21 0.12 0.25 0.34
MC2 Pd 0.51 0.64 0.35 0.56 0.79 0.83
Pf 0.24 0.49 0.09 0.38 0.54 0.29
MW1 Pd 0.21 0.29 0.49 0.25 0.61 0.79
Pf 0.04 0.09 0.19 0.11 0.25 0.25
PC1 Pd 0.66 0.38 0.36 0.46 0.54 0.86
Pf 0.19 0.09 0.11 0.13 0.17 0.29
PC3 Pd 0.64 0.34 0.28 0.41 0.65 0.77
Pf 0.41 0.08 0.09 0.13 0.25 0.28
PC4 Pd 0.72 0.49 0.39 0.48 0.66 0.89
Pf 0.16 0.07 0.13 0.06 0.18 0.28
PC5 Pd 0.71 0.50 0.32 0.37 0.79 0.84
Pf 0.22 0.02 0.14 0.13 0.08 0.06

Table 3.3 Average Pd value SVM CC4.5 NB CEL CBNN CDDL

of 10 NASA datasets
Average .0.47 .0.41 .0.35 .0.39 .0.64 .0.79

comparatively better results in contrast with other methods. We can also observe
that the Pd values of CDDL, which are presented with boldface, are higher than
the corresponding values of all other methods. CDDL achieves the highest Pd
values on all datasets. The results indicate that the proposed CDDL approach takes
the misclassification costs into consideration, which makes the prediction tend to
classify the defective-free modules as the defective ones in order to obtain higher
Pd values.
We calculate the average Pd values of 10 NASA datasets in Table 3.3. As
compared with other methods, the average Pd value of our approach is higher in
contrast with other related methods, and CDDL improves the average Pd value at
least by .0.15(= 0.79 − 0.64).
Table 3.4 shows the F-measure values of our approach and the compared methods
on 10 NASA datasets. In Table 3.4, F-measure values of CDDL are better than
other methods on all datasets, which means that our proposed approach outperforms
other methods and achieves the ideal prediction effects. According to the average F-
measure values shown in Table 3.4, CDDL improves the average F-measure value at
3.1 Basic WPDP 25

Table 3.4 F-measure values Datasets SVM CC4.5 NB CEL CBNN CDDL
on ten NASA datasets
CM1 .0.20 .0.25 .0.32 .0.27 .0.33 .0.38

JM1 .0.29 .0.34 .0.33 .0.33 .0.38 .0.40

KC1 .0.29 .0.39 .0.38 .0.36 .0.41 .0.47
KC3 .0.38 .0.38 .0.38 .0.33 .0.38 .0.44

MC2 .0.52 .0.48 .0.45 .0.49 .0.56 .0.63

MW1 .0.27 .0.27 .0.31 .0.27 .0.33 .0.38
PC1 .0.35 .0.32 .0.28 .0.32 .0.32 .0.41

PC3 .0.28 .0.29 .0.29 .0.36 .0.38 .0.42

PC4 .0.47 .0.49 .0.36 .0.48 .0.46 .0.55

PC5 .0.16 .0.48 .0.33 .0.36 .0.37 .0.59

Average .0.32 .0.37 .0.34 .0.35 .0.39 .0.47

Table 3.5 P -values between CDDL and other compared methods on ten NASA datasets
.CDDL

Dataset .s .SVM .CC4.5 .NB .CEL .CBNN

.CM1 .1.23 × 10−8 .3.51 × 10−6 .4.24 × 10−4 .1.80 × 10−4 .1.01 × 10−4
.JM1 .7.51 × 10
−18 .2.33 × 10
−13 .1.27 × 10
−14 .1.58 × 10
−13 .0.0564
.KC1 .1.20 × 10
−14 .1.23 × 10
−9 .8.38 × 10
−13 .2.80 × 10
−11 .9.69 × 10
−6

.KC3 .0.0265 .0.0089 .3.22 × 10

−4 .1.61 × 10
−4 .4.24 × 10
−4

.MC2 .1.26 × 10
−4 .2.61 × 10
−5 .1.13 × 10
−8 .7.58 × 10
−6 .1.01 × 10
−4

.MW1 .1.14 × 10
−3 .2.31 × 10
−4 .1.10 × 10
−3 .1.84 × 10
−5 .2.20 × 10
−3

.PC1 .2.64 × 10
−4 .2.41 × 10
−5 .1.60 × 10
−8 .1.69 × 10
−5 .1.68 × 10
−8

.PC3 .7.79 × 10
−14 .7.73 × 10
−9 .1.04 × 10
−8 .4.03 × 10
−5 .4.31 × 10
−5

.PC4 .7.32 × 10
−8 .7.26 × 10
−4 .2.81 × 10
−16 .4.26 × 10
−6 .1.75 × 10
−10

.PC5 .3.01 × 10
−18 .7.00 × 10
−9 .1.90 × 10
−14 .1.30 × 10
−12 .2.13 × 10
−11

least by (.0.47 − 0.39 = 0.08). To sum up, Tables 3.3 and 3.4 show that our approach
has the best achievement in the Pd and F-measure values.
To statistically analyze the F-measure results given in Table 3.4, we conduct a sta-
tistical test, that is, Mcnemar’s test [9]. This test can provide statistical significance
between CDDL and other methods. Here, the Mcnemar’s test uses a significance
level of 0.05. If the p-value is below 0.05, the performance difference between two
compared methods is considered to be statistically significant. Table 3.5 shows the
p-values between CDDL and other compared methods on 10 NASA datasets, where
only one value is slightly above 0.05. According to Table 3.5, the proposed approach
indeed makes a significant difference in comparison with other methods for software
defect prediction.
26 3 Within-Project Defect Prediction

3.1.2 Collaborative Representation Classification Based

Software Defect Prediction
3.1.2.1 Methodology

Figure 3.3 shows the flowchart of defect prediction in our approach, which includes
three steps. The first step is Laplace sampling process for the defective-free modules
to construct the training dataset. Second, the prediction models is trained by
using the CRC based learner. Finally, the CRC based predictor classifies whether
new modules are defective or defective-free. In the metric based software defect
prediction, the number of defective-free modules is much larger than that of
defective ones, that is, the class imbalance problem may occur. In this section, we
conduct the Laplace score sampling for training samples, which solves the class
imbalance problem effectively.
Sparse representation classification (SRC) represents a testing sample collabo-
ratively by samples of all classes. In SRC, there are enough training samples for
each class so that the dictionary is over-completed. Unfortunately, the number of
defective modules is usually much small. If we use this under-complete dictionary
to represent a defective module, the representation error may be much big and the
classification will be unstable. Fortunately, one fact in software defect prediction
is that software modules share similarities. Some samples from one class may be
very helpful to represent the testing sample of other classes. In CRC, this “lack of
samples” problem is solved by taking the software modules from the other class as
the possible samples of each class.
The main idea of CRC technique is that information of a signal can be
collaboratively represented by a linear combination of a few elementary signals.
We utilize .A = [A1 , A2 ] ∈ R m×n to denote the set of training samples which
is processed by Laplace sampling, and y denotes a testing sample. In order to
collaboratively represent the query sample using A with low computational burden,
we use the regularized least square method as follows:
 
X̂ = arg min ‖y − A · X‖22 + λ‖X‖22
. (3.7)
X

Training Instances
Test Instances

Laplace
Sampling
Building a Prediction Results
CRC_RLS (defective/defective-free)
Prediction Model
Software Defect Prediction
Database CRC Based Learner CRC Based Predictor

Fig. 3.3 CRC based software defect prediction flowchart

3.1 Basic WPDP 27

where .λ is the regularization parameter. The role of the regularization term is

twofold. First, it makes the least square solution stable. Second, it introduces a
certain amount of “sparsity” to the solution .X̂ while this sparsity is much weaker
than that by .l1 -norm.
The solution of collaborative representation with regularized least square in
Eq. 3.7 can be easily and analytically derived as
 −1
X̂ = AT A + λ · I
. AT y (3.8)

Let .P = (AT A + λ · I )−1 AT . Clearly, P is independent of y so that it can be pre-

calculated as a projection matrix. Hence, a query sample y can be simply projected
onto P via P y, which makes the collaborative representation very fast.
After training the CRC based learner, we can use the collaborative representation
classification with regularized least square .CRCR LS algorithm to do prediction.
For a test sample y, we code y over A and  get .X̂. In addition to the class
 
specific representation residual .y − Ai · X̂i  , where .X̂i is the coefficient vector
2  
 
associated with class i (.i = 1, 2), the .l -norm “sparsity” .X̂i  can also bring
2
2
some discrimination information for classification. Thus we use both of them in
classificationandcalculate
  the regularized residual of each class by using .ri =
   
y − Ai · X̂i  / X̂i  . The test sample y is assigned to the ith class corresponding
2 2
to the smallest regularized residual .ri .

3.1.2.2 Experiments

In the experiment, ten datasets from NASA Metrics Data Program are taken as the
test data. We compare the proposed approach with several representative software
defect prediction methods, including Compressed C4.5 decision tree (CC4.5),
weighted Naïve Bayes (NB), cost-sensitive boosting neural network (CBNN), and
coding based ensemble learning (CEL).

3.1.2.3 Discussions

We use recall (Pd), false positive rate (Pf), precision (Pre), and F-measure as
prediction accuracy evaluation indexes. A good prediction model desires to achieve
high value of recall rate and precision. However, there exists trade-off between
precision and recall. F-measure is the harmonic mean of precision and recall rate.
Note that these quality indexes are commonly used in the field of software defect
prediction. Table 3.6 shows the average Pd, Pf, Pre, and F-measure values of our
CSDP approach and other compared methods on ten NASA datasets, where each
value is the mean of 20 random runs. Our approach can acquire better prediction
28 3 Within-Project Defect Prediction

Table 3.6 Average Pd, Pf, Evaluation Prediction methods

Pre, and F-measure values of
indexes CC4.5 NB CEL CBNN CSDP
20 random runs on ten NASA
datasets Pd .0.408 .0.354 .0.394 .0.637 .0.745
Pf .0.140 .0.152 .0.148 .0.260 .0.211
Pre .0.342 .0.347 .0.324 .0.288 .0.343

F-measure .0.371 .0.342 .0.354 .0.390 .0.465

accuracy than other methods. In particular, our approach improves the average Pd at
least by 16.95% (.= (0.745 − 0.637)/0.637) and the average F-measure at least by
19.23% (.= (0.465 − 0.390)/0.390).

3.2 Semi-supervised WPDP

3.2.1 Sample-Based Software Defect Prediction with Active

and Semi-supervised Learning
3.2.1.1 Methodology

Software defect prediction, which aims to predict whether a particular software

module contains any defects, can be cast into a classification problem in machine
learning, where software metrics are extracted from each software module to form
an example with manually assigned labels defective (having one or more defects)
and non-defective (no defects). A classifier is then learned from these training
examples in the purpose of predicting the defect-proneness of unknown software
modules. In this section, we propose a sample-based defect prediction approach
which does not rely on the assumption that the current project has the same defect
characteristics as the historical projects.
Given a newly finished project, unlike the previous studies that leverage the
modules in historical projects for classifier learning, sample-based defect prediction
manages to sample a small portion of modules for extensive testing in order
to reliably label the sampled modules, while the defect-proneness of unsampled
modules remains unknown. Then, a classifier is constructed based on the sample of
software modules (the labeled data) and expected to provide accurate predictions
for the unsampled modules (unlabeled data). Here, conventional machine learners
(e.g., logistic regression, decision tree, Naive Bayes, etc.) can be applied to the
classification.
In practice, modern software systems often consist of hundreds or even thousands
of modules. An organization is usually not able to afford extensive testing for
all modules especially when time and resources are limited. In this case, the
organization can only manage to sample a small percentage of modules and test
them for defect-proneness. Classifier would have to be learned from a small training
3.2 Semi-supervised WPDP 29

set with the defect-proneness labels. Thus, the key for the sample-based defect
prediction to be cost-effective is to learn a well-performing classifier while keeping
the sample size small.
To improve the performance of sample-based defect prediction, we propose to
apply semi-supervised learning for classifier construction, which firstly learns an
initial classifier from a small sample of labeled training set and refines it by further
exploiting a larger number of available unlabeled data.
In semi-supervised learning, an effective paradigm is known as disagreement-
based semi-supervised learning, where multiple learners are trained for the same
task and the disagreements among the learners are exploited during learning. In
this paradigm, unlabeled data can be regarded as a special information exchange
“platform.” If one learner is much more confident on a disagreed unlabeled example
than other learner(s), then this learner will teach other(s) with this example; if
all learners are comparably confident on a disagreed unlabeled example, then this
example may be selected for query. Many well-known disagreement-based semi-
supervised learning methods have been developed.
In this study, we apply CoForest for defect prediction. It works based on a
well-known ensemble learning algorithm named random forest [10] to tackle the
problems of determining the most confident examples to label and producing the
final hypothesis. The pseudocode of CoForest is presented in Table 3.1. Briefly,
it works as follows. Let L denote the labeled dataset and U denote the unlabeled
dataset. First, N random trees are initiated from the training sets bootstrap-sampled
from the labeled dataset L for creating a random forest. Then, in each learning
iteration, each random tree is refined with the original labeled examples L and the
newly labeled examples .L' selected by its concomitant ensemble (i.e., the ensemble
of the other random trees except for the current tree). The learning process iterates
until certain stopping criterion is reached. Finally, the prediction is made based
on the majority voting from the ensemble of random trees. Note that in this way,
CoForest is able to exploit the advantage of both semi-supervised learning and
ensemble learning simultaneously, as suggested in Xu et al. [11].
In CoForest, the stopping criterion is essential to guarantee a good performance
Li and Zhou [12] derived a stopping criterion based on the theoretical findings
in Angluin and Laird [13]. By enforcing the worst case generalization error of a
random tree in the current round to be less than that in the preceded round, they
derived that semi-supervised learning process will be beneficial if the following
condition is satisfied

êi,t Wi,t−1
. < <1 (3.9)
êi,t−1 Wi,t

where .êi,t and .êi,t−1 denote the estimated classification error of the i-th random tree
in the t-th and (.t − 1)-th round, respectively, and .Wi,t and .Wi,t−1 denote the total
weights of its newly labeled sets L .i, t and L.i, t − 1 in the t-th and (.t − 1)-th round,
respectively, and .i ∈ {1, 2, . . . , N}. For detailed information on the derivation,
please refer to Li and Zhou [12].
30 3 Within-Project Defect Prediction

The CoForest has been successfully applied to the domain of computer-aided

medical diagnosis, where conducting a large amount of routine examinations places
heavy burden on medical experts. The CoForest algorithm was applied to help learn
hypothesis from diagnosed and undiagnosed samples in order to assist the medical
experts in making diagnosis.
Although a random sample can be used to approximate the properties of all the
software modules in the current projects, a random sample is apparently not data-
efficient since random sample neglects the “needs” of the learners for achieving
good performance and hence may contain redundant information that the learner has
already captured during the learning process. Intuitively, if a learner is trained using
the data that the learner needs most for improving its performance, it may require
less labeled data than the learners trained without caring its needs for learning. Put
it another way, if the same number of labeled data is used, the learner that is trained
using the labeled data it needs most would achieve better performance than the
learner that is trained without caring its needs for learning.
Active learning, which is another major approach for learning in presence of a
large number of unlabeled data, aims to achieve good performance by learning with
as few labeled data as possible. It assumes that the learner has some control over
the data sampling process by allowing the learner to actively select and query the
label of some informative unlabeled example which, if the labels are known, may
contribute the most for improving the prediction accuracy. Since active learning
and semi-supervised learning exploit the merit of unlabeled data from different
perspective, they have been further combined to achieve better performance in
image retrieval [86], Email spam detection [39], etc. Recently, Wang and Zhou [68]
analytically showed that combining active learning and semi-supervised learning is
beneficial in exploiting unlabeled data.
In this study, we extend CoForest to incorporate the idea of active learning into
the sample-based defect prediction. We propose a novel active semi-supervised
learning method called ACoForest, which leverages the advantages from both
disagreement-based active learning and semi-supervised learning. In detail, let L
and U denote the labeled set and unlabeled set, respectively. Similar to CoForest,
ACoForest is firstly initiated by constructing a random forest with N random trees
over L. Then, ACoForest iteratively exploits the unlabeled data via both active
learning and semi-supervised learning. In each iteration, ACoForest firstly labels
all the unlabeled examples and computes the degree of agreement of the ensemble
on each unlabeled example. Then, it reversely ranks all the unlabeled data according
to the degree of agreement and selects the M top-most disagreed unlabeled data to
query their labels from the user. These unlabeled data as well as their corresponding
labels are then used to augment L. After that, ACoForest exploits the remaining
unlabeled data just as CoForest does.
3.2 Semi-supervised WPDP 31

3.2.1.2 Experiments

To evaluate the effectiveness of sample-based defect prediction methods, we

perform experiments using datasets available at the PROMISE website. We have
collected the Eclipse, Lucene, and Xalan datasets.
The Eclipse datasets contain 198 attributes, including code complexity metrics
(such as LOC, cyclomatic complexity, number of classes, etc.) and metrics about
abstract syntax trees (such as number of blocks, number of if statements, method
references, etc.) (Zimmermann et al. 2007). The Eclipse defect data was collected
by mining Eclipse’s bug databases and version archives.
In this study, we experiment with Eclipse 2.0 and 3.0. To show the generality of
the results, we use the package-level data for Eclipse 3.0 and the file-level data for
Eclipse 2.0. We also choose two Eclipse components: JDT.Core and SWT in Eclipse
3.0 to evaluate the defect prediction performance for smaller Eclipse projects. We
only examine the pre-release defects, which are defects reported in the last six
months before release.
The Lucene dataset we use contains metric and defect data for 340 source
files in Apache Lucene v2.4. The Xalan dataset contains metric and defect data
for 229 source files in Apache Xalan v2.6. Both datasets contain 20 attributes,
including code complexity metrics (e.g., average cyclomatic complexity), object-
oriented metrics (e.g., depth of inheritance tree), and program dependency metrics
(e.g., number of dependent classes).
Having collected the data, we then apply the three methods described in Sect. 2
to construct defect prediction models from a small sample of modules and use them
to predict defect-proneness of unsampled modules. We evaluate the performance
of all the methods in terms of precision (P ), recall (R), F-measure (F ), and
Balancemeasure (B), which are defined as follows:

tp
P =
. (3.10)
tp + fp

tp
. R= (3.11)
tp + f n

2P R
F =
. (3.12)
P +R
 

1 !2 !2 "
 fn fp
.B = 1 − + (3.13)
2 tp + f n tn + fp

where tp, fp, tn, f n are the number of defective modules that are predicted as
defective, the number non-defective modules that are predicted as defective, the
number non-defective modules that are predicted as non-defective, and the number
defective module that are predicted as non-defective, respectively.
32 3 Within-Project Defect Prediction

3.2.1.3 Discussions

Our experiments show that a smaller sample can achieve similar defect prediction
performance as larger samples do. The sample can serve as an initial labeled training
set that represents the underlying data distribution of the entire dataset. Thus if there
is no sufficient historical datasets for building an effective defect prediction model
for a new project, we can randomly sample a small percentage of modules to test,
obtain their defect status (defective or non-defective), and then use the collected
sample to build a defect prediction for this project.
Our experiments also show that, in general, sampling with semi-supervised
learning and active learning can achieve better prediction performance than sam-
pling with conventional machine learning techniques. A sample may contain much
information that a conventional machine learner has already learned well but may
contain little information that the learner needs for improving the current prediction
accuracy. The proposed CoForest and ACoForest learners take the needs for learning
into account and obtain information needed for improving performance from the
unsampled modules.
Both CoForest and ACoForest methods work well for sample-based defect
prediction. ACoForest also supports the active selection of the modules—it can
actively suggest the QA team which modules to be chosen in order to increase the
prediction performance. Thus in order to apply ACoForest, interactions with test
engineers are required. If such interactions is allowed (which implies that more time
and efforts are allowed), we can apply the ACoForest method. If such interaction is
not allowed due to limited time and resources, we can apply the CoForest method.
In our approach, we draw a random sample from the population of modules.
To ensure proper statistical inference and to ensure the cost effectiveness of the
proposed method, the population size should be large enough. Therefore, the
proposed method is suitable for large-scale software systems.
The simple random sampling method requires that each individual in a sample
to be collected entirely by chance with the same probability. Selection bias may
be introduced if the module sample is collected simply by convenience, or from a
single developer/team. The selection bias can lead to non-sampling errors (errors
caused by human rather than sampling) and should be avoided.
The defect data for a sample can be collected through quality assurance activities
such as software testing, static program checking, and code inspection. As the
sample will be used for prediction, these activities should be carefully carried out so
that most of defects can be discovered. Incorrect sample data may lead to incorrect
estimates of the population.
In our experiments, we used the public defect dataset available at the PROMISE
dataset. Although this dataset has been used by many other studies [14–18], our
results may be under threat if the dataset is seriously flawed (e.g., there were
major problems in bug data collection and recording). Also, all the data used are
collected from open source projects. It is desirable to replicate the experiments on
industrial, in-house developed projects to further evaluate their validity. This will be
our important future work.
Exploring the Variety of Random
Documents with Different Content
marked, both as a restless and unbelieving man, he was more
assiduous than any one in his attendance at the court chapel, and
joined in singing hymns, which must have possessed all the charm
of novelty for him.[17]
How little feeling that most miserable of monarchs, Christian, really
had in the whole matter, is seen by a perusal of the Danish journals
at the time. The amusements of the court offer a most revolting
exhibition of apathy and want of sympathy.
On April 23, there was a masked ball, en domino, at which the king,
the queen dowager, and their suite were present; on the 24th,
instead of the play, a concert at the Danish theatre, where the royal
family were present; on the 25th, the sentence on Counts Struensee
and Brandt was pronounced in open court; in the evening, the opera
of Adrien en Syrie was performed. The small-pox continuing its
ravages, on the 26th, Sunday, profane amusements were interdicted
by the new government. On the 27th, the king dined with his court
at Charlottenlund, and returned to town at 7 P.M.; he signed the
sentences, and proceeded to the Italian Opera. On the 28th, the day
of the execution, there was a grand concert at court. Well may a
writer in the Annales Belgiques for May, 1772, remark:—
"If the king has unfortunately reached such a stage of unfeelingness,
what praise does not Caroline Matilda deserve for having succeeded
in captivating him so greatly that up to the present it was not even
suspected that he possessed such a disposition?"
In the meanwhile, Dr. Münter had informed Struensee, on April 26th,
of the promulgation of the sentences, and that they would be carried
into effect two days after. Struensee listened to him patiently, and
then remarked, as to the circumstances which were to throw infamy
upon his death—
"I am far above all this, and I hope my friend Brandt may be the
same. Here in this world—since I am on the point of leaving it—
neither honour nor infamy can affect me any more. It is equally the
same to me after death, whether my body putrifies under ground or
in the open air; whether it serves to feed the worms or the birds.
God will know very well how to preserve those particles of my body
which, on the day of resurrection, are to constitute my future
glorified body. It is not my all which is to be laid on the wheel.
Thank God! I know now very well that this dust is not my whole
being."
After this they conversed quietly about various matters concerning
Struensee's administration. The decision whether his government
had been politically bad he left to posterity, and many times
repeated his assurance that he was not conscious of any wrong
intentions. When Dr. Münter left him, Struensee handed him the
following letter for Frau von Berkentin at Pinneberg. This was the
patroness who, as chief gouvernante to the prince royal, had
recommended Struensee as physician in ordinary for the king's
foreign tour:—

I make use of the first moments which permit me to write to

you. Business, duties, and my late connexions have perhaps
lessened in me the remembrance of my former friends, but they
have been not able to obliterate their memory entirely. My
present leisure has revived it. If my silence has aroused
suspicion as to my former sentiments, I beg pardon of all those
who are entitled to my gratitude, and of you, gracious lady, in
particular. This however, is not the only advantage which the
change of my fate has produced. I owe my knowledge of truth
to it; it has procured me a happiness of which I had no further
expectation, as I had already lost sight of it. I entreat you to
consider my misfortunes in no other light but that of religion. I
gain more by them than I can ever lose; and I feel and assure
you of this with conviction, ease, and joy of heart. I beg you to
repeat what I now write in the house of Count Ahlefeldt and at
Rantzau. I am under great obligations to these two families, and
it has grieved me far more to have drawn with me into
misfortune persons who are related to them.
On the following day, April 27, Struensee also referred to his
administration, and assured Münter again, most sacredly, that he
had not falsified the accounts about the presents made by the king
to him and Brandt. Münter's remarks on this subject are worthy
quotation:—
"It is difficult to dismiss every suspicion on this head against
Struensee; and if he were guilty, of how little value would be his
conversion! It has made me uneasy, frequently, and even now, still,
after his death. All manner of appearances, his own confession that
he could not free himself from all suspicion, and many other
evidences, are against him. However, on the other side, it makes me
easy that he confessed greater and more punishable crimes without
constraint, but denied this with a firmness, calmness of mind and
confidence, which, inexplicable as the matter remains, makes it
difficult to believe him guilty."
Struensee then handed to Münter the following letter to Chamberlain
Christian Brandt, which he desired him to get delivered:—

Permit me to bewail with you and with the gracious lady your
mother, the fate of our dear Enevold. Do not think me unworthy
of sharing your grief with you, though, accidentally, I have been
the cause of it. You know how much I love him. He was the
man of all the world who possessed the largest share of my
friendship. His misfortunes cause me the greatest anxiety, and
my own have been on his account most painful to me. He has
shared my prosperity with me, and I trust that we shall now
together enjoy that happiness which our Redeemer has
promised us. I do not know anything wherewith I could comfort
you. You are acquainted with religion. In that I found a refuge
to comfort me on account of my misfortune. I pray to God that
he may at this very moment let you feel all its power. I shall not
cease to entertain a most lively sense of gratitude toward all
those persons who are dear to me at Rantzau. I am wholly
yours, &c.
April 27, 1772.
 P.S.—I have been in hopes, and still flatter myself, that the
sentence of my friend will be mitigated.

To Münter, Struensee declared that Brandt's sentence of death could

not be signed with a good conscience; for, he said, he could not
regard the action for which his friend's life was forfeited as a crime,
and he, Struensee, did not repent having taken part in it. On the
other hand, he reproached Brandt, because in his intercourse with
the king he neglected the reverence he owed him, which had also
been the reason why he attracted the king's displeasure on himself.
Of all the letters written by Struensee, the one he addressed to
Count von Rantzau is assuredly the most remarkable. Instead of the
reproaches with which he might have justly overwhelmed him, he
wrote in the following forgiving spirit:[18]—

This, Sir (Dr. Münter), is what I have begged you to say in my

behalf to Count von Rantzau. I never entertained any feeling
contrary to what his friendship had a right to expect. Though
convinced long ago that he was acting against me, I did not
venture to remove him from Copenhagen. The facilities I
possessed for doing so, the solicitations addressed to me, and
very powerful reasons not affecting me personally, could not
induce me to do so. The Russian affairs will inform him of the
measures taken against him, of which he is probably ignorant,
as I never spoke to him about them in detail. I had conceived
that his attachment to his master caused him to find the
conduct of his friend blamable, but it did not enter my mind that
he was capable of engaging any one to render his friend as
unfortunate as possible. Still, convinced by experience, I have
understood that the vivacity of zeal, circumstances, the
persuasion of the peril with which the king was believed to be
menaced, might stifle every other feeling. I have retained no
bitterness against the count. Having been since enlightened by
religion, I have preserved all the feelings of a personal
attachment for him which, through various signs, his memory
 will, doubtless, bring before his eyes. I offer up vows for his
prosperity. It is not in my power to give him stronger proofs
than by ardently wishing that he may find the happiness which
the truth of religion has taught me to know. I would desire the
count, on this point, to remember, by analogy, his prejudices
against medicine, and how he removed them by reading
"Zimmermann," and by experiencing the good effects of the
medicines I administered to him at Glückstadt. May these few
words efface everything that the count nourishes against me in
his mind! You will deliver this note to him, Sir, when no further
motives are in existence which may make him attribute this step
of mine to any other object.
STRUENSEE.
P.S. Having altered my mind, I have the honour to address this
note directly to the count, instead of entrusting it to Dr. Münter.
This 27th April, 1722. S.

Struensee did not wish to take a personal farewell of his brother,

Justiz-rath Struensee, because he was afraid that this might produce
a scene which would be too affecting for both of them. He therefore
begged Münter to do so for him. He entreated his brother's pardon
for drawing him into his misfortunes, but hoped and was certain that
his affairs would turn out well. He also assured him that he was
leaving the world with true brotherly affection for him. He also
wished his brother to be told of the sentiments in which he died.
This commission Dr. Münter discharged on the same evening, and
carried back the answer of the much afflicted brother.
Brandt also received on April 27, from his chaplain, Dean Hee, the
news of the confirmation of his sentence and the day of execution,
which he heard unconcerned, and said that he readily submitted to
the will of God.
A report had been spread that Brandt had spoken recklessly while in
prison, and sung merry songs. Hence the dean made a proposition
to him, which he left to him to accept or not, that he should make a
declaration of what his real sentiments were, in the presence of
witnesses. He readily complied with the proposal, and Hee went to
the commandant, who came with four officers, in whose presence
Brandt declared that he was ready to die, and was not afraid of it;
he likewise confessed before the omniscient God, that he had
without hypocrisy sought for God's mercy; he likewise confessed, as
he had done before, that he had acted very inconsiderately, that his
levity had been very great, and that he, on this account,
acknowledged God's mercy in suffering him to die, lest he should be
drawn away again from religion. He said, he knew very well that the
same levity of temper had induced him, in the beginning of his
imprisonment, to talk in a manner he was now ashamed of, though
he was sure in his conscience that many untruths were invented,
and propagated among the people, but he forgave those who had
been guilty of such a thing. Now, he wished that the gentlemen
present would bear testimony to what he should say. He
acknowledged himself a great sinner before God: a sinner who had
gone astray, but was brought back by Christ. He then begged the
commandant and the other officers to forgive him, if, by his levity, he
had offended any one of them, and wished that God's mercy in
Christ might always attend them as the greatest blessing. He said all
this with such a readiness, and in such moving terms, that all who
were present were affected by it, and every one of them wished that
God would preserve him in this situation of mind to the last.
In the meanwhile, the town council, the police, and military
authorities, were making preparations for the execution.
Copenhagen is surrounded on the land side, next the three suburbs,
by three large fields bordered by neat allées, which are used as
exercising grounds for the garrison, and for public festivities. On the
easternmost of these fields, situated on the Sound, a scaffold, 8
yards long and broad, and 27 feet in height, was erected; and on the
gallows hill, a mile distant, and situated in the western suburb, two
poles were planted, both of which were surrounded by four wheel-
posts. It took some trouble to complete this job, because no artisans
consented to undertake it. It was not until other workmen were
persuaded that a pleasure-house was to be built on the field that the
scaffold was completed. No wheelwright was willing either to supply
the wheels; so that the eight carriage wheels required had to be
begged from friends of the court party.
When dawn broke on the 28th of April, 1772, a day which inflicted
an eternal stain on the history of Denmark, the troops, consisting of
4,400 sailors belonging to the vessels in ordinary, and armed with
pikes, 1,200 infantry, 300 dragoons, and, strange to say, the corps of
military cadets, marched through the gates, in order to form a large
circle round the stage of blood on the Osterfeld, keep back the eager
countless mob, and be ready for any eventualities. General von
Eickstedt, town commandant, had the supreme command of all the
troops.
The two gates of the citadel were also kept shut till the departure of
the criminals; and the posts had been doubled in order to keep off
the pressure of the crowd, who also congregated eagerly here.
The two clergymen went at an early hour to the condemned men,
and found them both calm and easy in mind. When Münter entered,
Struensee was fully dressed, and lying on a couch. He was reading
Schlegel's sermons on Christ's Passion, and a religious conversation
began between the two, during which Münter looked very often
toward the cell door with a fearful expectation; but the count not
once.
At length the officer on duty came in and requested Münter to step
into the coach, and precede Struensee to the place of execution.
Münter was greatly moved, but Struensee, as if it did not concern
himself the least, comforted him by saying:—
"Make yourself easy, my dear friend, by considering the happiness I
am going to enter into, and with the consciousness that God has
made you a means for procuring it for me."
Soon after, the two delinquents were requested to get into their
coaches, Brandt going on first. The latter, after praying fervently, had
had his chains, which were fixed in the wall, taken off, and he put on
the clothes in which he intended to appear on the scaffold. He then
drank a dish of coffee and ate something, walking up and down the
room, which he had not been able to do before. As often as Dr. Hee
asked him how he found himself, he said that he was not afraid of
dying. He afterwards asked Hee whether he had seen anybody
executed before, and how far he was to lay his body bare for the
execution.
Struensee was dressed in a blue cut velvet coat with silver buttons;
Brandt in a green court dress richly embroidered with gold, and both
had costly fur pelisses thrown over them, but, as if in mockery, still
had a chain on their hand and foot. This gay attire had been given
them in order to remind the populace that the dizzy fall from the
greatest power to the scaffold was the just punishment of their
unparalleled crimes. By the side of each of the prisoners sat an
officer, and opposite to them two sergeants. The two coaches were
surrounded by 200 infantry soldiers with fixed bayonets, and an
equal number of dragoons with drawn sabres. The procession was
opened by a third coach, in which the Fiscal General and the king's
bailiff were seated, and, facing them, the latter's deputy, holding two
tin shields, on which the arms of the two counts were painted.
Half-past eight was striking from the tower of the citadel when the
three coaches began their progress to the scaffold, where they were
expected by upwards of 30,000 persons.
When the procession reached the spot, the Fiscal General and the
king's bailiff with his assistant first mounted the scaffold, on which
the executioner and his aids were awaiting their victims. They were
followed by Brandt; his features were so unchanged, and his bearing
was so perfectly calm, that it was generally supposed that a hope of
mercy was aroused in his mind at this supreme moment. Dean Hee
mounted the scaffold stairs immediately after him, and it was not till
they reached the top that the prisoner's fetters were removed. Even
here he assured Hee that his mind was composed, and that he was
not afraid of death. The dean, however, continued to encourage him,
and concluded with the words:—
"Son, be of good cheer, for thy sins are forgiven thee."
To which Brandt replied:—
"Yes; they are all cast into the depths of the sea."
The king's bailiff, Etats-rath Ortwed, now read the sentence; and
when he had finished, the executioner advanced to receive the
count's coat of arms. He asked Brandt whether it was his
escutcheon, to which the other replied by a nod; he then swung it in
the air, and broke it with the words:—
"This is not done in vain, but as a just punishment."
After the clergyman had read Brandt those things from the ritual
which are usual on such occasions, Hee asked him whether, in
addition to his other sins, he repented of his great crime of high
treason? Brandt answered in the affirmative, and then added:—
"I pray God, the king, and the country, for forgiveness, and only
wish that God may bless the king and the whole land for the sake of
Christ's blood."
After these words the clergyman gave him the benediction, and,
taking him by the hand, delivered him over to justice. When the
executioner approached to assist Brandt in undressing, the latter
said to him with firmness, though not without mildness, "Stand off,
and do not presume to touch me!" He quickly let his pelisse fall, took
off his hat, and himself removed his coat and waistcoat. After
previously feeling in all the pockets, which he doubtless did out of
habit, he also began to bare the right arm, from which the hand was
to be cut off, but the executioner now advanced, and helped him to
bare the whole arm as well as his neck.[19] After this, Brandt knelt
down, and laid his head on one block and his hand on another.
When the victim had thus offered himself for the execution of the
sentence, the clergyman reminded him of the posture of the Saviour
in the garden of Gethsemane, with his face on the ground, to which
Brandt, lying on the block, replied in a loud voice:—
"The blood of Christ intercedeth for me."
Hee stepped back, and while he was saying, "O Christ, in Thee I live,
in Thee I die! Oh! thou Lamb of God, that takest away the sins of
the world, be merciful!" the execution was over. Brandt did not die
as a hypocrite, but at the same time displayed no defiance.
Immediately after, the executioner's aids advanced, stripped the
body, and then divided it into four quarters with an axe. Each
quarter was let down separately by a rope into a cart standing
below, and the vessels with the entrails were also placed in it. Lastly,
the head was held up, shown to the spectators, and then let down
into the cart, together with the hand. After which the scaffold was
strown with fresh sand, in readiness for Struensee.[20]
During this awful tragedy Struensee sat in his coach, which was
standing near the scaffold. When Brandt went up, Münter ordered
the coach to be turned in such a way that they might not witness
Brandt's execution. But Struensee's eyes had already found his
unfortunate friend, and hence he said:—
"I have seen him already."
After some further exhortation, Münter said to the prisoner:—
"Christ prayed for his murderers even on the cross. May I rely upon
your leaving the world with the same sentiments of love toward
those whom you might have reason to think your enemies?"
"In the first place," Struensee replied, "I hope that there is no one
who has a personal hatred against me; but that those who have
promoted my misfortunes, have done it with the intention of doing
good. Secondly, I look upon myself already as a citizen of another
world, and consider that I am obliged to entertain sentiments
conformable with this dignity; and I am sure that if I were to see
those who might perhaps be my enemies here in the bliss of that
world which I hope to enter into, it would give me the highest
satisfaction. I pray to God that if my enemies hereafter repent of
their behaviour toward me, this repentance may induce them to look
out for that salvation which I confidently promise myself through the
mercy of God."
Struensee, during this conversation, suffered no other change than
that he appeared very pale, and thinking and speaking evidently cost
him more trouble than they had done earlier in the morning. Still he
retained perfect composure, and saluted some of those around the
coach by raising his hat, or by friendly glances. From the motion of
the spectators, Dr. Münter, though he could not see the scaffold,
guessed that Struensee's turn to ascend it had arrived, and that,
with Brandt's death, all hope of a pardon had disappeared.
When summoned by name, Struensee stepped out of the coach, and
went, led by Münter, with dignity though humbly, through the ranks
of favoured spectators, and bowed to them also. With difficulty he
ascended the fifteen steps leading to the scaffold. When they
reached the top, Münter spoke very concisely, and in a low voice,
upon the words, "He that believeth in me, though he were dead, yet
shall he live." After this the sentence of the Commission of Inquiry
and the royal confirmation were read to Struensee, and the king's
own signature was shown him. Then came the breaking of the coat
of arms, after which Struensee's chains were taken off. The
clergyman once again went up to Struensee, and asked him various
questions.
"Are you very truly sorry for all those actions by which you have
offended God and man?"
"You know my late sentiments on this point," Struensee replied, "and
I assure you they are this very moment just the same."
"Do you trust in the redemption of Christ as the only ground of your
being pardoned before God?"
"I know no other means of receiving God's mercy, and I trust in this
alone."
"Do you leave this world without hatred or malice against any person
whatever?"
"I hope nobody hates me personally; and as for the rest, you know
my sentiments on this head; they are the same as I told you
before."
Doctor Münter then laid his hand upon Struensee's head, and said
with deep emotion, before he delivered him up to justice:—
"Then go in peace whither God calls you! His grace be with you!"
Struensee then took off his fur pelisse, removed his hat, and tried to
undress himself, but his strength failed him in doing so, and he was
obliged to ask the executioner's help. After this he produced a white
handkerchief to bind his eyes with, but the executioner said that it
was not necessary, and then assisted him in removing his shirt.
Struensee then walked with hesitating steps the few yards leading to
the block, which still reeked with the blood of his dearest friend; a
stronger mind than Struensee ever possessed might have been
unhinged by the dreadful scene before his eyes. He knelt down, but
had great difficulty in placing himself in the proper position. As the
executioner raised the axe to cut off his right hand, Münter began
slowly pronouncing the words:—
"Remember Jesus Christ crucified, who died, but is risen again."
The first blow fell, and with it, Struensee was attacked by violent
convulsions, the result of which was, that the second blow intended
to behead the poor wretch, failed. He sprang up convulsively, but
the assistant seized him by the hair, and pulled him down on the
block by force; even when the head was removed, a portion of the
chin was left behind.
The same horrors were committed on his poor corpse as on
Brandt's, but I have no heart to dwell on them: let us rather agree
with the poet in saying,
 "Excidat illa dies ævo: nec postera credant
Sæcula: nos certé taceamus et obruta multa
Nocte tegi nostræ patiamur crimina gentis."

The mangled remains, after they had been thrown into the cart,
were conveyed all through the city to the field at the other extremity,
where they were to be left to moulder or be devoured by the fowls
of the air. For each, four stout balks were, at equal distances, driven
into the earth; a taller pole was fixed in the centre; the entrails, &c.,
were buried in a hole dug at the foot of the central pole; on the top
the head was fixed, the pole being forced up inside the skull,
through which a spike was driven to make it fast; the hand was
nailed on a piece of board, placed transversely below the head; a
cart wheel was fixed horizontally on the top of each of the four posts
or pillars, on which a quarter of the body was exposed, made fast to
the wheel by iron chains.
The countless crowd, whose curiosity was now fully satisfied,
returned to the city, shaken by the scenes they had witnessed, and
the deep impression produced by the awful drama could be noticed
for a long time. Convicts had to be employed on the next day in
removing the scaffold, as no honest man would have a hand in it;
but Gallows Hill preserved its decorations for some years, and even
in 1775, Mr. Coxe saw Struensee's and Brandt's skulls and bones
there.[21] All this was done to satiate the vengeance of the queen
dowager. With a telescope in her hand, Juliana Maria had witnessed
the whole execution from the tower of the Christiansborg, and when
the turn arrived for the special object of her hatred, Struensee, she
rubbed her hands joyously, and exclaimed, "Now comes the fat one."
[22]

But the queen did not neglect to observe decorum even in this affair,
and hence, soon after the execution, sent for Dr. Münter, in order to
hear all the details of the judicial murder from this immediate
witness of the fearful scene. When he had ended his report, the
queen burst into tears; but, as our Danish authority remarks, "it is
notorious that a crocodile can weep." Then she said to Münter—
"I feel sorry for the unhappy man. I have examined myself whether
in all I have done against him I have acted through any feeling of
personal enmity; but my conscience acquits me of it."
After this, the queen dowager gave Dr. Münter a snuff-box of rock
crystal, while a similar gift in porcelain was forwarded to Dean Hee.
[23]

But the historian, Suhm, who was attached to the court, and was
one of the most zealous enemies of Struensee, tells us how far we
are justified in believing the queen dowager's statement. As the
queen occupied the upper floor of Christiansborg Palace, whence a
view of the Gallows Hill was obtained, Chamberlain Suhm asked her
some years later, why her Majesty, who had so many splendid
palaces at her service, inhabited these unpretending rooms, and
received the answer:—
"And yet these rooms are dearer to me than all my most splendid
apartments; for from these windows I saw my bitterest foe exposed
on the wheel."
Such was the end of a man whose miserable story is indubitably one
of the most romantic episodes of his century; and it only required a
Danish Walter Scott, in order to make of it an historical romance of
the first class. For such a work the matter is fully sufficient. But for
the same reason all efforts must fail to convert Struensee into the
hero of a tragedy. Many poets, some of them in the first flight, have
undertaken this ungrateful task, but have not attained any success
worth mentioning. The reason can easily be found. Struensee was
no hero; not even an original: he possessed no distinct character,
but was merely a type of his age, and in spite of his undeniable
talents, he was an ordinary adventurer after all. Fortune is as much
the touchstone of minds as misfortune is. It subjected this man to a
trial, and he came out of it badly. Arrogant and unbridled in fortune,
he proved himself in misfortune despondent, cowardly, and even
worthless. The fortune which he at first did not turn to a bad use,
brought a king's sceptre into his hand, and he allowed it to be
shamefully torn from him by people far inferior to him in intellect. A
queen, young and beautiful as a May morning, supported him, and
he betrayed her. He had felt a pride in being an avowed free-thinker,
and he died with wailing and gnashing of teeth, as a penitent sinner.
No, he was not a tragic hero. Even the genius of a Shakspere would
have failed in rendering him one.
It is a fact worthy of attention that Struensee possessed none of the
qualities which generally presuppose success at court. He was not
an amiable man, in the conventional sense of the term. The English
envoy, Gunning, who was not ill-disposed toward him, expressly
stated, in a despatch of April, 1770, that Struensee did not at all
display in his conversation the liveliness and pleasantry by which
other men pave the way to fortune. "His mode of behaving and
expressing himself is dry and even unpleasant, so that it was a
subject of general surprise how he contrived to acquire such
unbounded influence over the king and queen." Further, the envoy
allows the favourite "no inconsiderable acquirements," but denies
him all statesman-like ability and political tact. At the same time he
was deficient in sufficient insight into Danish affairs. He was
tolerably free from vanity, but not from an immoderate self-
confidence, which not unfrequently degenerated into "impudence."
The envoy, however, supplies us with the key to the enigma of
Struensee's sudden elevation, when he mentions that he was "bold
and enterprising," and such a man is sure to make his way among
women.[24]
Still, in spite of Struensee's deficiency and all his mistakes, so much
justice must be done him as to allow that he desired the welfare of
the state. He originally possessed a not ignoble mind, which was
lowered and degraded by his fabulous elevation and sudden fall.
Being formed of much softer and more worthless stuff than the
metal out of which great, or even second-rate statesmen are
composed, he could not endure either fortune or misfortune. An
idealist, trained in the school of enlightened despotism, he did not
understand that a nation must be raised from the bottom to the top.
This was the mistake of the age. The reasons of state of a Frederick
the Great or a Joseph II. were, after all, only an improvement of the
breed. We have all due respect for those enlightened despots who
have so far freed themselves from the swaddling-clothes of the
Byzantine ideas about the divine right of kings, as to wish
themselves to be merely regarded as the first servants of the state;
but, at the same time, we are inclined to say with old Wieland, "May
Heaven protect us from the luck of being obliged to live under the
sceptre or stick of such first servants of the state." Struensee acted
on the principle that, in order to make nations progress, nothing
further was required than to realise by edicts the principles of the
French philosophers and German illuminati. After the fashion of
many other world-betterers of the age, he did not know or reflect
that it is far more difficult to lead the unjudging masses to what is
good, than to what is bad; that the most absurd prejudices of the
plebs must be humoured far more than the noblest human
privileges; that the coarse diplomacy of pot-house demagogues is
sufficient to make the ignorant mob throw away the diamonds of
truth and eagerly clutch at the strass of falsehood and absurdity;
and that, lastly, the people in all times are most willing, at the desire
of their enemies, to hate, persecute, stone, and crucify their friends.
It is possible, even probable, that, if Struensee had held the power
longer, he would have passed from the experimentalising stage to
really beneficial results. The beginning of his display of power was
not so bad. Denmark had long sighed under the brutal dictatorship
which the envoys of Russia exercised. Struensee broke this yoke,
and did it so cleverly, that the ambitious czarina in Petersburg was
obliged to give way, whether she liked it or not. The management of
the foreign policy by Struensee least of all deserves blame, because
it was based on the sensible principle that Denmark must live in
peace and amity with all states, but not be subject to any one of
them. The same praise cannot be afforded to Struensee's home
administration. The tendency generally was good and reasonable
here, but the execution left much, very much, to be desired. We find
everywhere hasty attempts, but no thorough carrying through. A
despotic theorising, which was followed by no energetic practice,
and the most correct designs destroyed and confused by the
interference of personal interests, sympathies, and antipathies,
characterised the administration.
Struensee's great fault was that he did not, and would not,
understand that in statesman-like calculations, not abstract ideas but
men are the figures employed in reckoning—men with all their
weaknesses, follies, prejudices, and passions. Through mistaking this
great fact, he contrived to embitter all classes of the nation. He
offended the nobles without winning the peasants; he made the
officers, soldiers, and sailors his enemies, without making the
citizens his friends. And he did this among a people whose education
was behindhand, and to whom he was an object of hatred, from the
fact of his being a foreigner.
After his fall, which every one but himself had foreseen—and we
may fairly say that he signed his own death-warrant by the maniacal
cabinet decree which placed all the authority in his hands—
Struensee behaved like a miserable coward and traitor. It has been
said that his judges, or, more correctly, executioners, terrified the
ruined man by a menace of the torture, and, at the same time,
deluded him by the idea that his sole chance of salvation was in
compromising Caroline Matilda most deeply. But, for all this, a man
would never do, and only a weakling and coward would do what he
did, when he confessed, on February 21, that he had been the
queen's lover. From this moment he could only lay claim to a feeling
of contempt. It would not even excuse him were it true, as has been
alleged, that a pretended confession of Caroline Matilda's guilt was
shown him.
Still the means employed to get rid of the favourites were most
reprehensible. It is true that the queen dowager and Prince
Frederick had a right to feel irritated at having no credit at a court
where a Struensee domineered, and that they wished to remove him
and his partisans. We can understand that Queen Juliana Maria, who
had no experience of business, and Prince Frederick, who had scarce
emerged from boyhood, should not suspect the extremities to which
Guldberg's faction would lead them; and it may be true that it was
owing to their generosity that the children of Caroline Matilda were
not deprived of their rights. Nor can we positively condemn Guldberg
for wishing to tear from Struensee powers which Struensee had torn
from others. Perhaps Guldberg possessed more capacity, or a better
claim to hold the power than he. But, as to the means employed in
gaining the object, we cannot help agreeing with Falckenskjold when
he says:
"To make Struensee perish in order to seize on his office, was not
this purchasing it very dearly? and especially to add the punishment
of the unfortunate Brandt to that of Struensee, and to assail the
liberties and fortunes of so many persons who were innocent of the
ambition of these two men. And, in order to give a legal appearance
to these proscriptions, they do not hesitate to abuse whatever is
most sacred in human laws; they convert private intrigues into
judicial proceedings; they employ calumnious libels as authentic
documents and sentences; they raise the veil and expose to the
public the domestic secrets of the king's house! They do not fear
violently to break the happy union of the king with his consort; to
render doubtful the rights of the issue of that union, by
compromising the future tranquillity of the state; and, lastly, to cast
on a young queen the affront of a mortal stain, and to condemn her
to expire in a lengthened agony!
"Was the post of a principal minister of the King of Denmark so
important, or desirable at such a price?"
 CHAPTER IV.
THE HIGH COMMISSION.

THE TEN PRISONERS—THE REPORT—LIEUT.-COLONEL VON HESSELBERG—

ETATS-RATH WILLEBRANDT—PROFESSOR BERGER—UNJUST SENTENCES—
VON GAHLER—FALCKENSKJOLD AND STRUENSEE—SERIOUS CRIMES—THE
SENTENCE—THE ROYAL APPROVAL—THE FORTRESS OF MUNKHOLM—THE
COMMANDANT—RESIGNATION—THE ORDER OF RELEASE—CURIOUS
CONDITIONS—DEATH OF FALCKENSKJOLD.

After the execution of the two counts, the turn came for the other
ten prisoners of state to be disposed of. In order to justify in the
sight of Europe the sanguinary treatment of the two principal
victims, these prisoners must also appear in the light of state
criminals, although no actual offence could be proved against a
single one of them. Although the grounds for their sentences are of
the most paltry description, it is easy to discover in them that, after
the removal of their real enemies, the dominant faction wished to
affect a display of mercy, probably through fear of public opinion,
which was beginning to be loudly expressed both at home and
abroad, about the cruel fate of the two counts.
The court believed it necessary to lay before the nation in print the
sentences of Counts Struensee and Brandt, and with them a list of
the crimes for which they were tried. This measure, however, had
the contrary effect to what was expected. Nothing was seen in this
sentence but a desire to prove the counts guilty, and a long series of
absurd conclusions, by which it was sought to attain this object: the
sentence on Count Brandt, more especially, revolted every one, and
the general dissatisfaction was unmistakable.[25] Hence, although it
was proposed in the council to lop off one or two more heads, one
of the judges, Kofod Ancher, thought it was time to say to the king,
as was said to Augustus, siste carnifex. It was therefore decided that
the only victim of judicial vengeance should be Falckenskjold, who
had dared to be an honest admirer of Struensee, and was really at
the same time an enlightened statesman and politician.
On April 21 a royal command was sent to the commissioners to hand
in an accurate representation of the offences of all the other persons
arrested on January 17, and more especially those who had been
guilty of minor crimes. This order was executed in a "most humble"
report, dated May 5. The poor wretches had been allowed to pass all
this time in prison, suffering from the most painful uncertainty.
Frau von Gähler was placed at the head of the list as the least guilty
sinner. She had attracted suspicion—so it was stated in the report—
by her intercourse with Struensee, by an uninterrupted
correspondence during the Holstein progress with her husband in
Copenhagen, by her being selected to be constantly in attendance
on the queen, by her and her husband's decoration with the queen's
Order of Matilda; and lastly, by the rumour that important papers
had been entrusted to her care. As, however, the closest
investigation had not produced a tittle of proof against her, and her
correspondence with her husband proved to be of the most innocent
nature, while other ladies who stood under no suspicion had enjoyed
equal favour at court, the commission had discovered nothing that
could be brought as a charge against her.
A similar declaration was issued by the commission in the matter of
Lieutenant-Colonel von Hesselberg. This officer who—on account of
the acquirements he had gained on foreign service, and the
intelligence displayed by him while holding an appointment at the
Academy of Military Cadets, had been proposed by Falckenskjold as
lieutenant-colonel of the king's regiment, because the latter wished
to have an efficient representative, while he went on diplomatic
business to Russia—had received orders to have an eye on
Struensee's youngest brother, who was appointed to his regiment.
Such was the state crime brought against this man of honour by the
exalted commissioners.
As an example of the condition of the Danish army at that day, the
circumstances connected with Hesselberg's appointment will be
perused with interest. When Falckenskjold, shortly after his arrival
from Russia, was appointed colonel of the king's regiment, the
quartermaster delivered to him an account of the state of the
regimental chest, according to which there ought to be 25,000
dollars in it. Falckenskjold replied, that he would take over the chest
after a committee of inquiry had sat. The quartermaster objected
that this was displaying an unusual suspicion, but Falckenskjold
stuck to his text, and lo and behold! there were but 95 dollars in the
chest. On his threatening to report the affair, 5,000 dollars were
produced, and the quartermaster bolted. A close investigation
proved that the missing sum had been stolen by frauds, in which the
officers were mixed up, and Falckenskjold was at last obliged to let
the matter drop, but for this very reason selected Hesselberg, in
whom he could trust, as his second in command.
The third ill-used man, against whom the commission could not
produce a scintilla of evidence either, was Rear-Admiral Hansen. He
had first formed Struensee's acquaintance when he received orders
to join the Algerine Committee, and afterwards assumed a special
command in the expedition. He had never spoken on any matters
but those connected with the navy to Struensee, and yet was kept
for months in prison as a dangerous criminal.
Equally little could be alleged against Councillor of Legation Sturtz.
His connection with Struensee had never possessed an intimate
character, which was proved by the fact, that he had remained
attached to Bernstorff. The sole reason for his lengthened residence
at Hirschholm, in the summer of 1771, was, that he, as a clever
amateur in portrait-painting, had received a commission to take the
likeness of the queen and prince royal, and the diamond ring
presented to him by the queen for his bride, was only an
acknowledgment given him for these works of art, which are still in
existence, and said to be excellent likenesses. Nor was anything in
the slightest degree incriminating found among his papers.
Lieutenant Aboe, of the navy, was the next prisoner declared to be
innocent. When a cadet he had formed the acquaintance of Brandt,
and became intimate with him, partly on account of the pecuniary
assistance the latter gave him, partly to be recommended to
Chamberlain Struensee, through whose application he obtained the
interim post of master of the crews at the navy docks, and of
adjutant to the Algerine expedition. In the latter capacity he
frequently waited on Struensee, and handed in some pretended
ameliorations in the administration of the navy, which, however, only
proved his ignorance. He had no further connection with Struensee,
and was, indeed, offended with the cabinet minister, because the
latter did not sufficiently recognise his merits.
The examination had thus proved that five innocent persons had
been kept in a state of torturing uncertainty as to their fate, and in
cruel imprisonment for four months and a half, while their relatives
were exposed to public contumely.
Of Etats-rath Willebrandt it was alleged that he had been so simple
and bold as to wish to reform the Admiralty College, without having
gained a proper insight of the administration of the navy, much less
of the difference in the business of the department during peace and
war, as he had solely obeyed the order given him to establish the
Admiralty College after the pattern of that of the Generalty. The
commissioners, it is true, declared that it was not within their
competence to judge the value of a plan which in any case had been
approved by his Majesty, and for this reason did not dare to express
dissatisfaction with it. Still they thought themselves at liberty to state
that which the result proved, that a portion of the plan, owing to
Willebrandt's ignorance, produced irregularity and inconveniences,
which could only be prevented by alterations and fresh expenses for
the king's treasury. After this, an allusion was made to the affair with
the enrolled sailors, who marched on Hirschholm in 1771.
Willebrandt's offence, therefore—the report went on to say—
consisted in his having undertaken to reform things of which he had
no thorough knowledge, and he thus produced scenes which might
easily have had dangerous consequences. It was not mentioned in
the report, however, that the king had given him this order, that
Count Haxthausen took part in it, and that both, when they drew up
the plan, expressly requested that the new scheme might be
previously examined by professional men.
Professor Berger, the physician in ordinary, also appeared to the
commission to have committed an offence. It was true that all the
medicines found in his house proved to be innocuous; there was no
proof that the steel cure attempted on his Majesty was improper; in
the matter of the rearing of the crown prince he had not agreed in
all points with Struensee, but rather had been the cause that
warmer clothing and better food were granted his royal highness
toward the close: it was also true that he expressed his anger at
Struensee having acquired a power which he considered prejudicial
to the nation; but he had given serious offence by concurring with
Struensee, and giving his advice and propositions in reforms of
which he probably knew nothing, more especially in things which did
not concern his trade as physician, or the res literariæ generally. As
a proof of this, it was alleged that he had proposed some persons to
fill the places of the dismissed members of the magistracy, and that
Struensee requested him to mention a person who would be suitable
for the chief post in the navy yards. Still the commission would not
venture to judge of the use or disadvantage of such propositions, as
the papers found on the accused and the other prisoners, on which
their argument must be founded, contained no information about
them.
This was everything that could be brought against a professional
man, who, in spite of his zealous attention to the king's health, and
his well-earned reputation, was dragged to the fortress, and, like a
murderer, prohibited the use of knife and fork, and was not allowed
to shave himself, or sleep on his own mattress.
If the conduct of the commission had hitherto retained a varnish of
justice, the royal resolution, minuted by Councillor of Conference
Schumacher, and issued on May 18, was a strange proof of tyranny,
which smote truth on the face with open falsehoods. For it was
stated in this resolution, that the persons who had been guilty of the
"smallest crimes" should be punished in the following way:—
Frau von Gähler would be set at liberty, but must refrain from
appearing at court so long as her husband's affair was not
concluded.
Rear-Admiral Hansen and Lieutenant Aboe would be discharged from
arrest, and report themselves to the Admiralty College, where they
would learn the king's commands with respect to them.
Legations-rath Sturtz would also be released from arrest, and
ordered to proceed to Holstein. He would retain his pension of 500
dollars, which had been granted him by the royal resolution of
January 26, but must expend it away from the court.
Etats-rath Willebrandt, after being discharged, would proceed to one
of the small towns in Seeland, where an annual pension of 300
dollars would be paid him.
Professor Berger, lastly, after being set at liberty, would go to
Aalborg, in northern Jütland, where a pension of 300 dollars would
be paid him, until a post of provincial surgeon became vacant in
Jütland.
These decisions the commissioners made known to the persons
concerned with the solemn warning that, after the king had
pardoned them this time, through special mercy, for their incautious,
thoughtless, and criminal conduct, they must be very careful not to
give rise to greater suspicion by word or writing, as, in that case,
they would be subjected to a further examination, and might expect
the king's most serious displeasure.
The sovereign lord over the life and death of his subjects was
consequently of a different opinion from the commissioners, who
had found the accused guilty of no offence. But it was considered
desirable to get rid of those persons most hated, and, in order to
convict them, it was requisite to accuse them of offences at the
expense of truth.
Lieutenant-Colonel von Hesselberg, who was referred to the
Commissariat College, learnt there that the king had appointed him
Commander of the 2nd National Battalion of Schleswig-Holstein. This
distinguished officer afterwards became colonel of an infantry
regiment in Norway, where he died in 1808, a lieutenant-general,
and commandant of the fortress of Bergenhuus.
Rear-Admiral Hansen was informed by the Admiralty that he had
forfeited his post as deputy of the latter college, but would continue
to serve the state. He died a few years after the catastrophe.
Lieutenant Aboe, who also learnt his future fate from the Admiralty,
received orders to pass two years abroad, but retained his
commission and pay. Eventually, he left the service with the rank of
captain, set up as a merchant in Copenhagen, failed, made voyages
to the East Indies, and died after many hard adventures in
Copenhagen.
The three exiles, Legations-rath Sturtz, Etats-rath Willebrandt, and
Professor Berger, in obedience to the royal commands, quitted the
capital, and proceeded to their several destinations. When
Falckenskjold was recalled from Switzerland by the crown prince in
1788, he found Willebrandt still in exile: Berger was a practising
physician at Kiel, but Sturtz had died of grief.[26]
The public of Copenhagen were astonished at this mild treatment of
persons who had been kept in such close arrest. Much worse had
been anticipated. But three state criminals still remained in prison,
and what had been spared their associates, who were punished for
having been proved innocent, could be done to them.
Lieutenant-General von Gähler, Colonel and Chamberlain von
Falckenskjold, and Justiz-rath Struensee, were still awaiting their
sentence. But on May 10 an order had been issued to the
commission to lay before the king a full report of the crimes of these
men, for his Majesty's most gracious consideration and resolution. In
obedience with this command, the commission sent in its report on
May 30.
With respect to Gähler, it was alleged that he was mixed up both in
the Traventhal league and the abolition of the council. It was true
that he had denied both, but, on the first point, the letters found at
his house contradicted him. As regards the council, he had not, as
his duty ordered, sufficiently represented the value of the council in
his answers to the questions laid before him for explanation on Sept.
24, 1770, and there were even strong reasons for conjecturing that
he proposed and promoted the abolition of the council, because he
was Struensee's principal adviser about this time. In the same way
he had recommended to his friend Struensee, the abolition of the
verbal reports of the colleges. By this, the general had helped to
conceal Struensee's audacious conduct from the king, and given
him, Struensee, opportunity for filching all the power and authority.
It was allowed that Von Gähler, by his propositions, had no intention
of sustaining Struensee in his situation and promoting his autocracy.
Still, he ought, and must have noticed Struensee's boundless
ambition, when he perceived that the latter "wished to apply the
practice of his profession to the state, and began by amputating
from it so important a limb as the council was." General von Gähler
ought the less to have attempted to promote Struensee's views, as
he was not adapted either by nature or Providence to regulate or
remodel a state. He ought not to have furnished Struensee with
projects, all the consequences of which he could not foresee. More
especially, he ought not to have advised the suppression of verbal
reports, but to have always opposed it. But he appeared to have
been possessed by a mania for reformation. As a proof, it might be
mentioned that he proposed the reform for which was introduced
into Norway by the regulation of January 14, 1771, that lands, after
ten years' tenure, should become freehold, which no man of
perspicuity could have advised.[27] The commission found a second
instance in the reform of the two Chanceries, although the division
of business, according to provinces, had had the best results.
Before all, however, Von Gähler wished to remodel the navy. It was
quite incredible what tricks he employed to get it into his hands, and
the commission reports that the misfortune which befel the Algerine
expedition gave Von Gähler the desired opportunity for effecting it.
Herr von Gähler's crime, therefore, principally consisted in the fact,
that he interfered in everything, and wished to reform all the
regulations of the state, without possessing the requisite knowledge
and insight, without knowing the advantages or defects of what
existed, and without sufficiently pondering over the consequences of
his propositions. The commission, however, could find no excuse in
the circumstance that Von Gähler's proposed reforms only consisted
of ideas and thoughts, whose trial by experiments injured nobody,
because most of the affairs in which he interfered in no way
concerned him, and the trouble he took in order to obtain a
justification for doing so, proved a greater offence; for he had
applied to Struensee, a man who was even more ignorant than
himself in such things, and blindly followed everything that was
proposed to him, especially when such propositions suggested
radical changes. The commission, however, would not omit
mentioning that the general, since May, 1771, had possessed no
special influence over Struensee, because he had joined the
opposition against the reduction of the Horse Guards, and besides,
he had not commended himself to the cabinet minister, by
representing to him how little the power he had appropriated agreed
with the royal law. Lastly, the general also displayed firmness when
the Foot Guards were disbanded; he had likewise resisted the
removal of the two regiments, and in the Generalty represented to
Chamberlain von Falckenskjold how improper it was to propose the
regiment of the hereditary prince for such a dislocation, without first
asking whether this would be agreeable to the prince.
The second of the criminal three was Colonel and Chamberlain von
Falckenskjold. According to the opinion of the commission, he was
the man who, next to Brandt, stood in the closest intimacy with
Struensee. Perhaps, however, Falckenskjold's notorious dislike of the
hereditary prince, and his bold and manly behaviour in the presence
of the commission, had their share in prejudicing his judges against
him, so that they, through personal hatred, behaved in the most
unscrupulous way toward a man of honour. Professor Sevel acted as
inquisitor, and seemed to find a pleasure in insulting the fallen friend
of Struensee by all sorts of cruel questions. We can form an idea of
this man's moral value on seeing that Sevel, in his examination, so
far forgot what he owed to himself as a judge, as to express his
regret that Struensee had not been murdered by the sailors. In their
report, the commissioners first made Falckenskjold's intimacy with
Struensee a capital offence, and asserted that he had sought to
maintain this intimacy so eagerly, because he and Brandt had
received the greatest benefactions from Struensee. The latter not
only conferred on him offices and honorary posts, but also gave him
money out of the royal treasury.
Thus, Falckenskjold, although on May 2, 1771, he had received from
the cabinet treasury, in payment of his travelling expenses to
Petersburg, the usual sum of 400 dollars, obtained on the 19th of
the same month 2,000 dollars more, under the same excuse, from
the private treasury, and, after his return, or in a period of three
months, a further sum of 3,500 dollars. Of these amounts, Struensee
paid him 1,000 dollars under the false allegation that Falckenskjold
had spent them on the journey from his private means, while the
2,000 dollars were paid him without the king's cognizance. How he
had earned these presents, neither he nor Struensee would have
been able to specify, and the assertion that the king had promised to
pay Von Falckenskjold's debts by degrees, was only an empty
pretext, for the latter had deposited 2,000 dollars with the minister
of finances, and therefore could only have had debts to the amount
of 400 dollars; and moreover, he never expressed the proper thanks
to his Majesty for such large gifts in money. The commission
consequently assumed that Struensee desired to acquire
Falckenskjold's gratitude, and declared in their report that they had
strong grounds for believing that it had been arranged between the
couple, that Struensee should be supported under all circumstances,
and guarded against any possible surprise, on which Falckenskjold's
own fortunes also depended.
In understanding with Struensee, he proposed the abolition of the
Chevalier guard, and no other had been more busy than
Falckenskjold in setting at work the cabinet order of December 21. If
any event occurred, and Struensee believed himself in danger,
Falckenskjold was immediately at hand; he had not merely proved
his devotedness to Struensee in this way, but also, for the sake of
pleasing the minister, had neglected the reverence due to the
hereditary prince, and in this had gone so far that he had furnished
proofs of it in the presence of the entire public. Two facts had
convinced the commission of this daring sentiment of Falckenskjold,
which was based on affection for Struensee, namely, the removal of
the prince's regiment, and more especially the occurrence on the
walls. The latter event, the commission represented as follows:—
In the spring of 1771, the prince was, one day, riding along the walls
at the moment when Colonel Falckenskjold was "exercising" his
band, composed of hautboists and fifers; the colonel blocked the
road, and marched straight upon the prince. Both majors of the
regiment called his attention to the fact, that the prince was coming
toward them; but Falckenskjold let his men march on. A groom of
the prince's now rode up, and requested room for his royal highness
to pass. The adjutant reported it to the colonel, and asked whether
the band should not leave off playing, and room be made for the
prince? but Falckenskjold answered: "No, not even if the good God
were to come along Himself." He allowed the band to continue
playing, and the prince, in order to pass, was compelled to ride close
to the parapet.
Falckenskjold alleged, in his excuse, that he had orders to let the
band play in public places, especially when the king came past, and
for this reason there would have been an impropriety in his stopping
the band on the arrival of the prince. The colonel also observed that,
in France, where he had served a long time, an officer was rarely on
guard without hearing the sentinel shout, "Aux armes! le bon Dieu
arrive!" when the Catholic priests passed with the host, and hence it
had grown into a habit to confirm a negative by saying, "No, I would
not do it, even if le bon Dieu were to come." The commission,
however, considered that this sort of defence contradicted itself, and
the colonel ought certainly to have made way for the prince.
Colonel Falckenskjold's crime, consequently, consisted in his having
sold himself to Struensee, in having always had an understanding
with him, in having advised the reduction of both Guards, in having
expressed himself for Struensee's conservation, in having given
proofs, on every occasion, how anxious he was that Struensee
should escape a surprise, and lastly, in having tried, for the sake of
pleasing Struensee, to cause annoyance to his Royal Highness Prince
Frederick, and, for this object, having been so audacious as to
neglect the respect due to his royal highness. Without dwelling on
Falckenskjold's numerous and high services to the king and country,
the commission concluded their report with the disgraceful
statement, that they could not refrain from seeing in Colonel and
Chamberlain von Falckenskjold a foolhardy, detrimental, and the
more dangerous man, because he would do anything for money.
The third and last upon whom the commission had to express an
opinion was Justiz-rath Struensee. As he had only been released
from his chains by special favour, after his brother's execution, it
might have been expected that the commission would depict him as
a great criminal. It was quite different, however, though through no
love of truth and justice on the part of the Inquisition, but in
consequence of commands from higher quarters. For Frederick II. of
Prussia, who had kept Struensee's place, as professor at Liegnitz,
open for him, while he went to try his luck, allowed his minister, Herr
von Arnim, whose tutor Struensee had been, to employ his master's
name in claiming him.[28]
The report on this prisoner of state, after opening with the
statement that Justiz-rath Struensee, though he only possessed a
theoretical knowledge of the laws and constitution of Denmark,
readily accepted a seat as deputy in the Financial Department, and
had the special inspection of the Mint, the Bank, and the course of
Exchange—in the hope of obtaining a better knowledge of these
branches of the administration by industry and work—acknowledged
his irreproachable conduct in the latter respect, and added the
remark, that the commission could express this with the greater
confidence, as the College of Finances, at their request, had had the
matter examined by an authority on the subject. As concerned his
functions as deputy of the finances generally, however, the
commission must blame Justiz-rath Struensee for a tendency to
foolhardy boasting about services which he had not rendered, and
arbitrariness in financial matters, as he wrote to a friend that all the
others in the Financial College understood nothing, and eventually
strove to become Contrôleur Général des Finances.
Although, as regarded his official administration as deputy, there
were no positive proofs that he had abused his charge to the injury
of the king and country for his own interest, yet there was a
tolerably strong presumption that, with the help of his brother and
his office, he wished to render all the subjects in Denmark tributary
to a few Brandenburg partners, who were to hold the salt and
tobacco monopolies, in exchange for profits promised to him and his
brother. As regards the salt-farming, the commission allowed that
the Justiz-rath had accepted no benefit for himself, but annually paid
over to the king the 10,000 dollars intended for himself. In the
matter of the tobacco-farming, however, some suspicion clung to
him that he, contrary to his brother's advice, accepted the two
shares offered him to promote the affair. But although, in spite of all
Justiz-rath Struensee's allegations to the contrary, the strongest
presumptions existed against him and his integrity, the
commissioners felt themselves bound to state that it was not his
intention to deprive the Danish tobacco-planters and spinners of
their livelihood, as the farming proposition was not carried into
effect.[29]
However—thus runs the forced conclusion of the report:—the reports
as to the Justiz-rath's former life in Prussia are generally satisfactory,
and he is honoured with the distinguished favour of Prince Henry,
the brother of Frederick II., who takes a lively interest in him, and
considers him incapable of any serious crime. Besides, no traces had
been found that the Justiz-rath had intrigued with his brother, and
just as few that he had interfered in things that did not concern him.
Finally, it could not be alleged as an offence that he received 4,000
dollars from the private treasury, under the title of gratifications,
because the first 2,000 were given him with the king's assent, and
there was no evidence of his being aware that the other 2,000 were
paid him without the cognizance of the king.
After the commissioners, probably with a heavy heart, had closed
their report in so mild a way, there followed, on June 12, 1772, a
royal resolution, containing his Majesty's "will and commands"
respecting the crimes of Lieutenant-General von Gähler, Colonel and
Chamberlain von Falckenskjold, and Justiz-rath and Deputy of
Finances Struensee. The king had learned from the report of the
commissioners that—
Von Gähler generally undertook to remodel and reform the whole
arrangements of the state, though he possessed no vocation or
knowledge for it; further, that Von Falckenskjold devoted himself
entirely to Struensee, was joined with him in a portion of his
injurious enterprises, and declared himself in favour of his
conservation, and on all occasions furnished proof of this, and
displayed a further audacious mode of behaviour; and that, lastly,
Strong presumptions existed against Justiz-rath Struensee and his
honest performance of his duties, and that he had not fully
consulted with the other persons concerned on matters that came
before the College of Finances.
For this reason, the commission would make known to Lieutenant-
General von Gähler, that, on account of his thoughtless and improper
undertaking, he was dismissed from the king's service, had forfeited
the royal favour granted him in the decree of March 26, 1767,[30]
and must at once select a spot in the Danish monarchy—Seeland,
Fühnen, and Schleswig excepted—where he would permanently