Eyper Sinh Convolutional Neural Network For Early Detection of Parkinsons Disease From Spiral Drawings
Eyper Sinh Convolutional Neural Network For Early Detection of Parkinsons Disease From Spiral Drawings
net/publication/350527156
CITATIONS READS
13 496
4 authors, including:
Mansour Youseffi
University of Bradford
154 PUBLICATIONS 1,219 CITATIONS
SEE PROFILE
All content following this page was uploaded by Luca Parisi on 03 April 2021.
Abstract—Modern developments in the state-of-the-art open-source activation functions for Convolutional Neural
Networks (CNNs) have broadened the selection of benchmark activations for Deep Learning (DL)-aided
classification. Nevertheless, achieving discrimination of non-linear input image data in CNN is still not
straightforward and it is unclear how such novel activation functions can have translational applications with
tangible impact. hyper-sinh, made freely available in TensorFlow and Keras, was demonstrated as a benchmark
activation function on five (N=5) datasets in its ground-breaking paper. Measuring the value from deploying this
activation in a specific application is pivotal to supply the required evidence of its performance on real-life
supervised DL-based image classification tasks. In this study, a CNN was for the first time combined with hyper-
sinh to aid early detection of Parkinson’s Disease (PD) from discriminating pathophysiological patterns extracted
from spiral drawings. Thus, the hyper-sinh activation was deployed to maximise the separability of the input
features from spiral drawings via automated pattern recognition. We demonstrate the accuracy and reliability of
hyper-sinh-CNN to aid early diagnosis of PD, evaluated against other gold standard activation functions, including
the recent Quantum ReLU (QReLU) and the modified Quantum ReLU (m-QReLU) that solved the ‘dying ReLU’
problem for the first time in the literature of DL. Two (N=2) benchmark datasets from the database of the Botucatu
Medical School, São Paulo State University in Brazil, scaled to be in 28 by 28 pixels as the MNIST benchmark
data, were used to discriminate between input image patterns of 158 subjects (53 healthy controls and 105 patients
with PD) from spirals drawn on graphics tablets. Overtraining was avoided via early stopping and the models were
developed and tested in TensorFlow and Keras (Python 3.6). The supervised model (hyper-sinh-CNN) could detect
early Parkinson’s Disease with 81% and 91% classification accuracy from the two datasets respectively (F1-scores:
73% and 91% correspondingly). Furthermore, the model achieved high sensitivity (81% and 91%). Thus, this study
validates the application of hyper-sinh to aid real-life supervised DL-based image classification, in particular early
diagnosis of PD from spiral drawings.
Keywords: Convolutional Neural Network; Parkinson’s Disease; m-arcsinh; hyper-sinh; QReLU; m-QReLU.
Received: February 12, 2021. Revised: March 3, 2021. Accepted: March 19, 2021. Published: March 31, 2021.
Both datasets were represented as one-dimensional Given its ability to cope with non-linearly separable
(1D) vectors of 784 features for each image (28*28 pixels); data, the hyper-sinh (Fig. 1) activation function was
training and testing datasets were used as per the coupled with CNN and evaluated against other gold
proportions already established from the two benchmark standard activations (ReLU, Leaky ReLU, sigmoid, tanh,
datasets used. QReLU and m-QReLU [16]).
The data were reshaped to match the picture format The equations for the hyper-sinh activation function for
(height * width * channel) and then, to a four-dimensional positive values of input (x) data, and negative values and
(4D) tensor input (batch size, height, width, channel). The zero, are available in (1) and (2) respectively.
image datasets underwent min-max normalisation to
prepare them for the subsequent CNN-based classification.
sinh 𝑥 (1)
The CNN model was developed and evaluated via the 𝑦 = 𝑖𝑓 x > 0 ,
Keras library in Python, whilst the hyper-sinh [7]
3
activation function was developed in both TensorFlow and 𝑤ℎ𝑒𝑟𝑒 𝑥 𝑖𝑠 𝑡ℎ𝑒 𝑖𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
Keras [20]. All other activation functions evaluated
(ReLU, Leaky ReLU, sigmoid and tanh) are available in
Keras, except for the custom QReLU and m-QReLU [16]
also developed and available in both TensorFlow and
Keras [18]. 𝑥3
𝑦 = 4
𝑖𝑓 x ≤ 0 (2)
2.2 hyper-sinh-based CNN Algorithm A flattening layer was applied to flatten the input data
to a one-dimensional (1D) vector for the fully connected
layer. The output layer had two neurons, as per the number
CNN is a deep neural network-based classifier that
of classes to discriminate (healthy vs PD-related image
leverages convolutional layers and filters to achieve
patterns). In Table I, these and other hyperparameters used
automated pattern recognition without the need for
in the CNN model tested in this study are listed, which
extrinsic feature engineering and separability of the input
were leveraged to classify both benchmark datasets, In binary classification problems, e.g., discriminating
regardless of the activation functions evaluated. between healthy and PD-related image patterns, classes can
be defined as ‘true’ or ‘false’, resulting in four scenarios
TABLE I. LEARNING HYPERPARAMETERS FOR THE CNN
MODEL USED FOR BOTH BENCHMARK DATASETS AND (Table II):
REGARDLESS OF THE ACTIVATION FUNCTIONS DEPLOYED.
Hyperparameter Value
true positive (TP), the occurrence of
Learning rate 0.001 pathophysiological image patterns correctly
Maximum number of epochs or 15
detected (e.g., due to PD).
training iterations true negative (TN), physiological/healthy speech
image correctly classified.
Batch size 128
false positive (FP), physiological/healthy image
Dropout rate prior to output layer 0.25 patterns misclassified as if they were
pathophysiological instead.
Number of convolutional layers 2
false negative (FN), the occurrence of a
Number of max pooling (down- 2 pathophysiological image patterns misclassified
sampling) layers as if they were physiological/healthy instead.
Number of convolutional filters in first 32
convolutional layer
TABLE II. CONFUSION MATRICES RELATING THE ACTUAL
Kernel size of first convolutional layer 5
AND PREDICTED IMAGE PATTERNS AS TO WHETHER THEY
Number of convolutional filters in 64 ARE INDICATORS OF PATHOPHYSIOLOGICAL TREMOR (E.G.,
second convolutional layer DUE TO EARLY PARKINSON’S DISEASE).
Negative
(physiological FN TN
2.3 Performance Measures to Validate image patterns)
Model Accuracy and Reliability
a) b)
Figure 1. The hyper-sinh activation function for positive values of input data (a)) and for negative values or zero of input data (b)).
References
[1] Chakravarthy, K. V., Chaturvedi, R., Agari, T., Iwamuro, H., [14] Parisi, L., RaviChandran, N., & Manaog, M. L. (2018).
Reddy, R., & Matsui, A. (2020). Single arm prospective Decision support system to improve postoperative
multicenter case series on the use of burst stimulation to discharge: A novel multi-class classification
improve pain and motor symptoms in Parkinson’s approach. Knowledge-Based Systems, 152, 1-10.
disease. Bioelectronic medicine, 6(1), 1-8.
[15] Dahl, G. E., Sainath, T. N., & Hinton, G. E. (2013, May).
[2] Parisi, L., RaviChandran, N., & Manaog, M. L. (2018). Improving deep neural networks for LVCSR using rectified
Feature-driven machine learning to improve early diagnosis linear units and dropout. In 2013 IEEE international
of Parkinson's disease. Expert Systems with conference on acoustics, speech, and signal processing (pp.
Applications, 110, 182-190. 8609-8613). IEEE.
[3] Carmeli, E., Patish, H., & Coleman, R. (2003). The aging [16] Parisi, L., Neagu, D., Ma, R., & Campean, F. (2020). QReLU
hand. The Journals of Gerontology Series A: Biological and m-QReLU: Two novel quantum activation functions to
Sciences and Medical Sciences, 58(2), M146-M152. aid medical diagnostics. arXiv preprint arXiv:2010.08031.
[4] Rosenblum, S., Samuel, M., Zlotnik, S., Erikh, I., & [17] Glorot, X., Bordes, A., & Bengio, Y. (2011, June). Deep
Schlesinger, I. (2013). Handwriting as an objective tool for sparse rectifier neural networks. In Proceedings of the
Parkinson’s disease diagnosis. Journal of neurology, 260(9), fourteenth international conference on artificial intelligence
2357-2361. and statistics (pp. 315-323). JMLR Workshop and
Conference Proceedings.
[5] Drotár, P., Mekyska, J., Rektorová, I., Masarová, L.,
Smékal, Z., & Faundez-Zanuy, M. (2016). Evaluation of [18] Parisi, L. (2020). QReLU and m-QReLU in TensorFlow and
handwriting kinematics and pressure for differential Keras. GitHub repository. https://github.com/luca-
diagnosis of Parkinson's disease. Artificial intelligence in parisi/QReLU_m-QReLU_TensorFlow_Keras.
Medicine, 67, 39-46.
[19] Parisi, L. (2020). m-arcsinh in scikit-learn. GitHub
[6] San Luciano, M., Wang, C., Ortega, R. A., Yu, Q., Boschung, repository. https://github.com/luca-parisi/m-arcsinh_scikit-
S., Soto-Valencia, J., ... & Saunders-Pullman, R. (2016). learn.
Digitized spiral drawing: a possible biomarker for early
Parkinson’s disease. PloS one, 11(10), e0162799. [20] Parisi, L. (2020). hyper-sinh in TensorFlow and
Keras. GitHub repository. https://github.com/luca-
[7] Parisi, L., Ma, R., RaviChandran, N., & Lanzillotta, M. parisi/hyper-sinh_TensorFlow_Keras.
(2020). hyper-sinh: An Accurate and Reliable Function from
Shallow to Deep Learning in TensorFlow and Keras. arXiv [21] Pereira, C. R., Pereira, D. R., Silva, F. A., Masieiro, J. P.,
preprint arXiv:2011.07661. Weber, S. A., Hook, C., & Papa, J. P. (2016). A new
computer vision-based approach to aid the diagnosis of
[8] Clevert, D. A., Unterthiner, T., & Hochreiter, S. (2015). Fast Parkinson's disease. Computer Methods and Programs in
and accurate deep network learning by exponential linear Biomedicine, 136, 79-88.
units (elus). arXiv preprint arXiv:1511.07289.
[22] Pereira, C. R., Weber, S. A., Hook, C., Rosa, G. H., & Papa,
[9] Pascanu, R., Mikolov, T., & Bengio, Y. (2012). J. P. (2016). Deep learning-aided Parkinson's disease
Understanding the exploding gradient problem. CoRR, diagnosis from handwritten dynamics. In 2016 29th
abs/1211.5063, 2(417), 1. SIBGRAPI Conference on Graphics, Patterns, and Images
(SIBGRAPI), 340-346. IEEE.
[10] Parisi, L. (2020). m-arcsinh: An Efficient and Reliable
Function for SVM and MLP in scikit-learn. arXiv preprint [23] LeCun, Y., & Bengio, Y. (1995). Convolutional networks for
arXiv:2009.07530. images, speech, and time series. The handbook of brain
theory and neural networks, 3361(10), 1995.
[11] Parisi, L., RaviChandran, N., & Manaog, M. L. (2020). A
novel hybrid algorithm for aiding prediction of prognosis in [24] Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B.,
patients with hepatitis. Neural Computing and & Chen, T. (2018). Recent advances in convolutional neural
Applications, 32(8), 3839-3852. networks. Pattern Recognition, 77, 354-377.
[12] Parisi, L., & RaviChandran, N. (2020). Evolutionary
Denoising-Based Machine Learning for Detecting Knee Creative Commons Attribution License 4.0
Disorders. Neural Processing Letters, 52(3), 2565-2581.
(Attribution 4.0 International, CC BY 4.0)
[13] Parisi, L., & RaviChandran, N. (2020). Evolutionary feature
transformation to improve prognostic prediction of This article is published under the terms of the Creative
hepatitis. Knowledge-Based Systems, 200, 106012. Commons Attribution License 4.0
https://creativecommons.org/licenses/by/4.0/deed.en_US