Abstract
In this paper, we propose an efficient Least Squares Support Vector Machine (LS-SVM) training algorithm, which incorporates sparse representation and dictionary learning. First, we formalize the LS-SVM training as a sparse representation process. Second, kernel parameters are adjusted by optimizing their average coherence. As such, the proposed algorithm addresses the training problem via generating the sparse solution and optimizing kernel parameters simultaneously. Experimental results demonstrate that the proposed algorithm is capable of achieving competitive performance compared to state-of-the-art approaches.
Article PDF
Similar content being viewed by others
References
A. Kuh, P.D. Wilde, Comments on “Pruning error minimization in least squares support vector machines.” IEEE Trans. Neural Netw. 18 (2007), 606–609.
J.A.K. Suykens, L. Lukas, J. Vandewalle, Sparse approximation using least squares support vector machines, IEEE Int. Symp. Circuits Syst. 2 (2000), 757–760.
L. Yuan, C. Lin, W. Zhang, Improved sparse least squares support vector machine classifiers, Neurocomputing. 69 (2006), 1655–1658.
R. Mall, J.A.K. Suykens, Very sparse LSSVM reductions for large-scale data, IEEE Trans. Neural Netw. Learn. Syst. 26 (2015), 1086–1097.
D.A. Silva, J.P. Silva, A.R.R. Neto, Novel approaches using evolutionary computation for sparse least squares support vector machines, Neurocomputing. 168 (2015), 908–916.
J. Yang, J. Ma, A sparsity-based training algorithm for least squares SVM, in 2014 IEEE Symposium on Computational Intelligence and Data Mining (CIDM), Orlando, FL, USA, 2014, pp. 345–350.
Y. Tan, Y.C. Fang, Y. Li, W. Dai, Adaptive kernel size selection for correntropy based metric, in: J. Park, J. Kim (Eds.), Computer Vision – ACCV 2012 Workshops, Springer, Berlin, Heidelberg, 2013, pp. 50–60.
C.L. Bcklin, C. Andersson, M.G. Gustafsson, Self-tuning density estimation based on Bayesian averaging of adaptive kernel density estimations yields state-of-the-art performance, Pattern Recognit. 78 (2018), 133–143.
S.G. Vega, X.J. Zeng, J. Keane, Learning from data streams using kernel least-mean-square with multiple kernel-sizes and adaptive step-size, Neurocomputing. 339 (2019), 105–115.
L.X. Yang, M. Wang, S.Y. Yang, R. Zhang, P.T. Zhang, Sparse spatio-spectral LapSVM with semisupervised kernel propagation for hyperspectral image classification, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10 (2017), 2046–2054.
Y.H. Shao, C.N. Li, M.Z.Liu, Z. Wang, N.Y. Deng, Sparse Lq-norm least squares support vector machine with feature selection, Pattern Recognit. 78 (2018), 167–181.
M. Yin, Z.Z. Wu, D.M. Shi, J.B. Gao, S.L. Xie, Locally adaptive sparse representation on Riemannian manifolds for robust classification, Neurocomputing. 310 (2018), 69–76.
J. Wang, S. Daming, D.S. Cheng, Y.Q. Zhang, J.B. Gao, LRSR: lowrank-sparse representation for subspace clustering, Neurocomputing. 214 (2016), 1026–1037.
J. Yang, J. Ma, Compressive sensing-enhanced feature selection and its application in travel mode choice prediction, Appl. Soft Comput. 75 (2019), 537–547.
H.F. Zhu, G.H. Yang, W. Chen, Efficient implementations of orthogonal matching pursuit based on inverse cholesky factorization, in 2013 IEEE 78th Vehicular Technology Conference (VTC Fall), Las Vegas, NV, USA, 2013, pp. 1–5.
M. Elad, Optimized projections for compressed sensing, IEEE Trans. Signal Process. 55 (2007), 5695–5702.
C.Y. Lu, H. Li, Z.C. Lin, Optimized projections for compressed sensing via direct mutual coherence minimization, Signal Process. 151 (2018), 45–55.
M.L. Xie, Z.X. Ji, G.Q. Zhang, T. Wang, Q.S. Sun, Mutually exclusive-KSVD: learning a discriminative dictionary for hyperspectral image classification, Neurocomputing. 315 (2018), 177–189.
I. Tosic, P. Frossard, Dictionary learning: what is the right representation for my signal?, IEEE Signal Process. Mag. 28 (2011), 27–38.
D. Dua, C. Graff, UCI Machine Learning Repository, School of Information and Computer Sciences, University of California, Irvine, 2017.
J. Yang, J. Ma, Feed-forward neural network training using sparse representation, Expert Syst. Appl. 116 (2019), 255–264.
L.X. Yang, S.Y. Yang, S.J. Li, R. Zhang, F. Liu, L.C. Jiao, Coupled compressed sensing inspired sparse spatial-spectral LSSVM for hyperspectral image classificatio, Knowl. Based Syst. 79 (2015), 80–89.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
This is an open access article distributed under the CC BY-NC 4.0 license (https://doi.org/creativecommons.org/licenses/by-nc/4.0/).
About this article
Cite this article
Yang, C., Yang, J. & Ma, J. Sparse Least Squares Support Vector Machine With Adaptive Kernel Parameters. Int J Comput Intell Syst 13, 212–222 (2020). https://doi.org/10.2991/ijcis.d.200205.001
Received:
Accepted:
Published:
Version of record:
Issue date:
DOI: https://doi.org/10.2991/ijcis.d.200205.001