May 2025: Top 10 Read

Articles in International
Journal of Artificial
Intelligence &Applications

International Journal of Artificial

Intelligence & Applications (IJAIA)

http://www.airccse.org/journal/ijaia/ijaia.html

ISSN: 0975-900X (Online); 0976-2191 (Print)

Contact Us: [email protected]

 HARDWARE DESIGN FOR MACHINE LEARNING
Pooja Jawandhiya

School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore

ABSTRACT

Things like growing volumes and varieties of available data, cheaper and more powerful
computational processing, data storage and large-value predictions that can guide better decisions
and smart actions in real time without human intervention are playing critical role in this age. All
of these require models that can automatically analyse large complex data and deliver quick
accurate results – even on a very large scale. Machine learning plays a significant role in
developing these models. The applications of machine learning range from speech and object
recognition to analysis and prediction of finance markets. Artificial Neural Network is one of the
important algorithms of machine learning that is inspired by the structure and functional aspects of
the biological neural networks. In this paper, we discuss the purpose, representation and
classification methods for developing hardware for machine learning with the main focus on neural
networks. This paper also presents the requirements, design issues and optimization techniques for
building hardware architecture of neural networks.

KEYWORDS

Artificial intelligence (AI), application specific integrated circuit (ASIC), artificial neural network
(ANN), central processing unit (CPU), field programmable gate array (FPGA), graphics processing
unit (GPU), machine learning (ML), neurochip

For More Details: https://aircconline.com/ijaia/V9N1/9118ijaia05.pdf

Volume Link: http://www.airccse.org/journal/ijaia/current2018.html

REFERENCES

[1] Jacques Bughin et. al., “How Artificial Intelligence Can Deliver Real Value to Companies”, McKinsey.
[Online] Available: https://www.mckinsey.com/business-functions/mckinsey-analytics/ our-insights/how-
artificial-intelligence-can-deliver-real-value-to-companies.

[2] Kevin Fogarty, (2017, Nov. 9), “The next Phase of Machine Learning”, Semiconductor Engineering.
[Online] Available: https://semiengineering.com/the-next-phase-of-machine-learning/.

[3] Eduard Sackinger et. al., “Application of the ANNA Neural Network Chip to High-Speed Character
Recognition”, IEEE Transactions on Neural Netsworks, Vol. 3, No. 3, May 1992.

[4] Patrick Bourke, Rob A. Rutenbar, “A High-Performance Hardware Speech Recognition System for Mobile
Applications”, 2005.

[5] Sergiu Nedevschi, Rabin K. Patra, Eric A. Brewer, “Hardware Speech Recognition for User Interfaces in
Low Cost, Low Power Devices”, Design Automation Conference, 2005.

[6] B.E. Boser et al, “Hardware requirements for neural network pattern classifiers”, IEEE Micro (Volume: 12,
Issue: 1, Feb. 1992), pp. 32-40.

[7] Bernard Marr, (2017, August 8). Forbes [Online]. Available:

https://www.forbes.com/sites/bernardmarr/2017/08/08/the-amazing-ways-how-google-uses-deep- learning-
ai/#711a9ea43204.

[8] Ryan Whitwam (2017, October 16). ExtremeTech [Online]. Available:

https://www.extremetech.com/extreme/257110-deepminds-wavenet-voice-synthesizer-live-google-
assistant.

[9] Vivienne Sze, Yu-Hsin Chen, Joel Emer, Amr Suleiman, Zhengdong Zhang, “Hardware for Machine
Learning: Challenges and Opportunities”, CICC 2017.

[10] R. Rojas, “Neural Networks”, Springer-Verlag, Berlin, 1996.

[11] Liao, Yihua, “Neural networks in hardware: A survey”, Davis, CA, 2017.

[12] Jihan Zhu and Peter Sutton, “FPGA Implementations of Neural Networks – A Survey of a Decade of
Progress”, Y. K. Cheung P., Constantinides G.A. (eds) Field Programmable Logic and Application, FPL
2003, Lecture Notes in Computer Science, vol. 2778. Springer, Berlin, Heidelberg.

[13] “Scikit-learn” [Online] Available: http://scikit-learn.org/stable/, Accessed on: Dec. 19, 2017.

[14] “Theano” [Online] Available: http://deeplearning.net/software/theano/, Accessed on: Dec. 19, 2017.

[15] “Apache Spark MLlib” [Online] Available: https://spark.apache.org/mllib/, Accessed on: Dec. 19, 2017.

[16] “H2O” [Online] Available: https://www.h2o.ai/, Accessed on: Dec. 19, 2017.

[17] “Tensorflow” [Online] Available: https://www.tensorflow.org/, Accessed on: Dec. 19, 2017.

[18] Arpan Chakraborty, (2016, April 7). Udacity [Online]. Available: https://blog.udacity.com/2016/04/5-
skills-you-need-to-become-a-machine-learning-engineer.html.

[19] McCartor, H., 1991, “A Highly Parallel Digital Architecture for Neural Network Emulation”, Delgado-
Frias, J. G. and Moore, W. R. (eds.), VLSI for Artificial Intelligence and Neural Networks, pp. 357- 366,
Plenum Press, New York, 1991.
[20] Lindsey, C. S., Lindblad, Th., Sekniaidze, G., Minerskjold, M., Szekely, S., and Eide, A., “Experience
with the IBM ZISC Neural Network Chip”. Proceedings of 3rd Int. Workshop on Software Engineering,
Artificial Intelligence, and Expert Systems, for High Energy and Nuclear Physics, Pisa, Italy, April 3-8,
1995.
[21] Nvidia, “Why GPUs?”. [Online] Available: http://www.fmslib.com/mkt/gpus.html, Accessed on: Dec. 20,
2017.

[22] Holt, J. and Hwang, J., “Finite Precision Error Analysis of the Neural Network Hardware
Implementations”. IEEE Trans. on Computers, 42:281-290, 1993.

[23] Dany Bradbury, (2017, July 24), “What sort of silicon brain do you need for artificial intelligence?”, The
Register. [Online]. Available: https://www.theregister.co.uk/2017/07/24/ai_hardware_development_plans/.

[24] Thiran, P., Peiris, V., Heim, P. and Hochet, B., “Quantization Effects in Digitally Behaving Circuit
Implementations of Kohonen Networks”. IEEE Trans. on Neural Networks, 5(3):450-458, 1994.

[25] Strey, A. and Avellana, N., “A New Concept for Parallel Neurocomputer Architectures”. Proceedings of
the Euro-Par'96 Conference, Lyon (France), Springer LNCS 1124, Berlin, 470-477, 1996.

[26] E. Won, “A hardware implementation of artificial neural networks using field programmable gate arrays”,
Elsevier, Nuclear Instruments and Methods in Physics Research A 581 (2007) pp. 816–820, 2007.

[27] Marchesi, M., et al., “Fast neural networks without multipliers”. IEEE Transactions on Neural Networks,
1993. 4(1): p. 53-62.

[28] Linda Barney, (2017, March 21), “Can FPGAs beat GPUs in accelerating next-generation deep learning?”,
The Next Platform. [Online]. Available: https://www.nextplatform.com/2017/03/21/can- fpgas-beat-gpus-
accelerating-next-generation-deep-learning/.

[29] Andre Xian Ming Chang, Eugenio Culurciello, “Hardware accelerators for Recurrent Neural Networks on
FPGA”, Circuits and Systems (ISCAS), 2017 IEEE International Symposium, ISSN: 2379-447X, 2017.

[30] Chao Wang, Qi Yu, Lei Gong, Xi Li, Yuan Xie, Xuehai Zhou, “DLAU: A Scalable Deep Learning
Accelerator Unit on FPGA”, IEEE Transactions on Computer-Aided Design of Integrated Circuits and
Systems (Volume: 36, Issue: 3, March 2017), pp. 513 – 517.

[31] Angshuman Parashar, Minsoo Rhu, Anurag Mukkara, Antonio Puglielli, Rangharajan Venkatesan,
BrucekKhailany, Joel Emer, Stephen W. Keckler, William J. Dally, “SCNN: An Accelerator for
Compressed-sparse Convolutional Neural Networks”, ISCA’17, Proceedings of the 44th Annual
International Symposium on Computer Architecture, pp. 27-40.

[32] Yijin Guan, Zhihang Yuan, Guangyu Sun, Jason Cong, “FPGA-based Accelerator for Long Short- Term
Memory Recurrent Neural Networks”, Design Automation Conference (ASP-DAC), 2017 22nd Asia and
South Pacific, ISSN: 2153-697X, 2017.

[33] Krste Asanovic, “Programmable Neurocomputing”, MIT Laboratory for Computer Science, Cambridge,
MA 02139. [Online]. Available: https://people.eecs.berkeley.edu/~krste/papers/neurocomputing.pdf,
Accessed on: Sept. 26, 2017.

[34] N. Morgan, J. Beck, P. Kohn, J. Bilmes, E. Allman, and J. Beer, “The Ring Array Processor (RAP): A
multiprocessing peripheral for connectionist applications”, Journal of Parallel and Distributed Computing,
14:248–259, April 1992.

[35] U. A. Muller, B. Baumie, P. Kohler, A. Gunzinger, and W. Guggenbuhl, “Achieving supercomputer

performance for neural net simulation with an array of digital signal processors”, IEEE Micro, 12(5):55–
64, October 1992.

[36] R. Means and L. Lisenbee, “Extensible linear floating-point SIMD neurocomputer array processor”,
Proceedings of the International Joint Conference on Neural Networks, pages I–587–592, New York, 1991.
IEEE Press.

[37] Ramacher, U., Raab, W., Anlauf, J., Hachmann, U., Beichter, J., Bruls, N., Webeling, M. and Sicheneder,
E., 1993, “Multiprocessor and Memory Architecture of the Neurocomputers SYNAPSE- 1”, Proceedings
of the 3rd International Conference on Microelectronics for Neural Networks (MicroNeuro), pp. 227-231,
 1993.

[38] J. Wawrzynek, K. Asanovi´c, B. Kingsbury, J. Beck, D. Johnson, and N. Morgan, “Spert-II: A vector
microprocessor syste”, IEEE Computer, 29(3):79–86, March 1996.

[39] M. Duranto, “Image processing by neural networks”, IEEE Micro, 16(5):12–19, October 1996.

[40] Fernando Morgado Dias, Ana Antunes, Alexandre Manuel Mota, “Commercial Hardware for Artificial
Neural Networks: A Survey”, IFAC Proceedings Volumes, Vol. 36, Issue 12, pp.189-196, 2003.

[41] Jung-Wook Cho and Soo-Young Lee, “Active Noise Cancelling using Analog NeuroChip with On- Chip
Learning Capability”, NIPS Proceedings, 1998.

[42] Mark Holler, Simon Tam, Hernan Castro, Ronald Benson, “An Electrically Trainable Artificial Neural
Network (ETANN) with 10240 "Floating Gate" Synapses”, Neural Networks, 1989, IJCNN., International
Joint Conference, 1989.

[43] Takeshi Kamio, Haruyasu Adachi, Hiroshi Ninomiya, Hideki Asai, “A Design Method of DWT Analog
Neuro Chip for VLSI Implementation”, Instrumentation and Measurement Technology Conference, 1997.
IMTC/97. Proceedings. Sensing, Processing, Networking., IEEE, 1997.

[44] Daiki Masumoto, Hiroki Ichiki, Hideki Yoshizawa, Hideki Kato, Kazuo Asakawa, “An Analog Neurochip
and Its Applications to Multilayered Artificial Neural Networks”, TOC, vol. 74, issue 9, pp. 92-103, 1991.

[45] Wikichip, “ETANN - Intel”. [Online] Available: https://en.wikichip.org/wiki/intel/etann, Accessed on: Oct.
19, 2017.

[46] Eduard Sackinger, Bernhard E. Boser, Lawrence D. Jackel, “A Neurocomputer Board Based on the ANNA
Neural Network Chip”, Advances in Neural Information Processing Systems 4 (NIPS 1994), pp. 773-780.

[47] Alan F. Murray et. al., “Pulse Stream VLSI Neural Networks”, IEEE Macro, Vol. 14, Issue 3, June 1994, p.
29-39.

[48] Karl Freund, (2017, March 3), “A machine learning landscape: where AMD, Intel, Nvidia, Qualcomm and
Xilinx AI engines live”, Forbes. [Online]. Available :
https://www.forbes.com/sites/moorinsights/2017/03/03/a-machine-learning-landscape-where-amd- intel-
nvidia-qualcomm-and-xilinx-ai-engines-live/#4436108a742f.

[49] Gaurav Nakhare, (2017, July 31), “Hardware options for machine/deep learning”, MS&E 238 Blog.
[Online]. Available: https://mse238blog.stanford.edu/2017/07/gnakhare/hardware-options-for-
machinedeep-learning/.

[50] Cade Metz, (2016, October 26), “How AI is shaking up the chip market”. [Online]. Available:
https://www.wired.com/2016/10/ai-changing-market-computer-chips/.

[51] “Intel Xeon Phi Processors”. [Online] Available:

https://www.intel.com/content/www/us/en/products/processors/xeon-phi/xeon-phi-processors.html,
Accessed on: Dec. 19, 2017.

[52] Nvidia, “Why GPUs?”. [Online] Available: http://www.fmslib.com/mkt/gpus.html, Accessed on: Dec. 20,
2017.

[53] Kevin Krewell, (2009, December 16), “What’s the difference between a CPU and a GPU?”. Nvivdia
[Online]. Available: https://blogs.nvidia.com/blog/2009/12/16/whats-the-difference-between-a-cpu- and-a-
gpu/.

[54] William Dally, (2015, July 12), “High performance hardware for machine learning”, NIPS Tutorial.
[Online]. Available: https://media.nips.cc/Conferences/2015/tutorialslides/Dally-NIPS-Tutorial- 2015.pdf.

[55] Nvidia, “Why GPUs?”. [Online] Available: http://www.fmslib.com/mkt/gpus.html, Accessed on: Dec. 20,
2017.
[56] Nvidia NVLink high-speed interconnect”, Nvidia. [Online]. Available:
http://www.nvidia.com/object/nvlink.html. Accessed on: Sept. 29, 2017.

[57] Nvidia, “Tegra Processors”. [Online] Available: http://www.nvidia.com/object/tegra-x1-

processor.html, Accessed on: Dec. 20, 2017.

[58] Nuno Edgar Nunes Fernandes, (2017, April 3), “FPGA chips will be the hardware future for deep learning
and AI”, Wordpress. [Online]. Available:
https://theintelligenceofinformation.wordpress.com/2017/04/03/fpga-chips-will-be-the-hardware- future-
for-deep-leaning-and-ai/.

[59] Nvidia, “Nvidia Introduces Nexus, The Industry’s First Integrated GPU/CPU Environment for Developers
Working with Microsoft Visual Studio”. [Online] Available:
http://www.nvidia.com/object/pr_nexus_093009.html.

[60] Kishore Kothapalli et. al., “CPU and/or GPU: Revisiting the GPU Vs. CPU Myth”. [Online] Available:
https://arxiv.org/pdf/1303.2171.pdf.

[61] William J., (2017, July 24), “Machine Learning on Intel FPGAs”, Intel. [Online]. Available:
https://software.intel.com/en-us/articles/machine-learning-on-intel-fpgas.

[62] Utku Aydonat, Shane O’Connell, Davor Capalija, Andrew C. Ling, Gordon R. Chiu, “An OpenCL Deep
Learning Accelerator on Arria 10”, 2017.

[63] Suhap Sahin, Yasar Becerikli, Suleyman Yazici, “Neural Network Implementation in Hardware Using
FPGAs”, Neural Network Implementation in Hardware Using FPGAs. In: King I., Wang J., Chan LW.,
Wang D. (eds) Neural Information Processing. ICONIP 2006. Lecture Notes in Computer Science, vol.
4234, Springer, Berlin, Heidelberg.

[64] Cox, C.E. and E. Blanz, “GangLion - a fast field-programmable gate array implementation of a
connectionist classifier”, IEEE Journal of Solid-State Circuits, 1992. 28(3): pp. 288-299.

[65] Pedro Ferreira, Pedro Ribeiro, Ana Antunes, Fernando Morgado Dias, “Artificial Neural Networks
Processor - a Hardware Implementation using a FPGA”, Becker J., Platzner M., Vernalde S. (eds) Field
Programmable Logic and Application. FPL 2004. Lecture Notes in Computer Science, vol. 3203, Springer,
Berlin, Heidelberg.

[66] Andrei Dinu, Marcian N. Cirstea, and Silvia E. Cirstea, “Direct Neural-Network Hardware-
Implementation Algorithm”, IEEE Transactions on Industrial Electronics (vol. 57, Issue: 5, May 2010).

[67] Seul Jung, Sung su Kim, “Hardware Implementation of a Real-Time Neural Network Controller with a
DSP and an FPGA for Nonlinear Systems”, IEEE Transactions on Industrial Electronics, vol. 54, No. 1,
February 2007.

[68] Intel FPGA and SoC, “Arria 10”. [Online] Available: https://www.altera.com/products/fpga/arria-
series/arria-10/overview.html.

[69] Intel FPGA and SoC, “Stratix 10”. [Online] Available: https://www.altera.com/products/fpga/stratix-
series/stratix-10/overview.html.

[70] Eriko Nurvitadhi et. al., “Accelerating Binarized Neural Networks: Comparison of FPGA, CPU, GPU, and
ASIC”, IEEE International Conference on Field-Programmable Technology, 7-9 Dec., 2016.

[71] Nvidia, “Jetson Automotive Development Platform”. [Online] Available:

http://www.nvidia.in/object/jetson-pro-automotive-development-platform-in.html.

[72] Nvidia, “Nvidia Drive PX”. [Online] Available: https://www.nvidia.com/en-us/self-driving-cars/drive-px/.

[73] Nicole Hemsoth (2017, April 5), “First In-depth Look at Google’s TPU Architecture”. [Online] Available:
https://www.nextplatform.com/2017/04/05/first-depth-look-googles-tpu-architecture/.
[74] Intel Nervana, [Online] Available: https://www.intelnervana.com/.

[75] J . Zhang, Z. Wang, N. Verma, “A machine-learning classifier implemented in a standard 6T SRAM

array,”, Sym. on VLSI, 2016.

[76] Z. Wang, R. Schapire, N. Verma, “Error-adaptive classifier boosting (EACB): Exploiting data-driven
training for highly fault-tolerant hardware,”, ICASSP, 2014.

[77] B. Murmann, D. Bankman, E. Chai, D. Miyashita, L. Yang, “Mixed-signal circuits for embedded machine-
learning applications”, Signals, Systems and Computers, 49th Asilomar Conference, 2015.

[78] Pai-Yu Chen, Deepak Kadetotad, Zihan Xu, Abinash Mohanty, Binbin Lin, Jieping Ye, Sarma Vrudhula,
Jae-sun Seo, Yu Cao, Shimeng Yu, “Technology-design co-optimization of resistive cross- point array for
accelerating learning algorithms on chip”, Design, Automation & Test in Europe Conference & Exhibition
(DATE), 2015.

[79] Junjie Lu, Steven Young, Itamar Arel, Jeremy Holleman, “A 1 TOPS/W Analog Deep Machine- Learning
Engine with Floating-Gate Storage in 0.13 µm CMOS”, IEEE Journal of Solid-State Circuits (Volume: 50,
Issue: 1, Jan. 2015).

[80] Y. Chen and et al., “DaDianNao: A Machine-Learning Supercomputer,”, MICRO, 2014.

[81] A. Shafiee, A. Nag, N. Muralimanohar, R. Balasubramonian, J. P. Strachan, M. Hu, R. S. Williams,

V. Srikumar, “ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in
Crossbars,”, ISCA, 2016.

[82] P. Chi, S. Li, Z. Qi, P. Gu, C. Xu, T. Zhang, J. Zhao, Y. Liu, Y. Wang, and Y. Xie, “PRIME: A Novel
Processing-In-Memory Architecture for Neural Network Computation in ReRAM-based Main Memory,”,
ISCA, 2016.

[83] Takashi Morie and Yoshihito Amemiya, “An All-Analog Expandable Neural Network LSI with On- Chip
Backpropagation Learning”, IEEE Journal of Solid-State Circuits, Vol. 29, No. 9, September, 1994.

[84] Arindam Basu, SunShuo, HongmingZhou, MengHiotLim, Guang-BinHuang, “Silicon spiking neurons for
hardware implementation of extreme learning machines”, Neurocomputing, 102, pp.125– 134, 2013.

[85] Jae-sun Seo et al, “A 45nm CMOS Neuromorphic Chip with a Scalable Architecture for Learning in
Networks of Spiking Neurons”, Custom Integrated Circuits Conference (CICC), 2011 IEEE.

[86] Yu-Hsin Chen, Joel Emer, Vivienne Sze, “Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow
for Convolutional Neural Networks”, Computer Architecture (ISCA), 2016 ACM/IEEE 43rd Annual
International Symposium, 2016, ISSN: 1063-6897.

[87] Joe Osborne, (2016, Aug. 22), “Google’s Tensor Processing Unit Explained: This is What the Future of
Computing Looks Like”. Techradar [Online] Available: http://www.techradar.com/news/computing-
components/processors/google-s-tensor-processing-unit- explained-this-is-what-the-future-of-computing-
looks-like-1326915.

[88] Kaz Sato, (2017, May 12), “An In-depth Look at Google’s First Tensor Processing Unit (TPU)”, Google
Cloud Platform. [Online] Available: https://cloud.google.com/blog/big-data/2017/05/an-in- depth-look-at-
googles-first-tensor-processing-unit-tpu.

[89] Google AI, “Cloud TPUs”. [Online] Available: https://ai.google/tools/cloud-tpus/.

AUTHOR

Pooja Jawandhiya was born in Nagpur, India on May 2, 1995. She received the Bachelor
of Engineering degree in Electronics and Telecommunication from University of Mumbai
in June, 2017. Currently, she is a student in Nanyang Technological University, Singapore
and is pursuing Master of Science (Electronics) from the School of Electrical and
Electronic Engineering
 PREDICTING STUDENT ACADEMIC PERFORMANCE IN
BLENDED LEARNING USING ARTIFICIAL NEURAL
NETWORKS
Nick Z. Zacharis

Department of Computer Systems Engineering, Technological Educational Institute of

Piraeus, Athens, Greece

ABSTRACT

Along with the spreading of online education, the importance of active support of students involved in
online learning processes has grown. The application of artificial intelligence in education allows
instructors to analyze data extracted from university servers, identify patterns of student behavior and
develop interventions for struggling students. This study used student data stored in a Moodle server
and predicted student success in course, based on four learning activities - communication via emails,
collaborative content creation with wiki, content interaction measured by files viewed and self-
evaluation through online quizzes. Next, a model based on the Multi-Layer Perceptron Neural
Network was trained to predict student performance on a blended learning course environment. The
model predicted the performance of students with correct classification rate, CCR, of 98.3%.

KEYWORDS

Artificial Neural Networks, Blended Learning, Student Achievement, Learning Analytics, Moodle
Data

For More Details:https://aircconline.com/ijaia/V7N5/7516ijaia02.pdf

Volume Link: http://www.airccse.org/journal/ijaia/current2016.html

REFERENCES

[1] Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for
educators: A proof of concept. Computers & Education, 54(2), 588–599.

[2] Zacharis, N. Z. (2015). A multivariate approach to predicting student outcomes in web-enabled blended
learning courses. Internet and Higher Education, 27, 44–53.

[3] Strang, D. K. (2016). Can online student performance be forecasted by learning analytics? International
Journal of Technology Enhanced Learning, 8(1), 26-47.

[4] Sabourin, J., Rowe, J., Mott, B., Lester, J. (2011). When Off-Task in On-Task: The Affective Role of Off-
Task Behavior in Narrative-Centered Learning Environments. Proceedings of the 15th International
Conference on Artificial Intelligence in Education, 534-536.

[5] Baker, R.S.J.d., Yacef, K. (2009). The State of Educational Data Mining in 2009: A Review and Future
Visions. Journal of Educational Data Mining, 1(1), 3-17.

[6] Lykourentzou, I., Giannoukos, I., Mpardis, G., Nikolopoulos, V. and Loumos, V. (2009), Early and
dynamic student achievement prediction in e-learning courses using neural networks. J. Am. Soc. Inf. Sci.,
60: 372–380. doi: 10.1002/asi.20970

[7] Paliwal, M., & Kumar, U. A. (2009). A study of academic performance of business school graduates using
neural network and statistical techniques. Expert Systems with Applications, 36(4), 7865–7872.

[8] Jayne C, Lanitis A, Christodoulou C (2011). Neural network methods for one-to-many multi-valued
mapping problems. Neural Comput Appl 20(6):775–785

[9] Kanakana, G.M., Olanrewaju, A.O. (2011). Predicting student performance in engineering education using
an artificial neural network at Tshwane university of technology. Proceedings of the International
Conference on Industrial Engineering, Systems Engineering and Engineering Management for Sustainable
Global Development, Stellenbosch, South Africa, pp. 1–7.

[10] Shahiri, A.M., Husain, W., Rashid, A.N. (2015). A review on predicting student's performance using data
mining techniques. Procedia Computer Science, 72, 414-422.

[11] McClelland, J.L., Rumelhart, D.E., and Hinton, G.E. (1986). The appeal of parallel distributed processing,
in Parallel Distributed Processing: Explorations in the Microstructure of Cognition - Foundations, Vol.1,
MIT Press, Cambridge, pp.3-44.

[12] Leverington, D. (2009). A Basic Introduction to Feedforward Backpropagation Neural Networks.

http://www.webpages.ttu.edu/dleverin/neural_network/neural_networks.html

[13] Rojas Raúl (1996). Neural Networks: A Systematic Introduction, Springer-Verlag, Berlin, New-York.

[14] Marwala, T. (2010). Finite Element Model Updating Using Computational Intelligence Techniques:
Applications to Structural Dynamics, Springer Publishing Company, Inc .

[15] IBM (2016). Knowledge Center. http://goo.gl/SuuMHu

[16] Møller, M.F., 1993. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6
(4),525–533.
 A CASE STUDY OF INNOVATION OF AN INFORMATION
COMMUNICATION SYSTEM AND UPGRADE OF THE
KNOWLEDGE BASE IN INDUSTRY
BY ESB, ARTIFICIAL INTELLIGENCE, AND BIG DATA
SYSTEM INTEGRATION
Alessandro Massaro1,*, Angelo Calicchio1, Vincenzo Maritati1, Angelo Galiano1, Vitangelo Birardi1, Leonardo
Pellicani1, Maria Gutierrez Millan2, Barbara Dalla
Tezza2, Mauro Bianchi2, Guido Vertua2, Antonello Puggioni2
1
Dyrecta Lab, IT Research Laboratory, Via Vescovo Simplicio, 45, 70014 Conversano
(BA), Italy.
2
Performance in Lighting S.p.A., Viale del Lavoro 9/11 - 37030 Colognola ai Colli (VR), Italy.

ABSTRACT

In this paper, a case study is analyzed. This case study is about an upgrade of an industry communication system
developed by following Frascati research guidelines. The knowledge Base (KB) of the industry is gained by
means of different tools that are able to provide data and information having different formats and structures
into an unique bus system connected to a Big Data. The initial part of the research is focused on the
implementation of strategic tools, which can able to upgrade the KB. The second part of the proposed study is
related to the implementation of innovative algorithms based on a KNIME (Konstanz Information Miner)
Gradient Boosted Trees workflow processing data of the communication system which travel into an Enterprise
Service Bus (ESB) infrastructure. The goal of the paper is to prove that all the new KB collected into a
Cassandra big data system could be processed through the ESB by predictive algorithms solving possible
conflicts between hardware and software. The conflicts are due to the integration of different database
technologies and data structures. In order to check the outputs of the Gradient Boosted Trees algorithm an
experimental dataset suitable for machine learning testing has been tested. The test has been performed on a
prototype network system modeling a part of the whole communication system. The paper shows how to
validate industrial research by following a complete design and development of a whole communication system
network improving business intelligence (BI).

KEYWORDS

Frascati Guideline, ESB, Data Mining, KNIME, Gradient Boosted Tree Algorithm, Big Data.

For More Details: https://aircconline.com/ijaia/V12N2/12221ijaia01.pdf

Volume Link: http://www.airccse.org/journal/ijaia/current2021.html

REFERENCES
[1]. Frascati Manual 2015: The Measurement of Scientific, Technological and Innovation ActivitiesGuidelines for
Collecting and Reporting Data on Research and Experimental Development. OECD (2015), ISBN 978-926423901-2
(PDF).
[2]. Hohpe, G., & Woolf, B. (2004) “Enterprise Integration Patterns Designing, Building, and Deploying Messaging
Solutions”, Addison-Wesley.
[3]. Polgar, J. (2009) “Open Source ESB in Action”, IGI Publishing.
[4]. Górski, T., & Pietrasik, K. (2016) “Performance analysis of Enterprise Service Buses”, Journal of Theoretical and
Applied Computer Science, Vol. 10, No. 2, pp 16-32.
[5]. Yenlo (2016) “ESB Comparison How to choose a reliable and fast ESB that fits your business needs”, white paper.
[6]. Massaro, A., Maritati, V., Galiano, A., Birardi, V., Pellicani, L. (2018) “ESB Platform Integrating KNIME Data
Mining Tool Oriented on Industry 4.0 based on Artificial Neural Network Predictive Maintenance”, International
Journal of Artificial Intelligence and Applications (IJAIA), Vol. 9, No. 3, pp1-17.
[7]. Bassil, Y. (2012) “A Simulation Model for the Waterfall Software Development Life Cycle”, International Journal of
Engineering & Technology (IJET), Vol. 2, No. 5, pp1-7.
[8]. Ragunath, P. K., Velmourougan, S., Davachelvan, P., Kayalvizhi, S., Ravimohan, R. (2010) “Evolving A New Model
(SDLC Model-2010) For Software Development Life Cycle (SDLC),” IJCSNS International Journal of Computer
Science and Network Security, Vol.10 No.1, January 2010.
[9]. Rather, M. A., Bhatnagar, V. (2015) “A Comparative Study of Software Development Life Cycle Models”, Vol. 4,
No. 10, pp23-29.
[10]. Dilek, S., Çakır, H., Aydın, M. (2015) “Applications of Artificial Intelligence Techniques to Combating Cyber
Crimes: a Review”, International Journal of Artificial Intelligence and Applications (IJAIA), Vol. 6, No. 1, pp21-39.
[11]. Linoff, G. S., Berry, M. J. (2011) “Data Mining Techniques: For Marketing, Sales, and Customer Relationship
Management”, 3rdEdition, John Wiley & Sons Inc, 2011.
[12]. Maimon, O., Rokach, L. (2006) “Data Mining and Knowledge Discovery Handbook”, 2nd edition, Springer US,
2010.
[13]. Kotu, V., Deshpande, B. (2015) “Predictive Analytics and Data Mining”, Elsevier book.
[14]. Adhikari, N. C. D. (2018) “Prevention of Heart Problem Using Artificial Intelligence”, International Journal of
Artificial Intelligence and Applications (IJAIA), Vo., 9, No. 2, pp21-35.
[15]. Khan, N., Yaqoob, I., Ibrahim, Hashem, A. T., Inayat, Z., Mahmoud Ali, W. K., Alam, M., Shiraz, M., Gani, A.
(2014) “Big Data: Survey, Technologies, Opportunities, and Challenges”, Hindawi Publishing Corporation The
Scientific World Journal, Vol. 2014, No. 712826, pp1-18.
[16]. Massaro, A., Maritati, V., Savino, N., Galiano, A., Convertini, D., De Fonte, E., Di Muro, M. (2018) “A Study of a
Health Resources Management Platform Integrating Neural Networks and DSS Telemedicine for Homecare
Assistance,” Information, Vol. 9, No. 176, pp1-20.
[17]. Prasad, B. R., Agarwal, S. (2016) “Comparative Study of Big Data Computing and Storage Tools: A Review”,
International Journal of Database Theory and Application, Vol. 9, No. 1, pp45-66.
[18]. Zheng, Z., Wang, P., Liu, J., Sun, S. (2015) “Real-Time Big Data Processing Framework: Challenges and Solutions,”
Applied Mathematics & Information Sciences An International Journal, Vol. 9, No. 6, pp3169-3190.
[19]. Hashem, I. A. T.,Yaqoo, I., Anuar, N. B., Mokhtar, S., Gani, A., Khan, A. U. (2015) “The Rise of ‘Big Data’ on
Cloud Computing: Review and Open Research Issues,” Information Systems, Vol. 47, pp98–115.
[20]. Pandove, K, Jindal, A., Kumar, R. (2010) “Email Security”, International Journal of Computer Applications, Vol. 5,
No. 1, pp23-26.
[21]. Ruotsalainen, P. (2013) “Endpoint Protection Security System for an Enterprise”, Master’s Thesis, Jamk University of
Applied Sciences.
[22]. Cantoni L., Di Blas, N., Bolchini, D. (2010) “Comunicazione, Qualità, Usabilità, una Nuova Prospettiva per la
Valutazione di Siti Web”, Maggiolini Editore, ISBN: 9788838788888.
[23]. Massaro, A., Maritati, V., Galiano, A. (2018) “Data Mining Model Performance of Sales Predictive Algorithms Based
on RapiMiner Workflow”, International Journal of Computer Science & Information Technology (IJCSIT),Vol. 10,
No. 3, pp39-56.
[24]. Massaro, A., Barbuzzi, D., Vitti, V., Galiano, A., Aruci, M., Pirlo, G. (2016) “Predictive Sales Analysis According to
the Effect of Weather”, Proceeding of the 2nd International Conference on Recent Trends and Applications in
Computer Science and Information Technology, Tirana, Albania, November 18 - 19, pp53-55.
[25]. Johansson, M., Roupé, M., Tallgren M. V. (2014) “From BIM to VR”, Proceedings of the 32nd eCAADe Conference,
Vol. 2 (eCAADe 2014), pp1-9.
[26]. “WSO2 Data Service Server” 2018. [Online]. Available: https://wso2.com/products/data-servicesserver/
[27]. “Getting Started with Apache Cassandra on Windows the Easy Way” [Online]. Available:
https://www.datastax.com/2012/01/getting-started-with-apache-cassandra-on-windows-the-easy-way
[28]. “Machine Learning Repository” [Online]. Available: https://archive.ics.uci.edu/ml/datasets/online+retail#
[29]. Chen, D., Sain, S. L., Guo, K. (2012) “Data Mining for the Online Retail Industry: A Case Study of RFM Model-
Based Customer Segmentation Using Data Mining”, Journal of Database Marketing and Customer Strategy
Management, Vol. 19, No. 3, pp197-208.
[30]. Friedman, J. H. (2001) “1999 REITZ LECTURE Greedy Function Approximation: A Gradient Boosting Machine”,
 The Annals of Statistics, Vol. 29, No. 5, pp1189-1232.
[31]. Breiman, L., Friedman, J., Stone, C. J., Olshen, R. A. (1984) “Classification and Regression Trees”, Taylor & Francis.

Corresponding Author
Alessandro Massaro: Research & Development Chief of Dyrecta Lab s.r.l.
 AN EXPLANATION FRAMEWORK FOR
INTERPRETABLE CREDIT SCORING
Lara Marie Demajo, Vince Vella and Alexiei Dingli

University of Malta, Malta

ABSTRACT

With the recent boosted enthusiasm in Artificial Intelligence (AI) and Financial Technology
(FinTech), applications such as credit scoring have gained substantial academic interest. However,
despite the evergrowing achievements, the biggest obstacle in most AI systems is their lack of
interpretability. This deficiency of transparency limits their application in different domains including
credit scoring. Credit scoring systems help financial experts make better decisions regarding whether
or not to accept a loan application so that loans with a high probability of default are not accepted.
Apart from the noisy and highly imbalanced data challenges faced by such credit scoring models,
recent regulations such as the `right to explanation' introduced by the General Data Protection
Regulation (GDPR) and the Equal Credit Opportunity Act (ECOA) have added the need for model
interpretability to ensure that algorithmic decisions are understandable and coherent. A recently
introduced concept is eXplainable AI (XAI), which focuses on making black-box models more
interpretable. In this work, we present a credit scoring model that is both accurate and interpretable.
For classification, state-of-the-art performance on the Home Equity Line of Credit (HELOC) and
Lending Club (LC) Datasets is achieved using the Extreme Gradient Boosting (XGBoost) model. The
model is then further enhanced with a 360-degree explanation framework, which provides different
explanations (i.e. global, local feature-based and local instance- based) that are required by different
people in different situations. Evaluation through the use of functionally-grounded, application-
grounded and human-grounded analysis shows that the explanations provided are simple and
consistent as well as correct, effective, easy to understand, sufficiently detailed and trustworthy

KEYWORDS

Credit Scoring, Explainable AI, BRCG, XGBoost, GIRP, SHAP, Anchors, ProtoDash, HELOC, Lending Club

For More Details: https://aircconline.com/ijaia/V12N1/12121ijaia02.pdf

Volume Link: https://www.airccse.org/journal/ijaia/current2021.html

REFERENCES
[1] Bekhet, H. A., & Eletter, S. F. K. (2014). Credit risk assessment model for Jordanian commercial
banks: Neural scoring approach. Review of Development Finance, 4(1), 20-28.
[2] Mester, L. J. (1997). What’s the point of credit scoring?. Business review, 3(Sep/Oct), 3-16.
[3] Bonacina, M. P. (2017, November). Automated Reasoning for Explainable Artificial Intelligence. In
ARCADE@ CADE (pp. 24-28).
[4] Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning.
arXiv preprint arXiv:1702.08608.
[5] Fernandez, A., Herrera, F., Cordon, O., del Jesus, M. J., & Marcelloni, F. (2019). Evolutionary fuzzy
systems for explainable artificial intelligence: why, when, what for, and where to?. IEEE
Computational Intelligence Magazine, 14(1), 69-81.
[6] Xu, F., Uszkoreit, H., Du, Y., Fan, W., Zhao, D., & Zhu, J. (2019, October). Explainable AI: A brief
survey on history, research areas, approaches and challenges. In CCF International Conference on
Natural Language Processing and Chinese Computing (pp. 563-574). Springer, Cham.
[7] Gilpin, L. H., Testart, C., Fruchter, N., & Adebayo, J. (2019). Explaining explanations to society.
arXiv preprint arXiv:1901.06560.
[8] Lu, J., Lee, D., Kim, T. W., & Danks, D. (2020, February). Good Explanation for Algorithmic
Transparency. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (pp. 93- 93).
[9] Došilović, F. K., Brčić, M., & Hlupić, N. (2018, May). Explainable artificial intelligence: A survey.
In 2018 41st International convention on information and communication technology, electronics and
microelectronics (MIPRO) (pp. 0210-0215). IEEE.
[10] Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., ... & Chatila, R.
(2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges
toward responsible AI. Information Fusion, 58, 82-115.
[11] Arya, V., Bellamy, R. K., Chen, P. Y., Dhurandhar, A., Hind, M., Hoffman, S. C., ... & Mourad,
S. (2019). One explanation does not fit all: A toolkit and taxonomy of ai explainability techniques.
arXiv preprint arXiv:1909.03012.
[12] Teach, R. L., & Shortliffe, E. H. (1981). An analysis of physician attitudes regarding computer- based
clinical consultation systems. Computers and Biomedical Research, 14(6), 542-558.
[13] Hendricks, L. A., Akata, Z., Rohrbach, M., Donahue, J., Schiele, B., & Darrell, T. (2016, October).
Generating visual explanations. In European Conference on Computer Vision(pp. 3- 19). Springer,
Cham.
[14] Core, M. G., Lane, H. C., Van Lent, M., Gomboc, D., Solomon, S., & Rosenberg, M. (2006, July).
Building explainable artificial intelligence systems. In AAAI (pp. 1766-1773).
[15] Van Lent, M., Fisher, W., & Mancuso, M. (2004, July). An explainable artificial intelligence system
for small-unit tactical behavior. In Proceedings of the national conference on artificial intelligence
(pp. 900-907). Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.
[16] Vellido, A., Martín-Guerrero, J. D., & Lisboa, P. J. (2012, April). Making machine learning models
interpretable. In ESANN(Vol. 12, pp. 163-172).
[17] Štrumbelj, E., & Kononenko, I. (2014). Explaining prediction models and individual predictions with
feature contributions. Knowledge and information systems, 41(3), 647-665.
[18] Trinkle, B. S., & Baldwin, A. A. (2007). Interpretable credit model development via artificial neural
networks. Intelligent Systems in Accounting, Finance & Management: International Journal, 15(3‐4),
123-147.
[19] Baesens, B., Setiono, R., Mues, C., & Vanthienen, J. (2003). Using neural network rule extraction
and decision tables for credit-risk evaluation. Management science, 49(3), 312-329.
[20] Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K. R., & Samek, W. (2015). On pixel-
wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one,
10(7), e0130140.
[21] Yang, C., Rangarajan, A., & Ranka, S. (2018, June). Global model interpretation via recursive
partitioning. In 2018 IEEE 20th International Conference on High Performance Computing and
Communications; IEEE 16th International Conference on Smart City; IEEE 4th International
Conference on Data Science and Systems (HPCC/SmartCity/DSS) (pp. 1563-1570). IEEE.
[22] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016, August). " Why should I trust you?" Explaining the
predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on
knowledge discovery and data mining (pp. 1135-1144).
[23] Ribeiro, M. T., Singh, S., & Guestrin, C. (2018, April). Anchors: High-precision model-agnostic
explanations. In Thirty-Second AAAI Conference on Artificial Intelligence.
[24] Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. In
Advances in neural information processing systems (pp. 4765-4774).
[25] Dash, S., Gunluk, O., & Wei, D. (2018). Boolean decision rules via column generation. In Advances
in Neural Information Processing Systems (pp. 4655-4665).
[26] Gomez, O., Holter, S., Yuan, J., & Bertini, E. (2020, March). ViCE: visual counterfactual
explanations for machine learning models. In Proceedings of the 25th International Conference on
Intelligent User Interfaces (pp. 531-535).
[27] Chen, C., Lin, K., Rudin, C., Shaposhnik, Y., Wang, S., & Wang, T. (2018). An interpretable model
with globally consistent explanations for credit risk. arXiv preprint arXiv:1811.12615.
[28] Serrano-Cinca, C., Gutiérrez-Nieto, B., & López-Palacios, L. (2015). Determinants of default in P2P
lending. PloS one, 10(10), e0139427.
[29] Gupta, D. K., & Goyal, S. (2018). Credit risk prediction using artificial neural network algorithm.
International Journal of Modern Education and Computer Science, 11(5), 9.
[30] Malekipirbazari, M., & Aksakalli, V. (2015). Risk assessment in social lending via random forests.
Expert Systems with Applications, 42(10), 4621-4631.
[31] Mancisidor, R. A., Kampffmeyer, M., Aas, K., & Jenssen, R. (2020). Deep generative models for
reject inference in credit scoring. Knowledge-Based Systems, 105758.
[32] Marceau, L., Qiu, L., Vandewiele, N., & Charton, E. (2019). A comparison of Deep Learning
performances with others machine learning algorithms on credit scoring unbalanced data. arXiv
preprint arXiv:1907.12363.
[33] Tsai, C. F., & Wu, J. W. (2008). Using neural network ensembles for bankruptcy prediction and
credit scoring. Expert systems with applications, 34(4), 2639-2649.
[34] Aithal, V., & Jathanna, R. D. (2019). Credit risk assessment using machine learning techniques.
International Journal of Innovative Technology and Exploring Engineering, 9(1), 3482-3486.
[35] Salvaire, P. A. J. M. (2019). Explaining the predictions of a boosted tree algorithm: application to
credit scoring (Doctoral dissertation).
[36] Gurumoorthy, K. S., Dhurandhar, A., Cecchi, G., & Aggarwal, C. (2019, November). Efficient Data
Representation by Selecting Prototypes with Importance Weights. In 2019 IEEE International
Conference on Data Mining (ICDM) (pp. 260-269). IEEE.
[37] Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on Explainable Artificial
Intelligence (XAI). IEEE Access, 6, 52138-52160.
[38] Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. (2007). Comprehensible credit scoring
models using rule extraction from support vector machines. European journal of operational research,
183(3), 1466-1476.
[39] Gacto, M. J., Alcalá, R., & Herrera, F. (2011). Interpretability of linguistic fuzzy rule-based systems:
An overview of interpretability measures. Information Sciences, 181(20), 4340-4360.
[40] Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W. K. (2013, September). Too
much, too little, or just right? Ways explanations impact end users' mental models. In 2013 IEEE
Symposium on Visual Languages and Human Centric Computing (pp. 3-10). IEEE.
[41] Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018). Metrics for explainable AI:
Challenges and prospects. arXiv preprint arXiv:1812.04608.
[42] Isaac, S., & Michael, W. B. (1995). Handbook in research and evaluation: A collection of principles,
methods, and strategies useful in the planning, design, and evaluation of studies in education and the
behavioral sciences. Edits publishers.
[43] Hill, R. (1998). What sample size is “enough” in internet survey research. Interpersonal Computing and
Technology: An electronic journal for the 21st century, 6(3-4), 1-12.
[43] Jain, S., Luthra, M., Sharma, S., & Fatima, M. (2020, March). Trustworthiness of Artificial Intelligence. In 2020 6th
International Conference on Advanced Computing and Communication Systems (ICACCS) (pp. 907-912). IEEE.
[45] Ashoori, M., & Weisz, J. D. (2019). In AI we trust? Factors that influence trustworthiness of AI- infused decision-
making processes. arXiv preprint arXiv:1912.02675.
[46] Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018, October). Explaining explanations:
An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and
advanced analytics (DSAA) (pp. 80-89). IEEE.

AUTHORS
Lara Marie Demajo has degrees in M.Sc. in Artificial Intelligence and B.Sc. in Information Technology
(Hons.) in AI, both from University of Malta. Her work has won various prizes including IEEE Best ICT
Project and first place in FICTeX Final Year Project awarded by the Dean of Faculty of ICT. She has over 5
years experience in software development and her interests include Artificial Intelligence, FinTech and
Model Interpretability.

Dr Vince Vella brings over 25 years of senior technical leadership and management experience. Currently,
he holds the position of CTO at Computime Software, BRSAnalytics and CTLabs. He holds a PhD from
the Centre for Computational Finance and Economic Agents (CCFEA), University of Essex. Vince is also
a lecturer within the Department of AI at University of Malta, mainly responsible for the MSc AI – Fintech
stream. His main interests overlap Artificial Intelligence, Machine Learning and Computational Finance,
particularly in the areas of AI Managed Funds, Algorithmic Trading, decentralized AI and AI for Anti
Money Laundering.

Prof Alexiei Dingli is a Professor of Artificial Intelligence (AI) at the Department of AI within the
University of Malta. He has been conducting research and working in the field of AI for the past two
decades. His work was rated World Class by international experts and won various prizes including; the
Semantic Web Challenge, the first prize by the European Space Agency, the e-Excellence Gold Seal award,
the First Prize in the Malta Innovation Awards, the World Intellectual Property Organization (WIPO) award
for Creativity and the first prize of the Energy Globe award by the UN, amongst others. He has published
several peer-reviewed papers and books in the field. He also formed part of the Malta.AI task-force aimed
at making Malta one of the top-AI countries in the world where he chaired the working- group on AI in
work & education. Prof Dingli also assists various local and international organizations during their transformation
towards becoming AI companies.
 FORGED CHARACTER DETECTION
DATASETS: PASSPORTS, DRIVING LICENCES
AND VISA STICKERS
Teerath Kumar1 , Muhammad Turab2 , Shahnawaz Talpur2 , Rob Brennan1 and Malika Bendechache1
1
CRT AI and ADAPT, School of Computing, Dublin City University, Ireland
2
Department of Computer Systems Engineering, Mehran University of Engineering and Technology,
Jamshoro, Pakistan

ABSTRACT

Forged documents specifically passport, driving licence and VISA stickers are used for fraud purposes
including robbery, theft and many more. So detecting forged characters from documents is a
significantly important and challenging task in digital forensic imaging. Forged characters detection
has two big challenges. First challenge is, data for forged characters detection is extremely difficult to
get due to several reasons including limited access of data, unlabeled data or work is done on private
data. Second challenge is, deep learning (DL) algorithms require labeled data, which poses a further
challenge as getting labeled is tedious, time-consuming, expensive and requires domain expertise. To
end these issues, in this paper we propose a novel algorithm, which generates the three datasets
namely forged characters detection for passport (FCD-P), forged characters detection for driving
licence (FCD-D) and forged characters detection for VISA stickers (FCD-V). To the best of our
knowledge, we are the first to release these datasets. The proposed algorithm starts by reading plain
document images, simulates forging simulation tasks on five different countries' passports, driving
licences and VISA stickers. Then it keeps the bounding boxes as a track of the forged characters as a
labeling process. Furthermore, considering the real world scenario, we performed the selected data
augmentation accordingly. Regarding the stats of datasets, each dataset consists of 15000 images
having size of 950 x 550 of each. For further research purpose we release our algorithm code 1 and,
datasets i.e. FCD-P 2 , FCD-D 3 and FCD-V 4 .

KEYWORDS

Character detection dataset, Deep learning forgery, Forged character detection

For More Details: https://aircconline.com/ijaia/V13N2/13222ijaia02.pdf

Volume Link: https://www.airccse.org/journal/ijaia/current2022.html

REFERENCES
[1]. Fake identity brits warned that their lives are in danger, Online
Available:https://www.independent.co.uk/news/world/middle-east/fake-identity-brits-warned-thattheir-
lives-are-in-danger-1905971.html .
[2]. Wu, L., Zhang, C., Liu, J., Han, J., Liu, J., Ding, E., & Bai, X. (2019, October). Editing text in the wild. In
Proceedings of the 27th ACM international conference on multimedia (pp. 1500-1508).
[3]. Yang, Q., Huang, J., & Lin, W. (2020). Swaptext: Image based texts transfer in scenes. In Proceedings of
the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 14700- 14709).
[4]. Roy, P., Bhattacharya, S., Ghosh, S., & Pal, U. (2020). STEFANN: scene text editor using font adaptive
neural network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
(pp. 13228-13237).
[5]. Korshunov, P., & Marcel, S. (2018). Deepfakes: a new threat to face recognition? assessment and
detection. arXiv preprint arXiv:1812.08685.
[6]. Zhao, L., Chen, C., & Huang, J. (2021). Deep Learning-based Forgery Attack on Document Images. arXiv
preprint arXiv:2102.00653.
[7]. Adadi, A. (2021). A survey on data‐efficient algorithms in big data era. Journal of Big Data, 8(1), 1- 54.
[8]. Aiman, A., Shen, Y., Bendechache, M., Inayat, I., & Kumar, T. (2021). AUDD: Audio Urdu Digits Dataset
for Automatic Audio Urdu Digit Recognition. Applied Sciences, 11(19), 8842.
[9]. Kumar, T., Turab, M., Talpur, S Brennan, R., Bendechache, M. (2022). Detection Datasets: Forged
Characters for Passport and Driving Licence. 6th International Conference on Artificial Intelligence, Soft
Computing and Applications (AISCA 2022), (pp. 45-54)
[10]. Bertrand, R., Gomez-Krämer, P., Terrades, O. R., Franco, P., & Ogier, J. M. (2013, August). A system
based on intrinsic features for fraudulent document detection. In 2013 12th International conference on
document analysis and recognition (pp. 106-110). IEEE.
[11]. Shang, S., Kong, X., & You, X. (2015). Document forgery detection using distortion mutation of geometric
parameters in characters. Journal of Electronic Imaging, 24(2), 023008.
[12]. Ryan, M., & Hanafiah, N. (2015). An examination of character recognition on ID card using template
matching approach. Procedia Computer Science, 59, 520-529.
[13]. Poddar, J., Parikh, V., & Bharti, S. K. (2020). Offline signature recognition and forgery detection using
deep learning. Procedia Computer Science, 170, 610-617.
[14]. Bertrand, R., Terrades, O. R., Gomez-Krämer, P., Franco, P., & Ogier, J. M. (2015, August). A conditional
random field model for font forgery detection. In 2015 13th International Conference on Document
Analysis and Recognition (ICDAR)(pp. 576-580). IEEE.
[15]. Cruz, F., Sidere, N., Coustaty, M., d'Andecy, V. P., & Ogier, J. M. (2017, November). Local binary
patterns for document forgery detection. In 2017 14th IAPR International Conference on Document
Analysis and Recognition (ICDAR) (Vol. 1, pp. 1223-1228). IEEE.
[16]. Sidere, N., Cruz, F., Coustaty, M., & Ogier, J. M. (2017, September). A dataset for forgery detection and
spotting in document images. In 2017 Seventh International Conference on Emerging Security
Technologies (EST) (pp. 26-31). IEEE.
[17]. Artaud, C., Doucet, A., Ogier, J. M., & d'Andecy, V. P. (2017, November). Receipt Dataset for Fraud
Detection. In First International Workshop on Computational Document Forensics.
[18]. Artaud, C., Sidère, N., Doucet, A., Ogier, J. M., & Yooz, V. P. D. A. (2018, August). Find it! fraud
detection contest report. In 2018 24th International Conference on Pattern Recognition (ICPR) (pp. 13-18).
IEEE.
[19]. Nandanwar, L., Shivakumara, P., Pal, U., Lu, T., Lopresti, D., Seraogi, B., & Chaudhuri, B. B. (2021). A
new method for detecting altered text in document images. International Journal of Pattern Recognition and
Artificial Intelligence, 35(12), 2160010
[20]. Nandanwar, L., Shivakumara, P., Mondal, P., Raghunandan, K. S., Pal, U., Lu, T., & Lopresti, D. (2021).
Forged text detection in video, scene, and document images. IET Image Processing, 14(17), 4744-4755.
[21]. Deshpande, P., & Kanikar, P. (2012). Pixel based digital image forgery detection techniques. International
Journal of Engineering Research and Applications (IJERA), 2(3), 539-543.
[22]. Van Beusekom, J., Shafait, F., & Breuel, T. M. (2013). Text-line examination for document forgery
detection. International Journal on Document Analysis and Recognition (IJDAR), 16(2), 189-207
[23]. Wilkinson, T. S., & Goodman, J. W. (1991, February). Slope histogram detection of forged handwritten
signatures. In High-Speed Inspection Architectures, Barcoding, and Character Recognition (Vol. 1384, pp.
293-304). SPIE.
[24]. Shivakumara, P., Basavaraja, V., Gowda, H. S., Guru, D. S., Pal, U., & Lu, T. (2018, August). A new RGB
based fusion for forged IMEI number detection in mobile images. In 2018 16th International Conference
on Frontiers in Handwriting Recognition (ICFHR) (pp. 386-391). IEEE
[25]. Kundu, S., Shivakumara, P., Grouver, A., Pal, U., Lu, T., & Blumenstein, M. (2019, November). A new
 forged handwriting detection method based on Fourier spectral density and variation. In Asian Conference
on Pattern Recognition (pp. 136-150). Springer, Cham.
[26]. Hashmi, S. A. A., Upadhyay, S., & Kumar, R. (2021). Comparative Study of Forged Urdu Signatures Done
By Persons Not Familiar To Language Belongs To Region of Sikkim And Kashmir. Journal of Punjab
Academy of Forensic Medicine & Toxicology, 21(1).
[27]. Lee, J., Kong, S. G., Lee, Y. S., Moon, K. W., Jeon, O. Y., Han, J. H., ... & Seo, J. S. (2012). Forged seal
detection based on the seal overlay metric. Forensic science international, 214(1-3), 200-206.
[28]. Tralic, D., Zupancic, I., Grgic, S., & Grgic, M. (2013, September). CoMoFoD—New database for copy-
move forgery detection. In Proceedings ELMAR-2013 (pp. 49-54). IEEE.
[29]. Cha, S. H., & Tappert, C. C. (2002, August). Automatic detection of handwriting forgery. In Proceedings
Eighth International Workshop on Frontiers in Handwriting Recognition (pp. 264-267). IEEE.
[30]. Nagel, R. N., & Rosenfeld, A. (1977). Computer detection of freehand forgeries. IEEE Transactions on
Computers, 26(09), 895-905.
[31]. Megahed, A., Fadl, S. M., Han, Q., & Li, Q. (2017, November). Handwriting forgery detection based on
ink colour features. In 2017 8th IEEE International Conference on Software Engineering and Service
Science (ICSESS) (pp. 141-144). IEEE.
[32]. Zhu, X., & Goldberg, A. B. (2009). Introduction to semi-supervised learning. Synthesis lectures on
artificial intelligence and machine learning, 3(1), 1-130.

AUTHORS

Teerath kumar received his Bachelor’s degree in Computer Science with

distinction from National University of Computer and Emerging Science
(NUCES), Islamabad, Pakistan, in 2018. Currently, he is pursuing PhD from
Dublin City University, Ireland. His research interests include advanced data
augmentation, deep learning for medical imaging, generative adversarial networks
and semi-supervised learning.

Muhammad Turab is an undergraduate final year student at Computer Systems

Engineering MUET, Jamshoro. He has done 60+ projects with java and python, all
projects can be found on GitHub. His research interests include deep learning,
computer vision and data augmentation for medical imaging.

Shahnawaz Talpur is the chairman of Computer Systems Engineering

Department at Muet Jamshoro. He has done his masters from MUET and PhD
from Beijing Institute of Technology, China. His research interests include high
performance computing, computer architecture and big data.

R. Brennan is an Assistant Professor in the School of Computing, Dublin City

University, founding Chair of the DCU MA in Data Protection and Privacy Law
and a Funded investigator in the Science Foundation Ireland ADAPT Centre for
Digital Content Technology which is funded under the SFI Research Centres
Programme (Grant 13/RC/2106) and is co-funded under the European Regional
Development Fund, His main research interests are data protection, data value,
data quality, data privacy, data/AI governance and semantics.

M. Bendechache is an Assistant Professor in the School of Computing at

Dublin City University, Ireland. She obtained her Ph.D. degree from University
College Dublin, Ireland in 2018. Malika’s research interests span the areas of
Big data Analytics, Machine Learning, Data Governance, Cloud Computing,
Blockchain, Security, and Privacy. She is an academic member and a Funded
Investigator of ADAPT and Lero research centres.
 TRANSFER LEARNING BASED IMAGE
VISUALIZATION USING CNN
Santosh Giri1 and Basanta Joshi2

1Department of Computer & Electronics Engineering, Kathford Int'l College ofEngineering and
Management, IOE, TU, Nepal
2Department of Electronics & Computer Engineering, Pulchowk Campus, IOE, TU,Nepal

ABSTRACT

Image classification is a popular machine learning based applications of deep learning. Deep learning
techniques are very popular because they can be effectively used in performing operations on image
data in large-scale. In this paper CNN model was designed to better classify images. We make use of
feature extraction part of inception v3 model for feature vector calculation and retrained the
classification layer with these feature vector. By using the transfer learning mechanism the
classification layer of the CNN model was trained with 20 classes of Caltech101 image dataset and 17
classes of Oxford 17 flower image dataset. After training, network was evaluated with testing dataset
images from Oxford 17 flower dataset and Caltech101 image dataset. The mean testing precision of
the neural network architecture with Caltech101 dataset was 98 % and with Oxford 17 Flower image
dataset was 92.27 %.

KEYWORDS

Image Classification, CNN, Deep Learning, Transfer Learning.

For More Details: https://aircconline.com/ijaia/V10N4/10419ijaia04.pdf

Volume Link:https://www.airccse.org/journal/ijaia/current2019.html
REFERENCES
[1] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel,
Backpropagation applied to handwritten zip code recognition," Neural computation, vol. 1, no. 4, pp. 541-
551, 1989.
[2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, Imagenet classification with deep convolutional neural
networks," in Advances in neural information processing systems, 2012, pp. 1097-1105.
[3] D. R. Reddy, Speech recognition by machine: A review," Proceedings of the IEEE, vol. 64, no. 4, pp. 501-
531, 1976.
[4] A. Coates, B. Carpenter, C. Case, S. Satheesh, B. Suresh, T. Wang, D. J. Wu, and A. Y. Ng, Text detection
and character recognition in scene images with unsupervised feature learning,"in Document Analysis and
Recognition (ICDAR), 2011 International Conference on. IEEE, 2011, pp. 440-445.
[5] T. Varga, D. Kilchhofer, and H. Bunke, Template-based synthetic handwriting generation for the training
of recognition systems," in Proceedings of the 12th Conference of the International Graphonomics Society,
2005, pp. 206-211.
[6] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, Rethinking the inception architecture for
computer vision," in Proceedings of the IEEE conference on computer vision and pattern recognition,
2016, pp. 2818-2826.
[7] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A.
Rabinovich, Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision
and pattern recognition, 2015, pp. 1-9.
[8] Q. Liu, A. Puthenputhussery, and C. Liu, Novel general knn classifier and general nearest mean classifier
for visual classification," in Image Processing (ICIP), 2015 IEEE International Conference on. IEEE, 2015,
pp. 1810-1814.
[9] J. M. Keller, M. R. Gray, and J. A. Givens, A fuzzy k-nearest neighbor algorithm," IEEE transactions on
systems, man, and cybernetics, no. 4, pp. 580-585, 1985.
[10] J. A. Tropp, Greed is good: Algorithmic results for sparse approximation," IEEE Transactions on
Information theory, vol. 50, no. 10, pp. 2231-2242, 2004.
[11] A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems,"
SIAM journal on imaging sciences, vol. 2, no. 1, pp. 183-202, 2009.
[12] K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman, Return of the devil in the details: Delving deep
into convolutional nets," arXiv preprint arXiv:1405.3531, 2014.
[13] Jolliffe, Principal component analysis," in International encyclopedia of statistical science. Springer, 2011,
pp. 1094-1096.
[14] Sun, X. Cai, F. Sun, and J. Zhang, Scene image classification method based on alex-net model," in
Informative and Cybernetics for Computational Social Systems (ICCSS), 2016 3rd International
Conference on. IEEE, 2016, pp. 363-367.
[15] A. Krizhevsky, I. Sutskever, and G. E. Hinton, Imagenet classification with deep convolutional neural
networks," in Advances in neural information processing systems, 2012, pp. 1097-1105.
[16] B. Cortes and V. Vapnik, Support-vector networks," Machine learning, vol. 20, no. 3, pp. 273-297, 1995.
[17] N. M. Nasrabadi, Pattern recognition and machine learning," Journal of electronic imaging, vol. 16, no. 4,
p. 049901, 2007.
[18] He, X. Zhang, S. Ren, and J. Sun, Spatial pyramid pooling in deep convolutional networks for visual
recognition," in European conference on computer vision.Springer, 2014, pp. 346-361.
[19] T. Tuytelaars, M. Fritz, K. Saenko, and T. Darrell, The nbnn kernel," in Computer Vision (ICCV), 2011
IEEE International Conference on. IEEE, 2011, pp. 1824-1831.
[20] P. Murphy et al., Naive bayes classifiers," University of British Columbia, vol. 18, 2006.
[21] Z. S. Harris, Distributional structure," Word, vol. 10, no. 2-3, pp. 146-162, 1954.
[22] C. Theriault, N. Thome, and M. Cord, Extended coding and pooling in the hmax model," IEEE
Transactions on Image Processing, vol. 22, no. 2, pp. 764-777, 2013.
[23] Riesenhuber and T. Poggio, Hierarchical models of object recognition in cortex," Nature neuroscience, vol.
2, no. 11, p. 1019, 1999.
[24] M.-E. Nilsback and A. Zisserman, A visual vocabulary for flower classification," in Computer Vision and
Pattern Recognition, 2006 IEEE Computer Society Conference on, vol. 2. IEEE, 2006, pp. 1447-1454.
[25] D. G. Lowe, Object recognition from local scale-invariant features," in Computer vision, 1999. The
proceedings of the seventh IEEE international conference on, vol. 2. Ieee, 1999, pp. 1150-1157.
[26] A. R. Smith, Color gamut transform pairs," ACM Siggraph Computer Graphics, vol. 12, no. 3, pp. 12-19,
1978.
[27] S. B. Kotsiantis, I. Zaharakis, and P. Pintelas, Supervised machine learning: A review of classification
techniques," Emerging artificial intelligence applications in computer engineering, vol.160, pp. 3-24, 2007.
[28] L. Fei-Fei, R. Fergus, and P. Perona, Learning generative visual models from few training examples: An
 incremental bayesian approach tested on 101 object categories," Computer vision and Image
understanding, vol. 106, no. 1, pp. 59-70, 2007.
[29] M.-E. Nilsback and A. Zisserman, A visual vocabulary for flower classification," in Computer Vision and
Pattern Recognition, 2006 IEEE Computer Society Conference on, vol. 2. IEEE, 2006, pp. 1447-1454.
[30] S. J. Pan, Q. Yang et al., A survey on transfer learning," IEEE Transactions on knowledge and data
engineering, vol. 22, no. 10, pp. 1345-1359, 2010.
[31] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, ImageNet: A Large-Scale Hierarchical Image
Database," in CVPR09, 2009.
[32] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et
al., Tensor flow: a system for large-scale machine learning." in OSDI, vol. 16, 2016, pp. 265-283.
[33] Y. Chai, V. Lempitsky, and A. Zisserman, Bicos: A bi-level co-segmentation method for image
classification," 2011.
AUTOMATIC TUNING OF PROPORTIONAL– INTEGRAL–
DERIVATIVE (PID) CONTROLLER USING PARTICLE
SWARM OPTIMIZATION (PSO) ALGORITHM
S. J. Bassi1, M. K. Mishra2 and E. E. Omizegba3
1
Department of Computer Engineering, University of Maiduguri, Borno State, Nigeria
2
Department of Computer Engineering, University of Maiduguri, Borno State, Nigeria
3
Electrical and Electronics Engineering Programme, Abubakar Tafawa Balewa University, P.M.B
0248, Bauchi, Bauchi State, Nigeria

ABSTRACT

The proportional-integral-derivative (PID) controllers are the most popular controllers used in industry
because of their remarkable effectiveness, simplicity of implementation and broad applicability.
However, manual tuning of these controllers is time consuming, tedious and generally lead to poor
performance. This tuning which is application specific also deteriorates with time as a result of plant
parameter changes. This paper presents an artificial intelligence (AI) method of particle swarm
optimization (PSO) algorithm for tuning the optimal proportional-integral derivative (PID) controller
parameters for industrial processes. This approach has superior features, including easy
implementation, stable convergence characteristic and good computational efficiency over the
conventional methods. Ziegler- Nichols, tuning method was applied in the PID tuning and results were
compared with the PSO-Based PID for optimum control. Simulation results are presented to show that
the PSO-Based optimized PID controller is capable of providing an improved closed-loop
performance over the Ziegler- Nichols tuned PID controller Parameters. Compared to the heuristic
PID tuning method of Ziegler-Nichols, the proposed method was more efficient in improving the step
response characteristics such as, reducing the steady-states error; rise time, settling time and maximum
overshoot in speed control of DC motor.

KEYWORDS

PID Controller, Particle swarm optimization algorithm, Ziegler- Nichols method, Simulation

For More Details:https://aircconline.com/ijaia/V2N4/1011ijaia03.pdf

Volume Link: http://www.airccse.org/journal/ijaia/current2011.html

REFERENCES

[1]. A.Varsek, T. Urbacic and B. Filipic, 1993, Genetic Algorithms in Controller Design and Tuning, IEEE
Trans. Sys. Man and Cyber, Vol. 23/5, pp1330-1339.
[2]. Astrom, K. J. and T., Hagglund, 1995, PID Controllers: Theory, Design and Tuning, ISA, Research
Triangle, Par, NC.
[3]. B.Nagaraj,S.Suba and B.Rampriya: Tuning Algorithms for PID Controller Using Soft Computing
Techniques; International Journal of Computer Science and Network Security(IJCSNS), VOL.8 No.4,
April 2008. pp278-281.
[4]. Clerc, M. ,1999, The Swarm and the queen: towards a deterministic and adaptive particle swarm
optimization. Proceedings of the Conference on Evolutionary Computation, pp. 1951-1957.
[5]. Cohen G.H. and Coon G.A., “Theoretical consideration of retarded control”, Trans. ASME, Vol. 75, pp.
827-834, 1953
[6]. Eberhart, R.C. and Shi, Y.H., 2000, Comparing inertia weights and constriction factors in particle swarm
optimization. Proc. The 2000 Congress on Evolutionary Computation. Vol.1, pp. 84-88.
[7]. Engineering and Information Technology 2011
[8]. Gaing, Z.L., 2004, A particle swarm optimization approach for optimum design of PID controller in AVR
system. IEEE Transaction on Energy Conversion, Vol.19(2), pp.384-391.
[9]. Hugo, J. Alan, 2002, Process Controller Performance Monitoring and Assessment,
http/www.controlartsinc.com/support/articles/PerformanceAssessment.PDF. Accessed 31/6/10
[10]. K Ogata, 1987, Modern Control Systems, University of Minnesota, Prentice Hall.
[11]. K Ogata, 2005, Modern Engineering (fifth edition), University of Minnesota, Prentice Hall.
[12]. K. Krishnakumar and D. E. Goldberg, 1992, Control System Optimization Using Genetic Algorithms,
Journal of Guidance, Control and Dynamics, Vol. 15, No. 3, pp. 735-740.
[13]. Kennedy, J. and Eberhart, R.C., 1995, Particle swarm optimization. Proc. IEEE International Conference
on Neural Networks (Perth,Australia), IEEE Service Center, Piscataway, NJ, pp. IV: 1942- 1948.
[14]. Krohling RA, Rey JP., 2001, Design of optimal disturbance rejection PID controllers using genetic
algorithm. IEEE Trans Evol Comput;5: pp. 78–82.
[15]. Kwok,D.P.,T.P.Leung and F.Sheng, 1993, Genetic algorithm for optimal dynamic control of robot arms.
Proceedings of the International Conference on Industrial Electronics,Control and Instrumentation.15-19
November,SanFrancisco,CA, pp. 380-385.
[16]. Mahmud Iwan Solihin, Lee Fook Tack and Moey Leap Kean, Tuning of PID Controller Using Particle
Swarm Optimization (PSO), Proceeding of the International Conference on Advanced Science,
[17]. Mitsukura Y, Yamamoto T, Kaneda M., June 1999, A design of self-tuning PID controllers using a genetic
algorithm. In: Proc Am Contr Conf, San Diego, CA, pp. 1361–5.
[18]. Nagaraj B, Subba S and Rampriya B, 2005, Tuning Algorithm for PID Using Soft-Computing Techniques.
International Journal of Computer Science and Network Security, Vol 8, No 4 April , pp. 278 – 289.
[19]. Ou, C. and Lin, W., 2006, Comparison between PSO and GA for parameters optimization of PID
controller. Proc. IEEE International Conference on Mechatronics and Automation. Luoyang, China.
[20]. Pillay N. and Govender P., 2007, A Particle Swarm Optimization Approach for Model Independent Tuning
of PID Control Loop, IEEE Africon 2007, IEEE Catalog: 04CH37590C, ISBN: 0-7803-8606-X.
[21]. Q.Wang, P Spronck and R Tracht, 2003, An Overview of Genetic Algorithms Applied to Control
Engineering Problems. Proceedings of the Second International Conference on Machine Learning and
Cybernetics.
[22]. Qu Sun, Renhou Li and Ping and Zhang, 2003, Stable and Optimal Adaptive Fuzzy Control of Complex
Systems using Fuzzy Dynamic Model. Fuzzy Sets and Systems, No 133 pp. 1 – 17.
[23]. Rasmussen H., 2002, Automatic Tuning of PID Regulator. http://www.volgas.dk/nores/auto.pdf
downloaded 30/6/10.
[24]. Schei, Tor Steiner, 1994, Automatic Tuning of PID Controllers Based on Transfer Function Estimation,
Automatica, pp. pp. 1983 – 1989.
[25]. Seng TL, Khalid MB, Yusof R., 1999, Tuning of a neuro-fuzzy controller by genetic algorithm. IEEE
Trans Syst Man Cybern B,29, pp. 226–36.
[26]. Shi, Y.H. and Eberhart, R.C., 1998, A modified particle swarm optimizer. IEEE International Conference
on Evolutionary Computation, Anchorage, Alaska.
[27]. Shi, Y.H. and Eberhart, R.C., 2001, Fuzzy Adaptive Particle Swarm Optimization. Proc. Congress on
Evolutionary Computation. Seoul,Korea.
[28]. T. O.Mahony, C J Downing and K Fatla, 2000, Genetic Algorithm for PID Parameter Optimization:
Minimizing Error Criteria, Process Control and Instrumentation, University of Stracthclyde, pp. 148- 153.
[29]. Visioli A., 2001, Tuning of PID controllers with fuzzy logic. Proc Inst Elect Eng Contr Theory Appl,
148(1), pp. 1–8.
[30]. Zhong Jinghua ,2006, PID Controller Tuning: A Short Tutorial,
http://saba.kntu.ac.ir/eecd/pd/download/PIDtutorial.pdf downloaded 1/7/2010
 A MODIFIED MAXIMUM RELEVANCE
MINIMUM REDUNDANCY FEATURE
SELECTION METHOD BASED ON TABU
SEARCH FOR PARKINSON’S DISEASE
MINING
Waheeda Almayyan
Computer Information Department, Collage of Business Studies, PAAET, Kuwait

ABSTRACT

Parkinson’s disease is a complex chronic neurodegenerative disorder of the central nervous

system. One of the common symptoms for the Parkinson’s disease subjects, is vocal performance
degradation. Patients usually advised to follow personalized rehabilitative treatment sessions with
speech experts. Recent research trends aim to investigate the potential of using sustained vowel
phonations for replicating the speech experts’ assessments of Parkinson’s disease subjects’
voices. With the purpose of improving the accuracy and efficiency of Parkinson’s disease
treatment, this article proposes a two-stage diagnosis model to evaluate an LSVT dataset. Firstly,
we propose a modified minimum Redundancy-Maximum Relevance (mRMR) feature selection
approach, based on Cuckoo Search and Tabu Search to reduce the features numbers. Secondly,
we apply simple random sampling technique to dataset to increase the samples of the minority
class. Promisingly, the developed approach obtained a classification Accuracy rate of 95% with
24 features by 10-fold CV method.

KEYWORDS

Parkinson’s disease; Medical data mining; maximum Relevance Minimum Redundancy, Tabu Search&
Simple Random Sampling

For More Details: https://aircconline.com/ijaia/V11N2/11220ijaia01.pdf

Volume Link: https://www.airccse.org/journal/ijaia/current2020.html

 REFERENCES

[1] Aarli, J.A.; Dua, T.; Janca, A.; Muscetta, A. Neurological Disorders: Public Health Challenges.World
Health Organization: Geneva, Switzerland, 2006.
[2] De Lau, L.M.; Breteler, M.M. Epidemiology of parkinsonʼs disease. Lancet Neurol. 2006, 5, 525– 535.
[3] Ho, A. K., Iansek, R., Marigliani, C., Bradshaw, J. L., & Gates, S. (1998). Speech impairment in a large
sample of patients with Parkinson’s disease. Behavioural Neurology, 11, 131–138.
[4] Ciosa, K.J. and Mooree, G.W. (2002) Uniqueness of medical data mining,Artif. Intell. Med. 26 1– 24.
[5] Huang, S.H. Wulsin, L.R. Li, H. and Guo, J. (2009) Dimensionality reduction for knowledge discovery
in medical claims database: application to antidepressant medication utilization study, Comput. Methods
Programs Biomed. 93, 115–123.
[6] Luukka, P. (2011) Feature selection using fuzzy entropy measures with similarity classifier, Expert
Systems with Applications, Volume 38, Issue 4, 4600-4607, ISSN 0957-4174,
http://dx.doi.org/10.1016/j.eswa.2010.09.133.
A. Tsanas, M.A. Little, C. Fox, L.O. Ramig: Objective automatic assessment of rehabilitative speech
treatment in Parkinson’s disease, IEEE Transactions on Neural Systems and Rehabilitation Engineering,
Vol. 22, pp. 181-190, January 2014.
[7] M. A. Little, P. E. McSharry, E. J. Hunter, J. Spielman, and L. O. Ramig, “Suitability of dysphonia
measurements for telemonitoring of Parkinson’s disease,” IEEE Transactions on Biomedical
Engineering, vol. .56, no. 4, pp. 1015-1022, 2009.
[8] Das, R. (2010). A comparison of multiple classification methods for diagnosis of Parkinson disease.
Expert Systems with Applications, 37, 1568–1572.
[9] Shahbaba, B., & Neal, R. (2009). Nonlinear models using Dirichlet process mixtures. The Journal of
Machine Learning Research, 10, 1829–1850.
[10] Sakar, C. Okan, and Olcay Kursun. "Telediagnosis of Parkinson’s disease using measurements of
dysphonia." Journal of medical systems 34.4 (2010): 591-599.
[11] Psorakis, I., Damoulas, T., & Girolami, M. A. (2010). Multiclass relevance vector machines: sparsity
and accuracy. Neural Networks, IEEE Transactions on, 21, 1588–1598.
[12] Guo, P. F., Bhattacharya, P., & Kharma, N. (2010). Advances in detecting Parkinson’s disease.
[13] Medical Biometrics, 306–314.
[14] Luukka, P. (2011). Feature selection using fuzzy entropy measures with similarity classifier.
[15] Expert Systems with Applications, 38, 4600–4607.
[16] Li, D. C., Liu, C. W., & Hu, S. C. (2011). A fuzzy-based data transformation for feature extraction to
increase classification performance with small medical data sets. Artificial Intelligence in Medicine, 52,
45–52.
[17] Ozcift, A., & Gulten, A. (2011). Classifier ensemble construction with rotation forest to improve
medical diagnosis performance of machine learning algorithms. Comput Methods Programs Biomed,
104, 443–451.
[18] AStröm, F., & Koker, R. (2011). A parallel neural network approach to prediction of Parkinson’s
Disease. Expert Systems with Applications, 38, 12470–12474.
[19] Spadoto, A. A., Guido, R. C., Carnevali, F. L., Pagnin, A. F., Falcao, A. X., & Papa, J. P. (2011).
Improving Parkinson’s disease identification through evolutionarybased feature selection. In
Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the
IEEE (pp. 7857–7860).
[20] Sriram, TarigoppulaV.S., Rao, M.Venkateswara, Narayana, G.V.Satya & Kaladhar, D.S.V.G.K. (2015).
Diagnosis of Parkinson Disease Using Machine Learning and Data Mining Systems from Voice Dataset.
In Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and
Applications (FICTA) 2014. (pp. 151-157).
[21] Chuan Xu, Jiajun Chen, Xia Xu, Yingyu Zhang, and Jia Li, “Potential Therapeutic Drugs for
Parkinson’s Disease Based on Data Mining and Bioinformatics Analysis,” Parkinson’s Disease, vol.
2018, Article ID 3464578, 8 pages, 2018. https://doi.org/10.1155/2018/3464578.
[22] Marras C, Beck JC, Bower JH, Roberts E, Ritz B, Ross GW, Abbott RD, Savica R, Van Den Eeden SK,
Willis AW, Tanner CM; Parkinson’s Foundation P4 Group. Prevalence of Parkinson's disease across
North America. NPJ Parkinsons Dis. 2018 Jul 10;4:21. doi: 10.1038/s41531-018- 0058-0. PMID:
30003140; PMCID: PMC6039505.
[23] C. Ding and H. Peng, “Minimum redundancy feature selection from microarray gene expression data,”
Journal of Bioinformatics and Computational Biology, vol. 3, no. 2, pp. 185–205, 2005.
[24] H. Peng, F. Long, and C. Ding, “Feature selection based on mutual information criteria of max-
dependency, max- relevance, and min-redundancy,” IEEE Transactions on Pattern Analysis and
Machine Intelligence, vol. 27, no. 8, pp. 1226–1238, 2005.
[25] El A, Aouatif A, El A, Driss O (2011) A two-stage gene selection scheme utilizing MRMR filter and
 GA wrapper. Knowl Inf Syst 26:487–500
[26] Li A, Hu L, Niu S, Cai Y, Chou K (2012) Predict and analyze Snitrosylation modification sites with the
mRMR and IFS approaches. J Proteom 75:1654–1665
[27] F. Glover, “Tabu search—part I,” ORSA Journal on Computing, vol. 1, no. 3, pp. 190–206, 1989. [ 27 ]
F. Glover, “Tabu search—part II,” ORSA Journal on Computing, vol. 2, no. 1, pp. 4–32, 1990.
[28] Tahir, M.A., Bouridane, A., Kurugollu, F., Amira, A., 2004a. Feature Selection using Tabu Search for
Improving the Classification Rate of Prostate Needle Biopsies. In: Proc. 17th International Conference
on Pattern Recognition (ICPR 2004), Cambridge, UK.
[29] Tahir, M.A., Bouridane, A., Kurugollu, F., 2004b. Simulataneous Feature Selection and Weigthing for
Nearest Neighbor Using Tabu Search. In: Lecture Notes in Computer Science (LNCS 3177), 5th
International Conference on Intelligent Data Engineering and Automated Learning (IDEAL 2004),
Exeter, UK.
[30] Korycinski, D., Crawford, M., Barnes, J.W, and Ghosh, J., 2003. Adaptive feature selection for
hyperspectral data analysis using a binary hierarchical classifier and Tabu Search. In: Proceedings of the
IEEE International Geoscience and Remote Sensing Symposium, IGARSS.
[31] Sait, S.M., Youssef, H., 1999. General iterative algorithms for combinatorial optimization. IEEE
Computer Society.
[32] G. Weiss and F. Provost, "Learning when Training Data are Costly: The Effect of Class Distribution on
Tree Induction," J. Artificial Intelligence Research, vol.19,315-354,2003.
[33] Park, B.-H., Ostrouchov, G., Samatova, N.F., Geist, A.: Reservoir-based random sampling with
replacement from data stream. In: SDM 2004, 492-496, (2004)
[34] Estabrooks, A., Jo, T. & Japkowicz, N., 2004. A Multiple Resampling Method for Learning from
Imbalanced Data Sets. Computational Intelligence, 20(1), pp.18– 36.
[35] Yang, Y.Y. et al., 2011. Adaptive neural-fuzzy inference system for classification of rail quality data
with bootstrapping-based over-sampling. IEEE International Conference on Fuzzy Systems (FUZZ-
IEEE 2011), pp.2205–2212.
[36] Mitra SK and Pathak PK. The nature of simple random sampling. Ann. Statist., 1984, 12:1536- 1542.
[37] Ramig L, Pawlas A, Countryman S. The Lee Silverman voice treatment (LSVT®): a practical guide to
treating the voice and speech disorders in Parkinson disease. Iowa City, IA: National Center for Voice
and Speech; 1995.
[38] F. Provost, T. Fawcett, and R. Kohavi. The case against accuracy estimation for comparing induction
algorithms, Proceedings of the Fifteenth International Conference on Machine Learning, San Francisco,
CA: Morgan Kaufmann, 445-453, 1998.
[39] M.A. Maloof, Learning when data sets are Imbalanced and when costs are unequal and unknown,
ICML-2003 Workshop on Learning from Imbalanced Data Sets II, 2003.
[40] Smialowski P, Doose G, Torkler P, Kaufmann S, Frishman D: PROSO II-a new method for protein
solubility prediction. FEBS J 2012, 279(12):2192-2200.
[41] Oshiro TM, Perez PS, Baranauskas JA (2012) How many trees in a Random Forest? In: Machine
learning and data mining in pattern recognition. Springer, Berlin, pp 154–168 .
 HOME APPLIANCE IDENTIFICATION FOR
NILM SYSTEMS BASED ON DEEP NEURAL
NETWORKS
Deyvison de Paiva Penha1 and Adriana Rosa Garcez Castro2

1 Institute of Technology, Federal University of Para, Belém, Brazil

2 Institute of Technology, Federal University of Para, Belém, Brazil

ABSTRACT

This paper presents the proposal for the identification of residential equipment in non-intrusive
load monitoring systems. The system is based on a Convolutional Neural Network to classify
residential equipment. As inputs to the system, transient power signal data obtained at the time an
equipment is connected in a residence is used. The methodology was developed using data from a
public database (REED) that presents data collected at a low frequency (1 Hz). The results
obtained in the test database indicate that the proposed system is able to carry out the
identification task, and presented satisfactory results when compared with the results already
presented in the literature for the problem in question.

KEYWORDS

Convolutional Neural Networks, identification of residential equipment, non-intrusive load monitoring,

NILM.

For More Details: https://aircconline.com/ijaia/V9N2/9218ijaia06.pdf

Volume Link: https://www.airccse.org/journal/ijaia/current2018.html

REFERENCES

[1] HART, George William. Nonintrusive appliance load monitoring. Proceedings of the IEEE, v. 80, n. 12, p.
1870-1891, 1992.

[2] FIGUEIREDO, Marisa. Contributions to Electrical Energy Disaggregation in a Smart Home. 2014. Tese de
Doutorado. APA. Disponível em: <www:http://hdl.handle.net/10316/24256>. Acessado em: novembro de
2017.

[3] WANG, Zhiguang; OATES, Tim. Encoding time series as images for visual inspection and classification
using tiled convolutional neural networks. In: Workshops at the Twenty-Ninth AAAI Conference on
Artificial Intelligence. 2015.

[4] ZHENG, Yi et al. Time series classification using multi-channels deep convolutional neural networks. In:
International Conference on Web-Age Information Management. Springer, Cham, 2014. p. 298-310.

[5] LECUN, Yann et al. Convolutional networks for images, speech, and time series. The handbook of brain
theory and neural networks, v. 3361, n. 10, p. 1995, 1995.

[6] LEE, Honglak et al. Unsupervised feature learning for audio classification using convolutional deep belief
networks. In: Advances in neural information processing systems. 2009. p. 1096 1104.

[7] International Journal of Artificial Intelligence and Applications (IJAIA), Vol.9, No.2, March 2018
LÄNGKVIST, Martin; KARLSSON, Lars; LOUTFI, Amy. A review of unsupervised feature learning and
deep learning for time-series modeling. Pattern Recognition Letters, v. 42, p. 11-24, 2014.

[8] ZHENG, Yi et al. Exploiting multi-channels deep convolutional neural networks for multivariate time
series classification. Frontiers of Computer Science, v. 10, n. 1, p. 96-112, 2016.

[9] KELLY, Jack; KNOTTENBELT, William. Neural nilm: Deep neural networks applied to energy
disaggregation. In: Proceedings of the 2nd ACM International Conference on Embedded Systems for
Energy-Efficient Built Environments. ACM, 2015. p. 55-64.

[10] DO NASCIMENTO, Pedro Paulo Marques. Applications of Deep Learning Techniques on NILM. 2016.
Tese de Doutorado. Universidade Federal do Rio de Janeiro.

[11] Wan He and Ying Chai. An Empirical Study on Energy Disaggregation via Deep Learning, in Advances in
Intelligent Systems Research, volume 133, 2nd International Conference on Artificial Intelligence and
Industrial Engineering (AIIE2016), pp338-341, 2016

[12] DE BAETS, Leen et al. Appliance classification using VI trajectories and convolutional neural networks.
Energy and Buildings, v. 158, p. 32-36, 2018.

[13] KOLTER, J. Zico; JOHNSON, Matthew J. REDD: A public data set for energy disaggregation research.
In: Workshop on Data Mining Applications in Sustainability (SIGKDD), San Diego, CA. 2011. p. 59-62.

[14] KATO, Takekazu et al. Appliance Recognition from Electric Current Signals for Information Energy
Integrated Network in Home Environments. ICOST, v. 9, p. 150-157, 2009.

[15] FIGUEIREDO, Marisa B.; DE ALMEIDA, Ana; RIBEIRO, Bernardete. An experimental study on
electrical signature identification of non-intrusive load monitoring (nilm) systems. In: International
Conference on Adaptive and Natural Computing Algorithms. Springer, Berlin, Heidelberg, 2011. p. 31-40.

[16] BATRA, Nipun et al. A comparison of non-intrusive load monitoring methods for commercial and
residential buildings. arXiv preprint arXiv:1408.6595, 2014.

[17] CARVALHO, Jorge Miguel Vidal. Metodologias de monitorização de consumos. 2013.

[18] NAJMEDDINE, Hala et al. State of art on load monitoring methods. In: Power and Energy Conference,
2008. PECon 2008. IEEE 2nd International. IEEE, 2008. p. 1256-1258.
[19] PARSON, Oliver. Unsupervised training methods for non-intrusive appliance load monitoring from smart
meter data. 2014. Tese de Doutorado. University of Southampton.

[20] WONG, Yung Fei et al. Recent approaches to non-intrusive load monitoring techniques in residential
settings. In: Computational Intelligence Applications In Smart Grid (CIASG), 2013 IEEE Symposium on.
IEEE, 2013. p. 73-79.

[21] PROVOST, Foster; KOHAVI, Ron. Guest editors' introduction: On applied research in machine learning.
Machine learning, v. 30, n. 2, p. 127-132, 1998.

[22] ABDEL-HAMID, Ossama et al. Convolutional neural networks for speech recognition. IEEE/ACM
Transactions on audio, speech, and language processing, v. 22, n. 10, p. 1533-1545, 2014.

[23] Atabay, H.A.: Binary shape classification using convolutional neural networks. IIOAB J. 7(5), 332–336
(2016)

[24] VARGAS, A. C. G.; PAES, A.; VASCONCELOS, C. N. Um estudo sobre redes neurais convolucionais e
sua aplicação em detecção de pedestres. In: Proceedings of the XXIX Conference on Graphics, Patterns
and Images. 2016. p. 1-4.

[25] WONG, Yung Fei; DRUMMOND, T.; ŞEKERCIOĞLU, Y. A. Real-time load disaggregation algorithm
using particle-based distribution truncation with state occupancy model. Electronics Letters, v. 50, n. 9, p.
697-699, 2014.

[26] ZHAO, Bochao; STANKOVIC, Lina; STANKOVIC, Vladimir. On a training-less solution for non-
intrusive appliance load monitoring using graph signal processing. IEEE Access, v. 4, p. 1784-1799, 2016.

[27] KONG, Weicong et al. Improving Nonintrusive Load Monitoring Efficiency via a Hybrid Programing
Method. IEEE Transactions on Industrial Informatics, v. 12, n. 6, p. 2148-2157, 2016.

[28] HIJAZI, Samer; KUMAR, Rishi; ROWEN, Chris. Using convolutional neural networks for image
recognition. Tech. Rep., 2015. [Online]. Available: http://ip. cadence. com/uploads/901/cnn-wp-pdf.

[29] DE PAIVA PENHA, Deyvison; CASTRO, Adriana Rosa Garcez. Convolutional neural network applied to
the identification of residential equipment in non-intrusive load monitoring systems.

[30] In: 3rd International Conference on Artificial Intelligence and Applications, pp. 11– 21, 2017. © CS & IT-
CSCP 2017

AUTHORS

B. Sc. Deyvison de Paiva Penha is a Master Student in the Electrical Engineering Graduate Program in Federal
University of Pará. He received his bachelor degree in 2009 at the Federal University of Pará. Prof. Dr. Adriana
Rosa Garcez Castro has a Master’s degree in Electrical Engineering from the Federal University of Pará in 1995
and a PhD in Electrical Engineering from the Faculty of Engineering of the University of Porto in 2004. She is
currently a Professor at the Federal University of Pará. His areas of interest are: Control of Electronic Processes
and Computational Intelligence applied to Energy Systems.
 A REVIEW ON OPTIMIZATION OF LEAST
SQUARES SUPPORT VECTOR MACHINE
FOR TIME SERIES FORECASTING
Yuhanis Yusof1 and Zuriani Mustaffa2

1 School of Computing, Universiti Utara Malaysia, Malaysia

2 Faculty of Computer Systems & Software Engineering, Universiti Malaysia Pahang, Malaysia

ABSTRACT

Support Vector Machine has appeared as an active study in machine learning community and
extensively used in various fields including in prediction, pattern recognition and many more.
However, the Least Squares Support Vector Machine which is a variant of Support Vector
Machine offers better solution strategy. In order to utilize the LSSVM capability in data mining
task such as prediction, there is a need to optimize its hyper parameters. This paper presents a
review on techniques used to optimize the parameters based on two main classes; Evolutionary
Computation and Cross Validation.

KEYWORDS

Least Squares Support Vector Machine, Evolutionary Computation, Cross Validation, Swarm
Intelligence

For More Details: https://aircconline.com/ijaia/V7N2/7216ijaia03.pdf

Volume Link: https://www.airccse.org/journal/ijaia/current2016.html

 REFERENCES

[1] J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector
Machines. Leuven, Belgium: World Scientific Publishing Co. Pte. Ltd., 2002.
[2] V. N. Vapnik, The Nature of Statistical Learning Theory 2nd ed. New York: Springer-Verlag, 1995.
[3] H.-L. Xie, N. Li, F.-C. Lu, and Q. Xie, "Application of LS-SVM by GA for Dissolved Gas Concentration Forecasting
in Power Transformer Oil," in Proceedings of the Asia-Pacific Power and Energy Engineering Conference
(APPEEC). , 2009, pp. 1-4.
[4] B. T. Jiang and F. Y. Zhao, "Particle Swarm Optimization-based Least Squares Support Vector Regression for
Critical Heat Flux Prediction," Annals of Nuclear Energy vol. 53, pp. 69-81, 2013.
[5] D. C. Montgomery, C. L. Jennings, and M. Kulahci, Itroduction to Time Series Analysis and Forecasting. New
Jersey: John, Wiley &Sons, 2008.
[6] S. Ismail, A. Shabri, and R. Samsudin, "A Hybrid Model of Self-Organizing Maps (SOM) and Least Squares Support
Vector Machines (LSSVM) for Time Series Forecasting," Expert Systems with Applications, vol. 38, pp. 10574-
10578, 2011.
[7] A. Khashman and N. I. Nwulu, "Intelligent Prediction of Crudei Oil Price Using Support Vector Machines," in
Proceedings of the 9th IEEE International Symposium on Applied Machine Intelligenc abd Informatics, Smolenice,
Slovakia, 2011, pp. 165-169.
[8] H. A. Khazem, "Using artificial neural networks to forecast the futures prices of crude oil," D.B.A. 3295968, Nova
Southeastern University, United States -- Florida, 2008.
[9] A. Alizadeh, M. Moghaddam, M. Khakzad, and V. Ebrahimipour, "A Flexible Neural Network-Fuzzy Mathematical
Programming Algorithm for Improvement of Oil Price Estimation and Forecasting," Computers and Industrial
Engineering, vol. 62, pp. 421-430, 2012.
[10] Y. Bao, X. Zhang, L. Yu, K. K. Lai, and S. Wang, "An Intergrated Model Using Wavelet Decomposition and Least
Squares Support Vector Machines for Monthly Crude Oil Prices Forecasting," New Mathematics and Natural
Computation, vol. 7, pp. 299-311, 2011.
[11] M. E. Malliaris and S. G. Malliaris, "Forecasting Inter-Related Energy Product Prices," The European Journal of
Finance, vol. 14, pp. 453-468, 2008.
[12] J. Wang, T. Li, and R. Ren, "A real time IDSs based on artificial Bee Colony-support vector machine algorithm," in
Proceedings of the Third International Workshop on Advanced Computational Intelligence (IWACI), 2010, pp. 91-96.
[13] Haidar and R. C. Wolff, "Forecasting of Crude Oil Price (Revisited)," in Proceedings of the 30th USAEE Conference,
Washington D.C, 2011.
[14] L. Liu and W. Wang, "Exchange Rates Forecasting with Least Squares Support Vector Machine," in Proceedings of
the International Conference on Computer Science and Software Engineering, 2008, pp. 1017-1019.
[15] C.-S. Lin, S.-H. Chiu, and T.-Y. Lin, "Empirical Mode Decomposition-based Least Squares Support Vector
Regression for Foreign Exchange Rate Forecasting," Economic Modelling, vol. 29, pp. 25832590, 2012.
[16] A. Mellit, A. Massi Pavan, and M. Benghanem, "Least Squares Support Vector Machine for ShortTerm Prediction of
Meteorological Time Series," Theor Appl Climatol, vol. 2013, pp. 297-307, 2013.
[17] Wu and D. Niu, "Short-Term Power Load Forecasting Using Least Squares Support Vector Machines(LS-SVM)," in
Proceedings of the Second International Workshop on Computer Science and Engineering (WCSE) 2009, pp. 246-
250.
[18] X. Wang and H. Li, "One-Month Ahead Prediction of Wind Speed and Output Power Based on EMD and LSSVM,"
in Proceedings of the International Conference on Energy and Environment Technology (ICEET) 2009, pp. 439-442.
[19] D.-c. Wang, C.-x. Wang, Y.-H. Xie, and T.-Y. Zhu, "Air Temperature Prediction Based on EMD and LSSVM," in
Proceedings of the Fourth International Conference on Genetic and Evolutionary Computing, 2010, pp. 177-180.
[20] M. T. Gencoglu and M. Uyar, "Prediction of Flashover Voltage of Insulators using Least Squares Support Vector
Machines," Expert Systems with Applications, vol. 36, pp. 10789-10798, 2009.
[21] T. Van Gestel, J. A. K. Suykens, D. E. Baestaens, A. Lambrechts, G. Lanckriet, B. Vandaele, B. De Moor, and J.
Vandewalle, "Financial time series prediction using least squares support vector machines within the evidence
framework," IEEE Transactions on Neural Networks, vol. 12, pp. 809
[22] 821, 2001.
A. S. Ahmad, M. Y. Hassan, and M. S. Majid, "Application of Hybrid GMDH and Least Squares Support Vector
Machines in Energy Consumption Forecasting," in IEEE International Conference on Power and Energy (PECon),
Kota Kinabalu, Sabah, Malaysia, 2012.
[23] Y. Wang and Y. Li, "Applying LS-SVM to Predict Primary Energy Consumption," in Proceedings of the International
Conference on E-Product E-Service and E-Entertainment (ICEEE), 2010, pp. 1-4.
[24] N. Wu, X. Liang-fa, and W. Ji-lin, "The Application of Particle Swarm Optimization-Based RBF Neural Network in
Fault Diagnosis of Power Transformer," in Proceedings of the 2nd IEEE International Conference on Computer
Science and Information Technology (ICCSIT), 2009, pp. 534-536.
[25] A. Lendasse, Y. Ji, N. Reyhani, and M. Verleysen, "LS-SVM Hyperparameter Selection with a Nonparametric Noise
 Estimator," in Proceedings of the 15th International Conference on Artificial Neural Networks: Formal Models and
Their Applications (ICANN) Warsaw, Poland, 2005.
[26] Yu, H. Chen, S. Wang, and K. K. Lai, "Evolving Least Squares Support Vector Machines for Stock Market Trend
Mining," IEEE Transactions on Evolutionary Computation, vol. 13, pp. 87-102, 2009.
[27] D. Karaboga, B. Gorkemli, C. Ozturk, and N. Karaboga, "A comprehensive survey: artificial bee colony (ABC)
algorithm and applications," Artificial Intelligence Review, pp. 1-37, 2012/03/01 2012.
[28] D. Karaboga and B. Akay, "A comparative study of Artificial Bee Colony," Applied Mathematics and Computation,
vol. 214, pp. 108-132, 2009.
[29] Z. Yang, X. S. Gu, X. Y. Liang, and L. C. Ling, "Genetic Algorithm-Least Squares Support Vector Regression based
Predicting and Optimizing Model on Carbon Fiber Composite Integrated Conductivity," Materials and Design, vol.
31, pp. 1042-1049, 2010.
[30] R. L. Haupt and S. E. Haupt, Practical Genetic Algorithms, Second ed. New Jersey: A John Wiley & Sons, Inc.,
Publication, 2004.
[31] W. Mustafa, M. H. Sulaiman, S. N. Abd. Khalid, H. Shareef, S. R. Abdul Rahim, and O. Aliman, "An Application of
Genetic Algorithm and Least Squares Support Vector Machine for Tracing the Transmission Loss in Deregulated
Power System," in Proceedings of the 5th International Power Engineering and Optimization Conference (PEOCO),
Shah Alam, Selangor, 2011.
[32] W. Mustafa, M. H. Sulaiman, H. Shareef, and S. N. Abd. Khalid, "Reactive power tracing in poolbased power system
utilising the hybrid genetic algorithm and least squares support vector machine," IET, Generation, Transmission &
Distribution, vol. 6, pp. 133-141, 2012.
[33] H. Fu, S. Liu, and F. Sun, "Ship Motion Prediction Based on AGA-LSSVM," in Proceedings of the International
Conference on Mechatronics and Automation (ICMA), 2010, pp. 202-206.
[34] W. Sun and J. Zhang, "Forecasting Day Ahead Spot Electricity Prices Based on GASVM," in Proceedings of the
International Conference on Internet Computing in Science and Engineering (ICICSE), 2008, pp. 73-78.
[35] Zou, "Condition Prediction of Hydroturbine Generating Units using Least Squares Support Vector Regression with
Genetic Algorithm," in Proceedings of the Eightth International Conference on Machine Learning and Cybernetics,
Baoding, China, 2009.
[36] X. Wang, H. Zhang, C. Zhang, X. Cai, J. Wang, and J. Wang, "Prediction of Chaotic Time Series using LS-SVM with
Automatic Parameter Selection," in Proceedings of the Sixth International Conference on Parallel and Distributed
Computing, Applications and Technologies (PDCAT), 2005.
[37] W. Liao and Z. Balzen, "LSSVM Network Flow Prediction Based on the Self-adaptive Genetic Algorithm
Optimization," Journal of Networks, vol. 8, pp. 507-512, 2013.
[38] Kennedy and R. Eberhart, "Particle Swarm Optimization," in Proceedings of the IEEE International Conference on
Neural Networks (ICNN), Australia, 1995, pp. pp. 1942-1948.
[39] Q. Chen, Y. Wu, and X. Chen, "Research on Customers Demand Forecasting for E-business Web Site Based on LS-
SVM," in Proceedings of the International Symposium on Electronic Commerce and Security, 2008, pp. 66-70.
[40] Y. Xiang and L. Jiang, "Water Quality Prediction Using LS-SVM and Particle Swarm Optimization," in Proceedings
of the Second International Workshop on Knowledge Discovery and Data Mining (WKDD) 2009, pp. 900-904.
[41] Y. Li, "Short-Term Load Forecasting Based on LS-SVM Optimized by BCC Algorithm," in Proceedings of the 15th
International Conference on Intelligent System Applications to Power Systems (ISAP), 2009, pp. 1-5.
[42] W. Shen, Y. Zhang, and X. Ma, "Stock Return Forecast with LS-SVM and Particle Swarm Optimization," in
Proceedings of the International Conference on Business Intelligence and Financial Engineering, 2009.
[43] R. Liao, H. Zheng, S. Grzybowski, and L. Yang, "Particle Swarm Optimization-Least Squares Support Vector
Regression based Forecasting model on Dissolved Gases in Oil-Filled Power Transformers " Electric Power Systems
Research, vol. 81, pp. 2074-2080, 2011.
[44] Liu and J. Yao, "Application of least square support vector machine based on particle swarm optimization to chaotic
time series prediction," in Proceedings of the IEEE International Conference on Intelligent Computing and Intelligent
Systems (ICIS). , 2009, pp. 458-462.
[45] B. Zhou and A. Shi, "LSSVM and Hybrid Particle Swarm Optimization for Ship Motion Prediction," in Proceedings
of the International Conference on Intelligent Control and Information Processing (ICICIP), 2010, pp. 183-186.
[46] B. Zhu and Y. Wei, "Carbon Price Forecasting with a Novel Hybrid ARIMA and Least Squares Support Vector
Machines Methodology," Omega, vol. 41, pp. 517-524, 2013.
[47] L. X. Li, Z. J. Shao, and J. X. Qian, "An optimizing method based on autonomous animals: fishswarm algorithm,"
System Engineering Theory and Practice, vol. 22, pp. 32-38, 2002.
[48] X. Li, Z. Shao, and J. Qian, "An Optimizing Method base on Autonomous Animates: Fish Swarm Algorithm.,"
Systems Engineering Theory and Practice, vol. 22, pp. 32-38, 2002.
[49] M. Dorigo and T. Stutzle, Ant Colony Optimization. Cambridge: MIT Press 2004.
[50] R. Storn and K. Price, "Differential Evolution- a simple and efficient heuristic for global optimization over continuous
spaces," Journal of Global Optimization, vol. 11, pp. 341-359, 1997.
[51] E. Rashedi, H. Nezamabadi-pour, and S. Saryazdi, "GSA: A Gravitational Search Algorithm," Information Sciences,
vol. 179, pp. 2232-2248, 2009.
[52] Y. Peng, "An Improved Artificial Fish Swarm Algorithm for Optimal Operation of Cascade Reservoirs," Journal of
 Computers, vol. 6, pp. 740-746, 2011.
[53] X. Chen, J. Wang, D. Sun, and J. Liang, "Time Series Forecasting Based on Novel Support Vector Machine Using
Artificial Fish Swarm Algorithm," in Proceedings of the Fourth International Conference on Natural Computation
(ICNC). , 2008, pp. 206-211.
[54] V. Selvi and R. Umarani, "Comparative Analysisi of Ant Colony and Particle Swarm Optimization Techniques,"
International Journal of Computer Applications, vol. 5, pp. 1-5, 2010.
[55] X.-Y. Fang and T. Bai, "Share Price Prediction using Wavelet Transform and Ant Colony Optimization for
Parameters Optimization in SVM," in Proceedings of the Global Congress on Intelligent System, 2009.
[56] G. S. Dos Santos, L. G. J. Luvizotto, V. C. Mariani, and L. Dos Santos Coelho, "Least Squares Support Vector
Machines with tuning based on Differential Evolution Approach Applied to the Identification of a Thermal Process,"
Expert Systems with Applications, vol. 39, pp. 4805-4812, 2012.
[57] V. Feoktistov and S. Janaqi, "Hybridization od Differential Evolution with Least Squares Support Vector Machines,"
in Proceedings of the Annual Machine Learning Conference of Belgium and The Netherlands (BENELEARN),
Belgium, 2004, pp. 53-57.
[58] W. Zhang, P. Niu, G. Li, and P. Li, "Forecasting of Turbine Heat Rate with Online Least Squares Support Vector
Machine based on Gravitational Search Algorithm " Knowledge-Based Systems, vol. 39, pp. 34-44, 2013.
[59] E. Rashedi, H. Nezahabadi-pour, and S. Saryazdi, "GSA: A Gravitational Search Algorithm," Information Sciences,
vol. 179, pp. 2232-2248, 2009.
[60] H. Li, S. Guo, H. Zhao, C. Su, and B. Wang, "Annual Electric Load Forecasting by a Least Squares Support Vector
Machines with a Fruit Fly Optimization Algorithm," Energies, vol. 2012, pp. 44304445, 2012.
[61] W. T. Pan, "A New Fruit Fly Optimization Algorithm: Taking the Financial Distress Model as an Example,"
Knowledge Based System, vol. 2012, pp. 69-74, 2012.
[62] J.-B. Park, Y.-W. Jeong, J.-R. Shin, and K. Y. Lee, "An Improved Particle Swarm Optimization for Nonconvex
Economic Dispatch Problems," IEEE Transactions on Power Systems, vol. 25, pp. 156165, 2010.
[63] D. Karaboga and B. Basturk, "A Powerful and Efficient Algorithm for Numerical Function Optimization:Artificial
Bee Colony (ABC) Algorithm," Journal Global Optimization, vol. 39, pp. 459-471, 2007.
[64] Z. Mustaffa and Y. Yusof, "Optimizing LSSVM using ABC for Non-Volatile Financial Prediction," Australian
Journal of Basic and Applied Sciences vol. 5, pp. 549-556, 2011.
[65] Y. Yusof, S. S. Kamaruddin, H. Husni, Ku-Mahamud, Ku Ruhana, , and Z. Mustaffa, "Enhanced ABC-LSSVM for
Energy Fuel Price Prediction," Journal of Information and Communication Technology., vol. 12, pp. 73-101, 2013.
[66] Z. Mustaffa, Y. Yusof, and S. S. Kamaruddin, "Enhanced Artificial Bee Colony for Training Least Squares Support
Vector Machines in Commodity Price Forecasting," Journal of Computational Science, vol. 5, pp. 196-205, 2014.
[67] Z. Mustaffa and Y. Yusof, "Levy Mutation in Artificial Bee Colony Algorithm for Gasoline Price Prediction," in
Proceedings of the Knowledge Management International Conference (KMICe), Johor Bahru, Johor, Malaysia, 2012.