Robotic Fish Localization and Tracking Using Simultaneous Perturbation-Neural Algorithm

ISSN: 2277-3754 ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 3, September 2013 Robotic Fish Localization and Tracking Using Simultaneous Perturbation-Neural Algorithm Ahmad Taha Abdulsadda, Communication Department, Al Najaf Technical College, Foundation of Technical Education Abstract— Fish and aquatic amphibians use the lateral line system, consisting of arrays of hair-like neuromasts, as an important sensory organ for prey/predator detection, communication, and navigation. In this paper a novel bio-inspired artificial lateral line system is proposed for underwater robots and vehicles by exploiting the inherent sensing capability of ionic polymer-metal composites (IPMCs). Analogous to its biological counterpart, the IPMC-based lateral line processes the sensor signals through a neural network. The effectiveness of the proposed lateral line was validated in localization of underwater motion source such as a flapping foil tail. In particular, as a proof of concept, a prototype with Body Length (BL) of 8 cm, comprising five millimeter-scale IPMC sensors, was constructed and tested. Experimental results showed that the IPMC-based lateral line could localize the sources from 4-5 BLs away, with a localization error comparable to source placement resolution at the source-sensor separation of 1 BL. In addition to the ease of fabrication, these results established the advantages of the proposed approach over other reported artificial lateral lines, in terms of both localization range and accuracy. Index Terms—Ionic polymer-metal composite (IPMC), lateral line system, robotic fish, source localization, neural networks. Fig.1. (a) Illustration of the structure of a neuromast (image credit: C.H. Mallery); (b) distribution superficial (small dots) and canal neuromasts (dots within shaded areas) on the Lake Michigan mottled sculpin [1]. I. INTRODUCTION Most fish and aquatic amphibians use the lateral line system as an important sensory organ to probe their environment [1], [2]. A lateral line consists of arrays of hair cell sensors, known as neuromasts. Each neuromast contains bundles of sensory hairs, encapsulated in a gelatinous structure called cupula, as illustrated in Fig. 1(a). An impinging flow deflects the cupula, and thus the hairs inside, eliciting firing of the hair cell neurons. Neuromasts can be divided into two types, superficial neuromasts, which are distributed on the skin surface, and canal neuromasts, which are recessed in the scales or in bony canals underneath the skin (Fig. 1(b)). With the same basic structure, the two types of neuromasts show distinct sensing characteristics [3]. The lateral line system allows an aquatic animal to identify near-field objects of interest and perform hydrodynamic imaging of the environment, typically within one to two Body Lengths (BLs) of the animal. Consequently, the lateral line is involved in various behaviors of aquatic animals, such as prey/predator detection [4], schooling [5], rheotaxis [6], courtship and communication [2]. In addition to the qualitative roles the lateral line plays in behaviors, there have been studies on how probed information is encoded and decoded in the nervous systems [7]. The biological lateral line system has inspired the effort to engineer artificial lateral lines for applications in underwater vehicles and robots. Providing a stealthy complement to existing sensing modules, such as cameras and sonar’s, artificial lateral lines can potentially provide information on flow conditions, obstacles, and moving objects for underwater robots and vehicles. This in turn can enable stabilization in response to turbulent currents or choppy waves, energy-saving in locomotion, navigation, and collaborative behaviors such as schooling. On the hardware side, arrays of flow sensors, explicitly motivated by the biological lateral line, have been fabricated based on various transduction principles, such as hot wire anemometry [8], piezoresistivity/strain gauge [9], [10], and capacitive sensing [11]. On the signal processing, side, researchers have mainly examined the problem of localizing a vibrating sphere, known as a dipole. Dagamseh et al. proposed the use of characteristic points (zero-crossings, maxima, etc.) in the measured velocity profile for dipole source localization [11], similar to what was proposed by Franosch et al. for modeling the localization by the clawed frog Xenopus [12]. However, this approach would require prohibitively many sensors to determine the characteristic points, and it is limited to a maximum detection 49 ISSN: 2277-3754 ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 3, September 2013 distance of 1/p2 BL. Data-matching approaches were presented by Pandya et al., where the measured signal pattern was compared with a large, pre-obtained set of templates or a model fitted with sufficient amount of data [13]. These approaches suffered from the need for excessive computing and storage resources, or the difficulty in system-level implementation [13]. Recently, a beam forming algorithm for array signal processing was used to localize a dipole source and a flicking fish tail [10]. With a sensor-source separation of 0.5 BL, the resulting mean estimation error is between 0.1 and 0.2 BL. The contribution of this paper is a novel approach to the development of artificial lateral lines based on a novel class of soft smart materials, called ionic polymer-metal composites (IPMCs). We report, to our best knowledge, the first IPMC-based artificial lateral line system, including prototype development, sensor signal processing, and Fig.2. Illustration of IPMC sensing principle. demonstration in the application of robot like fish localization. As a proof of concept, a prototype with Body Length (BL) of Fig. 3 shows the constructed lateral line prototype, 8 cm, comprising five millimeter-scale IPMC sensors, was constructed. In analogy to the neural system for biological consisting of five IPMC sensors. Each sensor, with fish, an artificial neural network was proposed for processing dimensions 8 mm × 2.5 mm × 200 m, was cut from an IPMC the signals from the IPMC sensor array because of its ability sheet fabricated by the Smart Microsystems Laboratory at to model the highly nonlinear relationship between the Michigan State University, following a recipe similar to the stimulus and the resulting flow field/sensor output. The use of one described in [14]. The sensor-to-sensor separation was 2 the proposed artificial lateral line was demonstrated in the cm, resulting in a total span of 8 cm, which will be regarded as localization of a fish tail source. One type of vibrating stimuli the Body Length (BL) in later analysis. Under a mechanical was considered, a flapping foil that emulates a fish tail. stimulus, an open-circuit voltage or a short-circuit current can Experimental results showed that the IPMC-based lateral line be measured across the two electrodes of an IPMC. We have could localize the sources from 4-5 BLs away, with a chosen to take the short-circuit current as the sensor output localization error comparable to source placement resolution because current measurement is less susceptible to noises. Fig. at the source-sensor separation of 1 BL. In addition to the ease 4 shows the schematic of the measurement circuit, which of fabrication, these results established the advantages of the proposed approach over other reported artificial lateral lines, consists of two cascaded operational amplifiers (op-amps). in terms of both localization range and accuracy. The results Since the “−” terminal of Op-amp 1 is virtually the ground, also confirmed the ability of the simultaneous perturbation the two electrodes of IPMC are short-circuited. The sensing neural network-based processing in capturing the highly current generated under this configuration, i(t), is nonlinear and complex relationship between source location proportional to the voltage output v1(t) = R1i(t). The second op-amp is introduced for gain adjustment, where the resistor and the sensor responses, with relatively few parameters. R3 is tunable. The output v2(t) is related to the current signal i(t) via v2(t) = R3R1/R2 i(t). In the circuit we used, R1 = 470 k, II. EXPERIMENTAL SETUP AND SENSOR CHARACTERIZATION R2 = 10 k, and R3 was adjustable from 0 to 50 k. Acquisition and processing of the IPMC sensor output were A. Experimental Setup conducted through a dSPACE system (DS1104, dSPACE, The proposed artificial lateral line uses ionic polymer-metal Germany). In particular, a digital low-pass filter was further composite (IPMC) material as the sensing elements. As implemented to remove high-frequency noises. illustrated in Fig. 2, an IPMC consists of three layers, with an ion ion-exchange polymer membrane (e.g., Nafion) sandwiched by metal electrodes. Inside the polymer, (negatively charged) anions covalently fixed to polymer chains are balanced by mobile, (positively charged) cations. An applied mechanical stimulus redistributes the cations inside an IPMC, producing a detectable electrical signal (typically open-circuit voltage or short-circuit current) that is correlated with the mechanical stimulus (Fig. 2), which explains the sensing principle of IPMCs. Fig.3. The IPMC-based lateral line prototype 50 ISSN: 2277-3754 ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 3, September 2013 Fig.4. Circuit for measuring the short-circuit current generated by the IPMC sensor. Fig.6. A typical IPMC sensor signal, showing clear periodicity of the source All experiments were conducted in a water tank measured 6×2×2 ft3. The source of stimulus used in the experiments was a flapping foil, driven with a servo motor and attached as the tail of a robotic fish, as shown in Fig. 5. The frequency and amplitude of the stimulating source can be controlled easily. The frequency range of the flapping tail spanned 0.5–5 Hz. The source location and vibration direction with respect to the IPMC lateral line could be adjusted by moving the stand holding the IPMC lateral line or by moving the source itself. B. Sensor Characterization Fig 6 shows a typical sensor response under the flapping tail stimulation (3 Hz), indicating that the current output from the IPMC sensor was at the order of nA, which could be captured very well by the sensing circuit. For a given IPMC sensor, we also placed the dipole and flapping tail at different locations in the tank and measured the corresponding amplitudes of sensor output. Fig 7 shows the amplitude of measured sensor output as a function of the stimulus location, for flapping tail stimuli. It can be seen that, while in general the signal gets stronger when the source gets closer, the overall amplitude landscapes have sophisticated profiles. The results in Fig. 7 clearly illustrate the challenge in underwater localization. In particular, it would be difficult to localize unambiguously a source with a single sensor; instead, a lateral line-like array structure will be needed. Fig.7. Measured sensor signal amplitude as a function of source location: flapping tail stimulation. The signal was from a single IPMC sensor, the location of which was marked in the figures III. SOURCE LOCALIZATION USING SIMULTANEOUS PERTURBATION NEURAL NETWORK The signals from IPMC sensors of the artificial lateral line are complex functions of the source location, because of nonideal fluid conditions and interactions of fluid with structures (e.g., the IPMC beams and the walls of the tank). In addition, the sensing characteristics of individual sensors could be different from each other in practice because of imperfect fabrication processes and variations in dimensions. The sensor outputs are further contaminated with noises due to ambient water movement and thermal fluctuations [15]. As a result, it is difficult to decode the sensor signals analytically. Biological fish are faced with similar challenges in extracting relevant sensing information from vast amount of data that are corrupted by noises. However, they manage to accomplish source localization and other missions robustly through neural network-based information processing. Taking this biological inspiration, we constructed an (artificial) neural Fig.5 Stimuli sources used for the lateral line sensor: a flapping foil. 51 ISSN: 2277-3754 ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 3, September 2013 network to process the signals acquired by the IPMC-based weights optimization. Let K be the total number of weights. lateral line. As illustrated in Fig. 8, we adopted the multilayer For each weight wk, 1 ≤ k ≤ K, the update rule is perceptron (MLP) architecture for the neural network. An MLP network consists of an input layer, a hidden layer, and an J new (1) wk  wkold   knew old , output layer, and is the most widely used network structure for wk nonlinear classification and prediction applications [16]. One could use different features extracted from the sensor output data as the input to the neural network. In this work, we used where the adaptive learning rate k is updated as the signal amplitude at the stimulus frequency because of its J  old robustness to measurement noises. The amplitude was  k  a, if w old  0 obtained through fast Fourier transform (FFT). The number of  new  k (2) ,  k J inputs was the same as the number of IPMC sensors old   , if  0 k considered. For comparison purposes, in this work we wkold  investigated the performance of the artificial lateral line when different numbers of sensors were adopted. The number of the and 0 < a, b < 1 are constants. hidden layer nodes was chosen through a simultaneous IV. EXPERIMENTAL RESULTS perturbation algorithm (SP)-based optimization process, which will be further described below. We performed localization experiments for the flapping tail stimulus. As shown in Fig. 9, the working area was extended to about 40 × 100 cm2 and a total of 60 training points was used. Validation was performed at 37 points along four tracks. The tail was flapping at 3 Hz. Fig. 10 shows more detailed validation results. The prediction error was roughly twice that of the dipole case [18], [19], which could be explained by the larger working area, fewer training points, and more sophisticated hydrodynamics created by the tail than that by the dipole. Fig.8. Schematic of the MLP neural network for signal processing of the IPMC lateral line Each hidden layer node represents the operation of nonlinear activation, which takes the form of a sigmoid function. The output layer has two nodes, representing the x and y coordinates of the vibrating source. The number of hidden-layer nodes and the connective weights between the layers were determined through a two phase training procedure, [17]. The training data were obtained by placing the stimulus at known locations (xi, yi), 1 ≤ i ≤M, measuring the corresponding sensor outputs and computing the signal amplitudes. Here M is the number of training data. In the first training phase, a simultaneous perturbation was used to find an appropriate value for the number of hidden layer nodes and a reasonable set of values for the connective weights. In particular, each genome encoded both the number of hidden-layer nodes and the weights of all connecting edges. The maximum number of hidden-layer nodes was limited to 8 based on the numbers of network inputs and outputs. The values of the connective weights obtained in the first training phase then served as the initial condition for weights refinement in the second phase, where the network structure was fixed as determined in the first phase. Delta-bar-delta learning [16], with adaptive learning rate, was used for Fig.9. Localization of the flapping tail source using the lateral line: (a) One IPMC sensor as the input to the neural network; (b) three IPMC sensors as the input to the neural network 52 ISSN: 2277-3754 ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 3, September 2013 lateral line on a swimming robotic fish, such as the one V. CONCLUSION The contribution of this paper was a new approach to the reported in [20], and investigate the processing schemes for realization of artificial lateral lines for underwater robots and the lateral line to decouple external signals from vehicles, including both the proposal of using IPMC materials self-motion-induced flow signals. as sensing elements and the neural network-based signal processing algorithm. The effectiveness of the proposed approach was validated in experiments involving localization of a flapping tail. Experimental results showed that, with relatively few sensor elements, the IPMC-based lateral line was able to localize robot like fish at least 4-5 BLs away, and the localization accuracy at source-sensor separation of 1 BL was comparable to the resolution of placing the source. The approach thus demonstrated advantages in both the range and precision of localization over other reported methods for realizing artificial lateral lines. ACKNOWLEDGMENT The authors would like to thank Prof. Xiabo Tan and Hong Lei for assisting in the revision of this paper, and preparation of IPMC material. REFERENCES [1] S. Coombs, “Smart skins: Information processing by lateral line flow sensors,” Autonomous Robots, vol. 11, pp. 255–261, 2001. [2] S. Coombs and C. B. Braun, “Information processing by the lateral line system,” in Sensory Processing in Aquatic Environments, S. P. Collin and N. J. Marshall, Eds. New York: Springer-Verlag, 2003, pp. 122–138. [3] A. B. A. Kroese and N. A. M. Schellart, “Velocity- and acceleration sensitive units in the trunk lateral line of the trout,” Journal of Neurophysiology, vol. 68, no. 6, pp. 2212–2221, 1992. [4] K. Pohlmann, J. Atema, and T. Breithaupt, “The importance of the lateral line in nocturnal predation of piscivorous catfish,” Journal of Experimental Biology, vol. 207, pp. 2971–2978, 2004. [5] T. Pitcher, B. L. Partridge, and C. S. Wardle, “A blind fish can school,” Science, vol. 194, pp. 963–965, 1976. [6] J. M. Gardiner and J. Atema, “Sharks need the lateral line to locate odor sources: Rheotaxis and eddy chemo taxis,” Journal of Experimental Biology, vol. 210, pp. 1925–1934, 2007. [7] B. Curcic-Blake and S. M. van Netten, “Source location encoding in the fish lateral line canal,” Journal of Experimental Biology, vol. 209, pp. 1548–1559, 2006. [8] Y. Yang, J. Chen, J. Engel, S. Pandya, N. Chen, C. Tucker, S. Coombs, D. L. Jones, and C. Liu, “Distant touch hydrodynamic imaging with an artificial lateral line,” Proceedings of the National Academy of Sciences, vol. 103, no. 50, pp. 18 891–18 895, 2006. [9] V. I. Fernandez, S. M. Hou, F. S. Hover, J. H. Lang, and M. Triantafyllou, “Development and application of distributed MEMS pressure sensor array for AUV object avoidance,” in Proceedings of the Unmanned Untethered Submersible Technology Symposium, Durham, NH, 2009. [10] Y. Yang, N. Nguyen, N. Chen, M. Lockwood, C. Tucker, H. Hu, H. Bleckmann, C. Liu, and D. L. Jones2, “Artificial lateral line with biomimetic neuromasts to emulate fish sensing,” Bioinspiration and Biomimetics, vol. 5, pp. 016 001:1–016 001:9, 2010. Fig.10. Error in localization of the flapping tail source using the lateral line: (a) Maximum error along the track; (b) average error along the track [11] A. Dagamseh, T. Lammerink, M. Kolster, C. Bruinink, R.Wiegerink, and G. Krijnen, “Dipole-source localization using biomimetic flow-sensor arrays positioned as lateral-line system,” Sensors and Actuators A, 2010, in press. For future work, the proposed approach will be extended in several directions. First, we will explore the additional phase information in the sensor signals to further improve the localization performance. Extension will also be made to track moving robotic fish, where the received signals may not be at their steady state and insufficient data are available for FFT calculation. Finally, we will mount the IPMC-based [12] J.-M. P. Franosch, A. B. Sichert, M. D. Suttner, and J. L. van Hemmen, “Estimating position and velocity of a submerged moving object by the clawed frog xenopus and by fishłA 53 ISSN: 2277-3754 ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 3, September 2013 cybernetic approach,” Biol. Cybern., vol. 93, pp. 231–238, 2005. [13] S. Pandya, Y. Yang, D. L. Jones, J. Engel, and C. Liu, “Multisensor processing algorithms for underwater dipole localization and tracking using MEMS artificial lateral-line sensors,” EURASIP Journal on Applied Signal Processing, vol. 2006, 2006, Article ID 76593, 8 pages, 2006. doi:10.1155/ASP/2006/76593. [14] K. J. Kim and M. Shahinpoor, “Ionic polymer-metal composites: II. manufacturing techniques,” Smart Materials and Structures, vol. 12, pp. 65–79, 2003. [15] T. Ganley, D. L. Hung, G. Zhu, and X. Tan, “Modeling and inverse compensation of temperature-dependent ionic polymer-metal composite sensor dynamics,” IEEE/ASME Transactions on Mechatronics, 2010, in press. [16] S. Samarasinghe, Neural Networks for Applied Sciences and Engineering: From Fundamentals to Complex Pattern Recognition. Boca Raton, FL: Auerbach, 2007. [17] NeuroSolutions, http://www.neurosolutions.com/. [18] A. T. Abdulsadda and X. Tan, “Underwater source localization using an IPMC-based artificial lateral line,” in Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, 2011, pp. 447–452. [19] A. T. Abdulsadda, F. Zhang, and X. Tan, “Localization of source with unknown amplitude using IPMC sensor arrays,” in Proceedings of Electro active Polymer Actuators and Devices (EAPAD) XIII, ser. Proceedings of SPIE, Y. Bar-Cohen, Ed., vol. 7976, San Diego, CA, 2011, p. 797627. [20] Z. Chen, S. Shatara, and X. Tan, “Modeling of biomimetic robotic fish propelled by an ionic polymer-metal composite caudal fin,” IEEE/ASME Transactions on Mechatronics, vol. 15, no. 3, pp. 448–459, 2010. AUTHOR’S PROFILE Ahmad T Abdulsadda received his B. Sc. Degrees in electrical engineering from the Tickrit University, Iraq, in 1997, and the M. Sc. degree in electrical engineering from Baghdad University, Iraq in 2000. In 2000-2006, he was a faculty member at Baghdad University, Iraq and since 2006 at Technical Najaf college, Iraq. He received his Ph.D. degree in Department of Electrical and Computer Engineering at Michigan State University, Michigan USA. Currently, he is working in Department of Communication at Al Najaf Technical College, Foundation of Technical Education. He has published about 25 refereed journal and conference papers. His research interest covers robotics fish, feedback control systems, on-linear estimation techniques, and control theory. Ahmad received scholarship award from ministry of higher education in Iraq to get Ph.D. from USA. He is a student member IEEE. E-mail: [email protected]. 54