Hostname: page-component-5d84bcc8dc-9p9wk Total loading time: 0 Render date: 2026-08-21T13:01:19.062Z Has data issue: false hasContentIssue false
  • English
  • Français

Three-Dimensional Reconstruction Based on Visual SLAM of Mobile Robot in Search and Rescue Disaster Scenarios

Published online by Cambridge University Press:  21 May 2019

Hongling Wang ,
Chengjin Zhang ,
Yong Song ,
Bao Pang  and
Guangyuan Zhang
Hongling Wang
Affiliation:
School of Control Science and Engineering, Shandong University, Ji’nan 250101, China E-mail: pang_bao11@163.com School of Information Science and Electronic Engineering, Shandong Jiao Tong University, Ji’nan 250357, China. E-mails: wanghongling@sdjtu.edu.cn, zhangguangyuan@sdjtu.edu.cn
Chengjin Zhang*
Affiliation:
School of Mechanical, Electrical and Information Engineering, Shandong University at Weihai, Weihai 264209, China
Yong Song*
Affiliation:
School of Mechanical, Electrical and Information Engineering, Shandong University at Weihai, Weihai 264209, China
Bao Pang
Affiliation:
School of Control Science and Engineering, Shandong University, Ji’nan 250101, China E-mail: pang_bao11@163.com
Guangyuan Zhang
Affiliation:
School of Information Science and Electronic Engineering, Shandong Jiao Tong University, Ji’nan 250357, China. E-mails: wanghongling@sdjtu.edu.cn, zhangguangyuan@sdjtu.edu.cn
*
*Corresponding authors. E-mails: cjzhang@sdu.edu.cn, songyong@sdu.edu.cn
*Corresponding authors. E-mails: cjzhang@sdu.edu.cn, songyong@sdu.edu.cn
Get access Rights & Permissions [Opens in a new window]

Summary

Conventional simultaneous localization and mapping (SLAM) has concentrated on two-dimensional (2D) map building. To adapt it to urgent search and rescue (SAR) environments, it is necessary to combine the fast and simple global 2D SLAM and three-dimensional (3D) objects of interest (OOIs) local sub-maps. The main novelty of the present work is a method for 3D OOI reconstruction based on a 2D map, thereby retaining the fast performances of the latter. A theory is established that is adapted to a SAR environment, including the object identification, exploration area coverage (AC), and loop closure detection of revisited spots. Proposed for the first is image optical flow calculation with a 2D/3D fusion method and RGB-D (red, green, blue + depth) transformation based on Joblove–Greenberg mathematics and OpenCV processing. The mathematical theories of optical flow calculation and wavelet transformation are used for the first time to solve the robotic SAR SLAM problem. The present contributions indicate two aspects: (i) mobile robots depend on planar distance estimation to build 2D maps quickly and to provide SAR exploration AC; (ii) 3D OOIs are reconstructed using the proposed innovative methods of RGB-D iterative closest points (RGB-ICPs) and 2D/3D principle of wavelet transformation. Different mobile robots are used to conduct indoor and outdoor SAR SLAM. Both the SLAM and the SAR OOIs detection are implemented by simulations and ground-truth experiments, which provide strong evidence for the proposed 2D/3D reconstruction SAR SLAM approaches adapted to post-disaster environments.

Keywords

SLAM3D reconstructionRGB-D visual systemSAR environmentsRobot mapping

Information

Type
Articles
Information
Robotica , Volume 38 , Issue 2 , February 2020 , pp. 350 - 373
Copyright
© Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Lin, G. Y. and Wang, Y. T., “Improvement of speeded-up robust features for robot visual simultaneous localization and mapping,” Robotica 32(2), 533–549 (2014).Google Scholar
Martins, H., Oakley, I. and Ventura, R., “Design and evaluation of a head-mounted display for immersive 3D teleoperation of field robots,” Robotica 33(10), 2166–2185 (2015).CrossRefGoogle Scholar
Murphy, R. R., “Trial by fire,” IEEE Rob. Autom. Mag. 11(9), 50–61 (2004).CrossRefGoogle Scholar
Yokokohji, Y., Kurisu, M., Takao, S., et al., “Constructing a 3D Map of Rubble by Teleoperated Mobile Robots with a Motion Canceling Camera System,” Proceedings of the 2003 IEEE/RSJ, International Conference on Intelligent Robots and Systems, Las Vegas, Nevada (2003) pp. 3118–3125.Google Scholar
Murphy, R. R., Kravitz, J., Stover, S. L. and Shoureshi, R., “Mobile robots in mine rescue and recovery,” IEEE Rob. Autom. Mag. 9(6), 91–103 (2009).CrossRefGoogle Scholar
Santos, J. M., Portugal, D. and Rocha, R. P., “An Evaluation of 2D SLAM Techniques Available in Robot Operating System,” Fundaçào para a Ciência e a Tecnologia, The Portuguese Science Agency (2013).CrossRefGoogle Scholar
Alboul, L. and Chliveros, G., “A System for Reconstruction from Point Clouds in 3D: Simplification and Mesh Representation,” 2010 11th International Conference on Control, Automation, Robotics and Vision, Singapore (2010) pp. 2301–2306.Google Scholar
Henry, P., Krainin, M., Herbst, E., Ren, X. and Fox, D., “RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments,” Int. J. Rob. Res. 31(5), 647–663 (2012).CrossRefGoogle Scholar
Weiss, S., Scaramuzza, D. and Seigwart, R., “Monocular-SLAM-based navigation for autonomous micro helicopters in GPS-denied environments,” J. Field Rob. 28(6), 854–874 (2011).CrossRefGoogle Scholar
Nagatani, K., Ishida, H., Yamanaka, S. and Tanaka, Y., “Three-Dimensional Localization and Mapping for Mobile Robot in Disaster Environments,” Proceedings of the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, Nevada (2003) pp. 3112–3117.Google Scholar
McGill, M., Selleh, R., Wiley, T., et al., “Virtual Reconstruction using an Autonomous Robot,” 2012 International Conference on Indoor Positioning and Indoor Navigation, Sydney, Australia (2012).CrossRefGoogle Scholar
Sonka, M., Hlavac, V. and Boyle, R., Image Processing, Analysis and Machine Vision, 3rd. ed. (Thomson Corporation, Toronto, 2008). http://www.thomsonlearning.com.Google Scholar
Miró, J. V., Zhou, W. Z. and Dissanayake, G., “A Strategy for Efficient Observation Pruning in Multiobjective 3D SLAM,” IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA (2011) pp. 1640–1646.Google Scholar
Riaz, Z., Linder, T., Behnke, S., Worst, R. and Surmann, H., “Efficient Transmission and Rendering of RGB-D Views,” Advances in Visual Computing, In: ISVC, Part I, LNSC (Bebis, G. et al. eds.), vol. 8033 Springer Berlin Heidelberg (2013) pp. 517–26.CrossRefGoogle Scholar
Fazli, S. and Kleeman, L., “Simultaneous landmark classification, localization and map building for an advanced sonar ring,” Robotica 25(3), 283–296 (2007).CrossRefGoogle Scholar
Veth, M. J., Martin, R. K. and Pachter, M., “Anti-temporal-aliasing constraints for image-based feature tracking applications with and without inertial aiding,” IEEE Trans. Veh. Technol. 59(8), 3744–3756 (2010).CrossRefGoogle Scholar
Wikipedia, the free encyclopedia, “HSL and HSV,” (2015). http://en.wikipedia.org/wiki/HSL_and_HSV.Google Scholar
Zhou, W. Z., Miró, J. V. and Dissanayake, G., “Information-efficient 3-D visual SLAM for unstructured domains,” IEEE Trans. Rob. 24(5), 1078–1087 (2008).CrossRefGoogle Scholar
Nüchter, A., Lingemann, K., Hertzberg, J. and Surmann, H., “6D SLAM - 3D mapping outdoor environments,” Fraunhofer Institute for Autonomous Intelligent Systems (AIS) Schloss Birlinghoven D-53754, Sankt Augustin, Germany (2007).Google Scholar
Nejat, G. and Zhang, Z., “Finding disaster victims: robot-assisted 3D mapping of urban search and rescue environments via landmark identification,” IEEE ICARCV. 1(6), 50–61 (2006).Google Scholar
Zhang, Z., Guo, H., Nejat, G. and Huang, P., “Finding Disaster Victims: A Sensory System for Robot- Assisted 3D Mapping,” IEEE International Conference on Robotics and Automation, Rome, Italy (2007) pp. 3889–3894.Google Scholar
Zhang, Z. and Nejat, G., “Robot-Assisted Intelligent 3D Mapping of Unknown Cluttered Search and Rescue Environments,” IEEE/RSJ International Conference on Intelligent Robots and System, Acropolis Convention Center, Nice, France (2008) pp. 2115–2120.Google Scholar
Liu, M., Colas, F., Oth, L. and Siegwart, R., “Incremental topological segmentation for semi-structured environments using discretized GVG,” Auton. Rob. 38(2), 143–160 (2015).CrossRefGoogle Scholar
Blum, R. S. and Liu, Z., Multi-Sensor Image Fusion and Its Applications (CRC Press, Taylor & Francis Group, Oxford, 2006).Google Scholar
Schleicher, D., Bergasa, L.M.,Ocaña, M., Barea, R. and López, M. E., “Real-time hierarchical outdoor SLAM based on stereovision and GPS fusion,” IEEE Trans. Intell. Transp. Syst. 10(3), 50–61 (2009).CrossRefGoogle Scholar
Bloch, I., Information Fusion in Signal and Image Processing (John Wiley & Sons, Inc., Hoboken, NJ, USA, 2008).CrossRefGoogle Scholar
Savkin, A. V. and Hoy, M., “Reactive and the shortest path navigation of a wheeled mobile robot in cluttered environments,” Robotica 31(2), 323–330 (2013).CrossRefGoogle Scholar
Aghili, F., “3D simultaneous localization and mapping using IMU and its observability analysis,” Robotica 29(10), 805–814 (2011).CrossRefGoogle Scholar
Deißler, T. and Thielecke, J., “UWB SLAM with Rao-Blackwellized Monte Carlo Data Association,” International Conference on Indoor Navigation (IPIN), Zürich, Switzerland (2010).CrossRefGoogle Scholar
Happold, M. and Ollis, M., “Using learned features from 3D data for robot navigation,” Stud. Comput. Intell. (SCI) 76, 61–69, Applied Perception, Inc., Cranberry Township, Pennsylvania (2007).Google Scholar
Surmann, H., Nüchter, A. and Hertzberg, J., “An autonomous mobile robot with a 3D laser range finder for 3D exploration and digitalization of indoor environments,” Rob. Auton. Syst. 45(9), 181–198 (2003).CrossRefGoogle Scholar
Bêdkowski, J., Majek, K., Musialik, P., Adamek, A., Andrzejewski, D. and Czekaj, D., “Towards terrestrial 3D data registration improved by parallel programming and evaluated with geodetic precision,” Autom. Constr. 47(8), 78–91 (2014).CrossRefGoogle Scholar
Ohno, K., Nomura, T. and Tadokoro, S., “Real-Time Robot Trajectory Estimation and 3D Map Construction Using 3D Camera,” Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China (2006) pp. 5279–5285.Google Scholar
Bücher, T., Curio, C., Edelbrunner, J., Igel, C., Kastrup, D., Leefken, I., Lorenz, G., Steinhage, A. and von Seelen, W., “Image processing and behavior planning for intelligent vehicles,” IEEE Trans. Ind. Electron. 50(1), 62–75 (2003).CrossRefGoogle Scholar
Ellekilde, L. P., Huang, S. D., Miró, J. V. and Dissanayake, G., “Dense 3D map construction for indoor search and rescue,” J. Field Rob. 24(1–2), 71–89 (2007).CrossRefGoogle Scholar
Jesus, F. and Ventura, R., “Combining monocular and stereo vision in 6D-SLAM for the localization of a tracked wheel robot,” IEEE Inst. Syst. Rob. 12(4), 50–61 (2012).Google Scholar
Wong, R. H., Xiao, J. Z. and Joseph, S. L., “An Adaptive Data Association for Robotic SLAM in Search and Rescue Operation,” Proceedings of the 2011 IEEE International Conference on Mechatronics and Automation, Beijing, China (2011) pp. 997–1003.Google Scholar
Weingarten, J. and Siegwar, R., “3D SLAM Using Planar Segments,” Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China (2006) pp. 3062–3067.Google Scholar
Carbone, A., Ciacelli, D., Finzi, A. and Pirri, F., “Autonomous Attentive Exploration in Search and Rescue Scenarios,” In: WAPCV (Paletta, L. and Rome, E., eds.) (2007) pp. 431–446. DOI: 10.1007/978-3-540-77343-6_28.Google Scholar
Schleicher, D., Bergasa, L. M., Ocaña, M., Barea, R. and López, E., “Real-time hierarchical stereo visual SLAM in large-scale environments,” Rob. Auton. Syst. 58(8), 991–1002 (2010).CrossRefGoogle Scholar
Zhou, W. Z., Miró, J. V. and Dissanayake, G., “Information-Driven 6D SLAM Based on Ranging Vision,” IEEE/RSJ International Conference on Intelligent Robots and Systems, Acropolis Convention Center, Nice, France (2008) pp. 2072–2077.Google Scholar
Stronger, D. and Stone, P., “Selective Visual Attention for Object Detection on a Legged Robot,” In: RoboCup, LANI (Lakemeyer et al. eds.), vol. 4434 Springer-Verlag (2007) pp. 158–170.Google Scholar
Mihankhah, E., Taghirad, H. D., Kalantari, A., Aboosaeedan, E. and Semsarilar, H., “Line Matching Localization and Map Building with Least Square,” IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Suntec Convention and Exhibition Center, Singapore (2009) pp. 1734–1739.Google Scholar
Kim, A. and Eustice, R. M., “Perception-Driven Navigation: Active Visual SLAM for Robotic Area Coverage,” IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany (2013) pp. 3196–3203.Google Scholar
Valavanis, K. P., Doitsidis, L., Long, M. and Murphy, R. R., “A case study of fuzzy-logic-based robot navigation,” IEEE Rob. Autom. Mag. 6(9), 93–107 (2006).CrossRefGoogle Scholar
Day, B., Bethel, C., Murphy, R. and Burke, J., “A Depth Sensing Display for Bomb Disposal Robots,” Proceedings of the 2008 IEEE International Workshop on Safety, Security and Rescue Robotics, Japan (2008) pp. 146–151.Google Scholar
Zhang, Z. and Nejat, G., “Intelligent sensing systems for rescue robots: landmark identification and threedimensional mapping of unknown cluttered urban search and rescue environments,” Adv. Rob. 23(11), 1159–1177 (2009).CrossRefGoogle Scholar
Yamamoto, Y., Pirjanian, P., Munich, M., DiBernardo, E., Goncalves, L., Ostrowski, J. and Karlsson, N., “Optical Sensing for Robot Perception and Localization,” IEEE Workshop on Advanced Robotics and its Social Impacts, Nagoya, Japan (2005) pp. 14–17.Google Scholar
Fujiwara, T., Kamegawa, T. and Gofuku, A., “Stereoscopic Presentation of 3D Scan Data Obtained by Mobile Robot,” Proceedings of the 2011 IEEE International Symposium on Safety, Security and Rescue Robotics, Kyoto, Japan (2011) pp. 178–183.Google Scholar
Knuth, J. and Barooah, P., “Distributed collaborative 3D pose estimation of robots from heterogeneous relative measurements: an optimization on manifold approach,” Robotica 33(7), 1507–1535 (2014).CrossRefGoogle Scholar
Pire, T., Baravalle, R., D’Alessandro, A. and Civera, J., “Real-time dense map fusion for stereo SLAM,” Robotica 36(10), 1510–1526 (2018). doi:10.1017/S0263574718000528 CrossRefGoogle Scholar
Saputra, M. R. U., Markham, A. and Trigoni, N.. “Visual SLAM and structure from motion in dynamic environments: a survey,” ACM Comput. Surv. 51(2), 1–37 (2018).CrossRefGoogle Scholar