Abstract
This paper describes a brain-inspired simultaneous localization and mapping (SLAM) system using oriented features from accelerated segment test and rotated binary robust independent elementary (ORB) features of RGB (red, green, blue) sensor for a mobile robot. The core SLAM system, dubbed RatSLAM, can construct a cognitive map using information of raw odometry and visual scenes in the path traveled. Different from existing RatSLAM system which only uses a simple vector to represent features of visual image, in this paper, we employ an efficient and very fast descriptor method, called ORB, to extract features from RGB images. Experiments show that these features are suitable to recognize the sequences of familiar visual scenes. Thus, while loop closure errors are detected, the descriptive features will help to modify the pose estimation by driving loop closure and localization in a map correction algorithm. Efficiency and robustness of our method are also demonstrated by comparing with different visual processing algorithms.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
E. C. Tolman. Cognitive maps in rats and men. Psychological Review, vol. 55, no. 4, pp. 189–208, 1948.
J. O’Keefe, J. Dostrovsky. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Research, vol. 34, no. 1, pp. 171–175, 1971.
M. Fyhn, S. Molden, M. P. Witter, E. I. Moser, M. B. Moser. Spatial representation in the entorhinal cortex. Science, vol. 305, no. 5688, pp. 1258–1264, 2004.
T. Hafting, M. Fyhn, S. Molden, M. B. Moser, E. I. Moser. Microstructure of a spatial map in the entorhinal cortex. Nature, vol. 436, no. 7052, pp. 801–806, 2005.
B. L. McNaughton, F. P. Battaglia, O. Jensen, E. I. Moser, M. B. Moser. Path integration and the neural basis of the cognitive map. Nature Reviews Neuroscience, vo.7, no. 8, pp. 663–678, 2006.
M. C. Fuhs, D. S. Touretzky. A spin glass model of path integration in rat medial entorhinal cortex. Journal of Neuroscience, vol. 26, no. 16, pp. 4266–4276, 2006.
P. Lavenex, D. G. Amaral. Hippocampal-neocortical interaction: A hierarchy of associativity. Hippocampus, vol. 10, pp. 420–430, 2000.
F. Savelli, J. J. Knierim. Hebbian analysis of the transformation of medial entorhinal grid-cell inputs to hippocampal place fields. Journal of Neurophysiology, vol. 103, no. 6, pp. 3167–3183, 2010.
T. Solstad, E. I. Moser, G. T. Einevoll. From grid cells to place cells: A mathematical model. Hippocampus, vol. 16, no. 12, pp. 1026–1031, 2006.
J. Hu, H. J. Tang, K. C. Tan, H. Z. Li. How the brain formulates memory: A spatio-temporal model research frontier. IEEE Computational Intelligence Magazine, vol. 11, no. 2, pp. 56–68, 2016.
E. T. Rolls, S. M. Stringer, T. Elliot. Entorhinal cortex grid cells can map to hippocampal place cells by competitive learning. Network: Computation in Neural Systems, vol. 17, no. 4, pp. 447–465, 2006.
E. I. Moser, E. Kropff, M. B. Moser. Place cells, grid cells, and the brain’s spatial representation system. Annual Review of Neuroscience, vol. 31, pp. 69–89, 2008.
M. L. Yuan, B. Tian, V. A. Shim, H. J. Tang, H. Z. Li. An entorhinal-hippocampal model for simultaneous cognitive map building. In Proceedings of the 29th AAAI Conference on Artificial Intelligence, AAAI, Austin, USA, pp. 586–592, 2015.
N. Cuperlier, M. Quoy, P. Gaussier. Neurobiologically inspired mobile robot navigation and planning. Frontiers in Neurorobotics, vol. 1, pp. 3, 2007.
A. Jauffret, N. Cuperlier, P. Gaussier, P. Tarroux. Multimodal integration of visual place cells and grid cells for navigation tasks of a real robot. In Proceedings of the 12th International Conference on Simulation of Adaptive Behavior, Springer, Odense, Denmark, pp. 136–145, 2012.
W. W. Huang, H. J. Tang, B. Tian. Vision enhanced neurocognitive structure for robotic spatial cognition. Neurocomputing, vol. 129, pp. 49–58, 2014.
W. Hess, D. Kohler, H. Rapp, D. Andor. Real-time loop closure in 2D LIDAR SLAM. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Stockholm, Sweden, pp. 1271–1278, 2016.
G. Grisetti, C. Stachniss, W. Burgard. Improved techniques for grid mapping with rao-blackwellized particle filters. IEEE Transactions on Robotics, vol. 23, no. 1, pp. 34–46, 2007.
M. J. Milford, G. F. Wyeth. Mapping a suburb with a single camera using a biologically inspired SLAM system. IEEE Transactions on Robotics, vol. 24, no. 5, pp. 1038–1053, 2008.
B. Tian, V. A. Shim, M. L. Yuan, C. Srinivasan, H. J. Tang, H. Z. Li. RGB-D based cognitive map building and navigation. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Tokyo, Japan, pp. 1562–1567, 2013.
V. A. Shim, B. Tian, M. L. Yuan, H. J. Tang, H. Z. Li. Direction-driven navigation using cognitive map for mobile robots. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Chicago, USA, pp. 2639–2646, 2014.
J. Engel, T. Schöps, D. Cremers. LSD-SLAM: Large-scale direct monocular SLAM. In Proceedings of the 13th European Conference on Computer Vision, Springer, Zurich, Switzerland, pp. 834–849, 2014.
F. Endres, J. Hess, N. Engelhard, J. Sturm, D. Cremers, W. Burgard. An evaluation of the RGB-D SLAM system. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Saint Paul, USA, pp. 1691–1696, 2012.
K. Konolige, M. Agrawal. FrameSLAM: From bundle adjustment to real-time visual mapping. IEEE Transactions on Robotics, vol. 24, no. 5, pp. 1066–1077, 2008.
D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.
H. Bay, T. Tuytelaars, L. Van Gool. SURF: Speeded up robust features. In Proceedings of the 9th European Conference on Computer Vision, Springer, Graz, Austria, pp. 404–417, 2006.
S. N. Sinha, J. M. Frahm, M. Pollefeys, Y. Genc. GPUbased video feature tracking and matching. In Proceeding of Workshop on Edge Computing Using New Commodity Architectures, Chapel Hill, USA, 2006.
E. Rosten, T. Drummond. Machine learning for highspeed corner detection. In Proceedings of the 9th European Conference on Computer Vision, Springer, Graz, Austria, pp. 430–443, 2006.
E. Rosten, R. Porter, T. Drummond. Faster and better: A machine learning approach to corner detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 105–119, 2010.
M. Calonder, V. Lepetit, C. Strecha, P. Fua. Brief: Binary robust independent elementary features. In Proceedings of the 11th European Conference on Computer Vision, Springer, Crete, Greece, pp. 778–792, 2010.
S. K. Vipparthi, S. K. Nagar. Local extreme complete trio pattern for multimedia image retrieval system. International Journal of Automation and Computing, vol. 13, no. 5, pp. 457–467, 2016.
Z. H. Wang, Q. F. Song, H. M. Liu, Z. Q. Huo. Absence importance and its application to feature detection and matching. International Journal of Automation and Computing, vol. 13, no. 5, pp. 480–490, 2016.
G. Klein, D. Murray. Parallel tracking and mapping for small AR workspaces. In Proceedings of the 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, IEEE, Nara, Japan, pp. 225–234, 2007.
D. Gálvez-López, J. D. Tardos. Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics, vol. 28, no. 5, pp. 1188–1197, 2012.
E. Rublee, V. Rabaud, K. Konolige, G. Bradski. ORB: An efficient alternative to SIFT or SURF. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Barcelona, Spain, pp. 2564–2571, 2011.
C. Harris, M. Stephens. A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference, Manchester, UK, pp. 147–151, 1988.
R. Mur-Artal, J. M. M. Montiel, J. D. Tardós. ORBSLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, vol. 31, no. 5, pp. 1147–1163, 2015.
R. Mur-artal, J. D. Tardós. Fast relocalisation and loop closing in keyframe-based SLAM. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Hong Kong, China, pp. 846–853, 2014.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by National Natural Science Foundation of China (No. 61673283).
Recommended by Associate Editor Hong Qiao
Sun-Chun Zhou received the B. Sc. degree from China University of Mining and Technology, China in 2015. He is currently a master student in the College of Computer Science, Sichuan University, China.
His research interests include cognitive robots, SLAM and robotic navigation.
Rui Yan received the B. Sc. and M. Sc. degrees from Department of Mathematics, Sichuan University, China in 1998 and 2001, respectively, and received the Ph.D. degree from Department of Electrical and Computer Engineering, National University of Singapore, Singapore in 2006. She was a postdoctoral research fellow in the University of Queensland, Australia, from 2006 to 2008, and a research scientist with the Institute for Infocomm Research, A*STAR from 2009 to 2014. Now she is a professor in the College of Computer Science, Sichuan University, China.
Her research interests include intelligent robots, neural computation, nonlinear control and complex systems analysis.
Jia-Xin Li received the B. Sc. degree from Sichuan University, China in 2016. He is currently a Ph. D. degree candidate in the College of Computer Science, Sichuan University, China.
His research interests include SLAM, robotic navigation and cognitive robots.
Ying-Ke Chen received the Ph.D. degree from Aalborg University, DK in 2013. He was a post-doctoral research associate in Queen’s University Belfast, USA and Georgia University, US, respectively, before joining Sichuan University. He is a lecturer at College of Computer Science, Sichuan University, China.
His research interests include intelligent agents, decision making and their applications in autonomous systems.
Huajin Tang received the B. Eng. degree from Zhejiang University, China in 1998, received the M. Eng. degree from Shanghai Jiao Tong University, China in 2001, and received the Ph.D. degree from the National University of Singapore, Singapore in 2005. He was a system engineer with STMicroelectronics, Singapore, from 2004 to 2006. From 2006 to 2008, he was a post-doctoral fellow with the Queensland Brain Institute, University of Queensland, Australia. Since 2008, he was head of the Cognitive Computing Group and the Robotic Cognition Lab, Institute for Infocomm Research, A*STAR, Singapore. He is currently a National Youth-1000 Talent Distinguished professor and the director of the Neuromorphic Computing Research Center, Sichuan University, China. He has co-authored two monographs by Springer and over 70 international journal and conference papers. His research work on Brain GPS has been reported by MIT Technology Review on 2015. He has received the 2011 Institute for Infocomm Research Role Model Award, 2016 IEEE CIS Outstanding TNNLS Paper Award. He was the Program Chair of the 6th and 7th IEEE CIS-RAM, and the Co-Chair of 2016 International Workshop on Neuromorphic Computing and Cyborg Intelligence. He is currently an associate editor of IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Cognitive and Developmental Systems and Frontiers in Neuromorphic Engineering.
His research interests include neuromorphic cognitive computing, neuromorphic hardware and neuro-cognitive robotics.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Zhou, SC., Yan, R., Li, JX. et al. A brain-inspired SLAM system based on ORB features. Int. J. Autom. Comput. 14, 564–575 (2017). https://doi.org/10.1007/s11633-017-1090-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11633-017-1090-y