research-article

Smartphone-Based Indoor Visual Navigation with Leader-Follower Mode

Authors:

Zheng YangAuthors Info & Claims

ACM Transactions on Sensor Networks (TOSN), Volume 17, Issue 2

Article No.: 18, Pages 1 - 22

https://doi.org/10.1145/3448417

Published: 04 May 2021 Publication History

Abstract

Existing indoor navigation solutions usually require pre-deployed comprehensive location services with precise indoor maps and, more importantly, all rely on dedicatedly installed or existing infrastructure. In this article, we present Pair-Navi, an infrastructure-free indoor navigation system that circumvents all these requirements by reusing a previous traveler’s (i.e., leader) trace experience to navigate future users (i.e., followers) in a Peer-to-Peer mode. Our system leverages the advances of visual simultaneous localization and mapping (SLAM) on commercial smartphones. Visual SLAM systems, however, are vulnerable to environmental dynamics in the precision and robustness and involve intensive computation that prohibits real-time applications. To combat environmental changes, we propose to cull non-rigid contexts and keep only the static and rigid contents in use. To enable real-time navigation on mobiles, we decouple and reorganize the highly coupled SLAM modules for leaders and followers. We implement Pair-Navi on commodity smartphones and validate its performance in three diverse buildings and two standard datasets (TUM and KITTI). Our results show that Pair-Navi achieves an immediate navigation success rate of 98.6%, which maintains as 83.4% even after 2 weeks since the leaders’ traces were collected, outperforming the state-of-the-art solutions by >50%. Being truly infrastructure-free, Pair-Navi sheds lights on practical indoor navigations for mobile users.

References

[1]

ROS Android. 2017. Retrieved from http://wiki.ros.org/android.

[2]

ROS Kinetic Kame. 2016. Retrieved from http://wiki.ros.org/kinetic.

[3]

Pablo Arbeláez, Jordi Pont-Tuset, Jonathan T. Barron, Ferran Marques, and Jitendra Malik. 2014. Multiscale combinatorial grouping. In Proceedings of IEEE CVPR.

Digital Library

[4]

M. Azizyan, I. Constandache, and R. Roy Choudhury. 2009. Surroundsense: Mobile phone localization via ambience fingerprinting. In Proceedings of ACM MobiCom.

[5]

Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. Surf: Speeded up robust features. In European Conference on Computer Vision. Springer, 404--417.

Digital Library

[6]

Sean L. Bowman, Nikolay Atanasov, Kostas Daniilidis, and George J. Pappas. 2017. Probabilistic data association for semantic slam. In Proceedings of IEEE ICRA.

[7]

Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, José Neira, Ian Reid, and John J. Leonard. 2016. Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age. IEEE Transactions on Robotics 32, 6 (2016), 1309--1332.

Digital Library

[8]

Javier Civera, Dorian Gálvez-López, Luis Riazuelo, Juan D. Tardós, and J. M. M. Montiel. 2011. Towards semantic SLAM using a monocular camera. In Proceedings of IEEE IROS.

[9]

Ionut Constandache, Xuan Bao, Martin Azizyan, and Romit Roy Choudhury. 2010. Did you see Bob?: Human localization using mobile phones. In Proceedings of ACM Mobicom.

Digital Library

[10]

John J. Craig. 2005. Introduction to Robotics: Mechanics and Control. Vol. 3. Pearson/Prentice Hall, Upper Saddle River, NJ.

Digital Library

[11]

Jifeng Dai, Kaiming He, Yi Li, Shaoqing Ren, and Jian Sun. 2016. Instance-sensitive fully convolutional networks. In Proceedings of ECCV. Springer.

[12]

Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. 2016. R-FCN: Object detection via region-based fully convolutional networks. In Advances in Neural Information Processing Systems.

[13]

Andrew J. Davison. 2003. Real-time simultaneous localisation and mapping with a single camera. In Proceedings of IEEE ICCV.

[14]

Erqun Dong, Jingao Xu, Chenshu Wu, Yunhao Liu, and Zheng Yang. 2019. Pair-Navi: Peer-to-peer indoor navigation with mobile visual SLAM. In Proceedings of the IEEE INFOCOM.

Digital Library

[15]

Liang Dong, Jingao Xu, Guoxuan Chi, Danyang Li, Xinglin Zhang, Jianbo Li, Qiang Ma, and Zheng Yang. 2020. Enabling surveillance cameras to navigate. In Proceedings of the IEEE ICCCN.

[16]

Christian Forster, Luca Carlone, Frank Dellaert, and Davide Scaramuzza. 2017. On-manifold preintegration for real-time visual--inertial odometry. IEEE Transactions on Robotics 33, 1 (2017), 1--21.

Digital Library

[17]

Christian Forster, Matia Pizzoli, and Davide Scaramuzza. 2014. SVO: Fast semi-direct monocular visual odometry. In 2014 IEEE International Conference on Robotics and Automation (ICRA’14). IEEE, 15--22.

[18]

Dorian Gálvez-López and Juan D. Tardos. 2012. Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics 28, 5 (2012), 1188--1197.

Digital Library

[19]

Xiang Gao, Rui Wang, Nikolaus Demmel, and Daniel Cremers. 2018. LDSO: Direct sparse odometry with loop closure. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’18). IEEE, 2198--2204.

Digital Library

[20]

Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for autonomous driving? The KITTI vision benchmark suite. In Proceedings of the IEEE CVPR.

[21]

Ross Girshick. 2015. Fast R-CNN. In Proceedings of IEEE ICCV.

[22]

Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of IEEE CVPR.

Digital Library

[23]

Bharath Hariharan, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. 2015. Hypercolumns for object segmentation and fine-grained localization. In Proceedings of IEEE CVPR.

[24]

Richard Hartley and Andrew Zisserman. 2003. Multiple View Geometry in Computer Vision. Cambridge University Press.

[25]

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask R-CNN. In Proceedings of IEEE ICCV.

[26]

Jan Hosang, Rodrigo Benenson, Piotr Dollár, and Bernt Schiele. 2016. What makes for effective detection proposals?IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 4 (2016), 814--830.

[27]

Georg Klein and David Murray. 2007. Parallel tracking and mapping for small AR workspaces. In Proceedings of IEEE ISMAR.

Digital Library

[28]

Rainer Kümmerle, Giorgio Grisetti, Hauke Strasdat, Kurt Konolige, and Wolfram Burgard. 2011. G²O: A general framework for graph optimization. In Proceedings of IEEE ICRA.

[29]

Vincent Lepetit, Francesc Moreno-Noguer, and Pascal Fua. 2009. EPnP: An accurate O(n) solution to the PnP problem. International Journal of Computer Vision 81, 2 (2009), 155.

Digital Library

[30]

Danyang Li, Yumeng Lu, Jingao Xu, Qiang Ma, and Zhuo Liu. 2019. iPAC: Integrate pedestrian dead reckoning and computer vision for indoor localization and tracking. IEEE Access 7 (2019), 183514--183523.

[31]

Yi Li, Haozhi Qi, Jifeng Dai, Xiangyang Ji, and Yichen Wei. 2017. Fully convolutional instance-aware semantic segmentation. In Proceedings of IEEE CVPR.

[32]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common objects in context. In Proceedings of ECCV. Springer.

[33]

Hongbo Liu, Yu Gan, Jie Yang, Simon Sidhom, Yan Wang, Yingying Chen, and Fan Ye. 2012. Push the limit of WiFi based localization for smartphones. In Proceedings of ACM MobiCom.

Digital Library

[34]

Song Liu and Tian He. 2017. SmartLight: Light-weight 3D indoor localization using a single LED lamp. In Proceedings of ACM Sensys.

Digital Library

[35]

Etienne Mouragnon, Maxime Lhuillier, Michel Dhome, Fabien Dekeyser, and Patrick Sayd. 2006. Real time localization and 3D reconstruction. In Proceedings of IEEE CVPR.

Digital Library

[36]

Raúl Mur-Artal, J. M. M. Montiel, and Juan D. Tardós. 2015. ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics 31, 5 (2015), 1147--1163.

Digital Library

[37]

Raúl Mur-Artal and Juan D. Tardós. 2017. Visual-inertial monocular SLAM with map reuse. IEEE Robotics and Automation Letters 2, 2 (2017), 796--803.

[38]

Richard A. Newcombe, Dieter Fox, and Steven M. Seitz. 2015. Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proceedings of IEEE CVPR.

[39]

Pauline C. Ng and Steven Henikoff. 2003. SIFT: Predicting amino acid changes that affect protein function. Nucleic Acids Research 31, 13 (2003), 3812--3814.

[40]

Pedro O. Pinheiro, Ronan Collobert, and Piotr Dollár. 2015. Learning to segment object candidates. In Advances in Neural Information Processing Systems.

[41]

Pedro O. Pinheiro, Tsung-Yi Lin, Ronan Collobert, and Piotr Dollár. 2016. Learning to refine object segments. In Proceedings of ECCV. Springer.

[42]

Kun Qian, Chenshu Wu, Yi Zhang, Guidong Zhang, Zheng Yang, and Yunhao Liu. 2018. Widar2.0: Passive human tracking with a single Wi-Fi link. In Proceedings of ACM MobiSys.

Digital Library

[43]

Anshul Rai, Krishna Kant Chintalapudi, Venkata N. Padmanabhan, and Rijurekha Sen. 2012. Zee: Zero-effort crowdsourcing for indoor localization. In Proceedings of ACM MobiCom.

Digital Library

[44]

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In Proceedings of IEEE CVPR.

[45]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems. 91--99.

[46]

Timothy H. Riehle, Shane M. Anderson, Patrick A. Lichter, Nicholas A. Giudice, Suneel I. Sheikh, Robert J. Knuesel, Daniel T. Kollmann, and Daniel S. Hedin. 2012. Indoor magnetic navigation for the blind. In Proceedings of IEEE EMBC.

[47]

Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2011. ORB: An efficient alternative to SIFT or SURF. In Proceedings of IEEE ICCV.

Digital Library

[48]

Renato F. Salas-Moreno, Richard A. Newcombe, Hauke Strasdat, Paul H. J. Kelly, and Andrew J. Davison. 2013. Slam++: Simultaneous localisation and mapping at the level of objects. In Proceedings of IEEE CVPR.

[49]

Longfei Shangguan, Zheng Yang, Alex X. Liu, Zimu Zhou, and Yunhao Liu. 2017. STPP: Spatial-temporal phase profiling-based method for relative RFID tag localization. IEEE/ACM Transactions on Networking 25, 1 (2017), 596--609.

Digital Library

[50]

Y. Shu, Y. Huang, J. Zhang, P. Coué, P. Cheng, J. Chen, and K. G. Shin. 2016. Gradient-based fingerprinting for indoor localization and tracking. IEEE Transactions on Industrial Electronics 63, 4 (April 2016), 2424--2433.

[51]

Yuanchao Shu, Kang G. Shin, Tian He, and Jiming Chen. 2015. Last-mile navigation using smartphones. In Proceedings of ACM MobiCom.

Digital Library

[52]

William Storms, Jeremiah Shockley, and John Raquet. 2010. Magnetic field navigation in an indoor environment. In Proceedings of IEEE UPINLBS.

[53]

Hauke Strasdat, José M. M. Montiel, and Andrew J. Davison. 2012. Visual SLAM: Why filter?Image and Vision Computing 30, 2 (2012), 65--77.

[54]

J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers. 2012. A benchmark for the evaluation of RGB-D SLAM systems. In Proceedings of the IEEE IROS.

[55]

Xiaohua Tian, Ruofei Shen, Duowen Liu, Yutian Wen, and Xinbing Wang. 2016. Performance analysis of RSS fingerprinting based indoor localization. IEEE Transactions on Mobile Computing 16, 10 (2016), 2847--2861.

Digital Library

[56]

Xiaohua Tian, Mei Wang, Wenxin Li, Binyao Jiang, Dong Xu, Xinbing Wang, and Jun Xu. 2017. Improve accuracy of fingerprinting localization with temporal correlation of the RSS. IEEE Transactions on Mobile Computing 17, 1 (2017), 113--126.

[57]

Jasper R. R. Uijlings, Koen E. A. Van De Sande, Theo Gevers, and Arnold W. M. Smeulders. 2013. Selective search for object recognition. International Journal of Computer Vision 104, 2 (2013), 154--171.

Digital Library

[58]

Jue Wang and Dina Katabi. 2013. Dude, where’s my card? RFID positioning that works with multipath and non-line of sight. In ACM SIGCOMM.

[59]

Chenshu Wu, Jingao Xu, Zheng Yang, Nicholas D. Lane, and Zuwei Yin. 2017. Gain without pain: Accurate WiFi-based localization with fingerprint spatial gradient. In PACM on Interactive, Mobile, Wearable and Ubiquitous Technologies.

[60]

Chenshu Wu, Zheng Yang, and Yunhao Liu. 2015. Smartphones based crowdsourcing for indoor localization. IEEE Transactions on Mobile Computing 14, 2 (2015), 444--457.

[61]

Chenshu Wu, Zheng Yang, Yunhao Liu, and Wei Xi. 2012. WILL: Wireless indoor localization without site survey. IEEE Transactions on Parallel and Distributed Systems 24, 4 (2012), 839--848.

[62]

Chenshu Wu, Zheng Yang, and Chaowei Xiao. 2018. Automatic radio map adaptation for indoor localization using smartphones. IEEE Transactions on Mobile Computing 17, 3 (2018), 517--528.

[63]

Jingao Xu, Hao Cao, Danyang Li, Kehong Huang, Chen Qian, Longfei Shangguan, and Zheng Yang. 2020. Edge assisted mobile semantic visual SLAM. In Proceedings of the IEEE INFOCOM.

Digital Library

[64]

Jingao Xu, Hengjie Chen, Kun Qian, Erqun Dong, Min Sun, Chenshu Wu, Li Zhang, and Zheng Yang. 2019. iVR: Integrated vision and radio localization with zero human effort. In PACM on Interactive, Mobile, Wearable and Ubiquitous Technologies.

[65]

Jingao Xu, Zheng Yang, Hengjie Chen, Yunhao Liu, Xianchun Zhou, Jinbo Li, and Nicholas Lane. 2018. Embracing spatial awareness for reliable WiFi-based indoor location systems. In Proceedings of the IEEE MASS.

[66]

Zheng Yang, Chenshu Wu, and Yunhao Liu. 2012. Locating in fingerprint space: Wireless indoor localization with little human intervention. In Proceedings of ACM MobiCom.

Digital Library

[67]

Zheng Yang, Chenshu Wu, Zimu Zhou, Xinglin Zhang, Xu Wang, and Yunhao Liu. 2015. Mobility increases localizability: A survey on wireless indoor localization using inertial sensors. ACM Computing Surveys 47, 3 (April 2015), Article 54, 34 pages.

Digital Library

[68]

Zheng Yang, Zimu Zhou, and Yunhao Liu. 2013. From RSSI to CSI: Indoor localization via channel response. ACM Computing Surveys (CSUR) 46, 2 (2013), 25.

Digital Library

[69]

Zuwei Yin, Chenshu Wu, Zheng Yang, and Yunhao Liu. 2017. Peer-to-peer indoor navigation using smartphones. IEEE Journal on Selected Areas in Communications 35, 5 (2017), 1141--1153.

Digital Library

[70]

Yuanqing Zheng, Guobin Shen, Liqun Li, Chunshui Zhao, Mo Li, and Feng Zhao. 2014. Travi-Navi: Self-deployable indoor navigation system. In Proceedings of ACM MobiCom.

Digital Library

[71]

Yue Zheng, Yi Zhang, Kun Qian, Guidong Zhang, Yunhao Liu, Chenshu Wu, and Zheng Yang. 2019. Widar3.0: Zero-effort cross-domain gesture recognition with Wi-Fi. In ACM International Conference on Mobile Systems, Applications, and Services.

Cited By

Li DZhang HCheng JLiu B(2024)Improving efficiency of DNN-based relocalization module for autonomous driving with server-side computingJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-024-00592-113:1Online publication date: 25-Jan-2024
https://dl.acm.org/doi/10.1186/s13677-024-00592-1
Shu CChen YTan CLuo YDou H(2024)Enhancing trust transfer in supply chain finance: a blockchain-based transitive trust modelJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-023-00557-w13:1Online publication date: 2-Jan-2024
https://dl.acm.org/doi/10.1186/s13677-023-00557-w
Zhang LZhou XLi DYang Z(2024)HCCNet: Hybrid Coupled Cooperative Network for Robust Indoor LocalizationACM Transactions on Sensor Networks10.1145/366564520:4(1-22)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3665645
Show More Cited By

Index Terms

Smartphone-Based Indoor Visual Navigation with Leader-Follower Mode
1. Human-centered computing
  1. Ubiquitous and mobile computing

Recommendations

Seamless outdoor/indoor navigation with WIFI/GPS aided low cost Inertial Navigation System

This paper describes an integrated navigation system that can be used for pedestrian navigation in both outdoor and indoor environments. With the aid of Global Positioning System (GPS) positioning solutions, an Inertial Navigation System (INS) can ...
Research and development of an indoor navigation system based on the digital processing of video images

The paper describes the development of an indoor navigation system of an autonomous mobile robot using image processing methods of industrial television. The system analyzes the surrounding space with a simple monocular TV camera. Color beacons placed ...
Vision for Mobile Robot Navigation: A Survey

This paper surveys the developments of the last 20 years in the area of vision for mobile robot navigation. Two major components of the paper deal with indoor navigation and outdoor navigation. For each component, we have further subdivided our ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Sensor Networks

ACM Transactions on Sensor Networks Volume 17, Issue 2

May 2021

296 pages

ISSN:1550-4859

EISSN:1550-4867

DOI:10.1145/3447946

Editor:
Yunhao Liu
Tsinghua University, China

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 04 May 2021

Accepted: 01 January 2021

Revised: 01 November 2020

Received: 01 September 2020

Published in TOSN Volume 17, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

NSFC
National Key R&D Program of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
362
Total Downloads

Downloads (Last 12 months)54
Downloads (Last 6 weeks)2

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li DZhang HCheng JLiu B(2024)Improving efficiency of DNN-based relocalization module for autonomous driving with server-side computingJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-024-00592-113:1Online publication date: 25-Jan-2024
https://dl.acm.org/doi/10.1186/s13677-024-00592-1
Shu CChen YTan CLuo YDou H(2024)Enhancing trust transfer in supply chain finance: a blockchain-based transitive trust modelJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-023-00557-w13:1Online publication date: 2-Jan-2024
https://dl.acm.org/doi/10.1186/s13677-023-00557-w
Zhang LZhou XLi DYang Z(2024)HCCNet: Hybrid Coupled Cooperative Network for Robust Indoor LocalizationACM Transactions on Sensor Networks10.1145/366564520:4(1-22)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3665645
Lyu YQin PXu TZhu CChen E(2024)InteractNet: Social Interaction Recognition for Semantic-rich VideosACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366366820:8(1-21)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3663668
Huang JRen HLiu SLiu YLv CLu JXie CLu H(2024)Real-Time Attentive Dilated U-Net for Extremely Dark Image EnhancementACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365466820:8(1-19)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3654668
Pang SLuo JZhao XChen JWang FYin J(2024)DTSSN: A Distributed Trustworthy Sensor Service Network Architecture for Smart CityACM Transactions on Sensor Networks10.1145/3649893Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3649893
Li DXu JYang ZTang C(2024)Train Once, Locate Anytime for Anyone: Adversarial Learning-based Wireless LocalizationACM Transactions on Sensor Networks10.1145/361409520:2(1-21)Online publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1145/3614095
Li NRen XCastiglione ALiu M(2024)Efficient and precise visual location estimation by effective priority matching-based pose verification in edge-cloud collaborative IoTFuture Generation Computer Systems10.1016/j.future.2024.02.001155(96-107)Online publication date: Jun-2024
https://doi.org/10.1016/j.future.2024.02.001
He HZhou FZhao YLi WFeng L(2023)Hypergraph convolution mix DDPG for multi-aerial base station deploymentJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-023-00556-x12:1Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1186/s13677-023-00556-x
Yang MGe CZhao XKou H(2023)FSPLO: a fast sensor placement location optimization method for cloud-aided inspection of smart buildingsJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-023-00410-012:1Online publication date: 6-Mar-2023
https://dl.acm.org/doi/10.1186/s13677-023-00410-0
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents