research-article

Amateur: Augmented Reality Based Vehicle Navigation System

Authors:

Mo LiAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 2, Issue 4

Article No.: 155, Pages 1 - 24

https://doi.org/10.1145/3287033

Published: 27 December 2018 Publication History

Abstract

This paper presents Amateur, an augmented reality based vehicle navigation system using commodity smart phones. Amateur reads the navigation information from a digital map, matches it into live road condition video captured by smart phone, and directly annotates the navigation instructions on the video stream. The Amateur design entails two major challenges, including the lane identification and the intersection inference so as to correctly annotate navigation instructions for lane-changing and intersection-turning. In this paper, we propose a particle filter based design, assisted by inertial motion sensors and lane markers, to tolerate incomplete and even erroneous detection of road conditions. We further leverage traffic lights as land markers to estimate the position of each intersection to accurately annotate the navigation instructions. We develop a prototype system on Android mobile phones and test our system in a total number of more than 300km travel distance on different taxi cabs in a city. The evaluation results suggest that our system can timely provide correct instructions to navigate drivers. Our system can identify lanes in 2s with 92.7% accuracy and detect traffic lights with 95.29% accuracy. Overall, the accuracy of the navigation signs placement is less than 105 pixels on the screen throughout the experiments. The feedback from 50 taxi drivers indicates that Amateur provides an improved experience compared to traditional navigation systems.

Supplementary Material

cao (cao.zip)

Supplemental movie, appendix, image and software files for, Amateur: Augmented Reality Based Vehicle Navigation System

Download
64.46 KB

References

[1]

Mohamed Aly. 2008. Real time detection of lane markers in urban streets. In Proceedings of IEEE IV. 7--12.

[2]

Donna R Berryman. 2012. Augmented reality: a review. Taylor & Francis Medical reference services quarterly 2 (2012), 212--218.

[3]

John Canny. 1986. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 6 (1986), 679--698.

Digital Library

[4]

Julie Carmigniani, Borko Furht, Marco Anisetti, Paolo Ceravolo, Ernesto Damiani, and Misa Ivkovic. 2011. Augmented reality technologies, systems and applications. Springer Multimedia tools and applications 1 (2011), 341--377.

Digital Library

[5]

Thanh-Son Dao, Keith Yu Kit Leung, Christopher Michael Clark, and Jan Paul Huissoon. 2007. Markov-based lane positioning using intervehicle communication. IEEE Transactions on Intelligent Transportation Systems 4 (2007), 641--650.

Digital Library

[6]

A David and Ponce Jean. 2002. Computer vision: a modern approach. (2002).

Digital Library

[7]

Fred D Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly (1989), 319--340.

Digital Library

[8]

Moises Diaz, Pietro Cerri, Giuseppe Pirlo, Miguel A Ferrer, and Donato Impedovo. 2015. A survey on traffic light detection. In Proceedings of Springer ICIAP. 201--208.

[9]

Exploride. 2018. Exploride website. (Jan. 2018). Retrieved May 4, 2018 from https://exploride.com/

[10]

Eric Foxlin, Thomas Calloway, and Hongsheng Zhang. 2014. Improved registration for vehicular AR using auto-harmonization. In Proceedings of IEEE ISMAR. 105--112.

[11]

Peter Froehlich, Raimund Schatz, Peter Leitner, Stephan Mantler, and Matthias Baldauf. 2010. Evaluating realistic visualizations for safety-related in-car information systems. In Proceedings of ACM CHI. 3847--3852.

Digital Library

[12]

Peter Fröhlich, Matthias Baldauf, Marion Hagen, Stefan Suette, Dietmar Schabus, and Andrew L Kun. 2011. Investigating safety services on the motorway: the role of realistic visualization. In Proceedings of ACM AutomotiveUI. 143--150.

Digital Library

[13]

Peter Fröhlich, Raimund Schatz, Peter Leitner, Matthias Baldauf, and Stephan Mantler. 2010. Augmenting the driver's view with real-time safety-related information. In Proceedings of ACM AH. 11.

Digital Library

[14]

Jianwei Gong, Yanhua Jiang, Guangming Xiong, Chaohua Guan, Gang Tao, and Huiyan Chen. 2010. The recognition and tracking of traffic lights based on color segmentation and CAMSHIFT for intelligent vehicles. In Proceedings of IEEE IV. 431--435.

[15]

Google. 2018. Google I/O Event. (May 2018). Retrieved May 9, 2018 from https://events.google.com/io/

[16]

Google. 2018. Google Maps. (Jan. 2018). Retrieved February 11, 2016 from https://itunes.apple.com/us/app/google-maps/id585027354?mt=8

[17]

Google. 2018. Google Maps Direction API. (Jan. 2018). Retrieved February 1, 2016 from https://developers.google.com/maps/documentation/directions/intro?hl=en

[18]

Aharon Bar Hillel, Ronen Lerner, Dan Levi, and Guy Raz. 2014. Recent progress in road and lane detection: a survey. Springer Machine Vision and Applications 3 (2014), 727--745.

Digital Library

[19]

Hudify. 2018. WAYRAY Website. (Jan. 2018). Retrieved May 4, 2018 from http://www.gethudify.com

[20]

Yurong Jiang, Hang Qiu, Matthew McCartney, Gaurav Sukhatme, Marco Gruteser, Fan Bai, Donald Grimm, and Ramesh Govindan. 2015. CARLOC: Precise Positioning of Automobiles. In Proceedings of ACM SenSys. 253--265.

Digital Library

[21]

Richie Jose, Gun A Lee, and Mark Billinghurst. 2016. A comparative study of simulated augmented reality displays for vehicle navigation. In Proceedings of ACM OzCHI. 40--48.

Digital Library

[22]

Soonhong Jung, Junsic Youn, and Sanghoon Sull. 2016. Efficient lane detection based on spatiotemporal images. IEEE Transactions on Intelligent Transportation Systems 1 (2016), 289--295.

[23]

Clemens Kaufmann, Ralf Risser, Arjan Geven, and Reinhard Sefelin. 2008. Effects of simultaneous multi-modal warnings and traffic information on driver behaviour. In Proceedings of European Conference on Human Centred Design for Intelligent Transport Systems. 33--42.

[24]

Emmanouil Koukoumidis, Li-Shiuan Peh, and Margaret Rose Martonosi. 2011. SignalGuru: leveraging mobile phones for collaborative traffic signal schedule advisory. In Proceedings of ACM MobiSys. 127--140.

Digital Library

[25]

Ye-Sheng Kuo, Pat Pannuto, Ko-Jen Hsiao, and Prabal Dutta. 2014. Luxapose: Indoor positioning with mobile phones and visible light. In Proceedings of ACM MobiCom. 447--458.

Digital Library

[26]

Vladimir I Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady. 707--710.

[27]

Masako Omachi and Shinichiro Omachi. 2009. Traffic light detection with color and edge information. In Proceedings of IEEE ICCSIT. 284--287.

[28]

OSM. 2018. Open Street Map. (Jan. 2018). Retrieved February 1, 2016 from https://www.openstreetmap.org

[29]

Oskar Palinko, Andrew L Kun, Zachary Cook, Adam Downey, Aaron Lecomte, Meredith Swanson, and Tina Tomaszewski. 2013. Towards augmented reality navigation using affordable technology. In Proceedings of ACM AutomotiveUI. 238--241.

Digital Library

[30]

Christian M Richard, Richard D Wright, Cheryl Ee, Steven L Prime, Yujiro Shimizu, and John Vavrik. 2002. Effect of a concurrent auditory task on visual search performance in a driving-related image-flicker task. SAGE Human Factors: The Journal of the Human Factors and Ergonomics Society 1 (2002), 108--119.

[31]

John TE Richardson. 2018. The use of Latin-square designs in educational and psychological research. Educational Research Review (2018), 84--97.

[32]

Michelle Krüger Silvéria Santos. 2013. Virtual windshields: merging reality and digital content to improve the driving experience. (2013).

[33]

H Sawano and M Okada. 2006. Real-time video processing by a car-mounted camera and its application to car navigation systems by an augmented reality-based display method. The Journal of the Society for Art and Science 2 (2006), 57--68.

[34]

Longfei Shangguan, Zheng Yang, Alex X Liu, Zimu Zhou, and Yunhao Liu. 2017. STPP: Spatial-temporal phase profiling-based method for relative RFID tag localization. IEEE/ACM Transactions on Networking 25, 1 (2017), 596--609.

Digital Library

[35]

Sygic. 2018. Sygic Official Website. (Jan. 2018). Retrieved May 2, 2018 from https://www.sygic.com/gps-navigation

[36]

Hwang Tae-Hyun, Joo In-Hak, and Cho Seong-Ik. 2006. Detection of traffics for vision-based car navigation system. In Springer Advances in Image and Video Technology. 682--691.

Digital Library

[37]

Zhenning Tao, Philippe Bonnifait, Vincent Fremont, and Javier Ibanez-Guzman. 2013. Lane marking aided vehicle localization. In Proceedings of IEEE ITSC. 1509--1515.

[38]

TESLA. 2018. Tesla official websit. (Jan. 2018). Retrieved March 2, 2016 from https://www.teslamotors.com/

[39]

Rafael Toledo-Moreo, David Bétaille, and François Peyret. 2010. Lane-level integrity provision for navigation and map matching with GNSS, dead reckoning, and enhanced maps. IEEE Transactions on Intelligent Transportation Systems 1 (2010), 100--112.

Digital Library

[40]

WAYRAY. 2018. WAYRAY Website. (Jan. 2018). Retrieved May 4, 2018 from https://wayray.com/navion

[41]

Yoshihisa Yamaguchi, Takashi Nakagawa, Kengo Akaho, Mitsushi Honda, Hirokazu Kato, and Shogo Nishida. 2007. AR-Navi: An in-vehicle navigation system using video-based augmented reality technology. In Springer Symposium on Human Interface and the Management of Information. 1139--1147.

Digital Library

[42]

Chuang-Wen You, Nicholas D Lane, Fanglin Chen, Rui Wang, Zhenyu Chen, Thomas J Bao, Martha Montes-de Oca, Yuting Cheng, Mu Lin, Lorenzo Torresani, et al. 2013. Carsafe app: Alerting drowsy and distracted drivers using dual cameras on smartphones. In Proceedings of ACM MobiSys. 13--26.

Digital Library

Cited By

Zhu QLiu RWang ZLiu QChen C(2024)Sensing-Communication Co-Design for UAV Swarm-Assisted Vehicular Network in Perspective of DopplerIEEE Transactions on Vehicular Technology10.1109/TVT.2023.331586873:2(2578-2592)Online publication date: Feb-2024
https://doi.org/10.1109/TVT.2023.3315868
Norval CCloete RSingh J(2023)Navigating the Audit Landscape: A Framework for Developing Transparent and Auditable XRProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594090(1418-1431)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594090
Yan JZhou XYang XShang ZLuo XGuan X(2023)Joint Design of Channel Estimation and Flocking Control for Multi-AUV-Based Maritime Transportation SystemsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.329296724:12(14520-14535)Online publication date: Dec-2023
https://doi.org/10.1109/TITS.2023.3292967
Show More Cited By

Recommendations

Invisible Sensing of Vehicle Steering with Smartphones
MobiSys '15: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services

Detecting how a vehicle is steered and then alarming drivers in real time is of utmost importance to the vehicle and the driver's safety, since fatal accidents are often caused by dan- gerous steering. Existing solutions for detecting dangerous maneuvers ...
BigRoad: Scaling Road Data Acquisition for Dependable Self-Driving
MobiSys '17: Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services

Advanced driver assistance systems and, in particular automated driving offers an unprecedented opportunity to transform the safety, efficiency, and comfort of road travel. Developing such safety technologies requires an understanding of not just common ...
Exploring head-up augmented reality interfaces for crash warning systems
AutomotiveUI '13: Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Crash warning systems are designed to help avoid vehicle accidents by notifying drivers of potential hazards. In typical crash warning systems, primary warning information is provided through visual, audible and/or haptic cues. In general, the use of ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 2, Issue 4

December 2018

1169 pages

EISSN:2474-9567

DOI:10.1145/3301777

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 December 2018

Accepted: 01 December 2018

Revised: 01 August 2018

Received: 01 May 2018

Published in IMWUT Volume 2, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Singapore MOE AcRF Tier 1
Tier 2
CoE
Research Grants Council of Hong Kong

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
708
Total Downloads

Downloads (Last 12 months)82
Downloads (Last 6 weeks)9

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhu QLiu RWang ZLiu QChen C(2024)Sensing-Communication Co-Design for UAV Swarm-Assisted Vehicular Network in Perspective of DopplerIEEE Transactions on Vehicular Technology10.1109/TVT.2023.331586873:2(2578-2592)Online publication date: Feb-2024
https://doi.org/10.1109/TVT.2023.3315868
Norval CCloete RSingh J(2023)Navigating the Audit Landscape: A Framework for Developing Transparent and Auditable XRProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594090(1418-1431)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594090
Yan JZhou XYang XShang ZLuo XGuan X(2023)Joint Design of Channel Estimation and Flocking Control for Multi-AUV-Based Maritime Transportation SystemsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.329296724:12(14520-14535)Online publication date: Dec-2023
https://doi.org/10.1109/TITS.2023.3292967
Liu QLiu RZhang YYuan YWang ZYang HYe LGuizani MThompson J(2023)Management of Positioning Functions in Cellular Networks for Time-Sensitive Transportation ApplicationsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.323453224:11(13260-13275)Online publication date: Nov-2023
https://doi.org/10.1109/TITS.2023.3234532
Cheliotis KLiarokapis FKokla MTomai EPastra KAnastopoulou NBezerianou MDarra AKavouras M(2023)A systematic review of application development in augmented reality navigation researchCartography and Geographic Information Science10.1080/15230406.2023.219403250:3(249-271)Online publication date: 9-May-2023
https://doi.org/10.1080/15230406.2023.2194032
Stefanucci JBrickler DFinney HWilson EDrew TCreem-Regehr S(2022)Effects of simulated augmented reality cueing in a virtual navigation taskFrontiers in Virtual Reality10.3389/frvir.2022.9713103Online publication date: 29-Sep-2022
https://doi.org/10.3389/frvir.2022.971310
Zhang WLin SBijarbooneh FCheng HBraud TZhou PLee LHui P(2022)EdgeXAR: A 6-DoF Camera Multi-target Interaction Framework for MAR with User-friendly Latency CompensationProceedings of the ACM on Human-Computer Interaction10.1145/35322026:EICS(1-24)Online publication date: 17-Jun-2022
https://dl.acm.org/doi/10.1145/3532202
Meteriz-Yildiran UYildiran NAwad AMohaisen D(2022)A Keylogging Inference Attack on Air-Tapping Keyboards in Virtual Environments2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR51125.2022.00098(765-774)Online publication date: Mar-2022
https://doi.org/10.1109/VR51125.2022.00098
Shi WLiu Y(2022)A Novel Navigation System on AR Helmet with Driving Attention Analysis2022 IEEE 7th International Conference on Intelligent Transportation Engineering (ICITE)10.1109/ICITE56321.2022.10101399(204-208)Online publication date: 11-Nov-2022
https://doi.org/10.1109/ICITE56321.2022.10101399
Riegler ARiener AHolzmann C(2021)Augmented Reality for Future Mobility: Insights from a Literature Review and HCI Workshopi-com10.1515/icom-2021-002920:3(295-318)Online publication date: 27-Nov-2021
https://doi.org/10.1515/icom-2021-0029
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents