research-article

Public Access

Low Bandwidth Offload for Mobile AR

Authors:

Justin Manweiler, and

Romit Roy ChoudhuryAuthors Info & Claims

CoNEXT '16: Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies

December 2016

Pages 237 - 251

https://doi.org/10.1145/2999572.2999587

Published: 06 December 2016 Publication History

Abstract

Environmental fingerprinting has been proposed as a key enabler to immersive, highly contextualized mobile computing applications, especially augmented reality. While fingerprints can be constructed in many domains (e.g., wireless RF, magnetic field, and motion patterns), visual fingerprinting is especially appealing due to the inherent heterogeneity in many indoor spaces. This visual diversity, however, is also its Achilles' heel -- matching a unique visual signature against a database of millions requires either impractical computation for a mobile device, or to upload large quantities of visual data for cloud offload. Further, most visual "features" tend to be low entropy -- e.g., homogeneous repetitions of floor and ceiling tiles. Our system VisualPrint, proposes a means to offload only the most distinctive visual data, that is, only those visual signatures which stand a good chance to yield a unique match. VisualPrint enables cloud-offloaded visual fingerprinting with efficacy comparable to using whole images, but with an order reduction in network transfer.

References

[1]

Amazon fire phone. https://developer.amazon.com/public/solutions/devices/fire-phone.

[2]

Dextro stream. https://www.dextro.co/.

[3]

Microsoft hololens. https://www.microsoft.com/microsoft-hololens/en-us.

[4]

Nest cam. https://nest.com/camera/meet-nest-cam/.

[5]

Oculus vr. https://www.oculus.com/en-us/.

[6]

Project tango. https://www.google.com/atap/projecttango/, 2014.

[7]

Meerkat: Live stream video. http://meerkatapp.co/, 2015.

[8]

Periscope: Explore the world through someone else's eyes. https://www.periscope.tv/, 2015.

[9]

Tango taking picture while sensing depth. http://stackoverflow.com/questions/29978991/how-to-take-high-res-picture-while-sensing-depth-using-project-tango, 2015.

[10]

P. F. Alcantarilla, A. Bartoli, and A. J. Davison. Kaze features. In ECCV, pages 214--227. Springer, 2012.

Digital Library

[11]

A. Andoni and P. Indyk. E2lsh: Exact euclidean locality-sensitive hashing. Implementation available at, 2004.

[12]

P. Bahl and V. N. Padmanabhan. Radar: An in-building rf-based user location and tracking system. In INFOCOM, volume 2, pages 775--784. Ieee, 2000.

[13]

X. Bao, S. Fan, A. Varshavsky, K. Li, and R. Roy Choudhury. Your reactions suggest you liked the movie: Automatic content rating via reaction sensing. In UbiComp, pages 197--206. ACM, 2013.

Digital Library

[14]

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool. Speeded-up robust features (surf). Computer vision and image understanding, 110(3):346--359, 2008.

Digital Library

[15]

E. Cuervo, A. Balasubramanian, D.-k. Cho, A. Wolman, S. Saroiu, R. Chandra, and P. Bahl. Maui: making smartphones last longer with code offload. In MobiSys, pages 49--62. ACM, 2010.

Digital Library

[16]

M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokni. Locality-sensitive hashing scheme based on p-stable distributions. In Proceedings of the twentieth annual symposium on Computational geometry, pages 253--262. ACM, 2004.

Digital Library

[17]

J. Fung and S. Mann. Openvidia: parallel gpu computer vision. In Proceedings of the 13th annual ACM international conference on Multimedia, pages 849--852. ACM, 2005.

Digital Library

[18]

M. Gong and Y.-H. Yang. Near real-time reliable stereo matching using programmable graphics hardware. In CVPR, volume 1, pages 924--931. IEEE, 2005.

Digital Library

[19]

S. Hodges, L. Williams, E. Berry, S. Izadi, J. Srinivasan, A. Butler, G. Smyth, N. Kapur, and K. Wood. Sensecam: A retrospective memory aid. In UbiComp, pages 177--193. Springer, 2006.

Digital Library

[20]

W. Hu, B. Amos, Z. Chen, K. Ha, W. Richter, P. Pillai, B. Gilbert, J. Harkes, and M. Satyanarayanan. The case for offload shaping. In HotMobile, pages 51--56. ACM, 2015.

Digital Library

[21]

Y. Hua, B. Xiao, B. Veeravalli, and D. Feng. Locality-sensitive bloom filter for approximate membership query. Computers, IEEE Transactions on, 61(6):817--830, 2012.

Digital Library

[22]

P. Jain, J. Manweiler, A. Acharya, and K. Beaty. Focus: clustering crowdsourced videos by line-of-sight. In SenSys, page 8. ACM, 2013.

Digital Library

[23]

P. Jain, J. Manweiler, and R. Roy Choudhury. Overlay: Practical mobile augmented reality. In MobiSys, pages 331--344. ACM, 2015.

Digital Library

[24]

N. Karlsson, E. Di Bernardo, J. Ostrowski, L. Goncalves, P. Pirjanian, and M. E. Munich. The vslam algorithm for robust localization and mapping. In ICRA, pages 24--29. IEEE, 2005.

[25]

R. Kemp, N. Palmer, T. Kielmann, and H. Bal. Cuckoo: a computation offloading framework for smartphones. In Mobile Computing, Applications, and Services, pages 59--79. Springer, 2012.

[26]

A. Kirsch and M. Mitzenmacher. Distance-sensitive bloom filters. In ALENEX, volume 6, pages 41--50. SIAM, 2006.

Digital Library

[27]

K. Kumar and Y.-H. Lu. Cloud computing for mobile users: Can offloading computation save energy? Computer, (4):51--56, 2010.

Digital Library

[28]

K. Lee, J. Lee, Y. Yi, I. Rhee, and S. Chong. Mobile data offloading: how much can wifi deliver? In CoNext, page 26. ACM, 2010.

Digital Library

[29]

L. Li, P. Hu, C. Peng, G. Shen, and F. Zhao. Epsilon: A visible light based positioning system. In NSDI, pages 331--343. USENIX Association, 2014.

Digital Library

[30]

R. LiKamWa, Y. Liu, N. D. Lane, and L. Zhong. Moodscope: Building a mood sensor from smartphone usage patterns. In MobiSys, pages 389--402. ACM, 2013.

Digital Library

[31]

R. LiKamWa, B. Priyantha, M. Philipose, L. Zhong, and P. Bahl. Energy characterization and optimization of image sensing toward continuous mobile vision. In MobiSys, pages 69--82. ACM, 2013.

Digital Library

[32]

Y. Lin, F. Lv, S. Zhu, M. Yang, T. Cour, K. Yu, L. Cao, and T. Huang. Large-scale image classification: fast feature extraction and svm training. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 1689--1696. IEEE, 2011.

Digital Library

[33]

D. G. Lowe. Object recognition from local scale-invariant features. In ICCV, volume 2, pages 1150--1157. Ieee, 1999.

Digital Library

[34]

B. D. Lucas, T. Kanade, et al. An iterative image registration technique with an application to stereo vision. In IJCAI, volume 81, pages 674--679, 1981.

Digital Library

[35]

Q. Lv, W. Josephson, Z. Wang, M. Charikar, and K. Li. Multi-probe lsh: efficient indexing for high-dimensional similarity search. In VLDB, pages 950--961. VLDB Endowment, 2007.

Digital Library

[36]

G. Papagiannakis, G. Singh, and N. Magnenat-Thalmann. A survey of mobile and wireless technologies for augmented reality systems. Computer Animation and Virtual Worlds, 19(1):3--22, 2008.

Digital Library

[37]

B. Rister, G. Wang, M. Wu, and J. R. Cavallaro. A fast and efficient sift detector using the mobile gpu. In ICASSP, pages 2674--2678. IEEE, 2013.

[38]

E. Rublee, V. Rabaud, K. Konolige, and G. Bradski. Orb: an efficient alternative to sift or surf. In ICCV, pages 2564--2571. IEEE, 2011.

Digital Library

[39]

T. Sattler, B. Leibe, and L. Kobbelt. Fast image-based localization using direct 2d-to-3d matching. In ICCV, pages 667--674. IEEE, 2011.

Digital Library

[40]

M. Satyanarayanan, P. Bahl, R. Caceres, and N. Davies. The case for vm-based cloudlets in mobile computing. Pervasive Computing, IEEE, 8(4):14--23, 2009.

Digital Library

[41]

S. Sen, J. Lee, K.-H. Kim, and P. Congdon. Avoiding multipath to revive inbuilding wifi localization. In MobiSys, pages 249--262. ACM, 2013.

Digital Library

[42]

C. Silpa-Anan and R. Hartley. Optimised kd-trees for fast image descriptor matching. In CVPR, pages 1--8. IEEE, 2008.

[43]

G. Takacs, V. Chandrasekhar, N. Gelfand, Y. Xiong, W.-C. Chen, T. Bismpigiannis, R. Grzeszczuk, K. Pulli, and B. Girod. Outdoors augmented reality on mobile phone using loxel-based visual feature organization. In ICMR, pages 427--434. ACM, 2008.

Digital Library

[44]

B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon. Bundle adjustment a modern synthesis. In Vision algorithms: theory and practice, pages 298--372. Springer, 2000.

[45]

H. Wang, S. Sen, A. Elgohary, M. Farid, M. Youssef, and R. R. Choudhury. No need to war-drive: unsupervised indoor localization. In MobiSys, pages 197--210. ACM, 2012.

Digital Library

[46]

Z. Zhang, R. Deriche, O. Faugeras, and Q.-T. Luong. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial intelligence, 78(1):87--119, 1995.

Digital Library

Cited By

Ma QYuan HHu ZWang XYang Z(2024)A Liquidity Analysis System for Large-scale Video Streams in the OilfieldACM Transactions on Sensor Networks10.1145/364922220:3(1-22)Online publication date: 13-Apr-2024
https://dl.acm.org/doi/10.1145/3649222
Zhang LWu XWang FSun ACui LLiu J(2024)Edge-Based Video Stream Generation for Multi-Party Mobile Augmented RealityIEEE Transactions on Mobile Computing10.1109/TMC.2022.323254323:1(409-422)Online publication date: Jan-2024
https://doi.org/10.1109/TMC.2022.3232543
Li ZXu CZhang ZWu R(2024)Deep reinforcement learning based trajectory design and resource allocation for task-aware multi-UAV enabled MEC networksComputer Communications10.1016/j.comcom.2023.11.006213(88-98)Online publication date: Jan-2024
https://doi.org/10.1016/j.comcom.2023.11.006
Show More Cited By

Index Terms

Low Bandwidth Offload for Mobile AR
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
  2. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing theory, concepts and paradigms
      1. Mobile computing
    2. Ubiquitous and mobile devices

Recommendations

Energy efficient offloading scheme for MEC-based augmented reality system
Abstract
Augmented reality (AR) is a recent communication paradigm that is considered one of the primary functions of the fifth-generation cellular system (5G). AR is one of the ultra-reliable low latency communications since it requires a total delay of ...
Read More
Methods of cloud-path selection for offloading in mobile cloud computing systems
CLOUDCOM '12: Proceedings of the 2012 IEEE 4th International Conference on Cloud Computing Technology and Science (CloudCom)

Recently, there emerge a variety of clouds in sky and thus, several similar cloud services (from different cloud venders) can be provided to a mobile end device. The goal of cloud-path selection is to find an optimal cloud among a certain class of ...
Read More
Efficient Mobile Computation Using the Cloud
FICLOUD '15: Proceedings of the 2015 3rd International Conference on Future Internet of Things and Cloud

Mobile devices have limited resources in terms of power and bandwidth. Cloud computing offers a way to reduce the power consumption of mobile devices by offloading computation to the cloud. However, offloading computation means an increase in ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CoNEXT '16: Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies

December 2016

524 pages

ISBN:9781450342926

DOI:10.1145/2999572

General Chairs:
Athina Markopoulou
University of California, Irvine, USA
,
Michalis Faloutsos
University of California, Riverside, USA
,
Program Chairs:
Vyas Sekar
Carnegie Mellon University, USA
,
Dejan Kostic
KTH Royal Institute of Technology, Sweden

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Google
Intel
NSF

Conference

CoNEXT '16

Sponsor:

SIGCOMM

CoNEXT '16: The 12th International Conference on emerging Networking EXperiments and Technologies

December 12 - 15, 2016

California, Irvine, USA

Acceptance Rates

CoNEXT '16 Paper Acceptance Rate 30 of 160 submissions, 19%;

Overall Acceptance Rate 198 of 789 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

44
Total Citations
View Citations
893
Total Downloads

Downloads (Last 12 months)54
Downloads (Last 6 weeks)7

Other Metrics

View Author Metrics

Citations

Cited By

Ma QYuan HHu ZWang XYang Z(2024)A Liquidity Analysis System for Large-scale Video Streams in the OilfieldACM Transactions on Sensor Networks10.1145/364922220:3(1-22)Online publication date: 13-Apr-2024
https://dl.acm.org/doi/10.1145/3649222
Zhang LWu XWang FSun ACui LLiu J(2024)Edge-Based Video Stream Generation for Multi-Party Mobile Augmented RealityIEEE Transactions on Mobile Computing10.1109/TMC.2022.323254323:1(409-422)Online publication date: Jan-2024
https://doi.org/10.1109/TMC.2022.3232543
Li ZXu CZhang ZWu R(2024)Deep reinforcement learning based trajectory design and resource allocation for task-aware multi-UAV enabled MEC networksComputer Communications10.1016/j.comcom.2023.11.006213(88-98)Online publication date: Jan-2024
https://doi.org/10.1016/j.comcom.2023.11.006
Deng CFang XWang X(2023)UAV-Enabled Mobile-Edge Computing for AI Applications: Joint Model Decision, Resource Allocation, and Trajectory OptimizationIEEE Internet of Things Journal10.1109/JIOT.2022.315161910:7(5662-5675)Online publication date: 1-Apr-2023
https://doi.org/10.1109/JIOT.2022.3151619
Lan TZhao J(2023)Optimization in Mobile Augmented Reality Systems for the Metaverse Over Wireless CommunicationsGLOBECOM 2023 - 2023 IEEE Global Communications Conference10.1109/GLOBECOM54140.2023.10437658(5439-5444)Online publication date: 4-Dec-2023
https://doi.org/10.1109/GLOBECOM54140.2023.10437658
Ben Ali AKouroshli MSemenova SHashemifar ZKo SDantu K(2022)Edge-SLAM: Edge-Assisted Visual Simultaneous Localization and MappingACM Transactions on Embedded Computing Systems10.1145/356197222:1(1-31)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3561972
Zhang WLin SBijarbooneh FCheng HBraud TZhou PLee LHui P(2022)EdgeXAR: A 6-DoF Camera Multi-target Interaction Framework for MAR with User-friendly Latency CompensationProceedings of the ACM on Human-Computer Interaction10.1145/35322026:EICS(1-24)Online publication date: 17-Jun-2022
https://dl.acm.org/doi/10.1145/3532202
Ko SKim SJung HChoi S(2022)Computation Offloading and Service Caching for Mobile Edge Computing Under Personalized Service PreferenceIEEE Transactions on Wireless Communications10.1109/TWC.2022.315113121:8(6568-6583)Online publication date: Aug-2022
https://doi.org/10.1109/TWC.2022.3151131
Zhang WHan BHui P(2022)SEAR: Scaling Experiences in Multi-user Augmented RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.315046728:5(1982-1992)Online publication date: May-2022
https://doi.org/10.1109/TVCG.2022.3150467
Yi JKim SKim JChoi S(2022)Supremo: Cloud-Assisted Low-Latency Super-Resolution in Mobile DevicesIEEE Transactions on Mobile Computing10.1109/TMC.2020.302530021:5(1847-1860)Online publication date: 1-May-2022
https://doi.org/10.1109/TMC.2020.3025300
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents