research-article

InFrame++: Achieve Simultaneous Screen-Human Viewing and Hidden Screen-Camera Communication

Authors:

Bing ZengAuthors Info & Claims

MobiSys '15: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services

Pages 181 - 195

https://doi.org/10.1145/2742647.2742652

Published: 18 May 2015 Publication History

Abstract

Recent efforts in visible light communication over screen-camera links have exploited the display for data communication. Such practices, albeit convenient, have led to contention between space allocated for users and content reserved for devices, in addition to their visual anti-aesthetics and distractedness. In this paper, we propose INFRAME++, a system that enables concurrent, dual-mode, full-frame communication for both users and devices. INFRAME++ leverages the spatial-temporal flicker-fusion property of human vision system and the fast frame rate of modern display. It multiplexes data onto full-frame video contents through novel complementary frame composition, hierarchical frame structure, and CDMA-like modulation. It thus ensures opportunistic and unobtrusive screen-camera data communication without affecting the primary video-viewing experience for human users. Our prototype and experiments have confirmed its effectiveness of delivering data to devices in its visual communication with imperceptible video artifacts for viewers. INFRAME++ is able to achieve 150-240 kbps at 120FPS over a 24? LCD monitor with one data frame per 12 display frames. It supports up to 360kbps while data:video is 1:6.

References

[1]

Cisco Visual Networking Index: Forecast and Methodology, 2013-2018. http://tinyurl.com/mev32z8.

[2]

Walsh function. http://en.wikipedia.org/wiki/Walsh_function.

[3]

Wechat.

[4]

D. A. Atchison, G. Smith, and G. Smith. Optics of the human eye. Butterworth-Heinemann Oxford, UK:, 2000.

[5]

R. Balaji and G. Naveen. Secure data transmission using video steganography. In IEEE International Conference on Electro/Information Technology (EIT), 2011.

[6]

G. Brindley, J. Du Croz, and W. Rushton. The flicker fusion frequency of the blue-sensitive mechanism of colour vision. The Journal of physiology, 183(2):497--500, 1966.

[7]

R. Carvalho, C.-H. Chu, and L.-J. Chen. IVC: Imperceptible video communication. 2014. Demo.

[8]

A. Cheddad, J. Condell, K. Curran, and P. McKevitt. Digital image steganography: Survey and analysis of current methods. Signal Processing, 90(3):727--752, 2010.

Digital Library

[9]

J. Fridrich and M. Goljan. Digital image steganography using stochastic modulation. In Electronic Imaging, pages 191--202, 2003.

[10]

D. G. Green. Sinusoidal flicker characteristics of the color-sensitive mechanisms of the eye. Vision research, 9(5):591--601, 1969.

[11]

T. Hao, R. Zhou, and G. Xing. Cobra: Color barcode streaming for smartphone systems. In MobiSys, 2012.

Digital Library

[12]

J. He, J. Huang, and G. Qiu. A new approach to estimating hidden message length in stochastic modulation steganography. In Digital Watermarking, pages 1--14. Springer, 2005.

Digital Library

[13]

W. A. Hershberger and J. S. Jordan. The phantom array: a perisaccadic illusion of visual direction. The Psychological Record, 48(1):2, 2012.

[14]

W. Hu, H. Gu, and Q. Pu. Lightsync: Unsynchronized visual communication over screen-camera links. In MobiCom, 2013.

Digital Library

[15]

W. Hu, J. Mao, Z. Huang, Y. Xue, J. She, K. Bian, and G. Shen. Strata: Layered Coding for Scalable Visual Communication. In MobiCom, 2014.

Digital Library

[16]

I18004:2000. Automatic identification and data capture techniques - Bar code symbology - QR Code.

[17]

R. Kavitha and A. Murugan. Lossless steganography on avi file using swapping algorithm. In Conference on Computational Intelligence and Multimedia Applications, volume 4, 2007.

Digital Library

[18]

D. Kelly. Flicker. In Visual psychophysics, pages 273--302, 1972.

[19]

T. Li, C. An, A. Campbell, and X. Zhoun. Hilight: Hiding bits in pixel translucency changes. In ACM Workshop on Visible Light Communication Systems (VLCS), 2014.

Digital Library

[20]

X. Liao, Q.-y. Wen, and J. Zhang. A steganographic method for digital images with four-pixel differencing and modified lsb substitution. Journal of Visual Communication and Image Representation, 22(1):1--8, 2011.

Digital Library

[21]

NVIDIA. Directcompute. https://developer.nvidia.com/directcompute.

[22]

S. D. Perli, N. Ahmed, and D. Katabi. Pixnet: interference-free wireless links using lcd-camera pairs. In MobiCom, 2010.

Digital Library

[23]

F. A. P. Petitcolas, R. J. Anderson, and M. G. Kuhn. Information hiding - a survey. Proceedings of the IEEE, 87(7):1062--1078, July 1999.

[24]

N. Rajagopal, P. Lazik, and A. Rowe. Visual light landmarks for mobile devices. In ISPN, 2014.

Digital Library

[25]

J. Roberts and A. Wilkins. Flicker can be perceived during saccades at frequencies in excess of 1 khz. Lighting Research and Technology, 45(1):124--132, 2013.

[26]

T. Sharp. An implementation of key-based digital signal steganography. In Information hiding, pages 13--26. Springer, 2001.

Digital Library

[27]

E. Simonson and J. Bro?ek. Flicker fusion frequency: background and applications. Physiological reviews, 1952.

[28]

R. D. Valois and K. D. Valois. Spatial Vision. Oxford University Press, 1988.

[29]

I. Vogels and I. Hernando. Effect of eye movements on perception of temporally modulated light. http://2012.experiencinglight.nl/doc/28.pdf.

[30]

A. Wang, S. Ma, C. Hu, J. Huai, C. Peng, and G. Shen. Enhancing Reliability to Boost the Throughput over Screen-camera Links. In MobiCom, 2014.

Digital Library

[31]

A. Wang, C. Peng, O. Zhang, G. Shen, and B. Zeng. InFrame: Multiflexing Full-Frame Visible Communication Channel for Humans and Devices. In HotNets-XIII, 2014.

Digital Library

[32]

S. B. Wicker. Reed-Solomon Codes and Their Applications. 1994.

Digital Library

[33]

wikipedia. Forward error correction.

[34]

G. Woo, A. Lippman, and R. Raskar. Vrcodes: Unobtrusive and active visual codes for interaction by exploiting rolling shutter. In IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2012.

Digital Library

[35]

H.-C. Wu, N.-I. Wu, C.-S. Tsai, and M.-S. Hwang. Image steganographic scheme based on pixel-value differencing and lsb replacement methods. IEE Proceedings-Vision, Image and Signal Processing, 152(5):611--615, 2005.

[36]

W. Yuan, K. Dana, A. Ashok, M. Gruteser, and N. Mandayam. Dynamic and invisible messaging for visual mimo. In IEEE Workshop on Applications of Computer Vision (WACV), 2012.

Digital Library

[37]

Y. Zhang. Digital watermarking technology: A review. In Future Computer and Communication (FCC), 2009.

Digital Library

Cited By

Kakui YAraki KHan CFukushima SNaemura T(2025)[Paper] Robust Recognition of Imperceptible QR Codes in Videos with a Dual-Camera SmartphoneITE Transactions on Media Technology and Applications10.3169/mta.13.16613:1(166-178)Online publication date: 2025
https://doi.org/10.3169/mta.13.166
Singh PKim YKim BJung S(2024)Display Field Communication: Enabling Seamless Data Exchange in Screen–Camera EnvironmentsPhotonics10.3390/photonics1111100011:11(1000)Online publication date: 24-Oct-2024
https://doi.org/10.3390/photonics11111000
Wang YShen YXu KHassan MZhao GXu CHu WShu YLiu JTan RHe YChen J(2024)Towards High-Speed Passive Visible Light Communication with Event Cameras and Digital Micro-MirrorsProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699368(704-717)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699368
Show More Cited By

Index Terms

InFrame++: Achieve Simultaneous Screen-Human Viewing and Hidden Screen-Camera Communication

Recommendations

InFrame: Multiflexing Full-Frame Visible Communication Channel for Humans and Devices
HotNets-XIII: Proceedings of the 13th ACM Workshop on Hot Topics in Networks

Recent efforts in visible light communication over screen-camera links have exploited the display for data communications. Such practices, albeit convenient, have led to contention between space allocated for users and content reserved for devices, in ...
ChromaCode: A Fully Imperceptible Screen-Camera Communication System
MobiCom '18: Proceedings of the 24th Annual International Conference on Mobile Computing and Networking

Hidden screen-camera communication techniques emerge as a new paradigm that embeds data imperceptibly into regular videos while remaining unobtrusive to human viewers. Three key goals on imperceptible, high rate, and reliable communication are desirable ...
Demo: Achieving Simultaneous Screen-Human Viewing and Hidden Screen-Camera Communication
MobiSys '15: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services

We present and demonstrate INFRAME++, a novel system that enables concurrent, dual-mode, full-frame communication for both users and devices. It achieves unobtrusive screen-camera data communication without affecting the primary video-viewing experience ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiSys '15: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services

May 2015

516 pages

ISBN:9781450334945

DOI:10.1145/2742647

General Chairs:
Gaetano Borriello
University of Washington, USA
,
Giovanni Pau
UPMC-LIP6, France / UCLA, USA
,
Program Chairs:
Marco Gruteser
Rutgers University, USA
,
Jason Hong
Carnegie Mellon University, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

In-Cooperation

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 May 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Conference

MobiSys'15

Sponsor:

SIGMOBILE

MobiSys'15: The 13th Annual International Conference on Mobile Systems, Applications, and Services

May 18 - 22, 2015

Florence, Italy

Acceptance Rates

MobiSys '15 Paper Acceptance Rate 29 of 219 submissions, 13%;

Overall Acceptance Rate 274 of 1,679 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

98
Total Citations
View Citations
1,070
Total Downloads

Downloads (Last 12 months)68
Downloads (Last 6 weeks)11

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kakui YAraki KHan CFukushima SNaemura T(2025)[Paper] Robust Recognition of Imperceptible QR Codes in Videos with a Dual-Camera SmartphoneITE Transactions on Media Technology and Applications10.3169/mta.13.16613:1(166-178)Online publication date: 2025
https://doi.org/10.3169/mta.13.166
Singh PKim YKim BJung S(2024)Display Field Communication: Enabling Seamless Data Exchange in Screen–Camera EnvironmentsPhotonics10.3390/photonics1111100011:11(1000)Online publication date: 24-Oct-2024
https://doi.org/10.3390/photonics11111000
Wang YShen YXu KHassan MZhao GXu CHu WShu YLiu JTan RHe YChen J(2024)Towards High-Speed Passive Visible Light Communication with Event Cameras and Digital Micro-MirrorsProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699368(704-717)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699368
Xiao RZhao LQian FYang LHan JOkoshi TKo JLiKamWa R(2024)Practical Optical Camera Communication Behind Unseen and Complex BackgroundsProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661866(113-126)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661866
Ben BZhang Z(2024)MicroFrame: A Multi-Complementary Color-Based Spatial Diversity System for Imperceptible and Robust Screen-Camera Communication2024 16th International Conference on Wireless Communications and Signal Processing (WCSP)10.1109/WCSP62071.2024.10826996(1368-1373)Online publication date: 24-Oct-2024
https://doi.org/10.1109/WCSP62071.2024.10826996
Kibayashi YAraki KAkashi YHan CNaemura T(2024)Quadrature Color Vibration Method for Improving Robustness of Imperceptible Markers Embedded in Video2024 IEEE International Conference on Visual Communications and Image Processing (VCIP)10.1109/VCIP63160.2024.10849839(1-5)Online publication date: 8-Dec-2024
https://doi.org/10.1109/VCIP63160.2024.10849839
Dang FXu YXu RChen XLiu Y(2024)LSync: A Universal Timeline-Synchronizing Solution for Live StreamingIEEE/ACM Transactions on Networking10.1109/TNET.2024.340814732:5(4144-4159)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3408147
Liao ZLuo ZHuang QZhang LWu FZhang QChen G(2024)Gesture Recognition Using Visible Light on Mobile DevicesIEEE/ACM Transactions on Networking10.1109/TNET.2024.336999632:4(2920-2935)Online publication date: Aug-2024
https://doi.org/10.1109/TNET.2024.3369996
Han HXie KWang TZhu XZhao YXu F(2024)RescQR: Enabling Reliable Data Recovery in Screen-Camera Communication SystemIEEE Transactions on Mobile Computing10.1109/TMC.2023.3277212(1-13)Online publication date: 2024
https://doi.org/10.1109/TMC.2023.3277212
Zhang HYu XZhang ZZhu B(2024)Robust and Imperceptible Commercial Camera-Screen Communication with 60Hz Refresh RateICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446206(4910-4914)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10446206
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

EPUB

View this article in ePub.

Figures

Tables

Media

View Table of Conten