research-article

Open access

SunBox: Screen-to-Camera Communication with Ambient Light

Authors:

Miguel Chávez Tapia,

Marco Zúñiga ZamalloaAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 6, Issue 2

Article No.: 46, Pages 1 - 26

https://doi.org/10.1145/3534602

Published: 07 July 2022 Publication History

Abstract

A recent development in wireless communication is the use of optical shutters and smartphone cameras to create optical links solely from ambient light. At the transmitter, a liquid crystal display (LCD) modulates ambient light by changing its level of transparency. At the receiver, a smartphone camera decodes the optical pattern. This LCD-to-camera link requires low-power levels at the transmitter, and it is easy to deploy because it does not require modifying the existing lighting infrastructure. The system, however, provides a low data rate, of just a few tens of bps. This occurs because the LCDs used in the state-of-the-art are slow single-pixel transmitters. To overcome this limitation, we introduce a novel multi-pixel display. Our display is similar to a simple screen, but instead of using embedded LEDs to radiate information, it uses only the surrounding ambient light. We build a prototype, called SunBox, and evaluate it indoors and outdoors with both, artificial and natural ambient light. Our results show that SunBox can achieve a throughput between 2 kbps and 10 kbps using a low-end smartphone camera with just 30 FPS. To the best of our knowledge, this is the first screen-to-camera system that works solely with ambient light.

References

[1]

Basler AG. 2021. Overlapping Image Acquisition. Retrieved November 12, 2021 from https://docs.baslerweb.com/overlapping-image-acquisition

[2]

Karan Ahuja, Sujeath Pareddy, Robert Xiao, Mayank Goel, and Chris Harrison. 2019. LightAnchors: Appropriating Point Lights for Spatially-Anchored Augmented Reality Interfaces. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST '19). ACM, New York, NY, USA, 189--196. https://doi.org/10.1145/3332165.3347884

Digital Library

[3]

Rens Bloom, Marco Zúñiga Zamalloa and Chaitra Pai. 2019. LuxLink: Creating a Wireless Link from Ambient Light. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems (New York, New York, USA) (SenSys '19). ACM, New York, NY, USA, 166--178. https://doi.org/10.1145/3356250.3360021

Digital Library

[4]

Chun-Ling Chan, Hsin-Mu Tsai, and Kate Ching-Ju Lin. 2017. POLI: Long-Range Visible Light Communications Using Polarized Light Intensity Modulation. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services (Niagara Falls, New York, USA) (MobiSys '17). ACM, New York, NY, USA, 109--120. https://doi.org/10.1145/3081333.3081353

Digital Library

[5]

Ryan Chylinski. 2012. Time-lapse photography: A Complete Introduction to Shooting, Processing, and Rendering Time-lapse Movies with a DSLR Camera. Cedar Wings Creative.

[6]

Trong-Hop Do and Myungsik Yoo. 2017. Two-Dimensional Coding for Optical Camera Communication using CMOS Sensors. In International Conference on Information and Communications 2017 (Hanoi, Vietnam) (ICIC '17). IEEE, New York, NY, USA, 314. https://doi.org/10.1109/ICIC42290.2017

[7]

David Doroski, Stephen H Perlmutter, and Garret Moddel. 1994. Alignment layers for improved surface-stabilized ferroelectric liquid-crystal devices. Appl. Opt. 33, 13 (1994), 2608--2610. https://doi.org/10.1364/AO.33.002608

[8]

Wan Du, Jansen Christian Liando, and Mo Li. 2016. Softlight: Adaptive visible light communication over screen-camera links. In IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications (San Francisco, CA, USA) (INFOCOM '16). IEEE, New York, NY, USA, 1--9. https://doi.org/10.1109/INFOCOM.2016.7524510

Digital Library

[9]

Seyed Keyarash Ghiasi, Marco A. Zúñiga Zamalloa, and Koen Langendoen. 2021. A Principled Design for Passive Light Communication. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking (New Orleans, Louisiana) (MobiCom '21). ACM, New York, NY, USA, 121--133. https://doi.org/10.1145/3447993.3448629

Digital Library

[10]

Qi Guo, Kexin Yan, Vladimir Chigrinov, Huijie Zhao, and Michael Tribelsky. 2019. Ferroelectric Liquid Crystals: Physics and Applications. Crystals 9, 9, Article 470 (2019), 21 pages. https://doi.org/10.3390/cryst9090470

[11]

Tian Hao, Ruogu Zhou, and Guoliang Xing. 2012. COBRA: Color Barcode Streaming for Smartphone Systems. In Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services (Low Wood Bay, Lake District, UK) (MobiSys '12). ACM, New York, NY, USA, 85--98. https://doi.org/10.1145/2307636.2307645

Digital Library

[12]

Mostafa Izz, Zhongyuan Li, Hongbo Liu, Yingying Chen, and Feng Li. 2016. Uber-in-light: Unobtrusive visible light communication leveraging complementary color channel. In IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications (San Francisco, CA, USA) (INFOCOM '16). IEEE, New York, NY, USA, 1--9. https://doi.org/10.1109/INFOCOM.2016.7524513

Digital Library

[13]

Weiwei Jiang, Denzil Ferreira, Jani Ylioja, Jorge Goncalves, and Vassilis Kostakos. 2014. Pulse: Low Bitrate Wireless Magnetic Communication for Smartphones. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Seattle, Washington) (UbiComp '14). ACM, New York, NY, USA, 261--265. https://doi.org/10.1145/2632048.2632094

Digital Library

[14]

J.C. Jones, M.J. Towler, and J.R. Hughes. 1993. Fast, high-contrast ferroelectric liquid crystal displays and the role of dielectric biaxiality. Displays 14, 2 (April 1993), 86--93. https://doi.org/10.1016/0141-9382(93)90075-G

[15]

Sung-Yoon Jung, Ji-Hwan Lee, Wonwoo Nam, and Byung Wook Kim. 2020. Complementary Color Barcode-Based Optical Camera Communications. Wireless Communications and Mobile Computing 2020, 1, Article 3898427 (Feb. 2020), 8 pages. https://doi.org/10.1155/2020/3898427

[16]

Ye-Sheng Kuo, Pat Pannuto, Ko-Jen Hsiao, and Prabal Dutta. 2014. Luxapose: Indoor Positioning with Mobile Phones and Visible Light. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (Maui, Hawaii, USA) (MobiCom '14). ACM, New York, NY, USA, 447--458. https://doi.org/10.1145/2639108.2639109

Digital Library

[17]

Jiangtao Li, Angli Liu, Guobin Shen, Liqun Li, Chao Sun, and Feng Zhao. 2015. Retro-VLC: Enabling Battery-Free Duplex Visible Light Communication for Mobile and IoT Applications. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications (Santa Fe, New Mexico, USA) (HotMobile '15). ACM, New York, NY, USA, 21--26. https://doi.org/10.1145/2699343.2699354

Digital Library

[18]

Liqiong Liu, Rui Deng, Jin Shi, Jing He, and Lian-Kuan Chen. 2020. Beyond 100-kbit/s Transmission over Rolling Shutter Camera-based VLC Enabled by Color and Spatial Multiplexing. In Proceedings of the Optical Fiber Communication Conference (OFC) 2020 (San Diego, California, USA) (OFC '20). Optica Publishing Group, Washington DC, USA, M1J.4. https://doi.org/10.1364/OFC.2020.M1J.4

[19]

Wenkai Liu, Biqi Wang, Yan Li, and Menglong Wu. 2020. Screen-Camera Communication System Based on Dynamic QR Code. IOP Conference Series: Materials Science and Engineering 790, 1, Article 012012 (March 2020), 6 pages. https://doi.org/10.1088/1757-899X/790/1/012012

[20]

Control Electronic Co. Ltd. 2020. 0.2 inch FLCOS Micro Display Module for 720x540 HMD, Video Glasses. Retrieved November 12, 2021 from http://www.szcontrol.com/Index/pdetail/id/305.html

[21]

Antonio Martínez-García, Ignacio Moreno, María M. Sánchez-López, and Pascuala García-Martínez. 2009. Operational modes of a ferroelectric LCoS modulator for displaying binary polarization, amplitude, and phase diffraction gratings. Appl. Opt. 48, 15 (May 2009), 2903--2914. https://doi.org/10.1364/AO.48.002903

[22]

Hao Pan, Yi-Chao Chen, Guangtao Xue, and Xiaoyu Ji. 2017. MagneComm: Magnetometer-Based Near-Field Communication. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking (Snowbird, Utah, USA) (MobiCom '17). ACM, New York, NY, USA, 167--179. https://doi.org/10.1145/3117811.3117824

Digital Library

[23]

Samuel David Perli, Nabeel Ahmed, and Dina Katabi. 2010. PixNet: Interference-Free Wireless Links Using LCD-Camera Pairs. In Proceedings of the Sixteenth Annual International Conference on Mobile Computing and Networking (Chicago, Illinois, USA) (MobiCom '10). ACM, New York, NY, USA, 137--148. https://doi.org/10.1145/1859995.1860012

Digital Library

[24]

Samsung. 2017. Reflective Display Technology. Retrieved November 12, 2021 from https://pid.samsungdisplay.com/en/learning-center/blog/reflective-display-technology

[25]

Shuyu Shi, Lin Chen, Wenjun Hu, and Marco Gruteser. 2015. Reading between Lines: High-Rate, Non-Intrusive Visual Codes within Regular Videos via ImplicitCode. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Osaka, Japan) (UbiComp '15). ACM, New York, NY, USA, 157--168. https://doi.org/10.1145/2750858.2805824

Digital Library

[26]

Zhao Tian, Charles J. Carver, Qijia Shao, Monika Roznere, Alberto Quattrini Li, and Xia Zhou. 2020. PolarTag: Invisible Data with Light Polarization. In Proceedings of the 21st International Workshop on Mobile Computing Systems and Applications (Austin, TX, USA) (HotMobile '20). ACM, New York, NY, USA, 74--79. https://doi.org/10.1145/3376897.3377854

Digital Library

[27]

Anran Wang, Zhuoran Li, Chunyi Peng, Guobin Shen, Gan Fang, and Bing Zeng. 2015. InFrame++: Achieve Simultaneous Screen-Human Viewing and Hidden Screen-Camera Communication. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (Florence, Italy) (MobiSys '15). ACM, New York, NY, USA, 181--195. https://doi.org/10.1145/2742647.2742652

Digital Library

[28]

Anran Wang, Shuai Ma, Chunming Hu, Jinpeng Huai, Chunyi Peng, and Guobin Shen. 2014. Enhancing Reliability to Boost the Throughput over Screen-Camera Links. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (Maui, Hawaii, USA) (MobiCom '14). ACM, New York, NY, USA, 41--52. https://doi.org/10.1145/2639108.2639135

Digital Library

[29]

Purui Wang, Lilei Feng, Guojun Chen, Chenren Xu, Yue Wu, Kenuo Xu, Guobin Shen, Kuntai Du, Gang Huang, and Xuanzhe Liu. 2020. Renovating Road Signs for Infrastructure-to-Vehicle Networking: A Visible Light Backscatter Communication and Networking Approach. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking (London, United Kingdom) (MobiCom '20). ACM, New York, NY, USA, Article 6, 13 pages. https://doi.org/10.1145/3372224.3380883

Digital Library

[30]

Maury Wright. 2015. Philips Lighting deploys LED-based indoor positioning in Carrefour hypermarket. Retrieved November 12, 2021 from https://goo.gl/a0tGIj

[31]

Yue Wu, Purui Wang, Kenuo Xu, Lilei Feng, and Chenren Xu. 2020. Turboboosting Visible Light Backscatter Communication. In Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication (Virtual Event, USA) (SIGCOMM '20). ACM, New York, NY, USA, 186--197. https://doi.org/10.1145/3387514.3406229

Digital Library

[32]

Xieyang Xu, Yang Shen, Junrui Yang, Chenren Xu, Guobin Shen, Guojun Chen, and Yunzhe Ni. 2017. PassiveVLC: Enabling Practical Visible Light Backscatter Communication for Battery-Free IoT Applications. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking (Snowbird, Utah, USA) (MobiCom '17). ACM, New York, NY, USA, 180--192. https://doi.org/10.1145/3117811.3117843

Digital Library

[33]

Jackie (Junrui) Yang and James A. Landay. 2019. InfoLED: Augmenting LED Indicator Lights for Device Positioning and Communication. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST '19). ACM, New York, NY, USA, 175--187. https://doi.org/10.1145/3332165.3347954

Digital Library

[34]

Yanbing Yang, Jun Luo, Chen Chen, Zequn Chen, Wen-De Zhong, and Liangyin Chen. 2021. Pushing the Data Rate of Practical VLC via Combinatorial Light Emission. IEEE Transactions on Mobile Computing 20, 5 (May 2021), 1979--1992. https://doi.org/10.1109/TMC.2020.2971204

[35]

Zhice Yang, Zeyu Wang, Jiansong Zhang, Chenyu Huang, and Qian Zhang. 2015. Wearables Can Afford: Light-Weight Indoor Positioning with Visible Light. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (Florence, Italy) (MobiSys '15). ACM, New York, NY, USA, 317--330. https://doi.org/10.1145/2742647.2742648

Digital Library

[36]

Bingsheng Zhang, Kui Ren, Guoliang Xing, Xinwen Fu, and Cong Wang. 2015. SBVLC: Secure barcode-based visible light communication for smartphones. IEEE Transactions on Mobile Computing 15, 2 (Feb. 2015), 432--446. https://doi.org/10.1109/TMC.2015.2413791

Digital Library

[37]

Kai Zhang, Yi Zhao, Chenshu Wu, Chaofan Yang, Kehong Huang, Chunyi Peng, Yunhao Liu, and Zheng Yang. 2021. ChromaCode: A Fully Imperceptible Screen-Camera Communication System. IEEE Transactions on Mobile Computing 20, 3 (March 2021), 861--876. https://doi.org/10.1109/TMC.2019.2956493

[38]

Ouyang Zhang, Zhenzhi Qian, Yifan Mao, Kannan Srinivasan, and Ness B. Shroff. 2019. ERSCC: Enable Efficient and Reliable Screen-Camera Communication. In Proceedings of the Twentieth ACM International Symposium on Mobile Ad Hoc Networking and Computing (Catania, Italy) (Mobihoc '19). ACM, New York, NY, USA, 281--290. https://doi.org/10.1145/3323679.3326526

Digital Library

Cited By

Kim YJung S(2024)Interpolation-based reference image estimation for video display field communicationOptics Express10.1364/OE.52801332:14(24643)Online publication date: 21-Jun-2024
https://doi.org/10.1364/OE.528013
Zhou MZhou WHuang JYang JDu MLi Q(2024)Stealthy and Effective Physical Adversarial Attacks in Autonomous DrivingIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.342292019(6795-6809)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3422920
Ghiasi SKaldenbach MZuniga M(2024)Passive Screen-to-Camera Communication2024 20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT)10.1109/DCOSS-IoT61029.2024.00016(35-43)Online publication date: 29-Apr-2024
https://doi.org/10.1109/DCOSS-IoT61029.2024.00016
Show More Cited By

Index Terms

SunBox: Screen-to-Camera Communication with Ambient Light
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Embedded systems

Recommendations

Real-Time Screen-Camera Communication Behind Any Scene
MobiSys '15: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services

We present HiLight, a new form of real-time screen-camera communication without showing any coded images (e.g., barcodes) for off-the-shelf smart devices. HiLight encodes data into pixel translucency change atop any screen content, so that camera-...
Using consumer LED light bulbs for low-cost visible light communication systems
VLCS '14: Proceedings of the 1st ACM MobiCom workshop on Visible light communication systems

LED-to-LED Visible Light Communication (VLC) based on Light Emitting Diodes (LEDs) and microcontrollers provide a foundation for networking using visible light as communication medium. We describe a low-complexity smart LED light bulb prototype that is ...
LED-to-LED visible light communication networks
MobiHoc '13: Proceedings of the fourteenth ACM international symposium on Mobile ad hoc networking and computing

Visible Light Communication (VLC) with Light Emitting Diodes (LEDs) as transmitters and receivers enables low bitrate wireless adhoc networking. LED-to-LED VLC adhoc networks with VLC devices communicating with each other over free-space optical links ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 6, Issue 2

June 2022

1551 pages

EISSN:2474-9567

DOI:10.1145/3547347

Issue’s Table of Contents

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Published in IMWUT Volume 6, Issue 2

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
664
Total Downloads

Downloads (Last 12 months)212
Downloads (Last 6 weeks)31

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kim YJung S(2024)Interpolation-based reference image estimation for video display field communicationOptics Express10.1364/OE.52801332:14(24643)Online publication date: 21-Jun-2024
https://doi.org/10.1364/OE.528013
Zhou MZhou WHuang JYang JDu MLi Q(2024)Stealthy and Effective Physical Adversarial Attacks in Autonomous DrivingIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.342292019(6795-6809)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3422920
Ghiasi SKaldenbach MZuniga M(2024)Passive Screen-to-Camera Communication2024 20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT)10.1109/DCOSS-IoT61029.2024.00016(35-43)Online publication date: 29-Apr-2024
https://doi.org/10.1109/DCOSS-IoT61029.2024.00016
Ghiasi SDsouza VLangendoen KZuniga M(2023)SpectraLux: Towards Exploiting the Full Spectrum with Passive VLCProceedings of the 22nd International Conference on Information Processing in Sensor Networks10.1145/3583120.3586966(274-287)Online publication date: 9-May-2023
https://dl.acm.org/doi/10.1145/3583120.3586966
Han HXie KWang TZhu XZhao YXu F(2023)RescQR: Enabling Reliable Data Recovery in Screen-Camera Communication SystemIEEE Transactions on Mobile Computing10.1109/TMC.2023.327721223:5(3510-3522)Online publication date: 17-May-2023
https://dl.acm.org/doi/10.1109/TMC.2023.3277212
Ullah MGelli GVerde F(2023)Visible light backscattering with applications to the Internet of Things: State-of-the-art, challenges, and opportunitiesInternet of Things10.1016/j.iot.2023.10076822(100768)Online publication date: Jul-2023
https://doi.org/10.1016/j.iot.2023.100768
Kim YSingh PJung S(2022)Experimental Evaluation of Display Field Communication Based on Machine Learning and Modem DesignApplied Sciences10.3390/app12231222612:23(12226)Online publication date: 29-Nov-2022
https://doi.org/10.3390/app122312226

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents