research-article

DehazeGlasses: Optical Dehazing with an Occlusion Capable See-Through Display

Authors:

Takumi Kaminokado,

Yuta ItohAuthors Info & Claims

AHs '20: Proceedings of the Augmented Humans International Conference

Article No.: 3, Pages 1 - 11

https://doi.org/10.1145/3384657.3384781

Published: 06 June 2020 Publication History

Abstract

We present DehazeGlasses, a see-through visual haze removal system that optically dehazes the user's field of vision. Human vision suffers from a degraded view due to aspects of the scene environment, such as haze. Such degradation may interfere with our behavior or judgement in daily tasks. We focus on hazy scenes as one common degradation source, which whitens the view due to certain atmospheric conditions. Unlike typical computer vision systems that process recorded images, we aim to realize a see-through glasses system that can optically manipulate our field of view to dehaze the perceived scene. Our system selectively modulates the intensity of the light entering the eyes via occlusion-capable optical see-through head-mounted displays (OST-HMD). We built a proof-of-concept system to evaluate the feasibility of our haze removal method by combining a digital micromirror device (DMD) and an OST-HMD, and tested it with a user-perspective viewpoint camera. A quantitative evaluation with 80 scenes from a haze removal dataset shows that our system realizes a dehazed view that is significantly closer to the ground truth scene compared to the native view under a perceptual image similarity metric. This evaluation shows that our system achieves perceptually natural haze removal while maintaining the see-through view of actual scenes.

References

[1]

Cai Bolun, Xu Xiangmin, Jia Kui, Qing Chunmei, and Tao Dacheng. 2016. DehazeNet: An End-to-End System for Single Image Haze Removal. IEEE Transactions on Image Processing 25, 11 (2016), 5187--5198.

Digital Library

[2]

Ozan Cakmakci, Yonggang Ha, and Jannick P Rolland. 2004. A compact optical see-through head-worn display with occlusion support. In 3rd IEEE International Symposium on Mixed and Augmented Reality. IEEE, 16--25.

Digital Library

[3]

Earl. J. Cartney. 1977. Optics of the Atmosphere: Scattering by Molecules and Particles. Physics Bulletin 28, 11 (Nov 1977), 521--521. https://doi.org/10.1088/0031-9112/28/11/025

[4]

Dongdong Chen, Mingming He, Qingnan Fan, Jing Liao, Liheng Zhang, Dongdong Hou, Lu Yuan, and Gang Hua. 2018. Gated Context Aggregation Network for Image Dehazing and Deraining. CoRR abs/1811.08747 (2018). arXiv:1811.08747 http://arxiv.org/abs/1811.08747

[5]

T. Hamasaki and Y. Itoh. 2019. Varifocal Occlusion for Optical See-Through Head-Mounted Displays using a Slide Occlusion Mask. IEEE Transactions on Visualization and Computer Graphics 25, 5 (May 2019), 1961--1969. https://doi.org/10.1109/TVCG.2019.2899249

[6]

Takenori Hara, Hideo Saito, and Takeo Kanade. 2009. Removal of Glare Caused by Water Droplets. 2009 Conference for Visual Media Production (2009), 144--151.

[7]

Kaiming He, Jian Sun, and Xiaoou Tang. 2011. Single Image Haze Removal Using Dark Channel Prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 12 (Dec 2011), 2341--2353. https://doi.org/10.1109/TPAMI.2010.168

Digital Library

[8]

Yuichi Hiroi, Yuta Itoh, Takumi Hamasaki, and Maki Sugimoto. 2017. AdaptiVisor: Assisting Eye Adaptation via Occlusive Optical See-through Head-mounted Displays. In 8th Augmented Human International Conference (AH '17). ACM, Article 9, 9 pages. https://doi.org/10.1145/3041164.3041178

Digital Library

[9]

Isela Howlett and Quinn Smithwick. 2017. Perspective correct occlusion-capable augmented reality displays using cloaking optics constraints: Perspective correct occlusion-capable AR displays. Journal of the Society for Information Display (01 2017). https://doi.org/10.1002/jsid.545

[10]

Alex D Hwang and Eli Peli. 2014. An augmented-reality edge enhancement application for Google Glass. Optometry and vision science: official publication of the American Academy of Optometry 91, 8 (2014), 1021.

[11]

Yuta Itoh, Maksym Dzitsiuk, Toshiyuki Amano, and Gudrun Klinker. 2015. Semi-parametric color reproduction method for optical see-through head-mounted displays. IEEE Transactions on Visualization and Computer Graphics 21, 11 (2015), 1269--1278.

Digital Library

[12]

Yuta Itoh, Takumi Hamasaki, and Maki Sugimoto. 2017. Occlusion Leak Compensation for Optical See-Through Displays using a Single-layer Transmissive Spatial Light Modulator. IEEE Transaction on Visualization and Computer Graphics 23, 11 (2017), 2463--2473.

[13]

Yuta Itoh and Gudrun Klinker. 2014. Interaction-free calibration for optical see-through head-mounted displays based on 3d eye localization. In IEEE Symp. on 3D User Interfaces (3DUI). IEEE, 75--82.

[14]

Yuta Itoh, Tobias Langlotz, Daisuke Iwai, Kiyoshi Kiyokawa, and Toshiyuki Amano. 2019. Light Attenuation Display: Subtractive See-Through Near-Eye Display via Spatial Color Filtering. IEEE Transactions on Visualization and Computer Graphics 25, 5 (May 2019), 1951--1960. https://doi.org/10.1109/TVCG.2019.2899229

[15]

Kiyoshi Kiyokawa, Mark Billinghurst, Bruce Campbell, and Eric Woods. 2003. An occlusion-capable optical see-through head mount display for supporting co-located collaboration. In 2nd IEEE International Symposium on Mixed and Augmented Reality. IEEE Computer Society, 133.

Digital Library

[16]

Harald Koschmiere. 1924. Theorie der horizontalen sichtweite. Beitrage zur Physik der freien Atmosphare (1924), 33--53. https://ci.nii.ac.jp/naid/20001360955/

[17]

George Alex Koulieris, Kaan Akşit, Michael Stengel, Rafał K. Mantiuk, Katerina Mania, and Christian Richardt. 2019. Near-Eye Display and Tracking Technologies for Virtual and Augmented Reality. Computer Graphics Forum 38, 2 (May 2019), 493--519. https://doi.org/10.1111/cgf.13654

[18]

Tobias Langlotz, Matthew Cook, and Holger Regenbrecht. 2016. Real-Time Radiometric Compensation for Optical See-Through Head-Mounted Displays. IEEE Trans. on Visualization and Computer Graphics 22, 11 (2016), 2385--2394.

[19]

Tobias Langlotz, Jonathan Sutton, Stefanie Zollmann, Yuta Itoh, and Holger Regenbrecht. 2018. ChromaGlasses: Computational Glasses for Compensating Colour Blindness. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 390, 12 pages. https://doi.org/10.1145/3173574.3173964

Digital Library

[20]

Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, and Zhangyang Wang. 2019. Benchmarking Single-Image Dehazing and Beyond. IEEE Transactions on Image Processing 28, 1 (2019), 492--505.

Digital Library

[21]

Kai-En Lin, Kuang-Tsu Shih, and Homer Chen. 2017. Enhancing the perception of a hazy visual world using a see-through head-mounted device. In 2017 IEEE International Conference on Image Processing (ICIP). 4397--4401. https://doi.org/10.1109/ICIP.2017.8297113

[22]

Andrew Maimone and Henry Fuchs. 2013. Computational Augmented Reality Eyeglasses. In 12th IEEE International Symposium on Mixed and Augmented Reality. IEEE, 29--38.

[23]

Andrew Maimone, Douglas Lanman, Kishore Rathinavel, Kurtis Keller, David Luebke, and Henry Fuchs. 2014. Pinlight displays: wide field of view augmented reality eyeglasses using defocused point light sources. In ACM SIGGRAPH Emerging Technologies. ACM, 20.

[24]

Rafal Mantiuk, Karol Myszkowski, and Hans-Peter Seidel. 2006. A Perceptual Framework for Contrast Processing of High Dynamic Range Images. ACM Trans. Appl. Percept. 3, 3 (July 2006), 286--308. https://doi.org/10.1145/1166087.1166095

Digital Library

[25]

Srinivasa G. Narasimhan and Shree K. Nayar. 2002. Vision and the Atmosphere. International Journal of Computer Vision 48, 3 (01 Jul 2002), 233--254. https://doi.org/10.1023/A:1016328200723

Digital Library

[26]

Xu Qin, Zhilin Wang, Yuanchao Bai, Xiaodong Xie, and Huizhu Jia. 2019. FFA-Net: Feature Fusion Attention Network for Single Image Dehazing. arXiv:cs.CV/1911.07559

[27]

Kishore Rathinavel, Gordon Wetzstein, and Henry Fuchs. 2019. Varifocal Occlusion-Capable Optical See-through Augmented Reality Display based on Focus-tunable Optics. IEEE Transactions on Visualization and Computer Graphics 25, 11 (Nov 2019), 3125--3134. https://doi.org/10.1109/TVCG.2019.2933120

[28]

TJ Rhodes, Gavin Miller, Qi Sun, Daichi Ito, and Li-Yi Wei. 2019. A transparent display with per-pixel color and opacity control. In ACM SIGGRAPH 2019 Emerging Technologies. ACM, 5.

Digital Library

[29]

Jannick P Rolland and Henry Fuchs. 2000. Optical versus video see-through head-mounted displays in medical visualization. Presence: Teleoperators & Virtual Environments 9, 3 (2000), 287--309.

Digital Library

[30]

Jannick P Rolland, Richard L Holloway, and Henry Fuchs. 1995. Comparison of optical and video see-through, head-mounted displays. In Photonics for Industrial Applications. SPIE, 293--307.

[31]

Yoav Y. Schechner, Srinivasa G. Narasimhan, and Shree K. Nayar. 2001. Instant dehazing of images using polarization. In CVPR (1). 325--332.

[32]

Yoav Y Schechner, Srinivasa G Narasimhan, and Shree K Nayar. 2003. Polarization-based vision through haze. Applied optics 42, 3 (2003), 511--525.

[33]

Jonathan Sutton, Tobias Langlotz, and Yuta Itoh. 2019. Computational Glasses: Vision augmentations using computational near-eye optics and displays. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).

[34]

Robert Tamburo, Eriko Nurvitadhi, Abhishek Chugh, Mei Chen, Anthony Rowe, Takeo Kanade, and Srinivasa G Narasimhan. 2014. Programmable automotive headlights. In European Conference on Computer Vision. Springer, 750--765.

[35]

Gordon Wetzstein, Wolfgang Heidrich, and David Luebke. 2010. Optical image processing using light modulation displays. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 1934--1944.

[36]

Austin Wilson and Hong Hua. 2017. Design and prototype of an augmented reality display with per-pixel mutual occlusion capability. Optics express 25, 24 (2017), 30539--30549.

[37]

Yuta Yamaguchi and Yasuhiro Takaki. 2016. See-through integral imaging display with background occlusion capability. Applied Optics 55, 3 (2016), A144-A149.

[38]

Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

Cited By

Hong JLanglotz TSutton JRegenbrecht H(2024)Visual Noise Cancellation: Exploring Visual Discomfort and Opportunities for Vision AugmentationsACM Transactions on Computer-Human Interaction10.1145/363469931:2(1-26)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1145/3634699
Langlotz TSutton JRegenbrecht H(2024)A Design Space for Vision Augmentations and Augmented Human Perception using Digital EyewearProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642380(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642380
Hiroi YWatanabe AMikawa YItoh Y(2023)Low-Latency Beaming Display: Implementation of Wearable, 133 μs Motion-to-Photon Latency Near-Eye DisplayIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332021229:11(4761-4771)Online publication date: Nov-2023
https://doi.org/10.1109/TVCG.2023.3320212
Show More Cited By

Index Terms

DehazeGlasses: Optical Dehazing with an Occlusion Capable See-Through Display
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Mixed / augmented reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality

Recommendations

ChromaGlasses: Computational Glasses for Compensating Colour Blindness
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Prescription glasses are used by many people as a simple, and even fashionable way, to correct refractive problems of the eye. However, there are other visual impairments that cannot be treated with an optical lens in conventional glasses. In this work ...
AdaptiVisor: assisting eye adaptation via occlusive optical see-through head-mounted displays
AH '17: Proceedings of the 8th Augmented Human International Conference

Brightness adaptation is a fundamental ability in human visual system, and adjusts various levels of darkness and light. While this ability is continuously used, and it can mostly handle sudden lighting changes in the environment, the adaptation could ...
AR Timewarping: A Temporal Synchronization Framework for Real-Time Sensor Fusion in Head-Mounted Displays
AH '18: Proceedings of the 9th Augmented Human International Conference

A significant issue associated with the use of head-mounted displays for augmented reality is the presence of latency between the real world and the augmented images displayed to the headset, which, if uncompensated for, results in registration error ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AHs '20: Proceedings of the Augmented Humans International Conference

March 2020

296 pages

ISBN:9781450376037

DOI:10.1145/3384657

General Chairs:
Bo Zhou,
Jingyuan Cheng,
Program Chairs:
Paul Strohmeier,
Shoya Ishimaru,
Thomas Kosch

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AHs '20

AHs '20: Augmented Humans International Conference

March 16 - 17, 2020

Kaiserslautern, Germany

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
242
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hong JLanglotz TSutton JRegenbrecht H(2024)Visual Noise Cancellation: Exploring Visual Discomfort and Opportunities for Vision AugmentationsACM Transactions on Computer-Human Interaction10.1145/363469931:2(1-26)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1145/3634699
Langlotz TSutton JRegenbrecht H(2024)A Design Space for Vision Augmentations and Augmented Human Perception using Digital EyewearProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642380(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642380
Hiroi YWatanabe AMikawa YItoh Y(2023)Low-Latency Beaming Display: Implementation of Wearable, 133 μs Motion-to-Photon Latency Near-Eye DisplayIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332021229:11(4761-4771)Online publication date: Nov-2023
https://doi.org/10.1109/TVCG.2023.3320212
Sutton JLanglotz TPlopski A(2022)Seeing Colours: Addressing Colour Vision Deficiency with Vision Augmentations using Computational GlassesACM Transactions on Computer-Human Interaction10.1145/348689929:3(1-53)Online publication date: 30-Jun-2022
https://doi.org/10.1145/3486899
Itoh YLanglotz TSutton JPlopski A(2021)Towards Indistinguishable Augmented RealityACM Computing Surveys10.1145/345315754:6(1-36)Online publication date: 13-Jul-2021
https://dl.acm.org/doi/10.1145/3453157
Zhang YIsoyama NSakata NKiyokawa KHua H(2020)Super Wide-view Optical See-through Head Mounted Displays with Per-pixel Occlusion Capability2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR50242.2020.00056(301-311)Online publication date: Nov-2020
https://doi.org/10.1109/ISMAR50242.2020.00056

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents