research-article

Automatic noise modeling for ghost-free HDR reconstruction

Authors:

Miguel Granados,

Christian TheobaltAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 32, Issue 6

Article No.: 201, Pages 1 - 10

https://doi.org/10.1145/2508363.2508410

Published: 01 November 2013 Publication History

Abstract

High dynamic range reconstruction of dynamic scenes requires careful handling of dynamic objects to prevent ghosting. However, in a recent review, Srikantha et al. [2012] conclude that "there is no single best method and the selection of an approach depends on the user's goal". We attempt to solve this problem with a novel approach that models the noise distribution of color values. We estimate the likelihood that a pair of colors in different images are observations of the same irradiance, and we use a Markov random field prior to reconstruct irradiance from pixels that are likely to correspond to the same static scene object. Dynamic content is handled by selecting a single low dynamic range source image and hand-held capture is supported through homography-based image alignment. Our noise-based reconstruction method achieves better ghost detection and removal than state-of-the-art methods for cluttered scenes with large object displacements. As such, our method is broadly applicable and helps move the field towards a single method for dynamic scene HDR reconstruction.

Supplementary Material

ZIP File (a201-granados.zip)

Supplemental material.

Download
250.24 MB

References

[1]

Bay, H., Ess, A., Tuytelaars, T., and Gool, L. J. V. 2008. Speeded-up robust features (SURF). Computer Vision and Image Understanding 110, 3, 346--359.

Digital Library

[2]

Bogoni, L. 2000. Extending dynamic range of monochrome and color images through fusion. In Proc. ICPR, 3007--3016.

Digital Library

[3]

Boykov, Y., and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE TPAMI 26, 9, 1124--1137.

Digital Library

[4]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE TPAMI 23, 11.

Digital Library

[5]

Burt, P. J., and Kolczynski, R. J. 1993. Enhanced image capture through fusion. In Proc. ICCV, 173--182.

[6]

Drago, F., Myszkowski, K., Annen, T., and Chiba, N. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. CGF 22, 3, 419--426.

[7]

Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM TOG 21, 3.

Digital Library

[8]

Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6, 381--395.

Digital Library

[9]

Gallo, O., Gelfand, N., Chen, W.-C., Tico, M., and Pulli, K. 2009. Artifact-free high dynamic range imaging. In Proc. ICCP, 1--7.

[10]

Gallo, O., Tico, M., Manduchi, R., Gelfand, N., and Pulli, K. 2012. Metering for exposure stacks. CGF 31, 2.

Digital Library

[11]

Granados, M., Seidel, H.-P., and Lensch, H. P. A. 2008. Background estimation from non-time sequence images. In Proc. GI, 33--40.

Digital Library

[12]

Granados, M., Ajdin, B., Wand, M., Theobalt, C., Seidel, H.-P., and Lensch, H. P. A. 2010. Optimal HDR reconstruction with linear digital cameras. In Proc. CVPR, 215--222.

[13]

Grosch, T. 2006. Fast and robust high dynamic range image generation with camera and object movement. In Proc. VMV.

[14]

Heo, Y. S., Lee, K. M., Lee, S. U., Moon, Y., and Cha, J. 2010. Ghost-free high dynamic range imaging. In Proc. ACCV, vol. 4, 486--500.

Digital Library

[15]

Hu, J., Gallo, O., and Pulli, K. 2012. Exposure stacks of live scenes with hand-held cameras. In Proc. ECCV, 499--512.

Digital Library

[16]

Hubbard, W. M. 1970. The approximation of a Poisson distribution by a Gaussian distribution. Proc. IEEE 58, 9, 1374--1375.

[17]

Jacobs, K., Loscos, C., and Ward, G. 2008. Automatic high-dynamic range image generation for dynamic scenes. IEEE CGA 28, 2, 84--93.

Digital Library

[18]

Janesick, J. 2001. Scientific charge-coupled devices. SPIE Press.

[19]

Kang, S. B., Uyttendaele, M., Winder, S. A. J., and Szeliski, R. 2003. High dynamic range video. ACM TOG 22, 3, 319--325.

Digital Library

[20]

Khan, E. A., Akyüz, A. O., and Reinhard, E. 2006. Ghost removal in high dynamic range images. In Proc. ICIP.

[21]

Liu, C., Szeliski, R., Kang, S. B., Zitnick, C. L., and Freeman, W. T. 2008. Automatic estimation and removal of noise from a single image. IEEE TPAMI 30, 2, 299--314.

Digital Library

[22]

Mann, S., and Picard, R. 1995. Being 'undigital' with digital cameras: Extending dynamic range by combining differently exposed pictures. In Proc. IS&T, 422--428.

[23]

Menzel, N., and Guthe, M. 2007. Freehand HDR photography with motion compensation. In Proc. VMV, 127--134.

[24]

Min, T.-H., Park, R.-H., and Chang, S. 2009. Histogram based ghost removal in high dynamic range images. In Proc. ICME, 530--533.

Digital Library

[25]

Pece, F., and Kautz, J. 2010. Bitmap movement detection: HDR for dynamic scenes. In Proc. CVMP, 1--8.

Digital Library

[26]

Pedone, M., and Heikkilä, J. 2008. Constrain propagation for ghost removal in high dynamic range images. In Proc. VISAPP.

[27]

Raman, S., and Chaudhuri, S. 2010. Bottom-up segmentation for ghost-free reconstruction of a dynamic scene from multiexposure images. In Proc. ICVGIP, 56--63.

Digital Library

[28]

Reinhard, E., Ward, G., Pattanaik, S., and Debevec, P. 2005. High dynamic range imaging: Acquisition, display and image-based lighting. Morgan Kaufmann publishers.

Digital Library

[29]

Robertson, M., Borman, S., and Stevenson, R. 2003. Estimation-theoretic approach to dynamic range improvement using multiple exposures. J. Elec. Imag. 12, 2, 219--228.

[30]

Sen, P., Kalantari, N. K., Yaesoubi, M., Darabi, S., Goldman, D., and Shechtman, E. 2012. Robust patch-based hdr reconstruction of dynamic scenes. ACM TOG 31, 6.

Digital Library

[31]

Sidibé, D., Puech, W., and Strauss, O. 2009. Ghost detection and removal in high dynamic range images. In Proc. EUSIPCO.

[32]

Silk, S., and Lang, J. 2012. Fast high dynamic range image deghosting for arbitrary scene motion. In Proc. GI, 85--92.

Digital Library

[33]

Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In Proc. CVPR.

[34]

Srikantha, A., and Sidibé, D. 2012. Ghost detection and removal for high dynamic range images: Recent advances. Sig. Proc.: Image Comm. 27, 6, 650--662.

Digital Library

[35]

Tocci, M. D., Kiser, C., Tocci, N., and Sen, P. 2011. A versatile hdr video production system. ACM TOG 30, 4, 41.

Digital Library

[36]

Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839--846.

Digital Library

[37]

Tomaszewska, A. M., and Markowski, M. 2010. Dynamic scenes HDRI acquisition. In Proc. ICIAR, vol. 2, 345--354.

Digital Library

[38]

Veksler, O., Boykov, Y., and Mehrani, P. 2010. Superpixels and supervoxels in an energy optimization framework. In Proc. ECCV, vol. 6315, 211--224.

Digital Library

[39]

Zhang, W., and Cham, W.-K. 2012. Gradient-directed multiexposure composition. IEEE TIP 21, 4, 2318--2323.

[40]

Zimmer, H., Bruhn, A., and Weickert, J. 2011. Freehand HDR imaging of moving scenes with simultaneous resolution enhancement. CGF 30, 2, 405--414.

Cited By

Gan JHuo Y(2024)Ghost-free multi-exposure high dynamic range imaging based on feedback networkThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03072-w40:6(4115-4132)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s00371-023-03072-w
Xu FLiu JWang TWang XSun HRen H(2023)Multi-exposure image fusion of dynamic scenes using consistency detection and weighted least squares filteringOptical Engineering10.1117/1.OE.62.1.01310362:01Online publication date: 1-Jan-2023
https://doi.org/10.1117/1.OE.62.1.013103
Bolduc CGiroux JHébert MDemers CLalonde J(2023)Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00741(8037-8047)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.00741
Show More Cited By

Index Terms

Automatic noise modeling for ghost-free HDR reconstruction
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Video segmentation
      2. Computer vision tasks
        Scene understanding

Recommendations

An Analysis of Camera Noise

The class of cameras that are based on ionization sensors, which includes the most common charge-coupled device (CCD) and vidicon cameras, is examined. Camera signals are shown to be corrupted by direction-dependent stationary electronic noise sources ...
Real-time ghost free HDR video stream generation using weight adaptation based method
ICDSC '16: Proceedings of the 10th International Conference on Distributed Smart Camera

Temporal exposure bracketing is a simple and low cost technique to generate a high dynamic range (HDR) images. This technique is widely used to recover the whole dynamic range of a scene by selecting the adequate number of low dynamic range (LDR) images ...
Noise-Aware Merging of High Dynamic Range Image Stacks Without Camera Calibration
Computer Vision – ECCV 2020 Workshops
Abstract
A near-optimal reconstruction of the radiance of a High Dynamic Range scene from an exposure stack can be obtained by modeling the camera noise distribution. The latent radiance is then estimated using Maximum Likelihood Estimation. But this ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 32, Issue 6

November 2013

671 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2508363

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2013

Published in TOG Volume 32, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
1,021
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gan JHuo Y(2024)Ghost-free multi-exposure high dynamic range imaging based on feedback networkThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03072-w40:6(4115-4132)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s00371-023-03072-w
Xu FLiu JWang TWang XSun HRen H(2023)Multi-exposure image fusion of dynamic scenes using consistency detection and weighted least squares filteringOptical Engineering10.1117/1.OE.62.1.01310362:01Online publication date: 1-Jan-2023
https://doi.org/10.1117/1.OE.62.1.013103
Bolduc CGiroux JHébert MDemers CLalonde J(2023)Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00741(8037-8047)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.00741
Nie LLin JKang WShi YLin L(2023)Learning image blind denoisers without explicit noise modelingMultimedia Tools and Applications10.1007/s11042-023-14590-z82:18(27839-27859)Online publication date: 23-Feb-2023
https://dl.acm.org/doi/10.1007/s11042-023-14590-z
Yoon HUddin SJung Y(2022)Multi-Scale Attention-Guided Non-Local Network for HDR Image ReconstructionSensors10.3390/s2218704422:18(7044)Online publication date: 17-Sep-2022
https://doi.org/10.3390/s22187044
Xu FLiu JSong YSun HWang X(2022)Multi-Exposure Image Fusion Techniques: A Comprehensive ReviewRemote Sensing10.3390/rs1403077114:3(771)Online publication date: 7-Feb-2022
https://doi.org/10.3390/rs14030771
Zheng BPan XZhang HZhou XSlabaugh GYan CYuan SMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)DomainPlus: Cross Transform Domain Learning towards High Dynamic Range ImagingProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3547823(1954-1963)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3547823
Catley-Chandar STanay TVandroux LLeonardis ASlabaugh GPerez-Pellitero E(2022)FlexHDR: Modeling Alignment and Exposure Uncertainties for Flexible HDR ImagingIEEE Transactions on Image Processing10.1109/TIP.2022.320356231(5923-5935)Online publication date: 2022
https://doi.org/10.1109/TIP.2022.3203562
Chen JYang ZChan TLi HHou JChau L(2022)Attention-Guided Progressive Neural Texture Fusion for High Dynamic Range Image RestorationIEEE Transactions on Image Processing10.1109/TIP.2022.316007031(2661-2672)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1109/TIP.2022.3160070
Tan XChen HXu KXu CJin YZhu CZheng J(2022)High Dynamic Range Imaging for Dynamic Scenes With Large-Scale Motions and Severe SaturationIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2022.314420571(1-15)Online publication date: 2022
https://doi.org/10.1109/TIM.2022.3144205
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents