research-article

Layered Light Field Reconstruction for Defocus Blur

Authors:

Karthik Vaidyanathan,

Jacob Munkberg,

Petrik Clarberg,

Marco SalviAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 34, Issue 2

Article No.: 23, Pages 1 - 12

https://doi.org/10.1145/2699647

Published: 02 March 2015 Publication History

Abstract

We present a novel algorithm for reconstructing high-quality defocus blur from a sparsely sampled light field. Our algorithm builds upon recent developments in the area of sheared reconstruction filters and significantly improves reconstruction quality and performance. While previous filtering techniques can be ineffective in regions with complex occlusion, our algorithm handles such scenarios well by partitioning the input samples into depth layers. These depth layers are filtered independently and then combined together, taking into account inter-layer visibility. We also introduce a new separable formulation of sheared reconstruction filters that achieves real-time preformance on a modern GPU and is more than two orders of magnitude faster than previously published techniques.

Supplementary Material

JPG File (a23.jpg)

Download
6.08 KB

vaidyanathan (vaidyanathan.zip)

Supplemental movie, appendix, image and software files for, Layered Light Field Reconstruction for Defocus Blur

Download
669.96 MB

MP4 File (a23.mp4)

Download
20.20 MB

References

[1]

T. Akenine-Möller, E. Haines, and N. Hoffman. 2008. Real-Time Rendering 3^rd Ed. AK Peters Ltd.

[2]

T. Akenine-Möller, J. Munkberg, and J. Hasselgren. 2007. Stochastic rasterization using time-continuous triangles. In Proceedings of the 22^nd ACM SIGGRAPH/EUROGRAPHICS Symposium on Graphics Hardware (GH'07). 7--16.

Digital Library

[3]

L. Belcour, C. Soler, K. Subr, N. Holzschuch, and F. Durand. 2013. 5D covariance tracing for efficient defocus and motion blur. ACM Trans. Graph. 32, 3, 31:1--31:18.

Digital Library

[4]

J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum. 2000. Plenoptic sampling. In Proceedings of the 27^nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'00). ACM Press, New York, 307--318.

Digital Library

[5]

J. Demers. 2004. Depth of field: A survey of techniques. In GPU Gems, R. Fernando, Ed., Addison-Wesley, 375--390.

[6]

F. Durand, N. Holzschuch, C. Soler, E. Chan, and F. X. Sillion. 2005. A frequency analysis of light transport. ACM Trans. Graph. 24, 3, 1115--1126.

Digital Library

[7]

K. Egan, F. Durand, and R. Ramamoorthi. 2011a. Practical filtering for efficient ray-traced directional occlusion. ACM Trans. Graph. 30, 6, 180:1--180:10.

Digital Library

[8]

K. Egan, F. Hecht, F. Durand, and R. Ramamoorthi. 2011b. Frequency analysis and sheared filtering for shadow light fields of complex occluders. ACM Trans. Graph. 30, 2, 9:1--9:13.

Digital Library

[9]

K. Egan, Y.-T. Tseng, N. Holzschuch, F. Durand, and R. Ramamoorthi. 2009. Frequency analysis and sheared reconstruction for rendering motion blur. ACM Trans. Graph. 28, 3, 93:1--93:13.

Digital Library

[10]

T. Hachisuka, W. Jarosz, R. P. Weistroffer, K. Dale, G. Humphreys, M. Zwicker, and H. W. Jensen. 2008. Multidimensional adaptive sampling and reconstruction for ray tracing. ACM Trans. Graph. 27, 3, 33:1--33:10.

Digital Library

[11]

E. Hammon Jr. 2007. Practical post-process depth of field. In GPU Gems 3, H. Nguyen, Ed., Addison-Wesley, 583--606.

[12]

P. S. Heckbert. 1989. Fundamentals of texture mapping and image warping. M.S. thesis, University of California, Berkeley. https://www.cs.cmu.edu/~ph/texfund/texfund.pdf.

[13]

N. K. Kalantari and P. Sen. 2013. Removing the noise in monte carlo rendering with general image denoising algorithms. Comput. Graph. Forum 32, 2, 93--102.

[14]

C. Kolb, D. Mitchell, and P. Hanrahan. 1995. A realistic camera model for computer graphics. In Proceedings of the 22^nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'95). ACM Press, New York, 317--324.

Digital Library

[15]

D. Lanman, R. Raskar, A. Agrawal, and G. Taubin. 2008. Shield fields: Modeling and capturing 3D occluders. ACM Trans. Graph. 27, 5, 131:1--131:10.

Digital Library

[16]

S. Lee, E. Eisemann, and H.-P. Seidel. 2010. Real-time lens blur effects and focus control. ACM Trans. Graph. 29, 4, 65:1--65:7.

Digital Library

[17]

J. Lehtinen, T. Aila, J. Chen, S. Laine, and F. Durand. 2011. Temporal light field reconstruction for rendering distribution effects. ACM Trans. Graph. 30, 4, 55:1--55:12.

Digital Library

[18]

J. Lehtinen, T. Aila, S. Laine, and F. Durand. 2012. Reconstructing the indirect light field for global illumination. ACM Trans. Graph. 31, 4, 51:1--51:10.

Digital Library

[19]

K. Lei and J. F. Hughes. 2013. Approximate depth of field effects using few samples per pixel. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D'13). ACM Press, New York, 119--128.

Digital Library

[20]

S. Mehta, B. Wang, and R. Ramamoorthi. 2012. Axis-aligned filtering for interactive sampled soft shadows. ACM Trans. Graph. 31, 6, 163:1--163:10.

Digital Library

[21]

M. Pharr and G. Humphreys. 2010. Physically Based Rendering: From Theory to Implementation 2^nd Ed. Morgan Kaufmann, San Fransisco.

Digital Library

[22]

T. Porter and T. Duff. 1984. Compositing digital images. In Proceedings of the 11^th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'84). ACM Press, New York, 253--259.

Digital Library

[23]

M. Potmesil and I. Chakravarty. 1981. A lens and aperture camera model for synthetic image generation. In Proceedings of the 8^th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'81). ACM Press, New York, 297--305.

Digital Library

[24]

R. Ramamoorthi, J. Anderson, M. Meyer, and D. Nowrouzezahrai. 2012. A theory of Monte Carlo visibility sampling. ACM Trans. Graph. 31, 5, 121:1--121:16.

Digital Library

[25]

F. Rousselle, C. Knaus, and M. Zwicker. 2012. Adaptive rendering with non-local means filtering. ACM Trans. Graph. 31, 6, 195:1--195:11.

Digital Library

[26]

P. Sen and S. Darabi. 2012. On filtering the noise from the random parameters in Monte Carlo rendering. ACM Trans. Graph. 31, 3, 18:1--18:15.

Digital Library

[27]

P. Shirley, T. Aila, J. Cohen, E. Enderton, S. Laine, D. Luebke, and M. Mcguire. 2011. A local image reconstruction algorithm for stochastic rendering. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D'11). ACM Press, New York, 9--14.

Digital Library

[28]

C. Soler, K. Subr, F. Durand, N. Holzschuch, and F. Sillion. 2009. Fourier depth of field. ACM Trans. Graph. 28, 2, 18:1--18:12.

Digital Library

[29]

X. Yu, R. Wang, and J. Yu. 2011. Real-time depth of field rendering via dynamic light field generation and filtering. Comput. Graph. Forum 29, 7, 2099--2107.

Cited By

March JKrishnan AMantiuk RWatt S(2024)Impact of focus cue presentation on perceived realism of 3-D scene structure: Implications for scene perception and for display technologyJournal of Vision10.1167/jov.24.2.1324:2(13)Online publication date: 27-Feb-2024
https://doi.org/10.1167/jov.24.2.13
Chen PYang SHuang C(2023)A 250-mW 5.4G-Rendered-Pixel/s Realistic Refocusing Processor for High-Performance Five-Camera Mobile DevicesIEEE Open Journal of the Solid-State Circuits Society10.1109/OJSSCS.2023.32447593(52-62)Online publication date: 2023
https://doi.org/10.1109/OJSSCS.2023.3244759
Wang ZHuang LKodama K(2023)Robust extension of light fields with probable 3D distribution based on iterative scene estimation from multi-focus imagesSignal Processing: Image Communication10.1016/j.image.2022.116896111(116896)Online publication date: Feb-2023
https://doi.org/10.1016/j.image.2022.116896
Show More Cited By

Recommendations

Real-Time Defocus Rendering With Level of Detail and Sub-Sample Blur

This paper presents a GPU-based rendering algorithm for real-time defocus blur effects, which significantly improves the accumulation buffering. The algorithm combines three distinctive techniques: 1 adaptive discrete geometric level of detail LOD, made ...
Decoupled sampling for graphics pipelines

We propose a generalized approach to decoupling shading from visibility sampling in graphics pipelines, which we call decoupled sampling. Decoupled sampling enables stochastic supersampling of motion and defocus blur at reduced shading cost, as well as ...
Fast 4D Sheared Filtering for Interactive Rendering of Distribution Effects

Soft shadows, depth of field, and diffuse global illumination are common distribution effects, usually rendered by Monte Carlo ray tracing. Physically correct, noise-free images can require hundreds or thousands of ray samples per pixel, and take a long ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 34, Issue 2

February 2015

136 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2742222

Editor:
Holly Rushmeier
Yale University

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 March 2015

Received: 01 October 2014

Accepted: 01 September 2014

Revised: 01 August 2014

Published in TOG Volume 34, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
698
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

March JKrishnan AMantiuk RWatt S(2024)Impact of focus cue presentation on perceived realism of 3-D scene structure: Implications for scene perception and for display technologyJournal of Vision10.1167/jov.24.2.1324:2(13)Online publication date: 27-Feb-2024
https://doi.org/10.1167/jov.24.2.13
Chen PYang SHuang C(2023)A 250-mW 5.4G-Rendered-Pixel/s Realistic Refocusing Processor for High-Performance Five-Camera Mobile DevicesIEEE Open Journal of the Solid-State Circuits Society10.1109/OJSSCS.2023.32447593(52-62)Online publication date: 2023
https://doi.org/10.1109/OJSSCS.2023.3244759
Wang ZHuang LKodama K(2023)Robust extension of light fields with probable 3D distribution based on iterative scene estimation from multi-focus imagesSignal Processing: Image Communication10.1016/j.image.2022.116896111(116896)Online publication date: Feb-2023
https://doi.org/10.1016/j.image.2022.116896
Chen WZhu C(2022)Spectral analysis of a surface occlusion model for image-based rendering samplingDigital Signal Processing10.1016/j.dsp.2022.103697130(103697)Online publication date: Oct-2022
https://doi.org/10.1016/j.dsp.2022.103697
Munkberg JHasselgren J(2020)Neural Denoising with Layer EmbeddingsComputer Graphics Forum10.1111/cgf.1404939:4(1-12)Online publication date: 20-Jul-2020
https://doi.org/10.1111/cgf.14049
Chen YChang T(2020)Image Synthesis With Efficient Defocus Blur for Stereoscopic DisplaysIEEE Access10.1109/ACCESS.2020.30263328(176304-176312)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3026332
Luo PCai LZhou GChen Z(2019)Multiagent Light Field Reconstruction and Maneuvering Target Recognition via GANMathematical Problems in Engineering10.1155/2019/97109742019:1Online publication date: 24-Dec-2019
https://doi.org/10.1155/2019/9710974
Leimkühler TSeidel HRitschel T(2018)Laplacian kernel splatting for efficient depth-of-field and motion blur synthesis or reconstructionACM Transactions on Graphics10.1145/3197517.320137937:4(1-11)Online publication date: 30-Jul-2018
https://dl.acm.org/doi/10.1145/3197517.3201379
Zhu CZhang HYu L(2018)Structure Models for Image-Assisted Geometry Measurement in Plenoptic SamplingIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2017.275595867:1(150-166)Online publication date: Jan-2018
https://doi.org/10.1109/TIM.2017.2755958
Schied CKaplanyan AWyman CPatney AChaitanya CBurgess JLiu SDachsbacher CLefohn ASalvi MMcGuire MPatney A(2017)Spatiotemporal variance-guided filteringProceedings of High Performance Graphics10.1145/3105762.3105770(1-12)Online publication date: 28-Jul-2017
https://dl.acm.org/doi/10.1145/3105762.3105770
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents