research-article

Next generation image based lighting using HDR video

Authors:

Gunnar KaiserAuthors Info & Claims

SIGGRAPH '11: ACM SIGGRAPH 2011 Talks

Article No.: 60, Page 1

https://doi.org/10.1145/2037826.2037906

Published: 07 August 2011 Publication History

Get Access

Abstract

We present an overview of our recently developed systems pipeline for capture, reconstruction, modeling and rendering of real world scenes based on state-of-the-art high dynamic range video (HDRV). The reconstructed scene representation allows for photo-realistic Image Based Lighting (IBL) in complex environments with strong spatial variations in the illumination. The pipeline comprises the following essential steps:

1.) Capture - The scene capture is based on a 4MPixel global shutter HDRV camera with a dynamic range of more than 24 f-stops at 30 fps. The HDR output stream is stored as individual un-compressed frames for maximum flexibility. A scene is usually captured using a combination of panoramic light probe sequences [1], and sequences with a smaller field of view to maximize the resolution at regions of special interest in the scene. The panoramic sequences ensure full angular coverage at each position and guarantee that the information required for IBL is captured. The position and orientation of the camera is tracked during capture.

2.) Scene recovery - Taking one or more HDRV sequences as input, a geometric proxy model of the scene is built using a semi-automatic approach. First, traditional computer vision algorithms such as structure from motion [2] and Manhattan world stereo [3] are used. If necessary, the recovered model is then modified using an interaction scheme based on visualizations of a volumetric representation of the scene radiance computed from the input HDRV sequence. The HDR nature of this volume also enables robust extraction of direct light sources and other high intensity regions in the scene.

3.) Radiance processing - When the scene proxy geometry has been recovered, the radiance data captured in the HDRV sequences are re-projected onto the surfaces and the recovered light sources. Since most surface points have been imaged from a large number of directions, it is possible to reconstruct view dependent texture maps at the proxy geometries. These 4D data sets describe a combination of detailed geometry that has not been recovered and the radiance reflected from the underlying real surfaces. The view dependent textures are then processed and compactly stored in an adaptive data structure.

4.) Rendering - Once the geometric and radiometric scene information has been recovered, it is possible to place virtual objects into the real scene and create photo-realistic renderings as illustrated above. The extracted light sources enable efficient sampling and rendering times that are fully comparable to that of traditional virtual computer graphics light sources. No previously described method is capable of capturing and reproducing the angular and spatial variation in the scene illumination in comparable detail.

We believe that the rapid development of high quality HDRV systems will soon have a large impact on both computer vision and graphics. Following this trend, we are developing theory and algorithms for efficient processing HDRV sequences and using the abundance of radiance data that is going to be available.

Supplementary Material

session video (tk056_11.mp4)

Download
14.59 MB

References

[1]

P. Debevec: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In SIGGRAPH 98 Papers, ACM, New York, USA, 189--198, 1998.

Digital Library

Google Scholar

[2]

B. Triggs, P. Mclauchlan, R. Hartley, and A. Fitzgibbon: Bundle adjustment -- a modern synthesis, Vision Algorithms: Theory and Practice, LNCS, 298--375, Springer Verlag, 2000.

Digital Library

Google Scholar

[3]

Y. Furukawa, B. Curless, S. M. Seitz and R. Szeliski: Manhattan world stereo, IEEE Conference on Computer Vision and Pattern Recognition, 2009.

Crossref

Google Scholar

Cited By

View all

Mazurek P(2012)Estimation of Position and Radius of Light Probe ImagesProceedings of the International Conference on Computer Vision and Graphics - Volume 759410.5555/2942031.2942095(517-524)Online publication date: 24-Sep-2012
https://dl.acm.org/doi/10.5555/2942031.2942095
Mazurek P(2012)Estimation of Position and Radius of Light Probe ImagesComputer Vision and Graphics10.1007/978-3-642-33564-8_62(517-524)Online publication date: 2012
https://doi.org/10.1007/978-3-642-33564-8_62

Index Terms

Next generation image based lighting using HDR video
1. Computing methodologies
  1. Computer graphics
2. General and reference

Recommendations

Fast, Realistic Lighting for Video Games

Global lighting effects produced by diffuse interreflections are typically simulated using global illumination methods such as radiosity or ray tracing. Although diffuse interreflections are crucial to produce realistic images, radiosity like methods ...
Real-time image based lighting in software using HDR panoramas
GRAPHITE '03: Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia

We present a system allowing real-time image based lighting based on HDR panoramic images [Debevec 1998]. The system performs time-consuming diffuse light calculations in a pre-processing step, which is key to attaining interactivity. The real-time ...
Spatial image based lighting
SIGGRAPH '07: ACM SIGGRAPH 2007 posters

When compositing computer generated imagery into photographed scenes, image based lighting (IBL) [Debevec 1998] is used so that synthetic and real objects are consistently illuminated. Traditional IBL takes light captured from a single point in space ...

Comments

Information & Contributors

Information

Published In

SIGGRAPH '11: ACM SIGGRAPH 2011 Talks

August 2011

73 pages

ISBN:9781450309745

DOI:10.1145/2037826

Conference Chair:
Mark Elendt
Side Effects Software Inc.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SIGGRAPH '11

Sponsor:

SIGGRAPH

SIGGRAPH '11: Special Interest Group on Computer Graphics and Interactive Techniques Conference

August 7 - 11, 2011

British Columbia, Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
291
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Mazurek P(2012)Estimation of Position and Radius of Light Probe ImagesProceedings of the International Conference on Computer Vision and Graphics - Volume 759410.5555/2942031.2942095(517-524)Online publication date: 24-Sep-2012
https://dl.acm.org/doi/10.5555/2942031.2942095
Mazurek P(2012)Estimation of Position and Radius of Light Probe ImagesComputer Vision and Graphics10.1007/978-3-642-33564-8_62(517-524)Online publication date: 2012
https://doi.org/10.1007/978-3-642-33564-8_62

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Fast, Realistic Lighting for Video Games

Real-time image based lighting in software using HDR panoramas

Spatial image based lighting

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

Supplementary Material

References

Cited By

Index Terms

Recommendations

Fast, Realistic Lighting for Video Games

Real-time image based lighting in software using HDR panoramas

Spatial image based lighting

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations