other

Instant Neural Radiance Fields

Authors:

András Bódis-Szomorú,

Isaac Deutsch,

Michael Shelley,

Alexander KellerAuthors Info & Claims

SIGGRAPH '22: ACM SIGGRAPH 2022 Real-Time Live!

Article No.: 6, Pages 1 - 2

https://doi.org/10.1145/3532833.3538678

Published: 24 July 2022 Publication History

Get Access

Abstract

We extend our instant NeRF implementation [Müller et al. 2022] to allow training from an incremental stream of images and camera poses, provided by a realtime Simultaneous Localization And Mapping (SLAM) system. Camera poses are refined end-to-end by back-propagating the gradients from NeRF training. Reconstruction quality is further improved by compensating for various camera properties, such as rolling shutter, non-linear lens distortion, and variable exposure typical of digital cameras.

Static scenes can be scanned, the NeRF model trained, and the reconstruction verified in an interactive fashion, in under a minute.

References

[1]

Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, and Andrew Fitzgibbon. 2011. KinectFusion: Real-Time 3D Reconstruction and Interaction Using a Moving Depth Camera. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST ’11). Association for Computing Machinery, New York, NY, USA, 559–568. https://doi.org/10.1145/2047196.2047270

Digital Library

Google Scholar

[2]

Maik Keller, Damien Lefloch, Martin Lambers, Shahram Izadi, Tim Weyrich, and Andreas Kolb. 2013. Real-time 3d reconstruction in dynamic scenes using point-based fusion. In 2013 International Conference on 3D Vision-3DV 2013. IEEE, 1–8.

Digital Library

Google Scholar

[3]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 (June 2014).

Google Scholar

[4]

Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey. 2021. BARF: Bundle-Adjusting Neural Radiance Fields. CoRR abs/2104.06405(2021). arXiv:2104.06405https://arxiv.org/abs/2104.06405

Google Scholar

[5]

Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.

Google Scholar

[6]

Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph. 41, 4, Article 102 (July 2022), 15 pages. https://doi.org/10.1145/3528223.3530127

Digital Library

Google Scholar

[7]

Thomas Müller, Fabrice Rousselle, Jan Novák, and Alexander Keller. 2021. Real-time Neural Radiance Caching for Path Tracing. ACM Trans. Graph. 40, 4, Article 36 (Aug. 2021), 16 pages. https://doi.org/10.1145/3450626.3459812

Digital Library

Google Scholar

Cited By

View all

Pepe MAlfio VCostantino D(2023)Assessment of 3D Model for Photogrammetric Purposes Using AI Tools Based on NeRF AlgorithmHeritage10.3390/heritage60803016:8(5719-5731)Online publication date: 5-Aug-2023
https://doi.org/10.3390/heritage6080301
Partha MGupta SGao G(2023)Neural City Maps: A Case for 3D Urban Environment Representations Based on Radiance FieldsProceedings of the 36th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2023)10.33012/2023.19324(1953-1973)Online publication date: 5-Oct-2023
https://doi.org/10.33012/2023.19324
Harrer MFranke LFink LStamminger MWeyrich T(2023)Inovis: Instant Novel-View SynthesisSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618216(1-12)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618216
Show More Cited By

Instant Neural Radiance Fields
1. Computing methodologies
  1. Computer graphics

Recommendations

CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields From Imperfect Camera Poses
Existing volumetric neural rendering techniques, such as Neural Radiance Fields (NeRF), face limitations in synthesizing high-quality novel views when the camera poses of input images are imperfect. To address this issue, we propose a novel 3D ...
CamP: Camera Preconditioning for Neural Radiance Fields

Neural Radiance Fields (NeRF) can be optimized to obtain high-fidelity 3D scene reconstructions of objects and large-scale scenes. However, NeRFs require accurate camera parameters as input --- inaccurate camera parameters result in blurry renderings. ...
Structure-aware neural radiance fields without posed camera
Abstract
The neural radiance fields (NeRF) for realistic novel view synthesis require camera poses to be pre-acquired by a structure-from-motion (SfM) approach. This two-stage strategy is not convenient to use and degrades the performance because the ...
Highlights
- Our approach does not need the camera poses that required in NeRF.
- Our depth consistent constraint enables NeRF to be aware of the scene’s structure.
- The scene representation and camera’s pose extraction are jointly optimized.

Comments

Information & Contributors

Information

Published In

SIGGRAPH '22: ACM SIGGRAPH 2022 Real-Time Live!

July 2022

13 pages

ISBN:9781450393683

DOI:10.1145/3532833

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 July 2022

Check for updates

Qualifiers

Other
Research
Refereed limited

Conference

SIGGRAPH '22

Sponsor:

SIGGRAPH

SIGGRAPH '22: Special Interest Group on Computer Graphics and Interactive Techniques Conference

August 7 - 11, 2022

BC, Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
921
Total Downloads

Downloads (Last 12 months)125
Downloads (Last 6 weeks)4

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pepe MAlfio VCostantino D(2023)Assessment of 3D Model for Photogrammetric Purposes Using AI Tools Based on NeRF AlgorithmHeritage10.3390/heritage60803016:8(5719-5731)Online publication date: 5-Aug-2023
https://doi.org/10.3390/heritage6080301
Partha MGupta SGao G(2023)Neural City Maps: A Case for 3D Urban Environment Representations Based on Radiance FieldsProceedings of the 36th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2023)10.33012/2023.19324(1953-1973)Online publication date: 5-Oct-2023
https://doi.org/10.33012/2023.19324
Harrer MFranke LFink LStamminger MWeyrich T(2023)Inovis: Instant Novel-View SynthesisSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618216(1-12)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618216
Kang XLiu KDuan JGong YQiu GEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)P2I-NET: Mapping Camera Pose to Image via Adversarial Learning for New View Synthesis in Real Indoor EnvironmentsProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612356(2635-2643)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612356
Weder SGarcia-Hernando GMonszpart ÁPollefeys MBrostow GFirman MVicente S(2023)Removing Objects From Neural Radiance Fields2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01586(16528-16538)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01586
Chacón RRamonell CPosada HSierra PTomar RMartínez de la Rosa CRodriguez AKoulalis IIoannidis KWagmeister S(2023)Digital twinning during load tests of railway bridges - case study: the high-speed railway network, Extremadura, SpainStructure and Infrastructure Engineering10.1080/15732479.2023.226484020:7-8(1105-1119)Online publication date: 9-Oct-2023
https://doi.org/10.1080/15732479.2023.2264840
He T(2022)Neural Radiance Fields Methods and Improvement Approaches2022 2nd International Conference on Big Data, Artificial Intelligence and Risk Management (ICBAR)10.1109/ICBAR58199.2022.00026(98-103)Online publication date: Nov-2022
https://doi.org/10.1109/ICBAR58199.2022.00026

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Cited By

Recommendations

CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields From Imperfect Camera Poses

CamP: Camera Preconditioning for Neural Radiance Fields

Structure-aware neural radiance fields without posed camera

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

HTML Format

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations