High-Resolution View Synthesis In Camera-Projector Systems Using Compressive Dual Imaging
Pages 7 - 12
Abstract
In this paper, a method for high-resolution view synthesis in a calibrated camera-projector system is proposed. Camera’s resolution is significantly lower than the projector’s resolution. Compressive sensing and sum-to-one transformation are used for an efficient light transport matrix estimation. The light transport matrix and Helmholtz reciprocity enable dual imaging, i.e. generating virtual images from the projector’s viewpoint and virtual relighting of the observed scene. Virtual fringe profilometry enables 3D reconstruction of the observed scene. Performing 3D reconstruction in the dual configuration of the camera-projector system results in a significantly denser point cloud. Reconstructed 3D points are back-projected onto the camera sensor using the camera’s projective matrix, resulting in a high-resolution image, synthesized from the camera’s viewpoint. The proposed method overcomes the spatial resolution limit of the imaging sensor, resulting in significantly increased spatial resolution of the camera’s images.
References
[1]
M. Zeshan Alam and Bahadir K. Gunturk. 2018. Hybrid light field imaging for improved spatial resolution and depth range. Machine Vision and Applications 29, 1 (01 Jan 2018), 11–22. https://doi.org/10.1007/s00138-017-0862-2
[2]
Isaac Amidror. 2002. Scattered data interpolation methods for electronic imaging systems: a survey. Journal of Electronic Imaging 11, 2 (April 2002), 157. https://doi.org/10.1117/1.1455013
[3]
Naoya Chiba and Koichi Hashimoto. 2018. Ultra-Fast Multi-Scale Shape Estimation of Light Transport Matrix for Complex Light Reflection Objects. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, Brisbane, QLD, Australia, 6147–6152. https://doi.org/10.1109/icra.2018.8460892
[4]
Tom Goldstein, Lina Xu, Kevin F. Kelly, and Richard Baraniuk. 2015. The STOne Transform: Multi-Resolution Image Enhancement and Compressive Video. IEEE Transactions on Image Processing 24, 12 (Dec. 2015), 5581–5593. https://doi.org/10.1109/tip.2015.2474697
[5]
M. Shahzeb Khan Gul and Bahadir K. Gunturk. 2018. Spatial and Angular Resolution Enhancement of Light Fields Using Convolutional Neural Networks. IEEE Transactions on Image Processing 27, 5 (May 2018), 2146–2159. https://doi.org/10.1109/tip.2018.2794181
[6]
Richard Hartley and Andrew Zisserman. 2004. Multiple View Geometry in Computer Vision. Cambridge University Press, Australian National University, Canberra. https://doi.org/10.1017/cbo9780511811685
[7]
Ana Sović Kržić and Damir Seršić. 2018. L1 minimization using recursive reduction of dimensionality. Signal Processing 151 (Oct. 2018), 119–129. https://doi.org/10.1016/j.sigpro.2018.05.002
[8]
Jian Liang, Haijuan Ju, Liyong Ren, Wenfei Zhang, Liming Yang, Zhaofeng Bai, and Rongguang Liang. 2019. High-resolution reconstruction of shortwave infrared polarimetric images using the intensity information of visible images. Applied Optics 58, 18 (June 2019), 4866. https://doi.org/10.1364/ao.58.004866
[9]
Daniel Moreno and Gabriel Taubin. 2012. Simple, Accurate, and Robust Projector-Camera Calibration. In 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission. IEEE, Zurich, Switzerland, 464–471. https://doi.org/10.1109/3dimpvt.2012.77
[10]
M. Umair Mukati and Bahadir K. Gunturk. 2017. Hybrid-sensor high-resolution light field imaging. In 2017 25th Signal Processing and Communications Applications Conference (SIU). IEEE, Antalya, Turkey, 1–4. https://doi.org/10.1109/siu.2017.7960651
[11]
Naoya Chiba and Koichi Hashimoto. 2017. 3D measurement by estimating Homogeneous Light Transport (HLT) Matrix. In 2017 IEEE International Conference on Mechatronics and Automation (ICMA). IEEE, Takamatsu, Japan, 1763–1768. https://doi.org/10.1109/icma.2017.8016084
[12]
Pieter Peers, Dhruv K. Mahajan, Bruce Lamond, Abhijeet Ghosh, Wojciech Matusik, Ravi Ramamoorthi, and Paul Debevec. 2009. Compressive light transport sensing. ACM Transactions on Graphics 28, 1 (Jan. 2009), 1–18. https://doi.org/10.1145/1477926.1477929
[13]
Ivan Ralasic, Matea Donlic, and Damir Sersic. 2020. Dual Imaging–Can Virtual Be Better Than Real?IEEE Access 8(2020), 40246–40260. https://doi.org/10.1109/access.2020.2976870
[14]
Ivan Ralasic, Damir Sersic, and Davor Petrinovic. 2019. Off-the-Shelf Measurement Setup for Compressive Imaging. IEEE Transactions on Instrumentation and Measurement 68, 2 (Feb. 2019), 502–511. https://doi.org/10.1109/tim.2018.2847018
[15]
Pradeep Sen, Billy Chen, Gaurav Garg, Stephen R. Marschner, Mark Horowitz, Marc Levoy, and Hendrik P. A. Lensch. 2005. Dual photography. ACM Transactions on Graphics 24, 3 (July 2005), 745. https://doi.org/10.1145/1073204.1073257
[16]
Pradeep Sen and Soheil Darabi. 2009. Compressive Dual Photography. Computer Graphics Forum 28, 2 (April 2009), 609–618. https://doi.org/10.1111/j.1467-8659.2009.01401.x
[17]
Xiang Wang, Lin Li, and GuangQi Hou. 2016. High-resolution light field reconstruction using a hybrid imaging system. Applied Optics 55, 10 (March 2016), 2580. https://doi.org/10.1364/ao.55.002580
[18]
Youngjin Yoon, Hae-Gon Jeon, Donggeun Yoo, Joon-Young Lee, and In So Kweon. 2015. Learning a Deep Convolutional Network for Light-Field Image Super-Resolution. In 2015 IEEE International Conference on Computer Vision Workshop (ICCVW). IEEE, Santiago, Chile, 57–65. https://doi.org/10.1109/iccvw.2015.17
[19]
Song Zhang. 2016. High-Speed 3D Imaging with Digital Fringe Projection Techniques. CRC Press, Boca Raton. https://doi.org/10.1201/b19565
[20]
Shuo Zhang, Youfang Lin, and Hao Sheng. 2019. Residual Networks for Light Field Image Super-Resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. CVPR, Long Beach, CA, USA, USA, 11046–11055.
[21]
Z. Zhang. 2000. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 11(2000), 1330–1334. https://doi.org/10.1109/34.888718
[22]
Chao Zuo, Lei Huang, Minliang Zhang, Qian Chen, and Anand Asundi. 2016. Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review. Optics and Lasers in Engineering 85 (Oct. 2016), 84–103. https://doi.org/10.1016/j.optlaseng.2016.04.022
Recommendations
Large-Area, Multilayered, and High-Resolution Visual Monitoring Using a Dual-Camera System
Large-area, high-resolution visual monitoring systems are indispensable in surveillance applications. To construct such systems, high-quality image capture and display devices are required. Whereas high-quality displays have rapidly developed, as ...
Geometric calibration of a camera-projector 3D imaging system
VRCAI '11: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in IndustryUse of projector in structured light based 3D imaging systems has grown its popularity recently, due to rapid advance of DLP and LCOS chip technologies. Yet, simple and accurate calibration method for camera-projector has not received its deserved ...
Scalable alignment of large-format multi-projector displays using camera homography trees
VIS '02: Proceedings of the conference on Visualization '02This paper presents a vision-based geometric alignment system for aligning the projectors in an arbitrarily large display wall. Existing algorithms typically rely on a single camera view and degrade in accuracy1 as the display resolution exceeds the ...
Comments
Information & Contributors
Information
Published In
October 2020
95 pages
ISBN:9781450388283
DOI:10.1145/3441233
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 March 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Croatian Science Foundation
- European Regional Development Fund
Conference
SSIP 2020
SSIP 2020: 2020 3rd International Conference on Sensors, Signal and Image Processing
October 9 - 11, 2020
Prague, Czech Republic
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 52Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format