research-article

Switchable primaries using shiftable layers of color filter arrays

Authors:

Aditi Majumder,

Kazuhiro Hiwada,

Atsuto Maki, and

Ramesh RaskarAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 30, Issue 4

Article No.: 65, Pages 1 - 10

https://doi.org/10.1145/2010324.1964960

Published: 25 July 2011 Publication History

Abstract

We present a camera with switchable primaries using shiftable layers of color filter arrays (CFAs). By layering a pair of CMY CFAs in this novel manner we can switch between multiple sets of color primaries (namely RGB, CMY and RGBCY) in the same camera. In contrast to fixed color primaries (e.g. RGB or CMY), which cannot provide optimal image quality for all scene conditions, our camera with switchable primaries provides optimal color fidelity and signal to noise ratio for multiple scene conditions.

Next, we show that the same concept can be used to layer two RGB CFAs to design a camera with switchable low dynamic range (LDR) and high dynamic range (HDR) modes. Further, we show that such layering can be generalized as a constrained satisfaction problem (CSP) allowing to constrain a large number of parameters (e.g. different operational modes, amount and direction of the shifts, placement of the primaries in the CFA) to provide an optimal solution.

We investigate practical design options for shiftable layering of the CFAs. We demonstrate these by building prototype cameras for both switchable primaries and switchable LDR/HDR modes.

To the best of our knowledge, we present, for the first time, the concept of shiftable layers of CFAs that provides a new degree of freedom in photography where multiple operational modes are available to the user in a single camera for optimizing the picture quality based on the nature of the scene geometry, color and illumination.

Supplementary Material

MP4 File (tp061_11.mp4)

Download
26.21 MB

References

[1]

Alter, F., Matsushita, Y., and Tang, X. 2006. An intensity similarity measure in low-light conditions. In ECCV, 267--280.

[2]

Baone, G. A., and Qi, H. 2006. Demosaicking methods for multispectral cameras using mosaic focal plane array technology. Proc. SPIE 6062.

[3]

Bayer, B. Color imaging array. US Patent 3,971,065.

[4]

Cao, H., and Kot, A. C. 2008. A generalized model for detection of demosaicing characteristics. In ICME, 1513--1516.

[5]

Drago, F., Myszkowski, K., Annen, T., and Chiba, N. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum 22, 419--426.

[6]

Freeman, T. W. 1988. Median filter for reconstructing missing color samples. US Patent 4,724,395.

[7]

Gindele, E., and Gallagher, A. Sparsely sampled image sensing device with color and luminance photosites. US Patent 6,476,865.

[8]

Gu, J., Wolfe, P. J., and Hirakawa, K. 2010. Filterbank-based universal demosaicking. 1981--1984.

[9]

Gunturk, B., Member, S., Altunbasak, Y., Member, S., and Mersereau, R. M. 2002. Color plane interpolation using alternating projections. IEEE Trans. Image Processing 11, 997--1013.

Digital Library

[10]

Gunturk, B., Glotzbach, J., Altunbasak, Y., Schafer, R., and Mersereau, R. 2005. Demosaicking: color filter array interpolation. IEEE Signal Processing Magazine.

[11]

Hamilton, J., and Adams, J. 1997. Adaptive color plane interpolation in single sensor color electronic camera. US Patent 5,629,734.

[12]

Hirakawa, K., and Wolfe, P. 2008. Spatio-spectral color filter spatio-spectral color filter array design for optimal image recovery. IEEE TIP 17, 10.

[13]

Imai, F. H., Berns, R. S., and Tzeng, D.-Y. 2000. A comparative analysis of spectral reflectance estimated in various spaces using a trichromatic camera system. Journal of Imaging Science and Technology 44, 280--287.

[14]

Kumar, M., Morales, E., Adams, J., and Hao, W. 2009. New digital camera sensor architecture for low light imaging. IEEE ICIP, 2681--2684.

[15]

Langfelder, G., Zaraga, F., and Longoni, A. 2009. Tunable spectral responses in a color-sensitive cmos pixel for imaging applications. Electron Devices, IEEE Transactions on.

[16]

Li, X. 2005. Demosaicing by successive approximation. IEEE Trans. Image Process. 14, 3, 370--379.

Digital Library

[17]

Lu, W., and Tan, Y.-P. 2003. Color filter array demosaicking: new method and performance measures. Image Processing, IEEE Transactions on.

[18]

Lukac, R. 2008. Single-Sensor Imaging: Methods and Applications for Digital Cameras. CRC Press.

[19]

Mohan, A., Raskar, R., and Tumblin, J. 2008. Agile spectrum imaging: Programmable wavelength modulation for cameras and projectors. Computer Graphics Forum 27, 2, 709--717.

[20]

Ratner, N., and Schechner, Y. Y. 2007. Illumination multiplexing within fundamental limits. Computer Vision and Pattern Recognition, IEEE Computer Society Conference on 0, 1--8.

[21]

Schechner, Y. Y., Nayar, S. K., and Belhumeur, P. N. 2007. Multiplexing for Optimal Lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 8, 1339--1354.

Digital Library

[22]

Shogenji, R., Kitamura, Y., Yamada, K., Miyatake, S., and Tanida, J. 2004. Multispectral imaging using compact compound optics. Opt. Exp., 16431655.

[23]

Susanu, G., Peterescu, S., Nanu, F., Capata, A., and Corcoran, P. 2009. Rgbw sensor array. US Patent 2009/0,167,893.

[24]

Yamagami, T., Sasaki, T., and Suga, A. Image signal processing apparatus having a color filter with offset luminance filter elements. US Patent 5,323,233.

[25]

Yasuma, F., Mitsunaga, T., Iso, D., and Nayar, S. Generalized Assorted Pixel Camera: Post-Capture Control of Resolution, Dynamic Range and Spectrum. Tech. rep.

Cited By

Bae T(2020)Image-quality metric system for color filter array evaluationPLOS ONE10.1371/journal.pone.023258315:5(e0232583)Online publication date: 11-May-2020
https://doi.org/10.1371/journal.pone.0232583
Lei ZYu XStrobel M(2020)High-Dynamic-Range and Wide Color Gamut VideoNANO-CHIPS 203010.1007/978-3-030-18338-7_21(359-386)Online publication date: 9-Jun-2020
https://doi.org/10.1007/978-3-030-18338-7_21
Li YMajumder AZhang HGopi M(2018)Optimized Multi-Spectral Filter Array Based Imaging of Natural ScenesSensors10.3390/s1804117218:4(1172)Online publication date: 12-Apr-2018
https://doi.org/10.3390/s18041172
Show More Cited By

Index Terms

Switchable primaries using shiftable layers of color filter arrays
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
      2. Image and video acquisition
  2. Computer graphics
    1. Image manipulation

Recommendations

Switchable primaries using shiftable layers of color filter arrays
SIGGRAPH '11: ACM SIGGRAPH 2011 papers

We present a camera with switchable primaries using shiftable layers of color filter arrays (CFAs). By layering a pair of CMY CFAs in this novel manner we can switch between multiple sets of color primaries (namely RGB, CMY and RGBCY) in the same ...
Read More
Extracting depth and matte using a color-filtered aperture
SIGGRAPH Asia '08: ACM SIGGRAPH Asia 2008 papers

This paper presents a method for automatically extracting a scene depth map and the alpha matte of a foreground object by capturing a scene through RGB color filters placed in the camera lens aperture. By dividing the aperture into three regions through ...
Read More
Extracting depth and matte using a color-filtered aperture

This paper presents a method for automatically extracting a scene depth map and the alpha matte of a foreground object by capturing a scene through RGB color filters placed in the camera lens aperture. By dividing the aperture into three regions through ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 30, Issue 4

July 2011

829 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2010324

Issue’s Table of Contents

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2011

Published in TOG Volume 30, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Division of Information and Intelligent Systems

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
929
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Bae T(2020)Image-quality metric system for color filter array evaluationPLOS ONE10.1371/journal.pone.023258315:5(e0232583)Online publication date: 11-May-2020
https://doi.org/10.1371/journal.pone.0232583
Lei ZYu XStrobel M(2020)High-Dynamic-Range and Wide Color Gamut VideoNANO-CHIPS 203010.1007/978-3-030-18338-7_21(359-386)Online publication date: 9-Jun-2020
https://doi.org/10.1007/978-3-030-18338-7_21
Li YMajumder AZhang HGopi M(2018)Optimized Multi-Spectral Filter Array Based Imaging of Natural ScenesSensors10.3390/s1804117218:4(1172)Online publication date: 12-Apr-2018
https://doi.org/10.3390/s18041172
Hirai KOsawa NHori MHoriuchi TTominaga S(2018)High-Dynamic-Range Spectral Imaging System for Omnidirectional Scene CaptureJournal of Imaging10.3390/jimaging40400534:4(53)Online publication date: 23-Mar-2018
https://doi.org/10.3390/jimaging4040053
Hu XHeide FDai QWetzstein G(2018)Convolutional Sparse Coding for RGB+NIR ImagingIEEE Transactions on Image Processing10.1109/TIP.2017.278130327:4(1611-1625)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1109/TIP.2017.2781303
Prasad D(2016)Strategies for Resolving Camera Metamers Using 3+1 Channel2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW.2016.123(954-962)Online publication date: Jun-2016
https://doi.org/10.1109/CVPRW.2016.123
Jeong KCho H(2016)A Digitalized Recomposition Technique Based on Photo Quality Evaluation CriteriaWireless Personal Communications: An International Journal10.1007/s11277-015-2977-y86:1(301-314)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1007/s11277-015-2977-y
Jeong KCho H(2016)Photo quality enhancement by relocating subjectsCluster Computing10.1007/s10586-016-0547-z19:2(939-948)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.1007/s10586-016-0547-z
Kauvar IYang SShi LMcDowall IWetzstein G(2015)Adaptive color display via perceptually-driven factored spectral projectionACM Transactions on Graphics10.1145/2816795.281807034:6(1-10)Online publication date: 2-Nov-2015
https://dl.acm.org/doi/10.1145/2816795.2818070
Jeong KCho H(2015)Digital panning shot generator from photographsCluster Computing10.1007/s10586-014-0411-y18:2(667-676)Online publication date: 1-Jun-2015
https://dl.acm.org/doi/10.1007/s10586-014-0411-y
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents