“Switchable primaries using shiftable layers of color filter arrays” by Sajadi, Majumder, Hiwada, Maki and Raskar

We present a camera with switchable primaries using shiftable layers of color filter arrays (CFAs). By layering a pair of CMY CFAs in this novel manner we can switch between multiple sets of color primaries (namely RGB, CMY and RGBCY) in the same camera. In contrast to fixed color primaries (e.g. RGB or CMY), which cannot provide optimal image quality for all scene conditions, our camera with switchable primaries provides optimal color fidelity and signal to noise ratio for multiple scene conditions.Next, we show that the same concept can be used to layer two RGB CFAs to design a camera with switchable low dynamic range (LDR) and high dynamic range (HDR) modes. Further, we show that such layering can be generalized as a constrained satisfaction problem (CSP) allowing to constrain a large number of parameters (e.g. different operational modes, amount and direction of the shifts, placement of the primaries in the CFA) to provide an optimal solution.We investigate practical design options for shiftable layering of the CFAs. We demonstrate these by building prototype cameras for both switchable primaries and switchable LDR/HDR modes.To the best of our knowledge, we present, for the first time, the concept of shiftable layers of CFAs that provides a new degree of freedom in photography where multiple operational modes are available to the user in a single camera for optimizing the picture quality based on the nature of the scene geometry, color and illumination.

1. Alter, F., Matsushita, Y., and Tang, X. 2006. An intensity similarity measure in low-light conditions. In ECCV, 267–280. Google Scholar
2. Baone, G. A., and Qi, H. 2006. Demosaicking methods for multispectral cameras using mosaic focal plane array technology. Proc. SPIE 6062.Google Scholar
3. Bayer, B. Color imaging array. US Patent 3,971,065.Google Scholar
4. Cao, H., and Kot, A. C. 2008. A generalized model for detection of demosaicing characteristics. In ICME, 1513–1516.Google Scholar
5. Drago, F., Myszkowski, K., Annen, T., and Chiba, N. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum 22, 419–426.Google ScholarCross Ref
6. Freeman, T. W. 1988. Median filter for reconstructing missing color samples. US Patent 4,724,395.Google Scholar
7. Gindele, E., and Gallagher, A. Sparsely sampled image sensing device with color and luminance photosites. US Patent 6,476,865.Google Scholar
8. Gu, J., Wolfe, P. J., and Hirakawa, K. 2010. Filterbank-based universal demosaicking. 1981–1984.Google Scholar
9. Gunturk, B., Member, S., Altunbasak, Y., Member, S., and Mersereau, R. M. 2002. Color plane interpolation using alternating projections. IEEE Trans. Image Processing 11, 997–1013. Google ScholarDigital Library
10. Gunturk, B., Glotzbach, J., Altunbasak, Y., Schafer, R., and Mersereau, R. 2005. Demosaicking: color filter array interpolation. IEEE Signal Processing Magazine.Google Scholar
11. Hamilton, J., and Adams, J. 1997. Adaptive color plane interpolation in single sensor color electronic camera. US Patent 5,629,734.Google Scholar
12. Hirakawa, K., and Wolfe, P. 2008. Spatio-spectral color filter spatio-spectral color filter array design for optimal image recovery. IEEE TIP 17, 10. Google Scholar
13. Imai, F. H., Berns, R. S., and Tzeng, D.-Y. 2000. A comparative analysis of spectral reflectance estimated in various spaces using a trichromatic camera system. Journal of Imaging Science and Technology 44, 280–287.Google Scholar
14. Kumar, M., Morales, E., Adams, J., and Hao, W. 2009. New digital camera sensor architecture for low light imaging. IEEE ICIP, 2681–2684. Google Scholar
15. Langfelder, G., Zaraga, F., and Longoni, A. 2009. Tunable spectral responses in a color-sensitive cmos pixel for imaging applications. Electron Devices, IEEE Transactions on.Google ScholarCross Ref
16. Li, X. 2005. Demosaicing by successive approximation. IEEE Trans. Image Process. 14, 3, 370–379. Google ScholarDigital Library
17. Lu, W., and Tan, Y.-P. 2003. Color filter array demosaicking: new method and performance measures. Image Processing, IEEE Transactions on. Google Scholar
18. Lukac, R. 2008. Single-Sensor Imaging: Methods and Applications for Digital Cameras. CRC Press. Google Scholar
19. Mohan, A., Raskar, R., and Tumblin, J. 2008. Agile spectrum imaging: Programmable wavelength modulation for cameras and projectors. Computer Graphics Forum 27, 2, 709–717.Google ScholarCross Ref
20. Ratner, N., and Schechner, Y. Y. 2007. Illumination multiplexing within fundamental limits. Computer Vision and Pattern Recognition, IEEE Computer Society Conference on 0, 1–8.Google Scholar
21. Schechner, Y. Y., Nayar, S. K., and Belhumeur, P. N. 2007. Multiplexing for Optimal Lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 8, 1339–1354. Google ScholarDigital Library
22. Shogenji, R., Kitamura, Y., Yamada, K., Miyatake, S., and Tanida, J. 2004. Multispectral imaging using compact compound optics. Opt. Exp., 16431655.Google Scholar
23. Susanu, G., Peterescu, S., Nanu, F., Capata, A., and Corcoran, P. 2009. Rgbw sensor array. US Patent 2009/0,167,893.Google Scholar
24. Yamagami, T., Sasaki, T., and Suga, A. Image signal processing apparatus having a color filter with offset luminance filter elements. US Patent 5,323,233.Google Scholar
25. Yasuma, F., Mitsunaga, T., Iso, D., and Nayar, S. Generalized Assorted Pixel Camera: Post-Capture Control of Resolution, Dynamic Range and Spectrum. Tech. rep.Google Scholar