“Compact single-shot hyperspectral imaging using a prism” by Baek, Kim, Gutierrez and Kim
Conference:
Type(s):
Title:
- Compact single-shot hyperspectral imaging using a prism
Session/Category Title: High Performance Imaging
Presenter(s)/Author(s):
Abstract:
We present a novel, compact single-shot hyperspectral imaging method. It enables capturing hyperspectral images using a conventional DSLR camera equipped with just an ordinary refractive prism in front of the camera lens. Our computational imaging method reconstructs the full spectral information of a scene from dispersion over edges. Our setup requires no coded aperture mask, no slit, and no collimating optics, which are necessary for traditional hyperspectral imaging systems. It is thus very cost-effective, while still highly accurate. We tackle two main problems: First, since we do not rely on collimation, the sensor records a projection of the dispersion information, distorted by perspective. Second, available spectral cues are sparse, present only around object edges. We formulate an image formation model that can predict the perspective projection of dispersion, and a reconstruction method that can estimate the full spectral information of a scene from sparse dispersion information. Our results show that our method compares well with other state-of-the-art hyperspectral imaging systems, both in terms of spectral accuracy and spatial resolution, while being orders of magnitude cheaper than commercial imaging systems.
References:
1. Manya V. Afonso, José M. Bioucas-Dias, and Mário A. T. Figueiredo. 2011. An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems. IEEE Trans. Image Processing 20, 3 (2011), 681–695.
2. Sara Alvarez, Timo Kunkel, and Belen Masia. 2016. Practical Low-Cost Recovery of Spectral Power Distributions. Computer Graphics Forum 35, 1 (2016), 166–178.
3. Seung-Hwan Baek, Diego Gutierrez, and Min H Kim. 2016. Birefractive stereo imaging for single-shot depth acquisition. ACM Transactions on Graphics 35, 6 (2016), 194.
4. Nicola Brusco, S Capeleto, M Fedel, Anna Paviotti, Luca Poletto, Guido Maria Cortelazzo, and G Tondello. 2006. A system for 3D modeling frescoed historical buildings with multispectral texture information. Machine Vision and Applications 17, 6 (2006), 373–393.
5. Xun Cao, Hao Du, Xin Tong, Qionghai Dai, and Stephen Lin. 2011a. A prism-mask system for multispectral video acquisition. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 12 (2011), 2423–2435.
6. Xun Cao, Xin Tong, Qionghai Dai, and S. Lin. 2011b. High Resolution Multispectral Video Capture with a Hybrid Camera System. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’11). 297–304.
7. Inchang Choi, Daniel S. Jeon, Giljoo Nam, Diego Gutierrez, and Min H. Kim. 2017. High-Quality Hyperspectral Reconstruction Using a Spectral Prior. ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2017) 36, 6 (2017).
8. J. M. Bioucas Dias and M. A. T. Figueiredo. 2007. A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration. IEEE Trans. Image Processing 16, 12 (Dec. 2007), 2992–3004.
9. Piotr Dollár and C Lawrence Zitnick. 2015. Fast edge detection using structured forests. IEEE transactions on pattern analysis and machine intelligence 37, 8 (2015), 1558–1570.
10. Liang Gao, Robert T. Kester, Nathan Hagen, and Tomasz S. Tkaczyk. 2010. Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy. Opt. Express 18, 14 (Jul 2010), 14330–14344. Cross Ref
11. Nahum Gat. 2000. Imaging spectroscopy using tunable filters: a review. In AeroSense 2000. International Society for Optics and Photonics, 50–64.
12. M E Gehm, R John, D J Brady, R M Willett, and T J Schulz. 2007. Single-shot compressive spectral imaging with a dual-disperser architecture. OSA OE 15, 21 (2007), 14013–27. Cross Ref
13. Ralf Habel, Michael Kudenov, and Michael Wimmer. 2012. Practical spectral photography. In Computer graphics forum, Vol. 31. Wiley Online Library, 449–458.
14. Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 6 (2013), 1397–1409.
15. Felix Heide, Mushfiqur Rouf, Matthias B Hullin, Bjorn Labitzke, Wolfgang Heidrich, and Andreas Kolb. 2013. High-quality computational imaging through simple lenses. ACM Transactions on Graphics 32, 5 (2013), 149.
16. Daniel S Jeon, Inchang Choi, and Min H Kim. 2016. Multisampling Compressive Video Spectroscopy. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 467–477.
17. Jun Jiang, Dengyu Liu, Jinwei Gu, and Sabine Süsstrunk. 2013. What is the space of spectral sensitivity functions for digital color cameras?. In Applications of Computer Vision (WACV), 2013 IEEE Workshop on. IEEE, 168–179.
18. William R Johnson, Daniel W Wilson, Wolfgang Fink, Mark Humayun, and Greg Bearman. 2007. Snapshot hyperspectral imaging in ophthalmology. Journal of biomedical optics 12, 1 (2007), 014036–014036. Cross Ref
19. R. Kawakami, Y. Matsushita, J. Wright, M. Ben-Ezra, Y. W. Tai, and K. Ikeuchi. 2011. High-resolution hyperspectral imaging via matrix factorization. In CVPR 2011. 2329–2336.
20. Min H Kim. 2013. 3D Graphics Techniques for Capturing and Inspecting Hyperspectral Appearance. In Ubiquitous Virtual Reality (ISUVR), 2013 Int. Symp. on. IEEE, 15–18.
21. Min H Kim, Todd Alan Harvey, David S Kittle, Holly Rushmeier, Julie Dorsey, Richard O Prum, and David J Brady. 2012a. 3D imaging spectroscopy for measuring hyper-spectral patterns on solid objects. ACM Transactions on Graphics 31, 4 (2012), 38.
22. Min H Kim, Holly Rushmeier, John ffrench, and Irma Passeri. 2012b. Developing Open-Source Software for Art Conservators. In VAST12: The 13th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage. Eurographics Association, Brighton, England, 97–104.
23. Min H Kim, Holly Rushmeier, John ffrench, Irma Passeri, and David Tidmarsh. 2014. Hyper3D: 3D Graphics Software for Examining Cultural Artifacts. ACM Journal on Computingand Cultural Heritage 7, 3 (2014), 1:1–19.
24. David Kittle, Kerkil Choi, Ashwin Wagadarikar, and David J Brady. 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Applied Optics 49, 36 (2010), 6824–6833. Cross Ref
25. Haebom Lee and Min H Kim. 2014. Building a Two-Way Hyperspectral Imaging System with Liquid Crystal Tunable Filters. In Proc. Int. Conf. Image and Signal Processing (ICISP 2014) (LNCS), Vol. 8509. Springer, Normandy, France, 26–34. Cross Ref
26. Chengbo Li. 2011. Compressive sensing for 3D data processing tasks: applications, models and algorithms. Ph.D. Dissertation. Rice University.
27. Xing Lin, Yebin Liu, Jiamin Wu, and Qionghai Dai. 2014. Spatial-spectral encoded compressive hyperspectral imaging. ACM Transactions on Graphics 33, 6 (2014), 233.
28. Rang MH Nguyen, Dilip K Prasad, and Michael S Brown. 2014. Training-Based Spectral Reconstruction from a Single RGB Image. In Computer Vision—ECCV 2014. Springer, 186–201.
29. Takayuki Okamoto, Akinori Takahashi, and Ichirou Yamaguchi. 1993. Simultaneous Acquisition of Spectral and Spatial Intensity Distribution. Appl. Spectrosc. 47, 8 (Aug 1993), 1198–1202. Cross Ref
30. Patrick Pérez, Michel Gangnet, and Andrew Blake. 2003. Poisson image editing. In ACM Transactions on Graphics, Vol. 22. ACM, 313–318.
31. Wallace M Porter and Harry T Enmark. 1987. A system overview of the airborne visible/infrared imaging spectrometer (AVIRIS). In 31st Annual Technical Symposium. International Society for Optics and Photonics, 22–31. Cross Ref
32. Konstantinos Rapantzikos and Costas Balas. 2005. Hyperspectral imaging: potential in non-destructive analysis of palimpsests. In IEEE International Conference on Image Processing 2005, Vol. 2. IEEE, II-618. Cross Ref
33. Brian Smits. 1999. An RGB-to-spectrum conversion for reflectances. Journal of Graphics Tools 4, 4 (1999), 11–22.
34. Tsuyoshi Takatani, Takahito Aoto, and Yasuhiro Mukaigawa. 2017. One-shot Hyperspectral Imaging using Faced Reflectors. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). TBD. Cross Ref
35. Ashwin Wagadarikar, Renu John, Rebecca Willett, and David Brady. 2008. Single disperser design for coded aperture snapshot spectral imaging. Applied optics 47, 10 (2008), B44–B51.
36. Richard A Waltz, José Luis Morales, Jorge Nocedal, and Dominique Orban. 2006. An interior algorithm for nonlinear optimization that combines line search and trust region steps. Mathematical programming 107, 3 (2006), 391–408.
37. Fumihito Yasuma, Tomoo Mitsunaga, Daisuke Iso, and Shree K Nayar. 2010. Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE Transactions on Image Processing 19, 9 (2010), 2241–2253.
38. Zhengyou Zhang. 2000. A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence 22, 11 (2000), 1330–1334.
