“3D imaging spectroscopy for measuring hyperspectral patterns on solid objects” by Kim, Harvey, Kittle, Rushmeier, Dorsey, et al. …

Sophisticated methods for true spectral rendering have been developed in computer graphics to produce highly accurate images. In addition to traditional applications in visualizing appearance, such methods have potential applications in many areas of scientific study. In particular, we are motivated by the application of studying avian vision and appearance. An obstacle to using graphics in this application is the lack of reliable input data. We introduce an end-to-end measurement system for capturing spectral data on 3D objects. We present the modification of a recently developed hyperspectral imager to make it suitable for acquiring such data in a wide spectral range at high spectral and spatial resolution. We capture four megapixel images, with data at each pixel from the near-ultraviolet (359 nm) to near-infrared (1,003 nm) at 12 nm spectral resolution. We fully characterize the imaging system, and document its accuracy. This imager is integrated into a 3D scanning system to enable the measurement of the diffuse spectral reflectance and fluorescence of specimens. We demonstrate the use of this measurement system in the study of the interplay between the visual capabilities and appearance of birds. We show further the use of the system in gaining insight into artifacts from geology and cultural heritage.

1. 3DCoForm, 2012. MeshLab. http://meshlab.sourceforge.net/.Google Scholar
2. Andersson, S., ornborg, J., and Andersson, M. 1998. Ultraviolet sexual dimorphism and assortative mating in blue tits. In Proc. R. Soc. Lond., vol. 265, 445–450.Google ScholarCross Ref
3. Arnold, K. E., Owens, I. P. F., and Marshall, N. J. 2002. Fluorescent signaling in parrots. Science 295, 5552, 92.Google ScholarCross Ref
4. Attas, M., Cloutis, E., Collins, C., Goltz, D., Majzels, C., Mansfield, J., and Mantsch, H. 2003. Near-infrared spectroscopic imaging in art conservation: investigation of drawing constituents. J. Cultural Heritage 4, 2, 127–136.Google ScholarCross Ref
5. Battle, D. R. 1997. The measurement of colour. In Colour Physics for Industry, R. McDonald, Ed., 2nd ed. Soc. Dyers Col., Bradford, 57–80.Google Scholar
6. Bennett, A. T. D., Cuthill, I. C., Partridge, J. C., and Maier, E. J. 1996. Ultraviolet vision and mate choice in zebra finches. Nature 380, 433–435.Google ScholarCross Ref
7. Bernardini, F., and Rushmeier, H. 2002. The 3D model acquisition pipeline. Comput. Graph. Forum 21, 2, 149–149.Google ScholarCross Ref
8. Bernardini, F., Martin, I. M., and Rushmeier, H. E. 2001. High-quality texture reconstruction from multiple scans. IEEE Trans. Vis. Comput. Graph. 7, 4, 318–332. Google ScholarDigital Library
9. Bioucas-Dias, J., and Figueiredo, M. 2007. A new twist: two-step iterative shrinkage/thresholding for image restoration. IEEE Trans. Image Processing 16, 12, 2992–3004. Google ScholarDigital Library
10. Brady, D. J. 2008. Optical Imaging and Spectroscopy. Wiley-interscience, New Jersey.Google Scholar
11. Brusco, N., Capeleto, S., Fedel, M., Paviotti, A., Poletto, L., Cortelazzo, G. M., and Tondello, G. 2006. A system for 3D modeling frescoed historical buildings with multispectral texture information. Machine Vision and Appl. 17, 6, 373–393. Google ScholarDigital Library
12. Burns, P. D. 2002. Slanted-edge MTF for digital camera and scanner analysis. In Proc. PICS Conf., IS&T, 135–138.Google Scholar
13. Chambolle, A. 2004. An algorithm for total variation minimization and applications. J. Mathematical Imaging and Vision 20, 1–2, 89–97. Google ScholarDigital Library
14. Chen, D. M., and Goldsmith, T. H. 1986. Four spectral classes of cone in the retinas of birds. J. Comparative Physiology A 159, 4, 473–479.Google ScholarCross Ref
15. Chen, D. M., Collins, J. S., and Goldsmith, T. H. 1984. The ultraviolet receptor of bird retinas. Science 225, 4659, 337–340.Google Scholar
16. CIE. 2001. Improvement to industrial colour difference equation. CIE Pub. 142, Commission Internationale de l’Eclairage, Vienna.Google Scholar
17. Devlin, K., Chalmers, A., Wilkie, A., and Purgathofer, W. 2002. Tone reproduction and physically based spectral rendering. Eurographics 2002: State of the Art Reports, 101–123.Google Scholar
18. Du, H., Tong, X., Cao, X., and Lin, S. 2009. A prism-based system for multispectral video acquisition. In Proc. Int. Conf. Comput. Vision (ICCV), IEEE, 175–182.Google Scholar
19. Elgazzar, S., Liscano, R., Blais, F., and Miles, A. 1997. 3-D data acquisition for indoor environment modeling using a compact active range sensor. In Proc. the IEEE Instrumentation, Measurement and Technology Conference, 1–8.Google Scholar
20. Farouk, M., Rifai, I. E., Tayar, S. E., Shishiny, H. E., Hosny, M., Rayes, M. E., Gomes, J., Giordano, F., Rushmeier, H. E., Bernardini, F., and Magerlein, K. 2003. Scanning and processing 3D objects for web display. In Proc. Int. Conf. 3D Digital Imaging and Modeling (3DIM), 310–317.Google Scholar
21. Figueiredo, M., Nowak, R., and Wright, S. 2007. Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J. Selected Topics in Signal Processing 1, 4, 586–597.Google ScholarCross Ref
22. Fischer, C., and Kakoulli, I. 2006. Multispectral and hyper-spectral imaging technologies in conservation: current research and potential applications. Reviews in Conservation 7, 3–16.Google Scholar
23. Habel, R., Kudenov, M., and Wimmer, M. 2012. Practical spectral photography. Comput. Graph. Forum 31, 2, 1–10. Google ScholarDigital Library
24. Hart, N. S. 2001. The visual ecology of avian photoreceptors. Progress in Retinal and Eye Research 20, 5, 675–703.Google ScholarCross Ref
25. Holroyd, M., Lawrence, J., and Zickler, T. 2010. A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance. ACM Trans. Graph. (Proc. SIGGRAPH 2010) 29, 3, 99:1–12. Google ScholarDigital Library
26. ISO. 2000. Photography — electronic still-picture cameras — resolution measurements. Tech. Rep. ISO 12233:2000, International Organization for Standardization (ISO).Google Scholar
27. Kawakami, R., Wright, J., Tai, Y.-W., Matsushita, Y., Ben-Ezra, M., and Ikeuchi, K. 2011. High-resolution hyperspectral imaging via matrix factorization. In Proc. IEEE Conf. Comput. Vision and Pattern Recognition, 2329–2336. Google ScholarDigital Library
28. Kittle, D., Choi, K., Wagadarikar, A., and Brady, D. J. 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Appl. Opt. 49, 36, 6824–6833.Google ScholarCross Ref
29. Labshpere, 2011. Spectralon diffuse reflectance standards. http://www.labsphere.com/uploads/datasheets/diffuse-reflectance-standards-product-sheet.pdf.Google Scholar
30. Land, M. F., and Nilsson, D.-E., Eds. 2002. Animal Eyes. Oxford University Press, Oxford.Google Scholar
31. Lensch, H., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H. 2003. Image-based reconstruction of spatial appearance and geometric detail. ACM Trans. Graph. (Proc. SIGGRAPH 2003) 22, 2, 234–257. Google ScholarDigital Library
32. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S. E., Davis, J., Ginsberg, J., Shade, J., and Fulk, D. 2000. The digital Michelangelo project: 3D scanning of large statues. In Proc. SIGGRAPH 2000, 131–144. Google ScholarDigital Library
33. Mansouri, A., Lathuiliere, A., Marzani, F. S., Voisin, Y., and Gouton, P. 2007. Toward a 3D multispectral scanner: An application to multimedia. IEEE Multimedia 14, 1, 40–47. Google ScholarDigital Library
34. Mcgraw, K. J. 2006. Mechanics of uncommon colors in birds: pterins, porphyrins, and psittacofulvins. In Bird coloration, Vol. I, Mechanisms and measurements, G. E. Hill and K. J. Mcgraw, Eds. Harvard University Press, Cambridge, Massachusetts.Google Scholar
35. Newsome, D., and Modreski, P. 1981. The colors and spectral distributions of fluorescent minerals. J. the Fluorescent Mineral Society 10, 7–56.Google Scholar
36. Next-Limit-Technologies, 2012. Maxwell Render. http://www.maxwellrender.com/.Google Scholar
37. nVidia, 2012. About iRay, physically correct GPU rendering technology. http://www.mentalimages.com/products/iray/about-iray.html.Google Scholar
38. Qin, J. 2010. Hyperspectral imaging instruments. In Hyperspectral Imaging for Food Quality Analysis and Control, D.-W. Sun, Ed. Elsevier, 129–175.Google Scholar
39. Rapantzikos, K., and Balas, C. 2005. Hyperspectral imaging: potential in non-destructive analysis of palimpsests. In Proc. Int. Conf. Image Processing (ICIP), vol. 2, 618–621.Google Scholar
40. Saunders, D., and Cupitt, J. 1993. Image processing at the National Gallery: The VASARI project. National Gallery Tech. Bull. 14, 72–86.Google Scholar
41. Stoddard, M. C., and Prum, R. O. 2008. Evolution of avian plumage color in a tetrahedral color space: a phylogenetic analysis of new world buntings. The American Naturalist 171, 6, 755–776.Google ScholarCross Ref
42. Stoddard, M. C., and Prum, R. O. 2011. How colorful are birds? Evolution of the avian plumage color gamut. Behavioral Ecology 22, 5, 1042–1052.Google ScholarCross Ref
43. Sugiura, H., Kuno, T., Watanabe, N., Matoba, N., Hayashi, J., and Miyata, Y. 2000. Development of a multi-spectral camera system. In Proc. SPIE 3965, 331–339.Google Scholar
44. Sun, X., and Pitsianis, N. 2008. Solving non-negative linear inverse problems with the NeAREst method. Proc. SPIE 7074, 707402.Google Scholar
45. Thoury, M., Elias, M., Frigerio, J. M., and Barthou, C. 2005. Non-destructive identification of varnishes by UV fluorescence spectroscopy. In Proc. SPIE 5857, 1–11.Google Scholar
46. Vos, J. J. 1978. Colorimetric and photometric properties of a 2-deg fundamental observer. Color Res. Appl. 3, 125–128.Google ScholarCross Ref
47. Wagadarikar, A., John, R., Willett, R., and Brady, D. J. 2008. Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt. 47, 10, B44–B51.Google ScholarCross Ref
48. Wagadarikar, A. A., Pitsianis, N. P., Sun, X., and Brady, D. J. 2009. Video rate spectral imaging using a coded aperture snapshot spectral imager. Opt. Express 17, 8, 6368–6388.Google ScholarCross Ref
49. Ware, G., Chabries, D., Christiansen, R., Brady, J., and Martin, C. 2000. Multispectral analysis of ancient Maya pigments: implications for the Naj Tunich Corpus. In Proc. IEEE Geoscience and Remote Sensing Symposium, vol. 6, 2489–2491.Google Scholar
50. Wright, S., Nowak, R., and Figueiredo, M. 2009. Sparse reconstruction by separable approximation. IEEE Trans. Signal Processing 57, 7, 2479–2493. Google ScholarDigital Library
51. Zhao, Y., Berns, R. S., Taplin, L. A., and Coddington, J. 2008. An investigation of multispectral imaging for the mapping of pigments in paintings. In Proc. SPIE 6810, 1–9.Google Scholar