“3D imaging spectroscopy for measuring hyperspectral patterns on solid objects” by Kim, Harvey, Kittle, Rushmeier, Dorsey, et al. …

  • ©Min H. Kim, Todd Alan Harvey, David S. Kittle, Holly E. Rushmeier, Julie Dorsey, Richard O. Prum, and David J. Brady




    3D imaging spectroscopy for measuring hyperspectral patterns on solid objects



    Sophisticated methods for true spectral rendering have been developed in computer graphics to produce highly accurate images. In addition to traditional applications in visualizing appearance, such methods have potential applications in many areas of scientific study. In particular, we are motivated by the application of studying avian vision and appearance. An obstacle to using graphics in this application is the lack of reliable input data. We introduce an end-to-end measurement system for capturing spectral data on 3D objects. We present the modification of a recently developed hyperspectral imager to make it suitable for acquiring such data in a wide spectral range at high spectral and spatial resolution. We capture four megapixel images, with data at each pixel from the near-ultraviolet (359 nm) to near-infrared (1,003 nm) at 12 nm spectral resolution. We fully characterize the imaging system, and document its accuracy. This imager is integrated into a 3D scanning system to enable the measurement of the diffuse spectral reflectance and fluorescence of specimens. We demonstrate the use of this measurement system in the study of the interplay between the visual capabilities and appearance of birds. We show further the use of the system in gaining insight into artifacts from geology and cultural heritage.


    1. 3DCoForm, 2012. MeshLab. http://meshlab.sourceforge.net/.Google Scholar
    2. Andersson, S., ornborg, J., and Andersson, M. 1998. Ultraviolet sexual dimorphism and assortative mating in blue tits. In Proc. R. Soc. Lond., vol. 265, 445–450.Google ScholarCross Ref
    3. Arnold, K. E., Owens, I. P. F., and Marshall, N. J. 2002. Fluorescent signaling in parrots. Science 295, 5552, 92.Google ScholarCross Ref
    4. Attas, M., Cloutis, E., Collins, C., Goltz, D., Majzels, C., Mansfield, J., and Mantsch, H. 2003. Near-infrared spectroscopic imaging in art conservation: investigation of drawing constituents. J. Cultural Heritage 4, 2, 127–136.Google ScholarCross Ref
    5. Battle, D. R. 1997. The measurement of colour. In Colour Physics for Industry, R. McDonald, Ed., 2nd ed. Soc. Dyers Col., Bradford, 57–80.Google Scholar
    6. Bennett, A. T. D., Cuthill, I. C., Partridge, J. C., and Maier, E. J. 1996. Ultraviolet vision and mate choice in zebra finches. Nature 380, 433–435.Google ScholarCross Ref
    7. Bernardini, F., and Rushmeier, H. 2002. The 3D model acquisition pipeline. Comput. Graph. Forum 21, 2, 149–149.Google ScholarCross Ref
    8. Bernardini, F., Martin, I. M., and Rushmeier, H. E. 2001. High-quality texture reconstruction from multiple scans. IEEE Trans. Vis. Comput. Graph. 7, 4, 318–332. Google ScholarDigital Library
    9. Bioucas-Dias, J., and Figueiredo, M. 2007. A new twist: two-step iterative shrinkage/thresholding for image restoration. IEEE Trans. Image Processing 16, 12, 2992–3004. Google ScholarDigital Library
    10. Brady, D. J. 2008. Optical Imaging and Spectroscopy. Wiley-interscience, New Jersey.Google Scholar
    11. Brusco, N., Capeleto, S., Fedel, M., Paviotti, A., Poletto, L., Cortelazzo, G. M., and Tondello, G. 2006. A system for 3D modeling frescoed historical buildings with multispectral texture information. Machine Vision and Appl. 17, 6, 373–393. Google ScholarDigital Library
    12. Burns, P. D. 2002. Slanted-edge MTF for digital camera and scanner analysis. In Proc. PICS Conf., IS&T, 135–138.Google Scholar
    13. Chambolle, A. 2004. An algorithm for total variation minimization and applications. J. Mathematical Imaging and Vision 20, 1–2, 89–97. Google ScholarDigital Library
    14. Chen, D. M., and Goldsmith, T. H. 1986. Four spectral classes of cone in the retinas of birds. J. Comparative Physiology A 159, 4, 473–479.Google ScholarCross Ref
    15. Chen, D. M., Collins, J. S., and Goldsmith, T. H. 1984. The ultraviolet receptor of bird retinas. Science 225, 4659, 337–340.Google Scholar
    16. CIE. 2001. Improvement to industrial colour difference equation. CIE Pub. 142, Commission Internationale de l’Eclairage, Vienna.Google Scholar
    17. Devlin, K., Chalmers, A., Wilkie, A., and Purgathofer, W. 2002. Tone reproduction and physically based spectral rendering. Eurographics 2002: State of the Art Reports, 101–123.Google Scholar
    18. Du, H., Tong, X., Cao, X., and Lin, S. 2009. A prism-based system for multispectral video acquisition. In Proc. Int. Conf. Comput. Vision (ICCV), IEEE, 175–182.Google Scholar
    19. Elgazzar, S., Liscano, R., Blais, F., and Miles, A. 1997. 3-D data acquisition for indoor environment modeling using a compact active range sensor. In Proc. the IEEE Instrumentation, Measurement and Technology Conference, 1–8.Google Scholar
    20. Farouk, M., Rifai, I. E., Tayar, S. E., Shishiny, H. E., Hosny, M., Rayes, M. E., Gomes, J., Giordano, F., Rushmeier, H. E., Bernardini, F., and Magerlein, K. 2003. Scanning and processing 3D objects for web display. In Proc. Int. Conf. 3D Digital Imaging and Modeling (3DIM), 310–317.Google Scholar
    21. Figueiredo, M., Nowak, R., and Wright, S. 2007. Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J. Selected Topics in Signal Processing 1, 4, 586–597.Google ScholarCross Ref
    22. Fischer, C., and Kakoulli, I. 2006. Multispectral and hyper-spectral imaging technologies in conservation: current research and potential applications. Reviews in Conservation 7, 3–16.Google Scholar
    23. Habel, R., Kudenov, M., and Wimmer, M. 2012. Practical spectral photography. Comput. Graph. Forum 31, 2, 1–10. Google ScholarDigital Library
    24. Hart, N. S. 2001. The visual ecology of avian photoreceptors. Progress in Retinal and Eye Research 20, 5, 675–703.Google ScholarCross Ref
    25. Holroyd, M., Lawrence, J., and Zickler, T. 2010. A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance. ACM Trans. Graph. (Proc. SIGGRAPH 2010) 29, 3, 99:1–12. Google ScholarDigital Library
    26. ISO. 2000. Photography — electronic still-picture cameras — resolution measurements. Tech. Rep. ISO 12233:2000, International Organization for Standardization (ISO).Google Scholar
    27. Kawakami, R., Wright, J., Tai, Y.-W., Matsushita, Y., Ben-Ezra, M., and Ikeuchi, K. 2011. High-resolution hyperspectral imaging via matrix factorization. In Proc. IEEE Conf. Comput. Vision and Pattern Recognition, 2329–2336. Google ScholarDigital Library
    28. Kittle, D., Choi, K., Wagadarikar, A., and Brady, D. J. 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Appl. Opt. 49, 36, 6824–6833.Google ScholarCross Ref
    29. Labshpere, 2011. Spectralon diffuse reflectance standards. http://www.labsphere.com/uploads/datasheets/diffuse-reflectance-standards-product-sheet.pdf.Google Scholar
    30. Land, M. F., and Nilsson, D.-E., Eds. 2002. Animal Eyes. Oxford University Press, Oxford.Google Scholar
    31. Lensch, H., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H. 2003. Image-based reconstruction of spatial appearance and geometric detail. ACM Trans. Graph. (Proc. SIGGRAPH 2003) 22, 2, 234–257. Google ScholarDigital Library
    32. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S. E., Davis, J., Ginsberg, J., Shade, J., and Fulk, D. 2000. The digital Michelangelo project: 3D scanning of large statues. In Proc. SIGGRAPH 2000, 131–144. Google ScholarDigital Library
    33. Mansouri, A., Lathuiliere, A., Marzani, F. S., Voisin, Y., and Gouton, P. 2007. Toward a 3D multispectral scanner: An application to multimedia. IEEE Multimedia 14, 1, 40–47. Google ScholarDigital Library
    34. Mcgraw, K. J. 2006. Mechanics of uncommon colors in birds: pterins, porphyrins, and psittacofulvins. In Bird coloration, Vol. I, Mechanisms and measurements, G. E. Hill and K. J. Mcgraw, Eds. Harvard University Press, Cambridge, Massachusetts.Google Scholar
    35. Newsome, D., and Modreski, P. 1981. The colors and spectral distributions of fluorescent minerals. J. the Fluorescent Mineral Society 10, 7–56.Google Scholar
    36. Next-Limit-Technologies, 2012. Maxwell Render. http://www.maxwellrender.com/.Google Scholar
    37. nVidia, 2012. About iRay, physically correct GPU rendering technology. http://www.mentalimages.com/products/iray/about-iray.html.Google Scholar
    38. Qin, J. 2010. Hyperspectral imaging instruments. In Hyperspectral Imaging for Food Quality Analysis and Control, D.-W. Sun, Ed. Elsevier, 129–175.Google Scholar
    39. Rapantzikos, K., and Balas, C. 2005. Hyperspectral imaging: potential in non-destructive analysis of palimpsests. In Proc. Int. Conf. Image Processing (ICIP), vol. 2, 618–621.Google Scholar
    40. Saunders, D., and Cupitt, J. 1993. Image processing at the National Gallery: The VASARI project. National Gallery Tech. Bull. 14, 72–86.Google Scholar
    41. Stoddard, M. C., and Prum, R. O. 2008. Evolution of avian plumage color in a tetrahedral color space: a phylogenetic analysis of new world buntings. The American Naturalist 171, 6, 755–776.Google ScholarCross Ref
    42. Stoddard, M. C., and Prum, R. O. 2011. How colorful are birds? Evolution of the avian plumage color gamut. Behavioral Ecology 22, 5, 1042–1052.Google ScholarCross Ref
    43. Sugiura, H., Kuno, T., Watanabe, N., Matoba, N., Hayashi, J., and Miyata, Y. 2000. Development of a multi-spectral camera system. In Proc. SPIE 3965, 331–339.Google Scholar
    44. Sun, X., and Pitsianis, N. 2008. Solving non-negative linear inverse problems with the NeAREst method. Proc. SPIE 7074, 707402.Google Scholar
    45. Thoury, M., Elias, M., Frigerio, J. M., and Barthou, C. 2005. Non-destructive identification of varnishes by UV fluorescence spectroscopy. In Proc. SPIE 5857, 1–11.Google Scholar
    46. Vos, J. J. 1978. Colorimetric and photometric properties of a 2-deg fundamental observer. Color Res. Appl. 3, 125–128.Google ScholarCross Ref
    47. Wagadarikar, A., John, R., Willett, R., and Brady, D. J. 2008. Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt. 47, 10, B44–B51.Google ScholarCross Ref
    48. Wagadarikar, A. A., Pitsianis, N. P., Sun, X., and Brady, D. J. 2009. Video rate spectral imaging using a coded aperture snapshot spectral imager. Opt. Express 17, 8, 6368–6388.Google ScholarCross Ref
    49. Ware, G., Chabries, D., Christiansen, R., Brady, J., and Martin, C. 2000. Multispectral analysis of ancient Maya pigments: implications for the Naj Tunich Corpus. In Proc. IEEE Geoscience and Remote Sensing Symposium, vol. 6, 2489–2491.Google Scholar
    50. Wright, S., Nowak, R., and Figueiredo, M. 2009. Sparse reconstruction by separable approximation. IEEE Trans. Signal Processing 57, 7, 2479–2493. Google ScholarDigital Library
    51. Zhao, Y., Berns, R. S., Taplin, L. A., and Coddington, J. 2008. An investigation of multispectral imaging for the mapping of pigments in paintings. In Proc. SPIE 6810, 1–9.Google Scholar

ACM Digital Library Publication: