“Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting” by Wetzstein, Lanman, Hirsch and Raskar

We introduce tensor displays: a family of compressive light field displays comprising all architectures employing a stack of time-multiplexed, light-attenuating layers illuminated by uniform or directional backlighting (i.e., any low-resolution light field emitter). We show that the light field emitted by an N-layer, M-frame tensor display can be represented by an Nth-order, rank-M tensor. Using this representation we introduce a unified optimization framework, based on nonnegative tensor factorization (NTF), encompassing all tensor display architectures. This framework is the first to allow joint multilayer, multiframe light field decompositions, significantly reducing artifacts observed with prior multilayer-only and multiframe-only decompositions; it is also the first optimization method for designs combining multiple layers with directional backlighting. We verify the benefits and limitations of tensor displays by constructing a prototype using modified LCD panels and a custom integral imaging backlight. Our efficient, GPU-based NTF implementation enables interactive applications. Through simulations and experiments we show that tensor displays reveal practical architectures with greater depths of field, wider fields of view, and thinner form factors, compared to prior automultiscopic displays.

1. Akeley, K., Watt, S. J., Girshick, A. R., and Banks, M. S. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) 23, 804–813. Google ScholarDigital Library
2. Bader, G., Ott, P., Lueder, E., and Schmid, V. 1997. Hybrid shape recognition system with microlens array processor and direct optical input. In SPIE Optical Pattern Recognition VIII, vol. 3073, 277–287.Google ScholarCross Ref
3. Bilgili, A., Ozturk, A., and Kurt, M. 2011. A general BRDF representation based on tensor decomposition. Computer Graphics Forum 30, 8, 2427–2439.Google ScholarCross Ref
4. Blondel, V., Ho, N.-D., and van Dooren, P. 2008. Weighted nonnegative matrix factorization and face feature extraction. In Image and Vision Computing, 1–17.Google Scholar
5. Brott, R., and Schultz, J. 2010. Directional backlight light-guide considerations for full resolution autostereoscopic 3D displays. SID Digest, 218–221.Google Scholar
6. Chai, J.-X., Tong, X., Chan, S.-C., and Shum, H.-Y. 2000. Plenoptic sampling. In ACM SIGGRAPH, 307–318. Google ScholarDigital Library
7. Chien, K.-W., and Shieh, H.-P. D. 2006. Time-multiplexed three-dimensional displays based on directional backlights with fast-switching liquid-crystal displays. Applied Optics 45, 13, 3106–3110.Google ScholarCross Ref
8. Chu, Y. M., Chien, K. W., Shieh, H. P. D., Chang, J. M., A. Hu, Y. C. S., and Yang, V. 2005. 3D mobile display based on dual-directional light guides with a fast-switching liquid-crystal panel. J. Soc. Inf. Display 13, 10, 875–879.Google ScholarCross Ref
9. Cichocki, A., Zdunek, R., Phan, A. H., and ichi Amari, S. 2009. Nonnegative Matrix and Tensor Factorizations. Wiley. Google ScholarDigital Library
10. Cossairt, O. S., Napoli, J., Hill, S. L., Dorval, R. K., and Favalora, G. E. 2007. Occlusion-capable multiview volumetric three-dimensional display. Applied Optics 46, 8, 1244–1250.Google ScholarCross Ref
11. Durand, F., Holzschuch, N., Soler, C., Chan, E., and Sillion, F. X. 2005. A frequency analysis of light transport. ACM Trans. Graph. (SIGGRAPH) 24, 3, 1115–1126. Google ScholarDigital Library
12. Favalora, G. E. 2005. Volumetric 3D displays and application infrastructure. IEEE Computer 38, 37–44. Google ScholarDigital Library
13. Gotoda, H. 2010. A multilayer liquid crystal display for autostereoscopic 3D viewing. In SPIE Stereoscopic Displays and Applications XXI, vol. 7524, 1–8.Google Scholar
14. Gotoda, H. 2011. Reduction of image blurring in an autostereoscopic multilayer liquid crystal display. In SPIE Stereoscopic Displays and Applications XXII, vol. 7863, 1–7.Google Scholar
15. Holroyd, M., Baran, I., Lawrence, J., and Matusik, W. 2011. Computing and fabricating multilayer models. ACM Trans. Graph. (SIGGRAPH Asia) 30, 187:1–187:8. Google ScholarDigital Library
16. Ives, F. E., 1903. Parallax stereogram and process of making same. U. S. Patent 725,567.Google Scholar
17. Jacobs, A., Mather, J., Winlow, R., Montgomery, D., Jones, G., Willis, M., Tillin, M., Hill, L., Khazova, M., Stevenson, H., and Bourhill, G. 2003. 2D/3D switchable displays. Sharp Technical Journal, 4, 1–5.Google Scholar
18. Jones, A., McDowall, I., Yamada, H., Bolas, M., and Debevec, P. 2007. Rendering for an interactive 360° light field display. ACM Trans. Graph. (SIGGRAPH) 26, 40:1–40:10. Google ScholarDigital Library
19. Kim, Y., Kim, J., Kang, J.-M., Jung, J.-H., Choi, H., and Lee, B. 2007. Point light source integral imaging with improved resolution and viewing angle by the use of electrically movable pinhole array. Optics Express 15, 26, 18253–18267.Google ScholarCross Ref
20. Kolda, T. G., and Bader, B. W. 2009. Tensor decompositions and applications. SIAM Review 51, 3, 455–500. Google ScholarDigital Library
21. Kwon, H., and Choi, H.-J. 2012. A time-sequential multiview autostereoscopic display without resolution loss using a multi-directional backlight unit and an LCD panel. In SPIE Stereoscopic Displays and Applications XXIII, vol. 8288, 1–6.Google Scholar
22. Lanman, D., Hirsch, M., Kim, Y., and Raskar, R. 2010. Content-adaptive parallax barriers: Optimizing dual-layer 3D displays using low-rank light field factorization. ACM Trans. Graph. (SIGGRAPH Asia) 29, 163:1–163:10. Google ScholarDigital Library
23. Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization fields: Dynamic light field display using multi-layer LCDs. ACM Trans. Graph. (SIGGRAPH Asia) 3, 1–9. Google ScholarDigital Library
24. Lawrence, J., Ben-Artzi, A., DeCoro, C., Matusik, W., Pfister, H., Ramamoorthi, R., and Rusinkiewicz, S. 2006. Inverse shade trees for non-parametric material representation and editing. ACM Trans. Graph. (SIGGRAPH) 25, 3. Google ScholarDigital Library
25. Lippmann, G. 1908. Épreuves réversibles donnant la sensation du relief. Journal of Physics 7, 4, 821–825.Google Scholar
26. Lumière, L. 1920. Représentation photographique d’un solide dans l’espace. photo-stéréo-synthèse. Comptes rendus hebdomadaires des séances de l’Académie des sciences, 891–896.Google Scholar
27. Mather, J., Barratt, N., Kean, D. U., Walton, E. J., and Bourhill, G., 2009. Directional backlight, a multiple view display and a multi-direction display. U. S. Patent Application 11/814,383.Google Scholar
28. Matusik, W., and Pfister, H. 2004. 3D TV: A scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Trans. Graph. (SIGGRAPH) 23, 814–824. Google ScholarDigital Library
29. Peers, P., vom Berge, K., Matusik, W., Ramamoorthi, R., Lawrence, J., Rusinkiewicz, S., and Dutré, P. 2006. A compact factored representation of heterogeneous subsurface scattering. ACM Trans. Graph. (SIGGRAPH) 25, 3, 746–753. Google ScholarDigital Library
30. Perlin, K., Paxia, S., and Kollin, J. S. 2000. An autostereoscopic display. In ACM SIGGRAPH, 319–326. Google ScholarDigital Library
31. Peterka, T., Kooima, R. L., Sandin, D. J., Johnson, A., Leigh, J., and DeFanti, T. A. 2008. Advances in the Dynal-lax solid-state dynamic parallax barrier autostereoscopic visualization display system. IEEE TVCG 14, 3, 487–499. Google ScholarDigital Library
32. Putilin, A. N., Lukianitsa, A. A., and Kanashin, K. 2001. Stereodisplay with neural network image processing. In SPIE Advanced Display Technologies, vol. 4511, 245–250.Google Scholar
33. Steyn, J., Brosnihan, T., Fijol, J., Gandhi, J., Hagood, N., Halfman, M., Lewis, S., Payne, R., and Wu, J. 2010. A MEMS digital microshutter (DMS) for low-power high brightness displays. In Optical MEMS and Nanophotonics, 73–74.Google Scholar
34. Stolle, H., Olaya, J.-C., Buschbeck, S., Sahm, H., and Schwerdtner, A. 2008. Technical solutions for a full-resolution autostereoscopic 2D/3D display technology. In Proc. SPIE, 1–12.Google Scholar
35. Sullivan, A. 2003. A solid-state multi-planar volumetric display. In SID Digest, vol. 32, 207–211.Google Scholar
36. Toyooka, K., Miyashita, T., and Uchida, T. 2001. The 3D display using field-sequential LCD with light direction controlling backlight. SID Digest, 177–180.Google Scholar
37. Travis, A., Large, T., Emerton, N., and Bathiche, S. 2009. Collimated light from a waveguide for a display backlight. Optics Express 17, 22, 19714–19719.Google ScholarCross Ref
38. Travis, A. R. L. 1990. Autostereoscopic 3-D display. Applied Optics 29, 29, 4341–4342.Google ScholarCross Ref
39. Vasilescu, M. A. O., and Terzopoulos, D. 2004. TensorTextures: Multilinear image-based rendering. ACM Trans. Graph. (SIGGRAPH) 23, 336–342. Google ScholarDigital Library
40. Wang, H., Wu, Q., Shi, L., Yu, Y., and Ahuja, N. 2005. Out-of-core tensor approximation of multi-dimensional matrices of visual data. ACM Trans. Graph. (SIGGRAPH) 24, 527–535. Google ScholarDigital Library
41. Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic image synthesis for attenuation-based light field and high dynamic range displays. ACM Trans. Graph. (SIGGRAPH) 30, 1–11. Google ScholarDigital Library
42. Zwicker, M., Matusik, W., Durand, F., and Pfister, H. 2006. Antialiasing for automultiscopic 3D displays. In EGSR. Google ScholarDigital Library