“Optimal presentation of imagery with focus cues on multi-plane displays” by Narain, Albert, Bulbul, Banks, O’Brien, et al. …
Conference:
Type(s):
Title:
- Optimal presentation of imagery with focus cues on multi-plane displays
Session/Category Title: VR, Display, and Interaction
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a technique for displaying three-dimensional imagery of general scenes with nearly correct focus cues on multi-plane displays. These displays present an additive combination of images at a discrete set of optical distances, allowing the viewer to focus at different distances in the simulated scene. Our proposed technique extends the capabilities of multi-plane displays to general scenes with occlusions and non-Lambertian effects by using a model of defocus in the eye of the viewer. Requiring no explicit knowledge of the scene geometry, our technique uses an optimization algorithm to compute the images to be displayed on the presentation planes so that the retinal images when accommodating to different distances match the corresponding retinal images of the input scene as closely as possible. We demonstrate the utility of the technique using imagery acquired from both synthetic and real-world scenes, and analyze the system’s characteristics including bounds on achievable resolution.
References:
1. Akeley, K., Watt, S. J., Girshick, A. R., and Banks, M. S. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graph. 23, 3 (Aug.), 804–813. Google ScholarDigital Library
2. Andersen, A., and Kak, A. 1984. Simultaneous algebraic reconstruction technique (SART): A superior implementation of the ART algorithm. Ultrasonic Imaging 6, 1, 81–94.Google ScholarCross Ref
3. Banks, M. S., Bulbul, A., Albert, R. A., Narain, R., O’Brien, J. F., and Ward, G. 2014. The perception of surface material from disparity and focus cues. In Proc. Vision Sciences Society 14th Annual Meeting.Google Scholar
4. Buckley, D., and Frisby, J. P. 1993. Interaction of stereo, texture and outline cues in the shape perception of three-dimensional ridges. Vision Research 33, 7, 919–933.Google ScholarCross Ref
5. Campbell, F. 1957. The depth of field of the human eye. Optica Acta: International Journal of Optics 4, 4, 157–164.Google ScholarCross Ref
6. Chambolle, A., and Pock, T. 2011. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision 40, 1, 120–145. Google ScholarDigital Library
7. Coelho, J. M. P., Baião, A., and Vieira, P. 2013. Development of an optical simulator of the human eye. Proc. SPIE 8785, 8785CS–8785CS–8.Google Scholar
8. Cossairt, O. S., Napoli, J., Hill, S. L., Dorval, R. K., and Favalora, G. E. 2007. Occlusion-capable multiview volumetric three-dimensional display. Appl. Opt. 46, 8 (Mar), 1244–1250.Google ScholarCross Ref
9. Du, S.-P., Masia, B., Hu, S.-M., and Gutierrez, D. 2013. A metric of visual comfort for stereoscopic motion. ACM Trans. Graph. 32, 6 (Nov.), 222:1–222:9. Google ScholarDigital Library
10. Emoto, M., Niida, T., and Okano, F. 2005. Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television. Display Technology, Journal of 1, 2, 328–340.Google ScholarCross Ref
11. Esser, E., Zhang, X., and Chan, T. 2010. A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM Journal on Imaging Sciences 3, 4, 1015–1046. Google ScholarDigital Library
12. Favalora, G. E., Napoli, J., Hall, D. M., Dorval, R. K., Giovinco, M., Richmond, M. J., and Chun, W. S. 2002. 100-million-voxel volumetric display. In Proc. SPIE, vol. 4712, 300–312.Google Scholar
13. Granger, E., and Cupery, K. 1972. Optical merit function (SQF), which correlates with subjective image judgments. Photographic Science and Engineering 16, 3.Google Scholar
14. He, B., and Yuan, X. 2012. Convergence analysis of primal-dual algorithms for a saddle-point problem: From contraction perspective. SIAM Journal on Imaging Sciences 5, 1, 119–149. Google ScholarDigital Library
15. Held, R. T., Cooper, E. A., O’Brien, J. F., and Banks, M. S. 2010. Using blur to affect perceived distance and size. ACM Transactions on Graphics 29, 2 (Mar.), 19:1–16. Google ScholarDigital Library
16. Hirsch, M., and Lanman, D., 2010. Build your own 3D display. ACM SIGGRAPH ASIA Course Notes. Google ScholarDigital Library
17. Hoffman, D. M., Girshick, A. R., Akeley, K., and Banks, M. S. 2008. Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. Journal of Vision 8, 3.Google ScholarCross Ref
18. Hu, X., and Hua, H. 2013. An optical see-through multi-focal-plane stereoscopic display prototype enabling nearly correct focus cues. vol. 8648, 86481A.Google Scholar
19. Hu, X., and Hua, H. 2014. Design and assessment of a depth-fused multi-focal-plane display prototype. J. Display Technol. 10, 4 (Apr), 308–316.Google ScholarCross Ref
20. Hu, X., and Hua, H. 2014. High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics. Opt. Express 22, 11 (Jun), 13896–13903.Google ScholarCross Ref
21. Huang, F.-C., Lanman, D., Barsky, B. A., and Raskar, R. 2012. Correcting for optical aberrations using multilayer displays. ACM Transaction on Graphics 31 (Nov.). Google ScholarDigital Library
22. Huang, F.-C., Wetzstein, G., Barsky, B. A., and Raskar, R. 2014. Eyeglasses-free display: Towards correcting visual aberrations with computational light field displays. ACM Trans. Graph. 33, 4 (July), 59:1–59:12. Google ScholarDigital Library
23. Ives, F., 1903. Parallax stereogram and process of making same., Apr. 14. US Patent 725,567.Google Scholar
24. Jakob, W., 2010. Mitsuba renderer. http://www.mitsuba-renderer.org.Google Scholar
25. Jones, A., McDowall, I., Yamada, H., Bolas, M., and Debevec, P. 2007. Rendering for an interactive 360° light field display. ACM Trans. Graph. 26, 3 (July). Google ScholarDigital Library
26. Lambooij, M., Fortuin, M., Heynderickx, I., and Ijsselsteijn, W. 2009. Visual discomfort and visual fatigue of stereoscopic displays: A review. Journal of Imaging Science and Technology 53, 3, 30201-1–30201-14.Google ScholarCross Ref
27. Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., and Gross, M. 2010. Nonlinear disparity mapping for stereoscopic 3D. ACM Trans. Graph. 29, 4 (July), 75:1–75:10. Google ScholarDigital Library
28. Lanman, D., and Luebke, D. 2013. Near-eye light field displays. ACM Trans. Graph. 32, 6 (Nov.), 220:1–220:10. Google ScholarDigital Library
29. Lanman, D., Hirsch, M., Kim, Y., and Raskar, R. 2010. Content-adaptive parallax barriers: Optimizing dual-layer 3D displays using low-rank light field factorization. ACM Trans. Graph. 29, 6 (Dec.), 163:1–163:10. Google ScholarDigital Library
30. Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization fields: Dynamic light field display using multi-layer LCDs. ACM Trans. Graph. 30, 6. Google ScholarDigital Library
31. Levoy, M., Ng, R., Adams, A., Footer, M., and Horowitz, M. 2006. Light field microscopy. ACM Trans. Graph. 25, 3 (July), 924–934. Google ScholarDigital Library
32. Lippmann, G. 1908. Épreuves réversibles donnant la sensation du relief. J. Phys. Theor. Appl. 7, 1, 821–825.Google ScholarCross Ref
33. Liu, S., and Hua, H. 2010. A systematic method for designing depth-fused multi-focal plane three-dimensional displays. Opt. Express 18, 11 (May), 11562–11573.Google Scholar
34. Liu, S., Hua, H., and Cheng, D. 2010. A novel prototype for an optical see-through head-mounted display with addressable focus cues. IEEE Transactions on Visualization and Computer Graphics 16, 3, 381–393. Google ScholarDigital Library
35. Love, G. D., Hoffman, D. M., Hands, P. J., Gao, J., Kirby, A. K., and Banks, M. S. 2009. High-speed switchable lens enables the development of a volumetric stereoscopic display. Opt. Express 17, 18 (Aug), 15716–15725.Google ScholarCross Ref
36. MacKenzie, K. J., Hoffman, D. M., and Watt, S. J. 2010. Accommodation to multiple-focal-plane displays: Implications for improving stereoscopic displays and for accommodation control. Journal of Vision 10, 8.Google ScholarCross Ref
37. MacKenzie, K., Dickson, R., and Watt, S. 2012. Vergence and accommodation to multiple-image-plane stereoscopic displays: “real world” responses with practical image-plane separations? Journal of Electronic Imaging 21, 1.Google ScholarCross Ref
38. Maimone, A., Wetzstein, G., Lanman, D., Hirsch, M., Raskar, R., and Fuchs, H. 2013. Focus 3D: Compressive accommodation display. ACM Trans. Graph. 32, 5, 1–13. Google ScholarDigital Library
39. Mantiuk, R., Kim, K. J., Rempel, A. G., and Heidrich, W. 2011. HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph. 30, 4 (July), 40:1–40:14. Google ScholarDigital Library
40. Marshall, J. A., Burbeck, C. A., Ariely, D., Rolland, J. P., and Martin, K. E. 1996. Occlusion edge blur: a cue to relative visual depth. Journal of the Optical Society of America. A, Optics, image science, and vision 13, 4 (Apr.), 681–8.Google ScholarCross Ref
41. Masia, B., Wetzstein, G., Didyk, P., and Gutierrez, D. 2013. A survey on computational displays: Pushing the boundaries of optics, computation, and perception. Computers and Graphics 37, 8, 1012–1038. Google ScholarDigital Library
42. Mather, G., and Smith, D. R. R. 2002. Blur discrimination and its relation to blur-mediated depth perception. Perception 31, 10, 1211–1219.Google ScholarCross Ref
43. Mathews, S., and Kruger, P. 1994. Spatiotemporal transfer function of human accommodation. Vision Research 34, 15.Google ScholarCross Ref
44. Matusik, W., and Pfister, H. 2004. 3D TV: A scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Trans. Graph. 23, 3 (Aug.), 814–824. Google ScholarDigital Library
45. Mendiburu, B. 2009. 3D movie making: Stereoscopic digital cinema from script to screen. Focal Press, Elsevier.Google Scholar
46. Navarro, R. 2009. The optical design of the human eye: a critical review. Journal of Optometry 2, 1, 3–18.Google ScholarCross Ref
47. Owens, D. 1980. A comparison of accommodative responsiveness and contrast sensitivity for sinusoidal gratings. Vision Research 20, 2, 159–167.Google ScholarCross Ref
48. Palmer, S. E., and Brooks, J. L. 2008. Edge-region grouping in figure-ground organization and depth perception. Journal of Experimental Psychology: Human Perception and Performance 34, 6 (Dec), 1353–1371.Google ScholarCross Ref
49. Pamplona, V. F., Oliveira, M. M., Aliaga, D. G., and Raskar, R. 2012. Tailored displays to compensate for visual aberrations. ACM Trans. Graph. 31, 4 (July), 81:1–81:12. Google ScholarDigital Library
50. Perlin, K., Paxia, S., and Kollin, J. S. 2000. An autostereoscopic display. In Proc. 27th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’00, 319–326. Google ScholarDigital Library
51. Ravikumar, S., Akeley, K., and Banks, M. S. 2011. Creating effective focus cues in multi-plane 3D displays. Opt. Express 19, 21 (Oct), 20940–20952.Google ScholarCross Ref
52. Ryan, L., MacKenzie, K., and Watt, S. 2012. Multiple-focal-planes 3D displays: A practical solution to the vergence-accommodation conflict? In 3D Imaging (IC3D), 2012 International Conference on, 1–6.Google Scholar
53. Shibata, T., Kim, J., Hoffman, D. M., and Banks, M. S. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. Journal of Vision 11, 8.Google ScholarCross Ref
54. Spring, K., and Stiles, W. S. 1948. Variation of pupil size with change in the angle at which the light stimulus strikes the retina. British J. Ophthalmol. 32, 6, 340–346.Google ScholarCross Ref
55. Sullivan, A. 2004. DepthCube solid-state 3D volumetric display. In Proc. SPIE, vol. 5291, 279–284.Google Scholar
56. Takaki, Y., Tanaka, K., and Nakamura, J. 2011. Super multi-view display with a lower resolution flat-panel display. Opt. Express 19, 5 (Feb), 4129–4139.Google ScholarCross Ref
57. Takaki, Y. 2006. High-density directional display for generating natural three-dimensional images. Proc. IEEE 94, 3, 654–663.Google ScholarCross Ref
58. van Ee, R., Banks, M. S., and T., B. B. 1999. An analysis of binocular slant contrast. Perception 28, 9, 1121–1145.Google ScholarCross Ref
59. Watt, S. J., Akeley, K., Ernst, M. O., and Banks, M. S. 2005. Focus cues affect perceived depth. Journal of Vision 5, 10.Google ScholarCross Ref
60. Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic image synthesis for attenuation-based light field and high dynamic range displays. ACM Trans. Graph. 30, 4. Google ScholarDigital Library
61. Wetzstein, G., Lanman, D., Gutierrez, D., and Hirsch, M., 2012. Computational displays: Combining optical fabrication, computational processing, and perceptual tricks to build the displays of the future. ACM SIGGRAPH Course Notes. Google ScholarDigital Library
62. Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4, 1–11. Google ScholarDigital Library
63. Zannoli, M., Albert, R. A., Bulbul, A., Narain, R., O’Brien, J. F., and Banks, M. S. 2014. Correct blur and accommodation information is a reliable cue to depth ordering. In Proc. Vision Sciences Society 14th Annual Meeting.Google Scholar
64. Zhu, M., and Chan, T. 2008. An efficient primal-dual hybrid gradient algorithm for total variation image restoration. Tech. rep., University of California, Los Angeles.Google Scholar
65. Zwicker, M., Matusik, W., Durand, F., Pfister, H., and Forlines, C. 2006. Antialiasing for automultiscopic 3D displays. In ACM SIGGRAPH 2006 Sketches, SIGGRAPH ’06. Google ScholarDigital Library
