“Hiding of phase-based stereo disparity for ghost-free viewing without glasses”
Conference:
Type:
Title:
- Hiding of phase-based stereo disparity for ghost-free viewing without glasses
Session/Category Title: Computational Cameras & Displays
Presenter(s)/Author(s):
Moderator(s):
Abstract:
When a conventional stereoscopic display is viewed without stereo glasses, image blurs, or ‘ghosts’, are visible due to the fusion of stereo image pairs. This artifact severely degrades 2D image quality, making it difficult to simultaneously present clear 2D and 3D contents. To overcome this limitation (backward incompatibility), here we propose a novel method to synthesize ghost-free stereoscopic images. Our method gives binocular disparity to a 2D image, and drives human binocular disparity detectors, by the addition of a quadrature-phase pattern that induces spatial subband phase shifts. The disparity-inducer patterns added to the left and right images are identical except for the contrast polarity. Physical fusion of the two images cancels out the disparity-inducer components and makes only the original 2D pattern visible to viewers without glasses. Unlike previous solutions, our method perfectly excludes stereo ghosts without using special hardware. A simple algorithm can transform 3D contents from the conventional stereo format into ours. Furthermore, our method can alter the depth impression of a real object without its being noticed by naked-eye viewers by means of light projection of the disparity-inducer components onto the object’s surface. Psychophysical evaluations have confirmed the practical utility of our method.
References:
1. Edward. H. Adelson and James. R. Bergen. 1991. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing. MIT Press, 3–20.Google Scholar
2. Stuart Anstis and Alan Ho. 1998. Nonlinear combination of luminance excursions during flicker, simultaneous contrast, afterimages and binocular fusion. Vision Research 38, 4 (Feb. 1998), 523–539. Google ScholarCross Ref
3. Stuart M. Anstis, Ian P. Howard, and Brian Rogers. 1978. A Craik-O’Brien-Cornsweet illusion for visual depth. Vision Research 18, 2 (1978), 213–217. Google ScholarCross Ref
4. Martin S. Banks, Sergei Gepshtein, and Michael S. Landy. 2004. Why is spatial stereoresolution so low? The Journal of Neuroscience: The Official Journal of the Society for Neuroscience 24, 9 (March 2004), 2077–2089. Google ScholarCross Ref
5. Oliver Bimber, Daisuke Iwai, Gordon Wetzstein, and Anselm Grundhöfer. 2007. The Visual Computing of Projector-Camera Systems. In Eurographics 2007 – State of the Art Reports.Google Scholar
6. Randolph Blake and Hugh Wilson. 2011. Binocular vision. Vision Research 51, 7 (April 2011), 754–770. Google ScholarCross Ref
7. Mark F. Bradshaw and Brian J. Rogers. 1999. Sensitivity to horizontal and vertical corrugations defined by binocular disparity. Vision Research 39, 18 (Sept. 1999), 3049–3056. Google ScholarCross Ref
8. Allen Brookes and Kent A. Stevens. 1989. The Analogy between Stereo Depth and Brightness. Perception 18, 5 (Oct. 1989), 601–614. Google ScholarCross Ref
9. ITU-RRec. BT.500-13. 2012. Methodology for the subjective assessment of the quality of television pictures.Google Scholar
10. Tom Cornsweet. 2012. Visual Perception. Academic Press.Google Scholar
11. Bruce G. Cumming and Gregory C. DeAngelis. 2001. The Physiology of Stereopsis. Annual Review of Neuroscience 24 (2001), 203–238. Google ScholarCross Ref
12. H. de Lange Dzn. 1958. Research into the Dynamic Nature of the Human Fovea → Cortex Systems with Intermittent and Modulated Light I Attenuation Characteristics with White and Colored Light. Journal of the Optical Society of America 48, 11 (Nov. 1958), 777.Google Scholar
13. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2011. A Perceptual Model for Disparity. ACM Transactions on Graphics (Proceedings SIGGRAPH 2011, Vancouver) 30, 4 (2011).Google ScholarDigital Library
14. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2012. Apparent stereo: the Cornsweet illusion can enhance perceived depth. In Human Vision and Electronic Imaging XVII, IS&T/SPIE’s Symposium on Electronic Imaging. Burlingame, CA, 1–12. Google ScholarCross Ref
15. Piotr Didyk, Pitchaya Sitthi-Amorn, William Freeman, Frédo Durand, and Wojciech Matusik. 2013. Joint View Expansion and Filtering for Automultiscopic 3D Displays. ACM Transactions on Graphics (Proceedings SIGGRAPH Asia 2013, Hong Kong) 32, 6 (2013).Google Scholar
16. Jian Ding, Stanley A. Klein, and Dennis M. Levi. 2013. Binocular combination of phase and contrast explained by a gain-control and gain-enhancement model. Journal of Vision 13, 2 (2013), 1–37. Google ScholarCross Ref
17. Jian Ding and George Sperling. 2006. A gain-control theory of binocular combination. Proceedings of the National Academy of Sciences of the United States of America 103, 4 (Jan. 2006), 1141–1146. Google ScholarCross Ref
18. Monika A. Formankiewicz and J. D. Mollon. 2009. The psychophysics of detecting binocular discrepancies of luminance. Vision Research 49, 15 (July 2009), 1929–1938. Google ScholarCross Ref
19. Wataru Fujimura, Yukua Koide, Robert Songer, Takahiro Hayakawa, Akihiko Shirai, and Kazuhisa Yanaka. 2012. 2x3D: Real Time Shader for Simultaneous 2D/3D Hybrid Theater. In SIGGRAPH Asia 2012 Emerging Technologies (SA ’12). ACM, 1:1–1:2. Google ScholarDigital Library
20. Mark A. Georgeson, Stuart A. Wallis, Tim S. Meese, and Daniel H. Baker. 2016. Contrast and lustre: A model that accounts for eleven different forms of contrast discrimination in binocular vision. Vision Research 129 (Dec. 2016), 98–118. Google ScholarCross Ref
21. Richard L. Gregory. 1970. Intelligent Eye. Littlehampton Book Services Ltd, London.Google Scholar
22. Robert T. Held and Martin S. Banks. 2008. Misperceptions in Stereoscopic Displays: A Vision Science Perspective. ACM Transactions on Graphics (2008), 23–32. Google ScholarDigital Library
23. Sid Henriksen, Seiji Tanabe, and Bruce Cumming. 2016. Disparity processing in primary visual cortex. Philosophical Transactions of the Royal Society B 371, 1697 (2016). Google ScholarCross Ref
24. Ian P. Howard. 2012a. Perceiving in Depth Volume 1 Basic Mechanisms. Oxford University Press.Google Scholar
25. Ian P. Howard. 2012b. Perceiving in Depth Volume 2 Stereoscopic Vision. Oxford University Press.Google Scholar
26. Helmut. Jorke, Arnold. Simon, and Markus. Fritz. 2009. Advanced stereo projection using interference filters. Journal of the Society for Information Display 17, 5 (2009), 407–410.Google ScholarCross Ref
27. Takahiro Kawabe, Taiki Fukiage, Masataka Sawayama, and Shin’ya Nishida. 2016. Deformation Lamps: A Projection Technique to Make Static Objects Perceptually Dynamic. ACM Trans. Appl. Percept. 13, 2 (March 2016), 10:1–10:17. Google ScholarDigital Library
28. Petr Kellnhofer, Piotr Didyk, Karol Myszkowski, Mohamed M. Hefeeda, Hans-Peter Seidel, and Wojciech Matusik. 2016. GazeStereo3D: Seamless Disparity Manipulations. ACM Trans. Graph. 35, 4 (July 2016), 68:1–68:13.Google ScholarDigital Library
29. Frederick. A. Kingdom and Bernard. Moulden. 1988. Border effects on brightness: a review of findings, models and issues. Spatial Vision 3, 4 (1988), 225–262.Google ScholarCross Ref
30. Frederick. A. Kingdom and David. R. Simmons. 1996. Stereoacuity and colour contrast. Vision Research 36, 9 (1996), 1311–1319.Google ScholarCross Ref
31. Frank L. Kooi and Alexander Toet. 2004. Visual comfort of binocular and 3D displays. Displays 25, 2–3 (Aug. 2004), 99–108. Google ScholarCross Ref
32. Marc Lambooij, Wijnand IJsselsteijn, Marten Fortuin, and Ingrid Heynderickx. 2009. Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review. Journal of Imaging Science and Technology 53, 3 (2009), 1–14. Google ScholarCross Ref
33. Michael S. Landy, Laurence T. Maloney, Elizabeth B. Johnston, and Mark Young. 1995. Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Research 35, 3 (Feb. 1995), 389–412. Google ScholarCross Ref
34. Manuel Lang, Alexander Hornung, Oliver Wang, Steven Poulakos, Aljoscha Smolic, and Markus Gross. 2010. Nonlinear Disparity Mapping for Stereoscopic 3D. In ACM SIGGRAPH 2010 Papers (SIGGRAPH 10). ACM, NewYork, NY, USA, 75:1–75:10.Google ScholarDigital Library
35. Hwan S. Lee and Allan C. Dobbins. 2006. Perceiving surfaces in depth beyond the fusion limit of their elements. Perception 35, 1 (2006), 31–39. Google ScholarCross Ref
36. Gordon E. Legge. 1984. Binocular contrast summation-II. Quadratic summation. Vision Research 24, 4 (1984), 385–394. Google ScholarCross Ref
37. Gordon E. Legge and Gary S. Rubin. 1981. Binocular interactions in suprathreshold contrast perception. Perception & Psychophysics 30, 1 (Jan. 1981), 49–61. Google ScholarCross Ref
38. Ira Ludwig, Wolfgang Pieper, and Harald Lachnit. 2007. Temporal integration of monocular images separated in time: Stereopsis, stereoacuity, and binocular luster. Perception & Psychophysics 69, 1 (2007), 92–102. Google ScholarCross Ref
39. William R. Mark, Leonard McMillan, and Gary Bishop. 1997. Post-rendering 3D Warping. In Proceedings of 1997 Symposium on Interactive 3D Graphics. ACM, New York, NY, USA, 7–16.Google ScholarDigital Library
40. Belen Masia, Gordon Wetzstein, Carlos Aliaga, Ramesh Raskar, and Diego Gutierrez. 2013. Display adaptive 3D content remapping. Computer and Graphics 37, 8 (2013), 983–996. Google ScholarDigital Library
41. Rainer Mausfeld, Gunnar Wendt, and Jürgen Golz. 2014. Lustrous Material Appearances: Internal and External Constraints on Triggering Conditions for Binocular Lustre. i-Perception 5, 1 (Feb. 2014), 1–19. Google ScholarCross Ref
42. Tim S. Meese, Mark A. Georgeson, and Daniel H. Baker. 2006. Binocular contrast vision at and above threshold. Journal of Vision 6, 11 (Oct. 2006), 1224–1243. Google ScholarCross Ref
43. Ken Nakayama and Shinsuke Shimojo. 1990. Da Vinci stereopsis: Depth and subjective occluding contours from unpaired image points. Vision Research 30, 11 (1990), 1811–1825. Google ScholarCross Ref
44. Izumi. Ohzawa, Gregory. C. DeAngelis, and Ralph. D. Freeman. 1990. Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. Science (New York, N.Y.) 249, 4972 (Aug. 1990), 1037–1041.Google Scholar
45. Adriana Olmos and Frederick A. A. Kingdom. 2004. A biologically inspired algorithm for the recovery of shading and reflectance images. Perception 33 (2004), 1463–1473. Google ScholarCross Ref
46. D. A. Palmer. 1961. Measurement of the Horizontal Extent of Panum’s Area by a Method of Constant Stimuli. Optica Acta 8 (1961), 151–159. Google ScholarCross Ref
47. Javier Portila and Eero. P. Simoncelli. 2000. A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients. International Journal of Computer Vision 40, 1 (2000), 49–70.Google ScholarDigital Library
48. Carl Pulfrich. 1922. Die Stereoskopie im Dienste der isochromen und heterochromen Photometrie. Naturwissenschaften 10, 35 (1922), 751–761. Google ScholarCross Ref
49. Dale Purves, Amita Shimpi, and R. Beau Lotto. 1999. An Empirical Explanation of the Cornsweet Effect. The Journal of Neuroscience 19, 19 (Oct. 1999), 8542–8551.Google ScholarCross Ref
50. Ning. Qian and Ralph. D. Freeman. 2009. Pulfrich phenomena are coded effectively by a joint motion-disparity process. Journal of Vision 9, 5 (May 2009), 1–16. Google ScholarCross Ref
51. Jenny C. A. Read and Bruce G. Cumming. 2005. All Pulfrich-like illusions can be explained without joint encoding of motion and disparity. Journal of Vision 5, 11 (2005), 901–927. Google ScholarCross Ref
52. Steven Scher, Jing Liu, Rajan Vaish, Prabath Gunawardane, and James Davis. 2013. 3D+2DTV: 3D Displays with No Ghosting for Viewers Without Glasses. ACM Transactions on Graphics 32, 3 (2013), 21:1–10.Google ScholarDigital Library
53. Clifton Schor, Ivan Wood, and Jane Ogawa. 1984. Binocular sensory fusion is limited by spatial resolution. Vision Research 24, 7 (1984), 661–665. Google ScholarCross Ref
54. Takashi Shibata, Joohwan Kim, David M. Hoffman, and Martin S. Banks. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. Journal of Vision 11, 8 (2011), 1–29. Google ScholarCross Ref
55. Mel Siegel and Shojiro Nagata. 2000. Just Enough Reality: Comfortable 3D Viewing via Microstereopsis. IEEE Transanctions on Circuits and Systems for Video Technology 10, 3 (2000), 387–396. Google ScholarDigital Library
56. Eero. P. Simoncelli and William T. Freeman. 1995. The Steerable Pyramid: A Flexible Architecture for Multi-scale Derivative Computation. In Proceedings of the 2nd IEEE International Conference on Image Processing, Vol. 3. Washington, DC, USA, 444–447.Google Scholar
57. Masahiko Terao, Junji Watanabe, Akihiro Yagi, and Shin’ya Nishida. 2010. Smooth Pursuit Eye Movements Improve Temporal Resolution for Color Perception. PLoS ONE 5, 6 (June 2010), e11214. Google ScholarCross Ref
58. Neal Wadhwa, Michael Rubinstein, Frédo Durand, and W. T. Freeman. 2013. Phase-Based Video Motion Processing. ACM Transactions on Graphics 32, 4 (2013), 1–9. Google ScholarDigital Library
59. Neal Wadhwa, Michael Rubinstein, Frédo Durand, and William T. Freeman. 2014. Riesz Pyramids for Fast Phase-Based Video Magnification. (2014).Google Scholar
60. Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero. P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612.Google ScholarDigital Library
61. Junji Watanabe and Shin’ya Nishida. 2007. Veridical perception of moving colors by trajectory integration of input signals. Journal of Vision 7, 11 (2007). Google ScholarCross Ref
62. Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, and William Freeman. 2012. Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Transactions on Graphics 31, 4 (July 2012), 65:1–8.Google ScholarDigital Library
63. C. Lawrence Zitnick, Sing Bing Kang, Matthew Uyttendaele, Simon Winder, and Richard Szeliski. 2004. High-quality Video View Interpolation Using a Layered Representation. In ACM SIGGRAPH 2004 Papers (SIGGRAPH ’04). ACM, New York, NY, USA, 600–608. Google ScholarDigital Library
