“Wirtinger holography for near-eye displays” by Chakravarthula, Peng, Kollin, Fuchs and Heide
Conference:
Type(s):
Title:
- Wirtinger holography for near-eye displays
Session/Category Title: Thoughts on Display
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Near-eye displays using holographic projection are emerging as an exciting display approach for virtual and augmented reality at high-resolution without complex optical setups — shifting optical complexity to computation. While precise phase modulation hardware is becoming available, phase retrieval algorithms are still in their infancy, and holographic display approaches resort to heuristic encoding methods or iterative methods relying on various relaxations.In this work, we depart from such existing approximations and solve the phase retrieval problem for a hologram of a scene at a single depth at a given time by revisiting complex Wirtinger derivatives, also extending our framework to render 3D volumetric scenes. Using Wirtinger derivatives allows us to pose the phase retrieval problem as a quadratic problem which can be minimized with first-order optimization methods. The proposed Wirtinger Holography is flexible and facilitates the use of different loss functions, including learned perceptual losses parametrized by deep neural networks, as well as stochastic optimization methods. We validate this framework by demonstrating holographic reconstructions with an order of magnitude lower error, both in simulation and on an experimental hardware prototype.
References:
1. Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16). 265–283.Google ScholarDigital Library
2. Lukas Ahrenberg, Philip Benzie, Marcus Magnor, and John Watson. 2008. Computer generated holograms from three dimensional meshes using an analytic light transport model. Applied optics 47, 10 (2008), 1567–1574.Google Scholar
3. S. Bahmani and J. Romberg. 2017. Phase Retrieval Meets Statistical Learning Theory: A Flexible Convex Relaxation. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research), Aarti Singh and Jerry Zhu (Eds.), Vol. 54. PMLR, Fort Lauderdale, FL, USA, 252–260.Google Scholar
4. R Bates. 1982. Fourier phase problems are uniquely solvable in mute than one dimension. I: Underlying theory. Optik (Stuttgart) 61 (1982), 247–262.Google Scholar
5. Heinz H Bauschke, Patrick L Combettes, and D Russell Luke. 2003. Hybrid projection-reflection method for phase retrieval. JOSA A 20, 6 (2003), 1025–1034.Google ScholarCross Ref
6. Muharrem Bayraktar and Meriç Özcan. 2010. Method to calculate the far field of three-dimensional objects for computer-generated holography. Applied optics 49, 24 (2010), 4647–4654.Google Scholar
7. Stephen A Benton and V Michael Bove Jr. 2008. Holographic imaging. John Wiley & Sons.Google Scholar
8. Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie Line Alberi-Morel. 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. (2012).Google Scholar
9. Emmanuel J Candes, Xiaodong Li, and Mahdi Soltanolkotabi. 2015. Phase retrieval via Wirtinger flow: Theory and algorithms. IEEE Transactions on Information Theory 61, 4 (2015), 1985–2007.Google ScholarDigital Library
10. Emmanuel J Candes, Thomas Strohmer, and Vladislav Voroninski. 2013. Phaselift: Exact and stable signal recovery from magnitude measurements via convex programming. Communications on Pure and Applied Mathematics 66, 8 (2013), 1241–1274.Google ScholarCross Ref
11. Praneeth Chakravarthula, David Dunn, Kaan Akşit, and Henry Fuchs. 2018. Focusar: Auto-focus augmented reality eyeglasses for both real world and virtual imagery. IEEE transactions on visualization and computer graphics 24, 11 (2018), 2906–2916.Google Scholar
12. J-S Chen and DP Chu. 2015. Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications. Optics express 23, 14 (2015), 18143–18155.Google Scholar
13. Rick H-Y Chen and Timothy D Wilkinson. 2009. Computer generated hologram from point cloud using graphics processor. Applied optics 48, 36 (2009), 6841–6850.Google Scholar
14. Steven A Cholewiak, Gordon D Love, Pratul P Srinivasan, Ren Ng, and Martin S Banks. 2017. ChromaBlur: Rendering chromatic eye aberration improves accommodation and realism. ACM Transactions on Graphics (TOG) 36, 6 (2017), 210.Google ScholarDigital Library
15. James R Fienup. 1982. Phase retrieval algorithms: a comparison. Applied optics 21, 15 (1982), 2758–2769.Google Scholar
16. James R Fienup. 1993. Phase-retrieval algorithms for a complicated optical system. Applied optics 32, 10 (1993), 1737–1746.Google Scholar
17. Ralph W Gerchberg. 1972. A practical algorithm for the determination of the phase from image and diffraction plane pictures. Optik 35 (1972), 237–246.Google Scholar
18. Tom Goldstein and Christoph Studer. 2018. PhaseMax: Convex phase retrieval via basis pursuit. IEEE Transactions on Information Theory (2018).Google Scholar
19. RA Gonsalves. 1976. Phase retrieval from modulus data. JOSA 66, 9 (1976), 961–964.Google ScholarCross Ref
20. Joseph W Goodman. 2005. Introduction to Fourier optics. Roberts and Company Publishers.Google Scholar
21. Felix Heide, Douglas Lanman, Dikpal Reddy, Jan Kautz, Kari Pulli, and David Luebke. 2014. Cascaded displays: spatiotemporal superresolution using offset pixel layers. ACM Transactions on Graphics (TOG) 33, 4 (2014), 60.Google ScholarDigital Library
22. CK Hsueh and AA Sawchuk. 1978. Computer-generated double-phase holograms. Applied optics 17, 24 (1978), 3874–3883.Google Scholar
23. Fu-Chung Huang, Kevin Chen, and Gordon Wetzstein. 2015a. The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues. ACM Transactions on Graphics (TOG) 34, 4 (2015), 60.Google ScholarDigital Library
24. Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja. 2015b. Single Image Super-Resolution From Transformed Self-Exemplars. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5197–5206.Google ScholarCross Ref
25. Jia Jia, Juan Liu, Guofan Jin, and Yongtian Wang. 2014. Fast and effective occlusion culling for 3D holographic displays by inverse orthographic projection with low angular sampling. Applied optics 53, 27 (2014), 6287–6293.Google Scholar
26. Hwi Kim, Joonku Hahn, and Byoungho Lee. 2008. Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography. Applied optics 47, 19 (2008), D117–D127.Google Scholar
27. RG Lane. 1991. Phase retrieval using conjugate gradient minimization. journal of Modern Optics 38, 9 (1991), 1797–1813.Google Scholar
28. Douglas Lanman and David Luebke. 2013. Near-eye light field displays. ACM Transactions on Graphics (TOG) 32, 6 (2013), 220.Google ScholarDigital Library
29. Detlef Leseberg and Christian Frère. 1988. Computer-generated holograms of 3-D objects composed of tilted planar segments. Applied optics 27, 14 (1988), 3020–3024.Google Scholar
30. LB Lesem, PM Hirsch, and JA Jordan. 1969. The kinoform: a new wavefront reconstruction device. IBM Journal of Research and Development 13, 2 (1969), 150–155.Google ScholarDigital Library
31. Dong C Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical programming 45, 1-3 (1989), 503–528.Google ScholarDigital Library
32. Mark Lucente and Tinsley A Galyean. 1995. Rendering interactive holographic images. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. ACM, 387–394.Google ScholarDigital Library
33. Mark E Lucente. 1993. Interactive computation of holograms using a look-up table. Journal of Electronic Imaging 2, 1 (1993), 28–34.Google ScholarCross Ref
34. D Russell Luke. 2004. Relaxed averaged alternating reflections for diffraction imaging. Inverse problems 21, 1 (2004), 37.Google Scholar
35. Andrew Maimone, Andreas Georgiou, and Joel S Kollin. 2017. Holographic near-eye displays for virtual and augmented reality. ACM Transactions on Graphics (TOG) 36, 4 (2017), 85.Google ScholarDigital Library
36. Stefano Marchesini, Yu-Chao Tu, and Hau-tieng Wu. 2016. Alternating projection, ptychographic imaging and phase synchronization. Applied and Computational Harmonic Analysis 41, 3 (2016), 815–851.Google ScholarCross Ref
37. D. Martin, C. Fowlkes, D. Tal, and J. Malik. 2001. A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. In Proc. 8th Int’l Conf. Computer Vision, Vol. 2. 416–423.Google ScholarCross Ref
38. Nobuyuki Masuda, Tomoyoshi Ito, Takashi Tanaka, Atsushi Shiraki, and Takashige Sugie. 2006. Computer generated holography using a graphics processing unit. Optics Express 14, 2 (2006), 603–608.Google ScholarCross Ref
39. Kyoji Matsushima. 2005. Computer-generated holograms for three-dimensional surface objects with shade and texture. Applied optics 44, 22 (2005), 4607–4614.Google Scholar
40. Kyoji Matsushima and Sumio Nakahara. 2009. Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method. Applied optics 48, 34 (2009), H54–H63.Google Scholar
41. Kyoji Matsushima, Masaki Nakamura, and Sumio Nakahara. 2014. Silhouette method for hidden surface removal in computer holography and its acceleration using the switch-back technique. Optics express 22, 20 (2014), 24450–24465.Google Scholar
42. Yifan Peng, Xiong Dun, Qilin Sun, and Wolfgang Heidrich. 2017. Mix-and-match holography. ACM Transactions on Graphics (2017).Google Scholar
43. Christoph Petz and Marcus Magnor. 2003. Fast hologram synthesis for 3D geometry models using graphics hardware. In Proc. SPIE, Vol. 5005. 266–275.Google ScholarCross Ref
44. Reinhold Remmert. 2012. Theory of complex functions. Vol. 122. Springer Science & Business Media.Google Scholar
45. Mark Schmidt. 2005. minFunc: unconstrained differentiable multivariate optimization in Matlab. Software available at http://www.cs.ubc.ca/~schmidtm/Software/minFunc.htm (2005).Google Scholar
46. Liang Shi, Fu-Chung Huang, Ward Lopes, Wojciech Matusik, and David Luebke. 2017. Near-eye light field holographic rendering with spherical waves for wide field of view interactive 3d computer graphics. ACM Transactions on Graphics (TOG) 36, 6 (2017), 236.Google ScholarDigital Library
47. Quinn YJ Smithwick, James Barabas, Daniel E Smalley, and V Michael Bove. 2010. Interactive holographic stereograms with accommodation cues. In Practical Holography XXIV: Materials and Applications, Vol. 7619. International Society for Optics and Photonics, 761903.Google Scholar
48. Tullio Tommasi and Bruno Bianco. 1993. Computer-generated holograms of tilted planes by a spatial frequency approach. JOSA A 10, 2 (1993), 299–305.Google ScholarCross Ref
49. John Stephen Underkoffler. 1991. Toward accurate computation of optically reconstructed holograms. Ph.D. Dissertation. Massachusetts Institute of Technology.Google Scholar
50. Zhou Wang, Eero P Simoncelli, and Alan C Bovik. 2003. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, Vol. 2. Ieee, 1398–1402.Google ScholarCross Ref
51. James P Waters. 1966. Holographic image synthesis utilizing theoretical methods. Applied physics letters 9, 11 (1966), 405–407.Google Scholar
52. Zaiwen Wen, Chao Yang, Xin Liu, and Stefano Marchesini. 2012. Alternating direction methods for classical and ptychographic phase retrieval. Inverse Problems 28, 11 (2012), 115010.Google ScholarCross Ref
53. Masahiro Yamaguchi, Hideshi Hoshino, Toshio Honda, and Nagaaki Ohyama. 1993. Phase-added stereogram: calculation of hologram using computer graphics technique. In Proc. SPIE, Vol. 1914. 25–31.Google ScholarCross Ref
54. Hiroshi Yoshikawa, Takeshi Yamaguchi, and Hiroki Uetake. 2016. Image quality evaluation and control of computer-generated holograms. In Practical Holography XXX: Materials and Applications, Vol. 9771. International Society for Optics and Photonics, 97710N.Google Scholar
55. Roman Zeyde, Michael Elad, and Matan Protter. 2010. On single image scale-up using sparse-representations. In International conference on curves and surfaces. Springer, 711–730.Google Scholar
56. Hao Zhang, Yan Zhao, Liangcai Cao, and Guofan Jin. 2015. Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues. Optics express 23, 4 (2015), 3901–3913.Google Scholar
57. Hao Zhang, Yan Zhao, Liangcai Cao, and Guofan Jin. 2016. Layered holographic stereogram based on inverse Fresnel diffraction. Applied optics 55, 3 (2016), A154–A159.Google Scholar
58. Jingzhao Zhang, Nicolas Pégard, Jingshan Zhong, Hillel Adesnik, and Laura Waller. 2017. 3D computer-generated holography by non-convex optimization. Optica 4, 10 (2017), 1306–1313.Google ScholarCross Ref
59. Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. arXiv preprint (2018).Google Scholar
60. Yan Zhao, Liangcai Cao, Hao Zhang, Dezhao Kong, and Guofan Jin. 2015. Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method. Optics express 23, 20 (2015), 25440–25449.Google Scholar
