“Laplacian kernel splatting for efficient depth-of-field and motion blur synthesis or reconstruction” by Leimkühler, Seidel and Ritschel
Conference:
Type(s):
Entry Number: 55
Title:
- Laplacian kernel splatting for efficient depth-of-field and motion blur synthesis or reconstruction
Session/Category Title: Smart Integration for Real-Time Rendering
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Simulating combinations of depth-of-field and motion blur is an important factor to cinematic quality in synthetic images but can take long to compute. Splatting the point-spread function (PSF) of every pixel is general and provides high quality, but requires prohibitive compute time. We accelerate this in two steps: In a pre-process we optimize for sparse representations of the Laplacian of all possible PSFs that we call spreadlets. At runtime, spreadlets can be splat efficiently to the Laplacian of an image. Integrating this image produces the final result. Our approach scales faithfully to strong motion and large out-of-focus areas and compares favorably in speed and quality with off-line and interactive approaches. It is applicable to both synthesizing from pinhole as well as reconstructing from stochastic images, with or without layering.
References:
1. Tomas Akenine-Möller, Jacob Munkberg, and Jon Hasselgren. 2007. Stochastic rasterization using time-continuous triangles. In Proc. Graphics Hardware. 9. Google ScholarDigital Library
2. Laurent Belcour, Cyril Soler, Kartic Subr, Nicolas Holzschuch, and Fredo Durand. 2013. 5D covariance tracing for efficient defocus and motion blur. ACM Trans. Graph. 32, 3 (2013). Google ScholarDigital Library
3. Pravin Bhat, Brian Curless, Michael Cohen, and C Zitnick. 2008. Fourier analysis of the 2D screened Poisson equation for gradient domain problems. ECCV (2008), 114–28. Google ScholarDigital Library
4. Pravin Bhat, C Lawrence Zitnick, Michael Cohen, and Brian Curless. 2010. Gradientshop: A gradient-domain optimization framework for image and video filtering. ACM Trans. Graph. 29, 2 (2010). Google ScholarDigital Library
5. Robert L Cook, Loren Carpenter, and Edwin Catmull. 1987. The REYES image rendering architecture. In ACM SIGGRAPH Computer Graphics, Vol. 21. 95–102. Google ScholarDigital Library
6. Robert L Cook, Thomas Porter, and Loren Carpenter. 1984. Distributed ray tracing. In ACM SIGGRAPH Computer Graphics, Vol. 18. ACM, 137–145. Google ScholarDigital Library
7. Franklin C Crow. 1984. Summed-area tables for texture mapping. ACM SIGGRAPH Computer Graphics 18, 3 (1984), 207–12. Google ScholarDigital Library
8. Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. 2006. Image denoising with block-matching and 3D filtering. In Proc. SPIE, Vol. 6064.Google ScholarCross Ref
9. Philip Dutré, Eric P Lafortune, and Yves Willems. 1993. Monte Carlo light tracing with direct computation of pixel intensities. In Proc. Computational Graphics and Visualisation Techniques. 128–37.Google Scholar
10. Kevin Egan, Yu-Ting Tseng, Nicolas Holzschuch, Frédo Durand, and Ravi Ramamoorthi. 2009. Frequency analysis and sheared reconstruction for rendering motion blur. ACM Trans. Graph. (Proc. SIGGRAPH) 28, 3 (2009). Google ScholarDigital Library
11. Zeev Farbman, Raanan Fattal, and Dani Lischinski. 2011. Convolution pyramids. ACM Trans. Graph. 30, 6 (2011), 175–1. Google ScholarDigital Library
12. Kayvon Fatahalian, Edward Luong, Solomon Boulos, Kurt Akeley, William R Mark, and Pat Hanrahan. 2009. Data-parallel rasterization of micropolygons with defocus and motion blur. In Proc. HPG. 59–68. Google ScholarDigital Library
13. Alain Fournier and Eugene Fiume. 1988. Constant-time filtering with space-variant kernels. In ACM SIGGRAPH Computer Graphics, Vol. 22. ACM, 229–238. Google ScholarDigital Library
14. Irena Galić, Joachim Weickert, Martin Welk, Andrés Bruhn, Alexander Belyaev, and Hans-Peter Seidel. 2008. Image compression with anisotropic diffusion. J Math. Imaging and Vision 31, 2 (2008), 255–69. Google ScholarDigital Library
15. Jhonny Göransson and Andreas Karlsson. 2007. Practical post-process depth of field. In GPU Gems 3. 583–606.Google Scholar
16. Michael Goy. 2013. American cinematographer manual. Vol. 10. Am. Cinemat.Google Scholar
17. Paul Haeberli and Kurt Akeley. 1990. The accumulation buffer: hardware support for high-quality rendering. ACM SIGGRAPH Computer Graphics 24, 4 (1990), 309–318. Google ScholarDigital Library
18. Paul S Heckbert. 1986. Filtering by repeated integration. In ACM SIGGRAPH Computer Graphics, Vol. 20. 315–21. Google ScholarDigital Library
19. Matthias Hullin, Elmar Eisemann, Hans-Peter Seidel, and Sungkil Lee. 2011. Physically-based real-time lens flare rendering. ACM Trans Graph. (Proc. SIGGRAPH Asia) 30, 4 (2011), 108. Google ScholarDigital Library
20. Nima Khademi Kalantari and Pradeep Sen. 2013. Removing the noise in Monte Carlo rendering with general image denoising algorithms. Comp. Graph. Forum 32, 2 (2013), 93–102.Google ScholarCross Ref
21. Craig Kolb, Don Mitchell, and Pat Hanrahan. 1995. A realistic camera model for computer graphics. In SIGGRAPH. 317–24. Google ScholarDigital Library
22. Janne Kontkanen, Jussi Räsänen, and Alexander Keller. 2006. Irradiance filtering for Monte Carlo ray tracing., 259–272 pages.Google Scholar
23. Johannes Kopf, Fabian Langguth, Daniel Scharstein, Richard Szeliski, and Michael Goesele. 2013. Image-based rendering in the gradient domain. ACM Trans. Graph. (Proc. SIGGRAPH) 32, 6 (2013). Google ScholarDigital Library
24. Martin Kraus and Magnus Strengert. 2007. Depth-of-Field Rendering by Pyramidal Image Processing. In Comp. Graph Forum, Vol. 26. 645–54.Google ScholarCross Ref
25. Sungkil Lee, Elmar Eisemann, and Hans-Peter Seidel. 2009. Depth-of-field rendering with multiview synthesis. In ACM Trans. Graph. (Proc. SIGRAPH Asia), Vol. 28. Google ScholarDigital Library
26. Sungkil Lee, Elmar Eisemann, and Hans-Peter Seidel. 2010. Real-time lens blur effects and focus control. ACM Trans. Graph (Proc. SIGGRAPH) 29, 4 (2010). Google ScholarDigital Library
27. Sungkil Lee, Gerard Jounghyun Kim, and Seungmoon Choi. 2008. Real-Time Depth-of-Field Rendering Using Point Splatting on Per-Pixel Layers. Comp. Graph. Forum 27, 7 (2008), 1955–62.Google ScholarCross Ref
28. Jaakko Lehtinen, Timo Aila, Jiawen Chen, Samuli Laine, and Frédo Durand. 2011. Temporal Light Field Reconstruction for Rendering Distribution Effects. ACM Trans. Graph. (Proc. SIGGRAPH) 30, 4 (2011). Google ScholarDigital Library
29. Jaakko Lehtinen, Tero Karras, Samuli Laine, Miika Aittala, Frédo Durand, and Timo Aila. 2013. Gradient-domain metropolis light transport. ACM Trans. Graph. (Proc. SIGGRAPH) 32, 4 (2013), 95. Google ScholarDigital Library
30. Thomas Leimkühler, Hans-Peter Seidel, and Tobias Ritschel. 2017. Minimal Warping: Planning Incremental Novel-view Synthesis. Comp. Graph. Forum 36, 4 (2017), 1–14. Google ScholarDigital Library
31. Stuart Lloyd. 1982. Least squares quantization in PCM. IEEE Trans. Inf. Theory 28, 2 (1982), 129–37. Google ScholarDigital Library
32. Michael D McCool. 1999. Anisotropic diffusion for Monte Carlo noise reduction. ACM Trans. Graph. 18, 2 (1999), 171–94. Google ScholarDigital Library
33. Morgan McGuire, Padraic Hennessy, Michael Bukowski, and Brian Osman. 2012. A reconstruction filter for plausible motion blur. In i3D. 135–42. Google ScholarDigital Library
34. Jacob Munkberg, Karthik Vaidyanathan, Jon Hasselgren, Petrik Clarberg, and Tomas Akenine-Möller. 2014. Layered reconstruction for defocus and motion blur. Comp. Graph. Forum 33, 4 (2014), 81–92.Google ScholarDigital Library
35. Oliver Nalbach, Elena Arabadzhiyska, Dushyant Mehta, H-P Seidel, and Tobias Ritschel. 2017. Deep Shading: Convolutional Neural Networks for Screen Space Shading. Comp. Graph. Forum 36, 4 (2017), 65–78. Google ScholarDigital Library
36. Oliver Nalbach, Tobias Ritschel, and Hans-Peter Seidel. 2014. Deep Screen Space. In I3D. ACM. Google ScholarDigital Library
37. Alexandrina Orzan, Adrien Bousseau, Pascal Barla, Holger Winnemöller, Joëlle Thollot, and David Salesin. 2013. Diffusion curves: a vector representation for smooth-shaded images. Comm. ACM 56, 7 (2013), 101–8. Google ScholarDigital Library
38. Patrick Pérez, Michel Gangnet, and Andrew Blake. 2003. Poisson image editing. ACM Trans. Graph (Proc. SIGGRAPH) 22, 3 (2003), 313–18. Google ScholarDigital Library
39. Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2016. Physically based rendering: From theory to implementation. Morgan Kaufmann. Google ScholarDigital Library
40. Michael Potmesil and Indranil Chakravarty. 1981. A lens and aperture camera model for synthetic image generation. ACM SIGGRAPH Computer Graphics 15, 3 (1981), 297–305. Google ScholarDigital Library
41. Simon Premože, Michael Ashikhmin, Jerry Tessendorf, Ravi Ramamoorthi, and Shree Nayar. 2004. Practical rendering of multiple scattering effects in participating media. In Proc. EGWR. 363–74. Google ScholarDigital Library
42. Gilberto Rosado. 2007. Motion blur as a post-processing effect. 575–81.Google Scholar
43. Pradeep Sen and Soheil Darabi. 2012. On filtering the noise from the random parameters in Monte Carlo rendering. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 3 (2012), 18–1. Google ScholarDigital Library
44. Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. 1998. Layered depth images. In Proc. SIGRAPH. 231–42. Google ScholarDigital Library
45. Patrice Simard, Leon Bottou, Patrick Haffner, and Yann LeCun. 1999. Boxlets: a fast convolution algorithm for signal processing and neural networks. In NIPS. 571–7. Google ScholarDigital Library
46. Cyril Soler, Kartic Subr, Frédo Durand, Nicolas Holzschuch, and François Sillion. 2009. Fourier depth of field. ACM Trans. Graph. 28, 2 (2009), 18. Google ScholarDigital Library
47. Xin Sun, Guofu Xie, Yue Dong, Stephen Lin, Weiwei Xu, Wencheng Wang, Xin Tong, and Baining Guo. 2012. Diffusion curve textures for resolution independent texture mapping. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4 (2012), 74–1. Google ScholarDigital Library
48. Karthik Vaidyanathan, Jacob Munkberg, Petrik Clarberg, and Marco Salvi. 2015. Layered light field reconstruction for defocus blur. ACM Trans. Graph. 34, 2 (2015). Google ScholarDigital Library
49. Xuan Yu, Rui Wang, and Jingyi Yu. 2010. Real-time Depth of Field Rendering via Dynamic Light Field Generation and Filtering. Comp. Graph. Forum 29, 7 (2010), 2099–2107.Google ScholarCross Ref
