“Stylized Rendering as a Function of Expectation”
Conference:
Type(s):
Title:
- Stylized Rendering as a Function of Expectation
Presenter(s)/Author(s):
Abstract:
We propose a generalization of the rendering equation that captures both realistic light transport and a class of stylized visuals in the same formulation. This allows physical effects like glossy reflections and color bleed to affect stylized visuals, and vice versa.
References:
[1]
Ken-ichi Anjyo and Katsuaki Hiramitsu. 2003. Stylized Highlights for Cartoon Rendering and Animation. IEEE Comput. Graph. Appl. 23, 4 (jul 2003), 54–61.
[2]
Ken-ichi Anjyo, Shuhei Wemler, and William Baxter. 2006. Tweakable Light and Shade for Cartoon Animation. In Proceedings of the 4th International Symposium on Non-Photorealistic Animation and Rendering (Annecy, France) (NPAR ’06). Association for Computing Machinery, New York, NY, USA, 133–139.
[3]
Sai Praveen Bangaru, Tzu-Mao Li, and Fr?do Durand. 2020. Unbiased Warped-Area Sampling for Differentiable Rendering. ACM Trans. Graph. 39, 6, Article 245 (nov 2020), 18 pages.
[4]
Pascal Barla, Jo?lle Thollot, and Lee Markosian. 2006. X-Toon: An Extended Toon Shader. In Proceedings of the 4th International Symposium on Non-Photorealistic Animation and Rendering (Annecy, France) (NPAR ’06). Association for Computing Machinery, New York, NY, USA, 127–132.
[5]
Aner Ben-Artzi, Ryan Overbeck, and Ravi Ramamoorthi. 2006. Real-Time BRDF Editing in Complex Lighting. ACM Trans. Graph. 25, 3 (jul 2006), 945–954.
[6]
Jose H. Blanchet and Peter W. Glynn. 2015. Unbiased Monte Carlo for optimization and functions of expectations via multi-level randomization. In 2015 Winter Simulation Conference (WSC). 3656–3667.
[7]
Subrahmanyan Chandrasekhar. 1960. Radiative Transfer. NY.
[8]
Robert L. Cook, Thomas Porter, and Loren Carpenter. 1984. Distributed Ray Tracing. 18, 3 (July 1984), 137–145.
[9]
Ant?nio Cardoso Costa, Ant?nio Augusto Sousa, and Fernando Nunes Ferreira. 1999. Lighting Design: A Goal Based Approach Using Optimisation. In Eurographics Workshop on Rendering, Dani Lischinski and Greg Ward Larson (Eds.). The Eurographics Association.
[10]
J?r?mi Dauchet, Jean-Jacques Bezian, St?phane Blanco, Cyril Caliot, Julien Charon, Christophe Coustet, Mouna El Hafi, Vincent Eymet, Olivier Farges, Vincent Forest, Richard Fournier, Mathieu Galtier, Jacques Gautrais, Ana?s Khuong, Lionel Pelissier, Benjamin Piaud, Maxime Roger, Guillaume Terr?e, and Sebastian Weitz. 2018. Addressing nonlinearities in Monte Carlo. Scientific Reports 8, 1 (05 Sep 2018), 13302.
[11]
Philippe Decaudin. 1996. Cartoon-Looking Rendering of 3D-Scenes. Technical Report 2919. INRIA Rocquencourt. http://www.antisphere.com/Research/Cartoon.php
[12]
Xi Deng, Milo? Ha?an, Nathan Carr, Zexiang Xu, and Steve Marschner. 2021. Path Graphs: Iterative Path Space Filtering. ACM Trans. Graph. 40, 6, Article 276 (dec 2021), 15 pages.
[13]
Oliver Deussen, J?rg Hamel, Andreas Raab, Stefan Schlechtweg, and Thomas Strothotte. 1999. An Illustration Technique Using Hardware-Based Intersections and Skeletons. In Proceedings of the 1999 Conference on Graphics Interface ’99 (Kingston, Ontario, Canada). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 175–182.
[14]
Oliver Deussen and Tobias Isenberg. 2013. Halftoning and Stippling. Springer London, London, 45–61.
[15]
K. Doi, Yuki Morimoto, and R. Tsuruno. 2021. Global Illumination-Aware Stylised Shading. Computer Graphics Forum 40 (10 2021), 11–20.
[16]
Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2016. Image Style Transfer Using Convolutional Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[17]
Iliyan Georgiev, Zackary Misso, Toshiya Hachisuka, Derek Nowrouzezahrai, Jaroslav K?iv?nek, and Wojciech Jarosz. 2019. Integral Formulations of Volumetric Transmittance. ACM Trans. Graph. 38, 6, Article 154 (nov 2019), 17 pages.
[18]
Amy Gooch, Bruce Gooch, Peter Shirley, and Elaine Cohen. 1998. A Non-Photorealistic Lighting Model for Automatic Technical Illustration. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’98). Association for Computing Machinery, New York, NY, USA, 447–452.
[19]
Amy A. Gooch. 2010. Towards Mapping the Field of Non-Photorealistic Rendering. In Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering (Annecy, France) (NPAR ’10). Association for Computing Machinery, New York, NY, USA, 159–164.
[20]
Toshiya Hachisuka and Henrik Wann Jensen. 2009. Stochastic Progressive Photon Mapping. 28, 5 (Dec. 2009), 130:1–130:8.
[21]
Toshiya Hachisuka, Shinji Ogaki, and Henrik Wann Jensen. 2008. Progressive Photon Mapping. 27, 5 (Dec. 2008), 130:1–130:8.
[22]
P. Hall. 1999. Nonphotorealistic Rendering by Q-mapping. Computer Graphics Forum (1999).
[23]
Eric Heitz, Jonathan Dupuy, Stephen Hill, and David Neubelt. 2016. Real-Time Polygonal-Light Shading with Linearly Transformed Cosines. 35, 4 (July 2016), 41:1–41:8.
[24]
Eric Heitz, Stephen Hill, and Morgan McGuire. 2018. Combining Analytic Direct Illumination and Stochastic Shadows. New York, NY, USA, 2:1–2:11.
[25]
Aaron Hertzmann and Denis Zorin. 2000. Illustrating Smooth Surfaces. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’00). ACM Press/Addison-Wesley Publishing Co., USA, 517–526.
[26]
Tobias Isenberg, Petra Neumann, Sheelagh Carpendale, Mario Costa Sousa, and Joaquim A. Jorge. 2006. Non-Photorealistic Rendering in Context: An Observational Study. In Proceedings of the Fourth International Symposium on Non-Photorealistic Animation and Rendering (NPAR 2006, June 5–7, 2006, Annecy, France), Doug DeCarlo and Lee Markosian (Eds.). ACM Press, New York, 115–126.
[27]
James T. Kajiya. 1986. The Rendering Equation. 20, 4 (Aug. 1986), 143–150.
[28]
Alexander Keller, Ken Dahm, and Nikolaus Binder. 2014. Path Space Filtering (SIGGRAPH ’14). ACM, 68:1–68:1.
[29]
Markus Kettunen, Eugene D’Eon, Jacopo Pantaleoni, and Jan Nov?k. 2021. An Unbiased Ray-Marching Transmittance Estimator. ACM Trans. Graph. 40, 4, Article 137 (jul 2021), 20 pages.
[30]
Adam Lake, Carl Marshall, Mark Harris, and Marc Blackstein. 2000. Stylized Rendering Techniques for Scalable Real-Time 3D Animation. In Proceedings of the 1st International Symposium on Non-Photorealistic Animation and Rendering (Annecy, France) (NPAR ’00). Association for Computing Machinery, New York, NY, USA, 13–20.
[31]
Kai Lawonn, Ivan Viola, Bernhard Preim, and Tobias Isenberg. 2018. A Survey of Surface-Based Illustrative Rendering for Visualization. Computer Graphics Forum 37, 6 (2018), 205–234. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13322
[32]
A Lee, S Tiberi, and G Zanella. 2019. Unbiased approximations of products of expectations. Biometrika 106, 3 (04 2019), 708–715. arXiv:https://academic.oup.com/biomet/article-pdf/106/3/708/29133126/asz008.pdf
[33]
X. Liu, Y. Wu, and P. Hall. 2023. Painterly Style Transfer With Learned Brush Strokes. IEEE Transactions on Visualization and Computer Graphics 01 (nov 2023), 1–12.
[34]
H. B. Mann and A. Wald. 1943. On Stochastic Limit and Order Relationships. The Annals of Mathematical Statistics 14, 3 (1943), 217 — 226.
[35]
Domingo Martin, Germn Arroyo, Alejandro Rodrguez, and Tobias Isenberg. 2017. A Survey of Digital Stippling. Comput. Graph. 67, C (oct 2017), 24–44.
[36]
Don McLeish. 2011. A general method for debiasing a Monte Carlo estimator. Monte Carlo Methods and Applications 17, 4 (2011), 301–315.
[37]
Jonatas Medeiros, Mario Sousa, Luiz Velho, and Carla Freitas. 2009. Perspective Contouring in Illustrative Visualization. In Proceedings of the 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI ’09). IEEE Computer Society, USA, 48–55.
[38]
Barbara J. Meier. 1996. Painterly Rendering for Animation. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’96). Association for Computing Machinery, 477–484.
[39]
Zackary Misso, Benedikt Bitterli, Iliyan Georgiev, and Wojciech Jarosz. 2022. Unbiased and consistent rendering using biased estimators. ACM Transactions on Graphics (Proceedings of SIGGRAPH) 41, 4 (2022).
[40]
Adithya Pediredla, Yasin Karimi Chalmiani, Matteo Giuseppe Scopelliti, Maysamreza Chamanzar, Srinivasa Narasimhan, and Ioannis Gkioulekas. 2020. Path tracing estimators for refractive radiative transfer. ACM Trans. Graph. 39, 6, Article 241 (nov 2020), 15 pages.
[41]
Lohit Petikam, Ken Anjyo, and Taehyun Rhee. 2021. Shading Rig: Dynamic Art-Directable Stylised Shading for 3D Characters. ACM Trans. Graph. 40, 5, Article 189 (sep 2021), 14 pages.
[42]
Matt Pharr, Brent Burley, Per Christensen, Marcos Fajardo, Luca Fascione, and Christopher Kulla. 2018. Design and Implementation of Modern Production Renderers. In ACM SIGGRAPH 2018 Panels (Vancouver, British Columbia, Canada) (SIGGRAPH ’18). Association for Computing Machinery, New York, NY, USA, Article 4, 2 pages.
[43]
Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2023. Physically based rendering: From theory to implementation. MIT Press.
[44]
Emil Praun, Hugues Hoppe, Matthew Webb, and Adam Finkelstein. 2001. Real-Time Hatching. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’01). Association for Computing Machinery, New York, NY, USA, 581.
[45]
Hao Qin, Xin Sun, Qiming Hou, Baining Guo, and Kun Zhou. 2015. Unbiased Photon Gathering for Light Transport Simulation. ACM Trans. Graph. 34, 6, Article 208 (Oct. 2015), 14 pages.
[46]
Alexander Rath, Pascal Grittmann, Sebastian Herholz, Philippe Weier, and Philipp Slusallek. 2022. EARS: efficiency-aware russian roulette and splitting. ACM Trans. Graph. 41, 4, Article 81 (jul 2022), 14 pages.
[47]
Chang-Han Rhee and Peter Glynn. 2012. A new approach to unbiased estimation for SDE’s. Proceedings – Winter Simulation Conference (07 2012).
[48]
Szymon Rusinkiewicz, Michael Burns, and Doug DeCarlo. 2006. Exaggerated Shading for Depicting Shape and Detail. ACM Trans. Graph. 25, 3 (jul 2006), 1199–1205.
[49]
Thorsten-Walther Schmidt, Fabio Pellacini, Derek Nowrouzezahrai, Wojciech Jarosz, and Carsten Dachsbacher. 2016. State of the Art in Artistic Editing of Appearance, Lighting and Material. Computer Graphics Forum 35, 1 (2016), 216–233. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.12721
[50]
Mario Costa Sousa, Amy Ashurst Gooch, and Bruce Gooch. 2005. Illustrative Scientific Visualization Framework. In Computational Aesthetics in Graphics, Visualization and Imaging, Laszlo Neumann, Mateu Sbert, Bruce Gooch, and Werner Purgathofer (Eds.). The Eurographics Association.
[51]
Martin Spindler, Niklas R?ber, Robert D?hring, and Maic Masuch. 2006. Enhanced Cartoon and Comic Rendering. In EG Short Papers, Dieter Fellner and Charles Hansen (Eds.). The Eurographics Association.
[52]
Hideki Todo, Ken-ichi Anjyo, William Baxter, and Takeo Igarashi. 2007. Locally Controllable Stylized Shading. In ACM SIGGRAPH 2007 Papers (San Diego, California) (SIGGRAPH ’07). Association for Computing Machinery, New York, NY, USA, 17–es.
[53]
Hideki Todo, Kunihiko Kobayashi, Jin Katsuragi, Haruna Shimotahira, Shizuo Kaji, and Yonghao Yue. 2022. Stroke Transfer: Example-Based Synthesis of Animatable Stroke Styles. In ACM SIGGRAPH 2022 Conference Proceedings (Vancouver, BC, Canada) (SIGGRAPH ’22). Association for Computing Machinery, New York, NY, USA, Article 54, 10 pages.
[54]
Kenji Tojo and Nobuyuki Umetani. 2022. Recolorable Posterization of Volumetric Radiance Fields Using Visibility-Weighted Palette Extraction. Computer Graphics Forum 41, 4 (2022), 149–160. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14594
[55]
Eric Veach. 1997. Robust Monte Carlo Methods for Light Transport Simulation. Ph.D. Thesis. Stanford University, United States – California.
[56]
Can Wang, Ruixiang Jiang, Menglei Chai, Mingming He, Dongdong Chen, and Jing Liao. 2023. NeRF-Art: Text-Driven Neural Radiance Fields Stylization. IEEE Transactions on Visualization and Computer Graphics (2023), 1–15.
[57]
Guanyang Wang and Tianze Wang. 2022. Unbiased Multilevel Monte Carlo methods for intractable distributions: MLMC meets MCMC.
[58]
Abdul-Majid Wazwaz. 2011. Linear and nonlinear integral equations. Vol. 639. Springer.
[59]
Rex West. 2021. Physically-Based Feature Line Rendering. ACM Trans. Graph. 40, 6, Article 246 (dec 2021), 11 pages.
[60]
Rex West, Iliyan Georgiev, and Toshiya Hachisuka. 2022. Marginal Multiple Importance Sampling. In SIGGRAPH Asia 2022 Conference Papers (Daegu, Republic of Korea) (SA ’22). Association for Computing Machinery, New York, NY, USA, Article 42, 8 pages.
[61]
Georges Winkenbach and David H. Salesin. 1994. Computer-Generated Pen-and-Ink Illustration. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’94). Association for Computing Machinery, New York, NY, USA, 91–100.
[62]
Holger Winnem?ller, Jan Kyprianidis, and Sven Olsen. 2012. XDoG: An eXtended difference-of-Gaussians compendium including advanced image stylization. Computers & Graphics 36 (10 2012), 740–753.
[63]
Holger Winnem?ller, Sven C Olsen, and Bruce Gooch. 2006. Real-Time Video Abstraction. ACM Transactions on Graphics 25, 3 (2006), 1221–1226.
[64]
Tizian Zeltner, Iliyan Georgiev, and Wenzel Jakob. 2020. Specular manifold sampling for rendering high-frequency caustics and glints. ACM Trans. Graph. 39, 4, Article 149 (aug 2020), 15 pages.
[65]
Lvmin Zhang, Yi Ji, Xin Lin, and Chunping Liu. 2017. Style transfer for anime sketches with enhanced residual u-net and auxiliary classifier gan. In 2017 4th IAPR Asian conference on pattern recognition (ACPR). IEEE, 506–511.
[66]
LvMin Zhang, Chengze Li, Tien-Tsin Wong, Yi Ji, and ChunPing Liu. 2018. Two-stage Sketch Colorization. ACM Transactions on Graphics 37, 6 (Nov. 2018).
