“ICTree: automatic perceptual metrics for tree models” by Polasek, Hrusa, Benes and Čadík
Conference:
Type(s):
Title:
- ICTree: automatic perceptual metrics for tree models
Session/Category Title: Natural Phenomena
Presenter(s)/Author(s):
Abstract:
Many algorithms for virtual tree generation exist, but the visual realism of the 3D models is unknown. This problem is usually addressed by performing limited user studies or by a side-by-side visual comparison. We introduce an automated system for realism assessment of the tree model based on their perception. We conducted a user study in which 4,000 participants compared over one million pairs of images to collect subjective perceptual scores of a large dataset of virtual trees. The scores were used to train two neural-network-based predictors. A view independent ICTreeF uses the tree model’s geometric features that are easy to extract from any model. The second is ICTreeI that estimates the perceived visual realism of a tree from its image. Moreover, to provide an insight into the problem, we deduce intrinsic attributes and evaluate which features make trees look like real trees. In particular, we show that branching angles, length of branches, and widths are critical for perceived realism. We also provide three datasets: carefully curated 3D tree geometries and tree skeletons with their perceptual scores, multiple views of the tree geometries with their scores, and a large dataset of images with scores suitable for training deep neural networks.
References:
1. André Altmann, Laura Toloşi, Oliver Sander, and Thomas Lengauer. 2010. Permutation importance: a corrected feature importance measure. Bioinformatics 26, 10 (2010), 1340–1347.
2. Fabricio Anastacio, Mario Costa Sousa, Faramarz Samavati, and Joaquim A. Jorge. 2006. Modeling Plant Structures Using Concept Sketches. In Proc. of NPAR (NPAR ’06). Association for Computing Machinery, 105–113.
3. Pontus Andersson, Jim Nilsson, Tomas Akenine-Möller, Magnus Oskarsson, Kalle Åström, and Mark D Fairchild. 2020. FLIP: A Difference Evaluator for Alternating Images. ACM Trans. Graph. 3, 2 (2020), 23.
4. Tunç Ozan Aydın, Martin Čadík, Karol Myszkowski, and Hans-Peter Seidel. 2010. Video Quality Assessment for Computer Graphics Applications. In ACM Trans. on Graph. (Proc. of SIGGRAPH Asia). ACM, 1–10.
5. Bedrich Benes and Erik Uriel Millán. 2002. Virtual Climbing Plants Competing for Space. In Proceedings of the Computer Animation (CA ’02). IEEE Computer Society, USA, 33.
6. Jan Beneš, Tom Kelly, F Děchtěrenko, Jaroslav Křivánek, and Pascal Müller. 2017. On realism of architectural procedural models. In Comp. Graph. Forum, Vol. 36. Wiley Online Library, 225–234.
7. Rolf Borchert and Hisao Honda. 1984. Control of development in the bifurcating branch system of Tabebuia rosea: a computer simulation. Botanical Gazette 145, 2 (1984), 184–195.
8. Sebastian Bosse, Dominique Maniry, Klaus-Robert Müller, Thomas Wiegand, and Wojciech Samek. 2018. Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment. IEEE Transactions on Image Processing 27, 1 (2018), 206–219.
9. Anne-Laure Boulesteix, Andreas Bender, Justo Lorenzo Bermejo, and Carolin Strobl. 2012. Random forest Gini importance favours SNPs with large minor allele frequency: impact, sources and recommendations. Briefings in Bioinformatics 13, 3 (2012), 292–304.
10. Derek Bradley, Derek Nowrouzezahrai, and Paul Beardsley. 2013. Image-based Reconstruction Synthesis of Dense Foliage. ACM Trans. Graph. 32, 4, Article 74 (2013), 74:1–74:10 pages.
11. Leo Breiman. 2001. Random Forests. Machine Learning 45, 1 (2001), 5–32.
12. Martin Čadík, Robert Herzog, Rafal Mantiuk, Radoslaw Mantiuk, Karol Myszkowski, and Hans-Peter Seidel. 2013. Learning to Predict Localized Distortions in Rendered Images. Comp. Graph. Forum 32, 7 (2013), 401–410.
13. Guillaume Cerutti, Laure Tougne, Julien Mille, Antoine Vacavant, and Didier Coquin. 2013. Understanding leaves in natural images-a model-based approach for tree species identification. Computer Vision and Image Understanding 117, 10 (2013), 1482–1501.
14. Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. 2015. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012 (2015).
15. Massimiliano Corsini, Mohamed-Chaker Larabi, Guillaume Lavoué, Oldřich Petřík, Libor Váša, and Kai Wang. 2013. Perceptual Metrics for Static and Dynamic Triangle Meshes. Comp. Graph. Forum (2013).
16. Herbert Aron David. 1988. The Method of Paired Comparisons. C. Griffin. https://books.google.cz/books?id=bB21VsB_GyYC
17. Phillippe De Reffye, Claude Edelin, Jean Françon, Marc Jaeger, and Claude Puech. 1988. Plant models faithful to botanical structure and development. ACM Siggraph Computer Graphics 22, 4 (1988), 151–158.
18. Lisa M DeBruine, Benedict C Jones, Layla Unger, Anthony C Little, and David R Feinberg. 2007. Dissociating averageness and attractiveness: attractive faces are not always average. Journal of Exp. Psychology: Human Perc. and Performance 33, 6 (2007), 1420.
19. Oliver Deussen, Pat Hanrahan, Brend Lintermann, Radomír Měch, Max Pharr, and Przemyslaw Prusinkiewicz. 1998. Realistic Modeling and Rendering of Plant Ecosystems. ACM Trans. Graph. (1998), 275–286.
20. Shenglan Du, Roderik Lindenbergh, Hugo Ledoux, Jantien Stoter, and Liangliang Nan. 2019. AdTree: Accurate, Detailed, and Automatic Modelling of Laser-Scanned Trees. Remote Sensing 11, 18 (9 2019), 2074.
21. Shanghua Gao, Ming-Ming Cheng, Kai Zhao, Xin-Yu Zhang, Ming-Hsuan Yang, and Philip HS Torr. 2019. Res2net: A new multi-scale backbone architecture. IEEE transactions on pattern analysis and machine intelligence (2019).
22. Ned Greene. 1989. Voxel Space Automata: Modeling with Stochastic Growth Processes in Voxel Space. SIGGRAPH Comp. Graph. 23, 3 (1989), 175–184.
23. Jianwei Guo, Shibiao Xu, Dong-Ming Yan, Zhanglin Cheng, Marc Jaeger, and Xiaopeng Zhang. 2018. Realistic procedural plant modeling from multiple view images. IEEE Trans. on Vis. and Comp. Graphics 26, 2 (2018), 1372 — 1384.
24. Ralf Habel, Alexander Kusternig, and Michael Wimmer. 2009. Physically Guided Animation of Trees. Comp. Graph. Forum 28, 2 (2009), 523–532.
25. Torsten Hädrich, Bedrich Benes, Oliver Deussen, and Sören. Pirk. 2017. Interactive Modeling and Authoring of Climbing Plants. Comp. Graph. Forum 36, 2 (2017), 49–61.
26. Alexander Hapfelmeier and Kurt Ulm. 2013. A new variable selection approach using random forests. Computational Statistics & Data Analysis 60 (2013), 50–69.
27. Robert Herzog, Martin Čadík, Tunç O. Aydın, Kwawng In Kim, Karol Myszkowski, and Hans-Peter Seidel. 2012. NoRM: No-Reference Image Quality Metric for Realistic Image Synthesis. Comp. Graph. Forum 31, 2 (2012), 545–554.
28. Hui Huang, Shihao Wu, Daniel Cohen-Or, Minglun Gong, Hao Zhang, Guiqing Li, and Baoquan Chen. 2013. L1-medial skeleton of point cloud. ACM Trans. Graph. 32, 4 (7 2013), 1–8.
29. Mathieu Jung, D. Léger, and M. Gazalet. 2002. Univariant assessment of the quality of images. J. Electronic Imaging 11 (2002), 354–364.
30. Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Optimization. arXiv:cs.LG/1412.6980
31. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25. Curran Associates, Inc., 1097–1105.
32. Neeraj Kumar, Peter N Belhumeur, Arijit Biswas, David W Jacobs, W John Kress, Ida C Lopez, and Joäo VB Soares. 2012. Leafsnap: A computer vision system for automatic plant species identification. In ECCV. Springer, 502–516.
33. Harold L Kundel and Marcia Polansky. 2003. Measurement of observer agreement. Radiology 228, 2 (2003), 303–308.
34. Debarati Kundu, Lark Kwon Choi, Alan C. Bovik, and Brian L. Evans. 2018. Perceptual quality evaluation of synthetic pictures distorted by compression and transmission. Signal Processing: Image Communication 61 (2018), 54 — 72.
35. Guillaume Lavoué, Mohamed-Chaker Larabi, and Libor Vása. 2016. On the Efficiency of Image Metrics for Evaluating the Visual Quality of 3D Models. IEEE TVCG 22, 8 (2016), 1987–1999.
36. Guillaume Lavoué. 2011. A Multiscale Metric for 3D Mesh Visual Quality Assessment. Comp. Graph. Forum 30, 5 (2011), 1427–1437.
37. Bosheng Li, Jacek Kałużny, Jonathan Klein, Dominik Michels, Wojtek Pałubicki, Bedrich Benes, and Sören Pirk. 2021. Learning to Reconstruct Botanical Trees from Single Images. ACM Trans. Graph. 40, 6 (2021).
38. Yangyan Li, Xiaochen Fan, Niloy J. Mitra, Daniel Chamovitz, Daniel Cohen-Or, and Baoquan Chen. 2013. Analyzing Growing Plants from 4D Point Cloud Data. ACM Trans. Graph. 32, 6, Article 157 (Nov. 2013), 10 pages.
39. Yan Lin, Ji Liu, and Jianlin Zhou. 2020. A novel tree-structured point cloud dataset for skeletonization algorithm evaluation. CoRR abs/2001.02823 (2020). arXiv:2001.02823
40. Aristid Lindenmayer. 1968. Mathematical models for cellular interaction in development. J. Theor. Biol. Parts I and II, 18 (1968), 280–315.
41. Berndt Lintermann and Oliver Deussen. 1999. Interactive Modeling of Plants. IEEE Comput. Graph. Appl. 19, 1 (1999), 56–65.
42. Yanchao Liu, Jianwei Guo, Bedrich Benes, Oliver Deussen, Xiaopeng Zhang, and Hui Huang. 2021. TreePartNet: Neural Decomposition of Point Clouds for 3D Tree Reconstruction. ACM Trans. Graph. 40, 6 (2021).
43. Yotam Livny, Sören Pirk, Zhanglin Cheng, Feilong Yan, Oliver Deussen, Daniel Cohen-Or, and Baoquan Chen. 2011. Texture-lobes for Tree Modelling. ACM Trans. Graph. 30, 4, Article 53 (2011), 10 pages.
44. Yotam Livny, Feilong Yan, Matt Olson, Baoquan Chen, Hao Zhang, and Jihad El-Sana. 2010. Automatic reconstruction of tree skeletal structures from point clouds. In ACM Trans. Graph., Vol. 29. 1–8.
45. Steven Longay, Adam Runions, Francois Boudon, and Przemyslaw Prusinkiewicz. 2012. TreeSketch: interactive procedural modeling of trees on a tablet. In Proc. of the Intl. Symp. on SBIM. 107–120.
46. Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. 2013. Rectifier nonlinearities improve neural network acoustic models. In Proc. icml, Vol. 30. 3.
47. Rafal Mantiuk, Kil Joong Kim, Allan G. Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: A Calibrated Visual Metric for Visibility and Quality Predictions in All Luminance Conditions. ACM Trans. Graph. 30, 4, Article 40 (July 2011), 14 pages.
48. Ravish Mehra, Qingnan Zhou, Jeremy Long, Alla Sheffer, Amy Gooch, and Niloy Mitra. 2009. Abstraction of Man-Made Shapes. ACM Trans. Graph. 28, 5 (2009), 1–10.
49. Ryoko Minamino and Masaki Tateno. 2014. Tree branching: Leonardo da Vinci’s rule versus biomechanical models. PloS one 9, 4 (2014), e93535.
50. Anush Krishna Moorthy and Alan Conrad Bovik. 2010. A Two-Step Framework for Constructing Blind Image Quality Indices. IEEE Signal Processing Letters 17, 5 (2010), 513–516.
51. Radomír Měch and Przemyslaw Prusinkiewicz. 1996. Visual models of plants interacting with their environment. In Proc. of SIGGRAPH. ACM, 397–410.
52. Boris Neubert, Thomas Franken, and O. Deussen. 2007. Approximate Image-based Tree-modeling Using Particle Flows. ACM Trans. Graph. 26, 3, Article 88 (2007).
53. Makoto Okabe, Shigeru Owada, and Takeo Igarashi. 2007. Interactive Design of Botanical Trees Using Freehand Sketches and Example-based Editing. In ACM SIGGRAPH Courses. ACM, Article 26.
54. Peter E. Oppenheimer. 1986. Real Time Design and Animation of Fractal Plants and Trees. In Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’86). Association for Computing Machinery, New York, NY, USA, 55–64.
55. Wojciech Palubicki, Kipp Horel, Steven Longay, Adam Runions, Brendan Lane, Radomír Měch, and Przemyslaw Prusinkiewicz. 2009. Self-organizing Tree Models for Image Synthesis. ACM Trans. Graph. 28, 3, Article 58 (2009), 10 pages.
56. Yixin Pan, Irene Cheng, and Anup Basu. 2005. Quality metric for approximating subjective evaluation of 3-D objects. IEEE Transactions on Multimedia 7, 2 (2005), 269–279.
57. Maria Perez-Ortiz and Rafal K Mantiuk. 2017. A practical guide and software for analysing pairwise comparison experiments. arXiv preprint arXiv:1712.03686 (2017).
58. Sören Pirk, Michal Jarząbek, Torsten Hädrich, Dominik L. Michels, and Wojciech Palubicki. 2017. Interactive Wood Combustion for Botanical Tree Models. ACM Trans. Graph. 36, 6, Article 197 (Nov. 2017), 12 pages.
59. Sören Pirk, Till Niese, Torsten Hädrich, Bedrich Benes, and Oliver Deussen. 2014. Windy Trees: Computing Stress Response for Developmental Tree Models. ACM Trans. Graph. 33, 6, Article 204 (Nov. 2014), 11 pages.
60. Sören Pirk, Ondrej Stava, Julian Kratt, Michel Abdul Massih Said, Boris Neubert, Radomír Měch, Bedrich Benes, and Oliver Deussen. 2012. Plastic Trees: Interactive Self-Adapting Botanical Tree Models. ACM Trans. Graph. 31, 4 (2012).
61. Przemyslaw Prusinkiewicz. 1986. Graphical applications of L-systems. In Proc. on Graph. Interf. 247–253.
62. Przemyslaw Prusinkiewicz. 1998. In search of the right abstraction: the synergy between art, science, and information technology in the modeling of natural phenomena. Art @ Science. Springer, Wien. 60–68 pages.
63. Przemyslaw Prusinkiewicz, Mark S. Hammel, and Eric Mjolsness. 1993. Animation of plant development. In Proceedings SIGGRAPH (SIGGRAPH ’93). ACM, 351–360.
64. Przemyslaw Prusinkiewicz, Mark James, and Radomír Měch. 1994. Synthetic Topiary. Proceedings SIGGRAPH (1994), 351–358.
65. Przemyslaw Prusinkiewicz and Aristid Lindenmayer. 1990. The Algorithmic Beauty of Plants. Springer-Verlag New York, Inc.
66. Hongxing Qin, Jia Han, Ning Li, Hui Huang, and Baoquan Chen. 2020. Mass-Driven Topology-Aware Curve Skeleton Extraction from Incomplete Point Clouds. IEEE Trans. on Vis. and Comp. Graphics 26, 9 (2020), 2805–2817.
67. Suren Deepak Rajasekaran, Hao Kang, Bedrich Benes, Martin Čadík, Eric Galin, Eric Guérin, Adrien Peytavie, and Pavel Slavík. 2019. PTRM: Perceived Terrain Realism Metrics. arXiv preprint arXiv:1909.04610 (2019).
68. Ganesh Ramanarayanan, James Ferwerda, Bruce Walter, and Kavita Bala. 2007. Visual Equivalence: Towards a New Standard for Image Fidelity. ACM Trans. Graph. 26, 3 (2007), 12.
69. Alex Reche-Martinez, Ignacio Martin, and George Drettakis. 2004. Volumetric reconstruction and interactive rendering of trees from photographs. ACM Trans. Graph. 23, 3 (2004), 720–727.
70. Sashank J Reddi, Satyen Kale, and Sanjiv Kumar. 2019. On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237 (2019).
71. Bernice E. Rogowitz and Holly E. Rushmeier. 2001. Are image quality metrics adequate to evaluate the quality of geometric objects?. In Human Vision and Electronic Imaging VI, Vol. 4299. Intl. Soc. for Optics and Photonics, SPIE, 340 — 348.
72. Petr Sloup, Tomas Rebok, and Jan Hanus. 2013. Automatic Tree Reconstruction from its Laser Scan. In Global Change and Resilience. Global Change Research Centre, Academy of Sciences of the Czech Republic, Brno.
73. SpeedTree. 2021. SpeedTree. http://www.speedtree.com
74. Ondrej Stava, Sören Pirk, Julian Kratt, Baoquan Chen, Radomír Měch, Oliver Deussen, and Bedrich Benes. 2014. Inverse procedural modelling of trees. In Comp. Graph. Forum, Vol. 33. Wiley Online Library, 118–131.
75. Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. 2015. Multi-View Convolutional Neural Networks for 3D Shape Recognition. In ICCV.
76. Sundaram Suresh, R. Venkatesh Babu, and Hyoung Joong Kim. 2009. No-reference image quality assessment using modified extreme learning machine classifier. Applied Soft Computing 9, 2 (2009), 541 — 552.
77. Andrea Tagliasacchi, Ibraheem Alhashim, Matt Olson, and Hao Zhang. 2012. Mean Curvature Skeletons. Comp. Graph. Forum 31, 5 (2012), 1735–1744.
78. Ping Tan, Tian Fang, Jianxiong Xiao, Peng Zhao, and Long Quan. 2008. Single Image Tree Modeling. ACM Trans. Graph. 27, 5, Article 108 (Dec. 2008), 7 pages.
79. Ping Tan, Gang Zeng, Jingdong Wang, Sing Bing Kang, and Long Quan. 2007. Image-based Tree Modeling. ACM Trans. Graph. 26, 3, Article 87 (2007).
80. Huixuan Tang, Neel Joshi, and Ashish Kapoor. 2011. Learning a blind measure of perceptual image quality. In CVPR 2011. IEEE, 305–312.
81. Fakhri Torkhani, Kai Wang, and Jean-Marc Chassery. 2015. Perceptual quality assessment of 3D dynamic meshes: Subjective and objective studies. Signal Processing: Image Communication 31 (2015), 185–204.
82. Bohan Wang, Yili Zhao, and Jernej Barbič. 2017. Botanical Materials Based on Biome-chanics. ACM Trans. Graph. 36, 4, Article 135 (July 2017), 13 pages.
83. Guan Wang, Hamid Laga, Ning Xie, Jinyuan Jia, and Hedi Tabia. 2018. The shape space of 3D botanical tree models. ACM Trans. Graph. 37, 1 (2018).
84. Kai Wang, Fakhri Torkhani, and Annick Montanvert. 2012. A fast roughness-based approach to the assessment of 3D mesh visual quality. Computers and Graphics 36, 7 (Nov. 2012), 808–818.
85. Wenping Wang, Bert Jüttler, Dayue Zheng, and Yang Liu. 2008. Computation of Rotation Minimizing Frames. ACM Trans. Graph. 27, 1, Article 2 (March 2008), 18 pages.
86. Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image Quality Assessment: From Error Visibility to Structural Similarity. Trans. Img. Proc. 13, 4 (April 2004), 600–612.
87. Geoffrey B West, James H Brown, and Brian J Enquist. 1999. A general model for the structure and allometry of plant vascular systems. Nature 400, 6745 (1999), 664–667.
88. Krzysztof Wolski, Daniele Giunchi, Nanyang Ye, Piotr Didyk, Karol Myszkowski, Radoslaw Mantiuk, Hans-Peter Seidel, Anthony Steed, and Rafal K. Mantiuk. 2018. Dataset and Metrics for Predicting Local Visible Differences. ACM Trans. Graph. 37, 5 (2018), 172:1–172:14.
89. Jinliang Wu, Xiaoyong Shen, Wei Zhu, and Ligang Liu. 2013. Mesh Saliency with Global Rarity. Graph. Models 75, 5 (Sept. 2013), 255–264.
90. Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In CVPR. 1912–1920.
91. Ke Xie, Feilong Yan, Andrei Sharf, Oliver Deussen, Hui Huang, and Baoquan Chen. 2016. Tree Modeling with Real Tree-Parts Examples. IEEE Trans. on Vis. and Comp. Graphics 22, 12 (2016), 2608–2618.
92. Hui Xu, Nathan Gossett, and Baoquan Chen. 2007. Knowledge and heuristic-based modeling of laser-scanned trees. ACM Trans. Graph. 26, 4 (2007), Article 19.
93. Peng Ye, Jayant Kumar, and David Doermann. 2014. Beyond Human Opinion Scores: Blind Image Quality Assessment Based on Synthetic Scores. In IEEE CVPR. 4241–4248.
94. Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR. 586–595.
95. Yili Zhao and Jernej Barbič. 2013. Interactive Authoring of Simulation-ready Plants. ACM Trans. Graph. 32, 4, Article 84 (2013), 12 pages.
