“Image-based reconstruction and synthesis of dense foliage” by Bradley, Nowrouzezahrai and Beardsley
Conference:
Type(s):
Title:
- Image-based reconstruction and synthesis of dense foliage
Session/Category Title: Image-Based Reconstruction
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Flora is an element in many computer-generated scenes. But trees, bushes and plants have complex geometry and appearance, and are difficult to model manually. One way to address this is to capture models directly from the real world. Existing techniques have focused on extracting macro structure such as the branching structure of trees, or the structure of broad-leaved plants with a relatively small number of surfaces. This paper presents a finer scale technique to demonstrate for the first time the processing of densely leaved foliage – computation of 3D structure, plus extraction of statistics for leaf shape and the configuration of neighboring leaves. Our method starts with a mesh of a single exemplar leaf of the target foliage. Using a small number of images, point cloud data is obtained from multi-view stereo, and the exemplar leaf mesh is fitted non-rigidly to the point cloud over several iterations. In addition, our method learns a statistical model of leaf shape and appearance during the reconstruction phase, and a model of the transformations between neighboring leaves. This information is useful in two ways – to augment and increase leaf density in reconstructions of captured foliage, and to synthesize new foliage that conforms to a user-specified layout and density. The result of our technique is a dense set of captured leaves with realistic appearance, and a method for leaf synthesis. Our approach excels at reconstructing plants and bushes that are primarily defined by dense leaves and is demonstrated with multiple examples.
References:
1. Ahrends, H. E., Etzold, S., Eugster, W., Buchmann, N., Jeanneret, F., and Wanner, H. 2009. Use of digital images to observe forest phenology and drought stress. In EGU General Assembly Conference Abstracts, vol. 11, 10886.Google Scholar
2. Anastacio, F., Sousa, M. C., Samavati, F., and Jorge, J. A. 2006. Modeling plant structures using concept sketches. In Proceedings of NPAR, 105–113. Google ScholarDigital Library
3. Baranoski, G. V. G., and Rokne, J. G. 2001. Efficiently simulating scattering of light by leaves. The Visual Computer 17, 8, 491–505.Google ScholarCross Ref
4. Beeler, T., Bickel, B., Sumner, R., Beardsley, P., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graphics (Proc. SIGGRAPH 98). Google ScholarDigital Library
5. Besl, P. J., and McKay, N. D. 1992. A method for registration of 3-d shapes. IEEE Trans. on PAMI 14, 2, 239–256. Google ScholarDigital Library
6. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In SIGGRAPH, 187–194. Google ScholarDigital Library
7. Blinn, J. F. 1977. Models of light reflection for computer synthesized pictures. SIGGRAPH Comput. Graph. 11, 2 (July), 192–198. Google ScholarDigital Library
8. Bradley, D., Boubekeur, T., and Heidrich, W. 2008. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In Proc. CVPR.Google Scholar
9. Chen, X., Neubert, B., Xu, Y.-Q., Deussen, O., and Kang, S. B. 2008. Sketch-based tree modeling using markov random field. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 27, 109. Google ScholarDigital Library
10. COBWEB. 2013. Citizen Observatory Web,edina.ac.uk.Google Scholar
11. Deussen, O., and Lintermann, B. 2005. Digital Design of Nature: Computer Generated Plants and Organics. Springer-Verlag. Google ScholarDigital Library
12. Diener, J., Reveret, L., and Eugene, F. 2006. Hierarchical retargetting of 2d motion fields to the animation of 3d plant models. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation, 187–195. Google ScholarDigital Library
13. Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6, 381–395. Google ScholarDigital Library
14. Goesele, M., Curless, B., and Seitz, S. M. 2006. Multi-view stereo revisited. In CVPR. Google ScholarDigital Library
15. Greene, N. 1989. Voxel space automata: modeling with stochastic growth processes in voxel space. SIGGRAPH Comput. Graph. 23, 3, 175–184. Google ScholarDigital Library
16. Habel, R., Kusternig, A., and Wimmer, M. 2007. Physically based real-time translucency for leaves. In Proc. Eurographics Symposium on Rendering, 253–263. Google ScholarDigital Library
17. Jakob, W., 2012. Mitsuba renderer. www.mitsuba-renderer.org.Google Scholar
18. Li, C., Deussen, O., Song, Y.-Z., Willis, P., and Hall, P. 2011. Modeling and generating moving trees from video. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 30, 127. Google ScholarDigital Library
19. Lindenmayer, A. 1968. Mathematical models for cellular interactions in development ii. simple and branching filaments with two-sided inputs. Journal of Theoretical Biology 18, 3, 300–315.Google ScholarCross Ref
20. Lintermann, B., and Deussen, O. 1999. Interactive modeling of plants. IEEE Comput. Graph. Appl. 19, 56–65. Google ScholarDigital Library
21. Livny, Y., Yan, F., Olson, M., Chen, B., Zhang, H., and El-Sana, J. 2010. Automatic reconstruction of tree skeletal structures from point clouds. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 29, 151. Google ScholarDigital Library
22. Livny, Y., Pirk, S., Cheng, Z., Yan, F., Deussen, O., Cohen-Or, D., and Chen, B. 2011. Texture-lobes for tree modelling. ACM Trans. Graphics (Proc. SIGGRAPH) 30, 53. Google ScholarDigital Library
23. Ma, W., Zha, H., Liu, J., Zhang, X., and Xiang, B. 2008. Image-based plant modeling by knowing leaves from their apexes. In Proc. ICPR.Google Scholar
24. Mundermann, L., MacMurchy, P., Pivovarov, J., and Prusinkiewicz, P. 2003. Modeling lobed leaves. In Computer Graphics International.Google Scholar
25. Neubert, B., Franken, T., and Deussen, O. 2007. Approximate image-based tree-modeling using particle flows. ACM Trans. Graphics (Proc. SIGGRAPH) 26, 88. Google ScholarDigital Library
26. Okabe, M., Owada, S., and Igarashi, T. 2005. Interactive design of botanical trees using freehand sketches and example-based editing. Computer Graphics Forum 24, 3, 487–496.Google ScholarCross Ref
27. Palubicki, W., Horel, K., Longay, S., Runions, A., Lane, B., Měch, R., and Prusinkiewicz, P. 2009. Self-organizing tree models for image synthesis. ACM Trans. Graph. (Proc. SIGGRAPH) 28, 58. Google ScholarDigital Library
28. Papazov, C., and Burschka, D. 2011. An efficient ransac for 3d object recognition in noisy and occluded scenes. In Proceedings of the 10th Asian conference on Computer vision, 135–148. Google ScholarDigital Library
29. Pirk, S., Niese, T., Deussen, O., and Neubert, B. 2012. Capturing and animating the morphogenesis of polygonal tree models. ACM Trans. Graph. 31, 6, 169:1–169:10. Google ScholarDigital Library
30. Pirk, S., Stava, O., Kratt, J., Said, M. A. M., Neubert, B., Měch, R., Benes, B., and Deussen, O. 2012. Plastic trees: interactive self-adapting botanical tree models. ACM Trans. Graph. 31, 4, 50:1–50:10. Google ScholarDigital Library
31. Prusinkiewicz, P., and Lindenmayer, A. 1990. The algorithmic beauty of plants. Springer-Verlag. Google ScholarDigital Library
32. Prusinkiewicz, P., James, M., and Měch, R. 1994. Synthetic topiary. In SIGGRAPH ’94, 351–358. Google ScholarDigital Library
33. Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., and Kang, S. B. 2006. Image-based plant modeling. ACM Trans. Graphics (Proc. SIGGRAPH) 25, 599–604. Google ScholarDigital Library
34. Reche-Martinez, A., Martin, I., and Drettakis, G. 2004. Volumetric reconstruction and interactive rendering of trees from photographs. ACM Trans. Graphics (Proc. SIGGRAPH) 23, 720–727. Google ScholarDigital Library
35. Schnabel, R., Wahl, R., and Klein, R. 2007. Efficient ransac for point-cloud shape detection. Computer Graphics Forum 26, 2, 214–226.Google ScholarCross Ref
36. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR. Google ScholarDigital Library
37. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: Exploring image collections in 3d. ACM Transactions on Graphics (Proc. of SIGGRAPH) 25, 3, 835–846. Google ScholarDigital Library
38. Sonohat, G., Sinoquet, H., Kulandaivelu, V., Combes, D., and Lescourret, F. 2006. Three-dimensional reconstruction of partially 3d-digitized peach tree canopies. Tree Physiology 26, 3, 337–351.Google ScholarCross Ref
39. Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., and Seidel, H.-P. 2004. Laplacian surface editing. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, 175–184. Google ScholarDigital Library
40. Strecha, C., Fransens, R., and Gool, L. V. 2006. Combined depth and outlier estimation in multi-view stereo. In CVPR. Google ScholarDigital Library
41. Szeliski, R. 1999. A multi-view approach to motion and stereo. In CVPR.Google Scholar
42. Talton, J. O., Lou, Y., Lesser, S., Duke, J., Měch, R., and Koltun, V. 2011. Metropolis procedural modeling. ACM Trans. Graph. 30, 11:1–11:14. Google ScholarDigital Library
43. Tan, P., Zeng, G., Wang, J., Kang, S. B., and Quan, L. 2007. Image-based tree modeling. ACM Trans. Graphics (Proc. SIGGRAPH) 26, 87. Google ScholarDigital Library
44. Tan, P., Fang, T., Xiao, J., Zhao, P., and Quan, L. 2008. Single image tree modeling. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 27, 108. Google ScholarDigital Library
45. Wilson, E. 2009. Ant Lovers Unite,www.npr.org.Google Scholar
46. Wither, J., Boudon, F., Cani, M.-P., and Godin, C. 2009. Structure from silhouettes: a new paradigm for fast sketch-based design of trees. Computer Graphics Forum 28, 2, 541–550.Google ScholarCross Ref
47. Xu, H., Gossett, N., and Chen, B. 2007. Knowledge and heuristic-based modeling of laser-scanned trees. ACM Trans. Graphics 26, 19–31. Google ScholarDigital Library
