“Image-based reconstruction and synthesis of dense foliage” by Bradley, Nowrouzezahrai and Beardsley

  • ©Derek Bradley, Derek Nowrouzezahrai, and Paul Beardsley



Session Title:

    Image-Based Reconstruction


    Image-based reconstruction and synthesis of dense foliage




    Flora is an element in many computer-generated scenes. But trees, bushes and plants have complex geometry and appearance, and are difficult to model manually. One way to address this is to capture models directly from the real world. Existing techniques have focused on extracting macro structure such as the branching structure of trees, or the structure of broad-leaved plants with a relatively small number of surfaces. This paper presents a finer scale technique to demonstrate for the first time the processing of densely leaved foliage – computation of 3D structure, plus extraction of statistics for leaf shape and the configuration of neighboring leaves. Our method starts with a mesh of a single exemplar leaf of the target foliage. Using a small number of images, point cloud data is obtained from multi-view stereo, and the exemplar leaf mesh is fitted non-rigidly to the point cloud over several iterations. In addition, our method learns a statistical model of leaf shape and appearance during the reconstruction phase, and a model of the transformations between neighboring leaves. This information is useful in two ways – to augment and increase leaf density in reconstructions of captured foliage, and to synthesize new foliage that conforms to a user-specified layout and density. The result of our technique is a dense set of captured leaves with realistic appearance, and a method for leaf synthesis. Our approach excels at reconstructing plants and bushes that are primarily defined by dense leaves and is demonstrated with multiple examples.


    1. Ahrends, H. E., Etzold, S., Eugster, W., Buchmann, N., Jeanneret, F., and Wanner, H. 2009. Use of digital images to observe forest phenology and drought stress. In EGU General Assembly Conference Abstracts, vol. 11, 10886.Google Scholar
    2. Anastacio, F., Sousa, M. C., Samavati, F., and Jorge, J. A. 2006. Modeling plant structures using concept sketches. In Proceedings of NPAR, 105–113. Google ScholarDigital Library
    3. Baranoski, G. V. G., and Rokne, J. G. 2001. Efficiently simulating scattering of light by leaves. The Visual Computer 17, 8, 491–505.Google ScholarCross Ref
    4. Beeler, T., Bickel, B., Sumner, R., Beardsley, P., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graphics (Proc. SIGGRAPH 98). Google ScholarDigital Library
    5. Besl, P. J., and McKay, N. D. 1992. A method for registration of 3-d shapes. IEEE Trans. on PAMI 14, 2, 239–256. Google ScholarDigital Library
    6. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In SIGGRAPH, 187–194. Google ScholarDigital Library
    7. Blinn, J. F. 1977. Models of light reflection for computer synthesized pictures. SIGGRAPH Comput. Graph. 11, 2 (July), 192–198. Google ScholarDigital Library
    8. Bradley, D., Boubekeur, T., and Heidrich, W. 2008. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In Proc. CVPR.Google Scholar
    9. Chen, X., Neubert, B., Xu, Y.-Q., Deussen, O., and Kang, S. B. 2008. Sketch-based tree modeling using markov random field. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 27, 109. Google ScholarDigital Library
    10. COBWEB. 2013. Citizen Observatory Web,edina.ac.uk.Google Scholar
    11. Deussen, O., and Lintermann, B. 2005. Digital Design of Nature: Computer Generated Plants and Organics. Springer-Verlag. Google ScholarDigital Library
    12. Diener, J., Reveret, L., and Eugene, F. 2006. Hierarchical retargetting of 2d motion fields to the animation of 3d plant models. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation, 187–195. Google ScholarDigital Library
    13. Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6, 381–395. Google ScholarDigital Library
    14. Goesele, M., Curless, B., and Seitz, S. M. 2006. Multi-view stereo revisited. In CVPR. Google ScholarDigital Library
    15. Greene, N. 1989. Voxel space automata: modeling with stochastic growth processes in voxel space. SIGGRAPH Comput. Graph. 23, 3, 175–184. Google ScholarDigital Library
    16. Habel, R., Kusternig, A., and Wimmer, M. 2007. Physically based real-time translucency for leaves. In Proc. Eurographics Symposium on Rendering, 253–263. Google ScholarDigital Library
    17. Jakob, W., 2012. Mitsuba renderer. www.mitsuba-renderer.org.Google Scholar
    18. Li, C., Deussen, O., Song, Y.-Z., Willis, P., and Hall, P. 2011. Modeling and generating moving trees from video. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 30, 127. Google ScholarDigital Library
    19. Lindenmayer, A. 1968. Mathematical models for cellular interactions in development ii. simple and branching filaments with two-sided inputs. Journal of Theoretical Biology 18, 3, 300–315.Google ScholarCross Ref
    20. Lintermann, B., and Deussen, O. 1999. Interactive modeling of plants. IEEE Comput. Graph. Appl. 19, 56–65. Google ScholarDigital Library
    21. Livny, Y., Yan, F., Olson, M., Chen, B., Zhang, H., and El-Sana, J. 2010. Automatic reconstruction of tree skeletal structures from point clouds. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 29, 151. Google ScholarDigital Library
    22. Livny, Y., Pirk, S., Cheng, Z., Yan, F., Deussen, O., Cohen-Or, D., and Chen, B. 2011. Texture-lobes for tree modelling. ACM Trans. Graphics (Proc. SIGGRAPH) 30, 53. Google ScholarDigital Library
    23. Ma, W., Zha, H., Liu, J., Zhang, X., and Xiang, B. 2008. Image-based plant modeling by knowing leaves from their apexes. In Proc. ICPR.Google Scholar
    24. Mundermann, L., MacMurchy, P., Pivovarov, J., and Prusinkiewicz, P. 2003. Modeling lobed leaves. In Computer Graphics International.Google Scholar
    25. Neubert, B., Franken, T., and Deussen, O. 2007. Approximate image-based tree-modeling using particle flows. ACM Trans. Graphics (Proc. SIGGRAPH) 26, 88. Google ScholarDigital Library
    26. Okabe, M., Owada, S., and Igarashi, T. 2005. Interactive design of botanical trees using freehand sketches and example-based editing. Computer Graphics Forum 24, 3, 487–496.Google ScholarCross Ref
    27. Palubicki, W., Horel, K., Longay, S., Runions, A., Lane, B., Měch, R., and Prusinkiewicz, P. 2009. Self-organizing tree models for image synthesis. ACM Trans. Graph. (Proc. SIGGRAPH) 28, 58. Google ScholarDigital Library
    28. Papazov, C., and Burschka, D. 2011. An efficient ransac for 3d object recognition in noisy and occluded scenes. In Proceedings of the 10th Asian conference on Computer vision, 135–148. Google ScholarDigital Library
    29. Pirk, S., Niese, T., Deussen, O., and Neubert, B. 2012. Capturing and animating the morphogenesis of polygonal tree models. ACM Trans. Graph. 31, 6, 169:1–169:10. Google ScholarDigital Library
    30. Pirk, S., Stava, O., Kratt, J., Said, M. A. M., Neubert, B., Měch, R., Benes, B., and Deussen, O. 2012. Plastic trees: interactive self-adapting botanical tree models. ACM Trans. Graph. 31, 4, 50:1–50:10. Google ScholarDigital Library
    31. Prusinkiewicz, P., and Lindenmayer, A. 1990. The algorithmic beauty of plants. Springer-Verlag. Google ScholarDigital Library
    32. Prusinkiewicz, P., James, M., and Měch, R. 1994. Synthetic topiary. In SIGGRAPH ’94, 351–358. Google ScholarDigital Library
    33. Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., and Kang, S. B. 2006. Image-based plant modeling. ACM Trans. Graphics (Proc. SIGGRAPH) 25, 599–604. Google ScholarDigital Library
    34. Reche-Martinez, A., Martin, I., and Drettakis, G. 2004. Volumetric reconstruction and interactive rendering of trees from photographs. ACM Trans. Graphics (Proc. SIGGRAPH) 23, 720–727. Google ScholarDigital Library
    35. Schnabel, R., Wahl, R., and Klein, R. 2007. Efficient ransac for point-cloud shape detection. Computer Graphics Forum 26, 2, 214–226.Google ScholarCross Ref
    36. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR. Google ScholarDigital Library
    37. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: Exploring image collections in 3d. ACM Transactions on Graphics (Proc. of SIGGRAPH) 25, 3, 835–846. Google ScholarDigital Library
    38. Sonohat, G., Sinoquet, H., Kulandaivelu, V., Combes, D., and Lescourret, F. 2006. Three-dimensional reconstruction of partially 3d-digitized peach tree canopies. Tree Physiology 26, 3, 337–351.Google ScholarCross Ref
    39. Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., and Seidel, H.-P. 2004. Laplacian surface editing. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, 175–184. Google ScholarDigital Library
    40. Strecha, C., Fransens, R., and Gool, L. V. 2006. Combined depth and outlier estimation in multi-view stereo. In CVPR. Google ScholarDigital Library
    41. Szeliski, R. 1999. A multi-view approach to motion and stereo. In CVPR.Google Scholar
    42. Talton, J. O., Lou, Y., Lesser, S., Duke, J., Měch, R., and Koltun, V. 2011. Metropolis procedural modeling. ACM Trans. Graph. 30, 11:1–11:14. Google ScholarDigital Library
    43. Tan, P., Zeng, G., Wang, J., Kang, S. B., and Quan, L. 2007. Image-based tree modeling. ACM Trans. Graphics (Proc. SIGGRAPH) 26, 87. Google ScholarDigital Library
    44. Tan, P., Fang, T., Xiao, J., Zhao, P., and Quan, L. 2008. Single image tree modeling. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 27, 108. Google ScholarDigital Library
    45. Wilson, E. 2009. Ant Lovers Unite,www.npr.org.Google Scholar
    46. Wither, J., Boudon, F., Cani, M.-P., and Godin, C. 2009. Structure from silhouettes: a new paradigm for fast sketch-based design of trees. Computer Graphics Forum 28, 2, 541–550.Google ScholarCross Ref
    47. Xu, H., Gossett, N., and Chen, B. 2007. Knowledge and heuristic-based modeling of laser-scanned trees. ACM Trans. Graphics 26, 19–31. Google ScholarDigital Library

ACM Digital Library Publication: