“WorldBrush: interactive example-based synthesis of procedural virtual worlds” by Emilien, Vimont, Cani, Poulin and Benes

We present a novel approach for the interactive synthesis and editing of virtual worlds. Our method is inspired by painting operations and uses methods for statistical example-based synthesis to automate content synthesis and deformation. Our real-time approach takes a form of local inverse procedural modeling based on intermediate statistical models: selected regions of procedurally and manually constructed example scenes are analyzed, and their parameters are stored as distributions in a palette, similar to colors on a painter’s palette. These distributions can then be interactively applied with brushes and combined in various ways, like in painting systems. Selected regions can also be moved or stretched while maintaining the consistency of their content. Our method captures both distributions of elements and structured objects, and models their interactions. Results range from the interactive editing of 2D artwork maps to the design of 3D virtual worlds, where constraints set by the terrain’s slope are also taken into account.

1. Aliaga, D. G., Vanegas, C. A., and Benes, B. 2008. Interactive example-based urban layout synthesis. ACM Trans. Graph. (SIGGRAPH Asia) 27, 5, 160:1–10. Google ScholarDigital Library
2. Ashikhmin, M. 2001. Synthesizing natural textures. In Proc. Symp. on Interactive 3D Graphics (I3D), ACM, 217–226. Google ScholarDigital Library
3. Avidan, S., and Shamir, A. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. (SIGGRAPH) 26, 3, 10:1–9. Google ScholarDigital Library
4. Bokeloh, M., Wand, M., and Seidel, H.-P. 2010. A connection between partial symmetry and inverse procedural modeling. ACM Trans. Graph. (SIGGRAPH) 29, 4, 104:1–10. Google ScholarDigital Library
5. Dischler, J.-M., Maritaud, K., Lévy, B., and Ghazanfarpour, D. 2002. Texture particles. Computer Graphics Forum (Eurographics) 21, 3, 401–410.Google ScholarCross Ref
6. Emilien, A., Bernhardt, A., Peytavie, A., Cani, M.-P., and Galin, E. 2012. Procedural generation of villages on arbitrary terrains. The Visual Computer (CGI) 28, 6-8, 809–818. Google ScholarDigital Library
7. Gain, J., Marais, P., and Strasser, W. 2009. Terrain sketching. In Proc. Symp. on Interactive 3D Graphics and Games (I3D), ACM, 31–38. Google ScholarDigital Library
8. Galin, E., Peytavie, A., Gurin, E., and Benes, B. 2011. Authoring hierarchical road networks. Computer Graphics Forum (Eurographics) 30, 7, 2021–2030.Google ScholarCross Ref
9. Génevaux, J.-D., Galin, E., Guérin, E., Peytavie, A., and Benes, B. 2013. Terrain generation using procedural models based on hydrology. ACM Trans. Graphics (SIGGRAPH) 32, 4, 143:1–13. Google ScholarDigital Library
10. Geyer, C. J., and Møller, J. 1994. Simulation procedures and likelihood inference for spatial point processes. Scandinavian journal of statistics 21, 4, 359–373.Google Scholar
11. Hurtut, T., Landes, P.-E., Thollot, J., Gousseau, Y., Drouillhet, R., and Coeurjolly, J.-F. 2009. Appearance-guided synthesis of element arrangements by example. In Proc. Symp. on Non-Photorealistic Animation and Rendering (NPAR), ACM, 51–60. Google ScholarDigital Library
12. Ijiri, T., Měch, R., Igarashi, T., and Miller, G. 2008. An Example-based Procedural System for Element Arrangement. Computer Graphics Forum (Eurographics) 27, 2, 429–436.Google ScholarCross Ref
13. Kazi, R. H., Igarashi, T., Zhao, S., and Davis, R. 2012. Vignette: Interactive texture design and manipulation with freeform gestures for pen-and-ink illustration. In Proc. SIGCHI Conference on Human Factors in Computing Systems (CHI), ACM, 1727–1736. Google ScholarDigital Library
14. Lafarge, F., Gimel’farb, G., and Descombes, X. 2010. Geometric feature extraction by a multimarked point process. IEEE Trans. Pattern Analysis and Machine Intelligence 32, 9, 1597–1609. Google ScholarDigital Library
15. Landes, P.-E., Galerne, B., and Hurtut, T. 2013. A shapeaware model for discrete texture synthesis. Computer Graphics Forum (EGSR) 32, 4, 67–76. Google ScholarDigital Library
16. Lipp, M., Scherzer, D., Wonka, P., and Wimmer, M. 2011. Interactive modeling of city layouts using layers of procedural content. Computer Graphics Forum (Eurographics) 30, 2, 345–354.Google ScholarCross Ref
17. Longay, S., Runions, A., Boudon, F., and Prusinkiewicz, P. 2012. TreeSketch: Interactive procedural modeling of trees on a tablet. In Proc. Eurographics Symp. on sketch-based interfaces and modeling (SBIM), 107–120. Google ScholarDigital Library
18. Lu, J., Barnes, C., Wan, C., Asente, P., Měch, R., and Finkelstein, A. 2014. Decobrush: Drawing structured decorative patterns by example. ACM Trans. Graph. (SIGGRAPH) 33, 4, 90:1–9. Google ScholarDigital Library
19. Milliez, A., Noris, G., Baran, I., Coros, S., Cani, M.-P., Nitti, M., Marra, A., Gross, M., and Sumner, R. W. 2014. Hierarchical motion brushes for animation instancing. In Proc. Workshop on Non-Photorealistic Animation and Rendering (NPAR), ACM, 71–79. Google ScholarDigital Library
20. Müller, P., Wonka, P., Haegler, S., Ulmer, A., and Van Gool, L. 2006. Procedural modeling of buildings. ACM Trans. Graph. (SIGGRAPH) 25, 3, 614–623. Google ScholarDigital Library
21. Měch, R., and Prusinkiewicz, P. 1996. Visual models of plants interacting with their environment. In SIGGRAPH Comput. Graph., ACM, 397–410. Google ScholarDigital Library
22. Olsen, L., Samavati, F. F., Sousa, M. C., and Jorge, J. A. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1, 85–103. Google ScholarDigital Library
23. öztireli, A. C., and Gross, M. 2012. Analysis and synthesis of point distributions based on pair correlation. ACM Trans. Graph. (SIGGRAPH Asia) 31, 6, 170:1–10. Google ScholarDigital Library
24. Parish, Y. I. H., and Müller, P. 2001. Procedural modeling of cities. In SIGGRAPH Comput. Graph., ACM, 301–308. Google ScholarDigital Library
25. Read, A. L. 1999. Linear interpolation of histograms. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 425, 1, 357–360.Google ScholarCross Ref
26. Sibbing, D., Pavic, D., and Kobbelt, L. 2010. Image synthesis for branching structures. Computer Graphics Forum (Pacific Graphics) 29, 7, 2135–2144.Google ScholarCross Ref
27. Smelik, R., Tutenel, T., de Kraker, K., and Bidarra, R. 2011. A declarative approach to procedural modeling of virtual worlds. Computers & Graphics 35, 2, 352–363. Google ScholarDigital Library
28. Smelik, R. M., Tutenel, T., Bidarra, R., and Benes, B. 2014. A survey on procedural modelling for virtual worlds. Computer Graphics Forum 33, 6, 31–50.Google ScholarDigital Library
29. Smith, A. R. 1984. Plants, fractals, and formal languages. SIGGRAPH Comput. Graph. 18, 3 (Jan.), 1–10. Google ScholarDigital Library
30. Št’ava, O., Benes, B., Měch, R., Aliaga, D. G., and Krišstof, P. 2010. Inverse procedural modeling by automatic generation of L-systems. Computer Graphics Forum (Eurographics) 29, 2, 665–674.Google ScholarCross Ref
31. Št’ava, O., Pirk, S., Kratt, J., Chen, B., Měch, R., Deussen, O., and Benes, B. 2014. Inverse procedural modelling of trees. Computer Graphics Forum 33, 6, 118–131.Google ScholarDigital Library
32. Sun, Q., Zhang, L., Zhang, M., Ying, X., Xin, S.-Q., Xia, J., and He, Y. 2013. Texture Brush: An interactive surface texturing interface. In Proc. Symp. on Interactive 3D Graphics and Games (I3D), ACM, 153–160. Google ScholarDigital Library
33. Talton, J. O., Lou, Y., Lesser, S., Duke, J., Měch, R., and Koltun, V. 2011. Metropolis procedural modeling. ACM Trans. Graph. 30, 2, 11:1–14. Google ScholarDigital Library
34. Talton, J., Yang, L., Kumar, R., Lim, M., Goodman, N., and Měch, R. 2012. Learning design patterns with Bayesian grammar induction. In Proc. ACM Symp. on User Interface Software and Technology (UIST), ACM, 63–74. Google ScholarDigital Library
35. Tasse, F. P., Emilien, A., Cani, M.-P., Hahmann, S., and Bernhardt, A. 2014. First person sketch-based terrain editing. In Proc. Graphics Interface, 217–224. Google ScholarDigital Library
36. Vanegas, C. A., Garcia-Dorado, I., Aliaga, D. G., Benes, B., and Waddell, P. 2012. Inverse design of urban procedural models. ACM Trans. Graph. (SIGGRAPH Asia) 31, 6, 168:1–11. Google ScholarDigital Library
37. Wither, J., Boudon, F., Cani, M.-P., and Godin, C. 2009. Structure from silhouettes: a new paradigm for fast sketch-based design of trees. Computer Graphics Forum (Eurographics) 28, 2, 541–550.Google ScholarCross Ref
38. Xing, J., Chen, H.-T., and Wei, L.-Y. 2014. Autocomplete painting repetitions. ACM Trans. Graph. 33, 6, 172:1–11. Google ScholarDigital Library
39. Yeh, Y.-T., Yang, L., Watson, M., Goodman, N. D., and Hanrahan, P. 2012. Synthesizing open worlds with constraints using locally annealed reversible jump MCMC. ACM Trans. Graph. (SIGGRAPH) 31, 4, 56:1–11. Google ScholarDigital Library
40. Zhou, H., Sun, J., Turk, G., and Rehg, J. 2007. Terrain synthesis from digital elevation models. IEEE Trans. Visualization and Computer Graphics 13, 4, 834–848. Google ScholarDigital Library