“Fluid volume modeling from sparse multi-view images by appearance transfer”

  • ©Makoto Okabe, Yoshinori Dobashi, Ken-ichi (Ken) Anjyo, and Rikio Onai

Conference:


Type:


Title:

    Fluid volume modeling from sparse multi-view images by appearance transfer

Session/Category Title: Transfer & Capture

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We propose a method of three-dimensional (3D) modeling of volumetric fluid phenomena from sparse multi-view images (e.g., only a single-view input or a pair of front- and side-view inputs). The volume determined from such sparse inputs using previous methods appears blurry and unnatural with novel views; however, our method preserves the appearance of novel viewing angles by transferring the appearance information from input images to novel viewing angles. For appearance information, we use histograms of image intensities and steerable coefficients. We formulate the volume modeling as an energy minimization problem with statistical hard constraints, which is solved using an expectation maximization (EM)-like iterative algorithm. Our algorithm begins with a rough estimate of the initial volume modeled from the input images, followed by an iterative process whereby we first render the images of the current volume with novel viewing angles. Then, we modify the rendered images by transferring the appearance information from the input images, and we thereafter model the improved volume based on the modified images. We iterate these operations until the volume converges. We demonstrate our method successfully provides natural-looking volume sequences of fluids (i.e., fire, smoke, explosions, and a water splash) from sparse multi-view videos. To create production-ready fluid animations, we further propose a method of rendering and editing fluids using a commercially available fluid simulator.

References:


    1. Atcheson, B., Ihrke, I., Heidrich, W., Tevs, A., Bradley, D., Magnor, M., and Seidel, H.-P. 2008. Time-resolved 3d capture of non-stationary gas flows. ACM Trans. Graph. 27, 5, 132:1–132:9. Google ScholarDigital Library
    2. Bhat, K. S., Seitz, S. M., Hodgins, J. K., and Khosla, P. K. 2004. Flow-based video synthesis and editing. ACM Trans. Graph. 23, 3, 360–363. Google ScholarDigital Library
    3. Bridson, R., and Müller-Fischer, M. 2007. Fluid simulation: Siggraph 2007 course notes. In ACM SIGGRAPH 2007 Courses, 1–81. Google ScholarDigital Library
    4. Debevec, P. 2001. Light probe image gallery. Available at http://www.pauldebevec.com/Probes/.Google Scholar
    5. Debevec, P. 2006. High-resolution light probe image gallery. Available at http://gl.ict.usc.edu/Data/HighResProbes/.Google Scholar
    6. Dong, Y., Lefebvre, S., Tong, X., and Drettakis, G. 2008. Lazy solid texture synthesis. In Proc. of EGSR ’08, 1165–1174. Google ScholarDigital Library
    7. Gregson, J., Krimerman, M., Hullin, M. B., and Heidrich, W. 2012. Stochastic tomography and its applications in 3d imaging of mixing fluids. ACM Trans. Graph. 31, 4, 52:1–52:10. Google ScholarDigital Library
    8. Gregson, J., Heide, F., Hullin, M. B., Rouf, M., and Heidrich, W. 2013. Stochastic deconvolution. In Proc. of CVPR 2013, 1043–1050. Google ScholarDigital Library
    9. Gregson, J., Ihrke, I., Thuerey, N., and Heidrich, W. 2014. From capture to simulation: Connecting forward and inverse problems in fluids. ACM Trans. Graph. 33, 4, 139:1–139:11. Google ScholarDigital Library
    10. Gu, J., Nayar, S., Grinspun, E., Belhumeur, P., and Ramamoorthi, R. 2013. Compressive structured light for recovering inhomogeneous participating media. IEEE Trans. Pattern Anal. Mach. Intell. 35, 3, 555–566.Google Scholar
    11. Hasinoff, S. W., and Kutulakos, K. N. 2007. Photo-consistent reconstruction of semitransparent scenes by densitysheet decomposition. IEEE Trans. Pattern Anal. Mach. Intell. 29, 5, 870–885. Google ScholarDigital Library
    12. Hawkins, T., Einarsson, P., and Debevec, P. 2005. Acquisition of time-varying participating media. ACM Trans. Graph. 24, 3, 812–815. Google ScholarDigital Library
    13. Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. In Proc. of SIGGRAPH ’95, 229–238. Google ScholarDigital Library
    14. Ihrke, I., and Magnor, M. 2004. Image-based tomographic reconstruction of flames. In Proc. of SCA ’04, 365–373. Google ScholarDigital Library
    15. Jagnow, R., Dorsey, J., and Rushmeier, H. 2004. Stereological techniques for solid textures. ACM Trans. Graph. 23, 3, 329–335. Google ScholarDigital Library
    16. Kak, A. C., and Slaney, M. 1988. Principles of computerized tomographic imaging. IEEE Press.Google Scholar
    17. Klehm, O., Ihrke, I., Seidel, H.-P., and Eisemann, E. 2013. Volume stylizer: Tomography-based volume painting. In Proc of I3D ’13, 161–168. Google ScholarDigital Library
    18. Kopf, J., Fu, C.-W., Cohen-Or, D., Deussen, O., Lischinski, D., and Wong, T.-T. 2007. Solid texture synthesis from 2d exemplars. ACM Trans. Graph. 26, 3, 2:1–2:9. Google ScholarDigital Library
    19. Kwatra, V., Essa, I., Bobick, A., and Kwatra, N. 2005. Texture optimization for example-based synthesis. ACM Trans. Graph. 24, 3, 795–802. Google ScholarDigital Library
    20. Li, C., Pickup, D., Saunders, T., Cosker, D., Marshall, D., Hall, P., and Willis, P. 2013. Water surface modeling from a single viewpoint video. IEEE Trans. Vis. Comput. Graphics 19, 7, 1242–1251. Google ScholarDigital Library
    21. Liu, Z., Hu, Y., and Qi, Y. 2011. Modeling of smoke from a single view. In Proc. of ICVRV ’11, 291–294. Google ScholarDigital Library
    22. Ma, C., Wei, L.-Y., Lefebvre, S., and Tong, X. 2013. Dynamic element textures. ACM Trans. Graph. 32, 4, 90:1–90:10. Google ScholarDigital Library
    23. Portilla, J., and Simoncelli, E. P. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vis. 40, 1, 49–70. Google ScholarDigital Library
    24. Qin, X., and Yang, Y.-H. 2007. Aura 3d textures. IEEE Trans. Vis. Comput. Graph. 13, 2, 379–389. Google ScholarDigital Library
    25. Reche-Martinez, A., Martin, I., and Drettakis, G. 2004. Volumetric reconstruction and interactive rendering of trees from photographs. ACM Trans. Graph. 23, 3, 720–727. Google ScholarDigital Library
    26. Takayama, K., Okabe, M., Ijiri, T., and Igarashi, T. 2008. Lapped solid textures: Filling a model with anisotropic textures. ACM Trans. Graph. 27, 3, 53:1–53:9. Google ScholarDigital Library
    27. Thürey, N., Keiser, R., Pauly, M., and Rüde, U. 2006. Detail-preserving fluid control. In Proc. of SCA ’06, 7–12. Google ScholarDigital Library
    28. Wei, L.-Y. 2002. Texture Synthesis by Fixed Neighborhood Searching. PhD thesis. Google ScholarDigital Library
    29. Wenger, S., Lorenz, D., and Magnor, M. 2013. Fast image-based modeling of astronomical nebulae. Comput. Graph. Forum 32, 7, 93–100.Google ScholarCross Ref
    30. Werlberger, M., Pock, T., and Bischof, H. 2010. Motion estimation with non-local total variation regularization. In Proc. of CVPR 2010, 2464–2471.Google Scholar
    31. Wexler, Y., Shechtman, E., and Irani, M. 2007. Space-time completion of video. IEEE Trans. Pattern Anal. Mach. Intell. 29, 3, 463–476. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: