“Physically guided liquid surface modeling from videos” by Wang, Liao, Zhang, Yang and Turk

We present an image-based reconstruction framework to model real water scenes captured by stereoscopic video. In contrast to many image-based modeling techniques that rely on user interaction to obtain high-quality 3D models, we instead apply automatically calculated physically-based constraints to refine the initial model. The combination of image-based reconstruction with physically-based simulation allows us to model complex and dynamic objects such as fluid. Using a depth map sequence as initial conditions, we use a physically based approach that automatically fills in missing regions, removes outliers, and refines the geometric shape so that the final 3D model is consistent to both the input video data and the laws of physics. Physically-guided modeling also makes interpolation or extrapolation in the space-time domain possible, and even allows the fusion of depth maps that were taken at different times or viewpoints. We demonstrated the effectiveness of our framework with a number of real scenes, all captured using only a single pair of cameras.

1. Atcheson, B., Ihrke, I., Heidrich, W., Tevs, A., Bradley, D., Magnor, M., and Seidel, H.-P. 2008. Time-resolved 3d capture of non-stationary gas flows. In Proc. of ACM SIGGRAPH Asia 2008, vol. 27. Google ScholarDigital Library
2. Bhat, K. S., Seitz, S. M., Hodgins, J. K., and Khosla, P. K. 2004. Flow-based video synthesis and editing. In Proc. of ACM SIGGRAPH 2004, 360–363. Google ScholarDigital Library
3. Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. In Proc. of ACM SIGGRAPH 2008, vol. 27. Google ScholarDigital Library
4. Carlson, M., Mucha, P. J., Van Horn III, R. B., and Turk, G. 2002. Melting and flowing. In Proc. of SCA ’02, 167–174. Google ScholarDigital Library
5. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. In Proc. of ACM SIGGRAPH ’08, 1–10. Google ScholarDigital Library
6. Egnal, G., and Wildes, R. P. 2002. Detecting binocular half-occlusions: Empirical comparisons of five approaches. IEEE Trans. Pattern Anal. Mach. Intell. 24, 8, 1127–1133. Google ScholarDigital Library
7. Enright, D., Marschner, S., and Fedkiw, R. 2002. Animation and rendering of complex water surfaces. In Proc. of ACM SIGGRAPH ’02, 736–744. Google ScholarDigital Library
8. Fattal, R., and Lischinski, D. 2004. Target-driven smoke animation. In Proc. of ACM SIGGRAPH 2004. Google ScholarDigital Library
9. Foster, N., and Fedkiw, R. 2001. Practical animation of liquids. In Proc. of SIGGRAPH ’01, 23–30. Google ScholarDigital Library
10. Foster, N., and Metaxas, D. 1996. Realistic animation of liquids. Graph. Models Image Process. 58, 5. Google ScholarDigital Library
11. Grant, I. 1997. Particle image velocimetry: a review. In Proc. of the Institution of Mechanical Engineers, vol. 211, 55C76.Google Scholar
12. Hawkins, T., Einarsson, P., and Debevec, P. 2005. Acquisition of time-varying participating media. In Proc. of ACM SIGGRAPH 2005. Google ScholarDigital Library
13. Hullin, M. B., Fuchs, M., Ihrke, I., Seidel, H.-P., and Lensch, H. P. A. 2008. Fluorescent immersion range scanning. In Proc. of ACM SIGGRAPH 2008. Google ScholarDigital Library
14. Ihrke, I., Goldluecke, B., and Magnor, M. 2005. Reconstructing the geometry of flowing water. In ICCV ’05, IEEE Computer Society, Washington, DC, USA, 1055–1060. Google ScholarDigital Library
15. Kanade, T., Rander, P., Vedula, S., and Saito, H. 1999. Virtualized reality: Digitizing a 3d time-varying event as is and in real time. In Mixed Reality, Merging Real and Virtual Worlds. 41–57.Google Scholar
16. McNamara, A., Treuille, A., Popovic, Z., and Stam, J. 2004. Fluid control using the adjoint method. In Proc. of ACM SIGGRAPH 2004. Google ScholarDigital Library
17. Mitra, N. J., Flory, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Eurographics Symposium on Geometry Processing. Google ScholarDigital Library
18. Morris, N. J. W., and Kutulakos, K. N. 2005. Dynamic refraction stereo. In Proc. of International Conference on Computer Vision. Google ScholarDigital Library
19. Morris, N. J. W., and Kutulakos, K. N. 2007. Reconstructing the surface of inhomogeneous transparent scenes by scatter trace photography. In Proc. of 11th Int. Conf. Computer Vision.Google Scholar
20. Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., and Kang, S. B. 2006. Image-based plant modeling. In Proc. of ACM SIGGRAPH 2006. Google ScholarDigital Library
21. Schneider, R., and Kobbelt, L. 2001. Geometric fairing of irregular meshes for freeform surface design. Computer aided geometric design 18, 359–379. Google ScholarDigital Library
22. Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., and Sheffer, A. 2008. Space-time surface reconstruction using incompressible flow. In Proc. of ACM SIGGRAPH Asia 2008, vol. 27, 1–10. Google ScholarDigital Library
23. Shi, J., and Tomasi, C. 1994. Good features to track. In Proc. of CVPR 1994, 593–600.Google Scholar
24. Simon, S. V., Baker, S., Seitz, S., and Kanade, T. 2000. Shape and motion carving in 6d. In Computer Vision and Pattern Recognition.Google Scholar
25. Sinha, S. N., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. 2008. Interactive 3d architectural modeling from unordered photo collections. Proc. of SIGGRAPH Asia 2008 27. Google ScholarDigital Library
26. Stam, J. 1999. Stable fluids. In Proc. of ACM SIGGRAPH ’99, 121–128. Google ScholarDigital Library
27. Staniforth, A., and Côté, J. 1991. Semi-lagrangian integration schemes for atmospheric models. Monthly Weather Review 119, 9, 2206.Google ScholarCross Ref
28. Sun, J., Zheng, N.-N., and Shum, H.-Y. 2003. Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25, 7, 787. Google ScholarDigital Library
29. Tan, P., Zeng, G., Wang, J., Kang, S. B., and Quan, L. 2007. Image-based tree modeling. In Proc. of ACM SIGGRAPH 2007. Google ScholarDigital Library
30. Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Eurographics Symposium on Geometry Processing. Google ScholarDigital Library
31. Wei, Y., Ofek, E., Quan, L., and Shum, H.-Y. 2005. Modeling hair from multiple views. In Proc. of ACM SIGGRAPH 2005. Google ScholarDigital Library
32. White, R., Crane, K., and Forsyth, D. 2007. Capturing and animating occluded cloth. In Proc. of ACM SIGGRAPH 2007. Google ScholarDigital Library
33. Xiao, J., Fang, T., Tan, P., Zhao, P., and Quan, L. 2008. Image-based facade modeling. Proc. of SIGGRAPH Asia 2008 27. Google ScholarDigital Library
34. Yang, Q., Yang, R., Davis, J., and Nister, D. 2007. Spatial-depth super resolution for range images. In Proc. of CVPR 2007, vol. 0, 1–8.Google Scholar
35. Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. In Proc. of ACM SIGGRAPH ’04, 600–608. Google ScholarDigital Library
36. Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM Transactions on Graphics, 23, 3, 600–608. Google ScholarDigital Library