“SMASH: physics-guided reconstruction of collisions from videos”
Conference:
Type(s):
Title:
- SMASH: physics-guided reconstruction of collisions from videos
Session/Category Title: Smash & Splash
Presenter(s)/Author(s):
Abstract:
Collision sequences are commonly used in games and entertainment to add drama and excitement. Authoring even two body collisions in the real world can be difficult, as one has to get timing and the object trajectories to be correctly synchronized. After tedious trial-and-error iterations, when objects can actually be made to collide, then they are difficult to capture in 3D. In contrast, synthetically generating plausible collisions is difficult as it requires adjusting different collision parameters (e.g., object mass ratio, coefficient of restitution, etc.) and appropriate initial parameters. We present SMASH to directly read off appropriate collision parameters directly from raw input video recordings. Technically we enable this by utilizing laws of rigid body collision to regularize the problem of lifting 2D trajectories to a physically valid 3D reconstruction of the collision. The reconstructed sequences can then be modified and combined to easily author novel and plausible collisions. We evaluate our system on a range of synthetic scenes and demonstrate the effectiveness of our method by accurately reconstructing several complex real world collision events.
References:
1. Agarwal, S., Mierle, K., et al., 2016. Ceres solver. http://ceres-solver.org/.
2. Anand, A., Koppula, H. S., Joachims, T., and Saxena, A. 2013. Contextually guided semantic labeling and search for three-dimensional point clouds. IJRR 32, 1, 19–34.
3. Armstrong, W., and Green, M. 1985. The dynamics of articulated rigid bodies for purposes of animation. The visual computer 1, 4, 231–240.
4. Baraff, D. 1990. Curved surfaces and coherence for nonpenetrating rigid body simulation. ACM SIGGRAPH 24, 4, 19–28.
5. Bongard, J., and Lipson, H. 2007. Automated reverse engineering of nonlinear dynamical systems. Proceedings of the National Academy of Sciences 104, 24, 9943–9948. Cross Ref
6. Bradski, G., et al., 2016. Open source computer vision library. http://opencv.org.
7. Chenney, S., and Forsyth, D. A. 2000. Sampling plausible solutions to multi-body constraint problems. ACM SIGGRAPH.
8. Conotter, V., O’Brien, J. F., and Farid, H. 2012. Exposing digital forgeries in ballistic motion. IEEE Transactions on Information Forensics and Security 7, 1 (Feb.), 283 — 296.
9. Davis, T. A. 2005. Algorithm 849: A concise sparse cholesky factorization package. ACM Trans. Math. Softw. 31, 4, 587–591.
10. Dou, M., Taylor, J., Fuchs, H., Fitzgibbon, A., and Izadi, S. 2015. 3D scanning deformable objects with a single RGBD sensor. IEEE CVPR, 493–501.
11. Eberly, D. H. 2010. Game physics. Taylor and Francis.
12. Gilardi, G., and Sharf, I. 2002. Literature survey of contact dynamics modelling. Mechanism and machine theory 37, 10.
13. Gregson, J., Thuerey, N., Ihrke, I., and Heidrich, W. 2014. From Capture to Simulation – Connecting Forward and Inverse Problems in Fluids. ACM SIGGRAPH 33 (4) (August), 10.
14. Grzeszczuk, R., Terzopoulos, D., and Hinton, G. 1998. Neuroanimator: Fast neural network emulation and control of physics-based models. ACM SIGGRAPH, 9–20.
15. Guennebaud, G., Jacob, B., et al., 2010. Eigen v3. http://eigen.tuxfamily.org.
16. Gupta, A., Efros, A. A., and Hebert, M. 2010. Blocks world revisited: Image understanding using qualitative geometry and mechanics. ECCV.
17. Hartley, E., Kermgard, B., Fried, D., Bowdish, J., Pero, L. D., and Barnard, K. 2012. Bayesian geometric modeling of indoor scenes. IEEE CVPR, 2719–2726.
18. Huber, P. J. 1964. Robust estimation of a location parameter. Ann. Math. Statist. 35, 1, 73–101. Cross Ref
19. Jia, Z., Gallagher, A., Saxena, A., and Chen, T. 2013. 3D-based reasoning with blocks, support, and stability. IEEE CVPR.
20. Jiang, H., and Xiao, J. 2013. A linear approach to matching cuboids in RGBD images. IEEE CVPR.
21. Kim, Y. M., Mitra, N. J., Yan, D.-M., and Guibas, L. 2012. Acquiring 3d indoor environments with variability and repetition. ACM SIGGRAPH Asia 31, 6, 138:1–138:11.
22. Kleppner, D., and Kolenkow, R. 2013. Introduction to Mechanics, 2nd Ed. Cambridge University Press.
23. Koppula, H., Anand, A., Joachims, T., and Saxena, A. 2011. Semantic labeling of 3D point clouds for indoor scenes. NIPS.
24. Lafarge, F., and Alliez, P. 2013. Surface reconstruction through point set structuring. CGF.
25. Lee, D. C., Gupta, A., Hebert, M., and Kanade, T. 2010. Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces. NIPS 24.
26. Liu, Z., Gortler, S. J., and Cohen, M. F. 1994. Hierarchical spacetime control. ACM SIGGRAPH, 35–42.
27. Loper, M. M., and Black, M. J. 2014. OpenDR: An approximate differentiable renderer. ECCV, 154–169.
28. Martin, S., Thomaszewski, B., Grinspun, E., and Gross, M. 2011. Example-based elastic materials. ACM TOG 30, 4, 72.
29. Mattausch, O., Panozzo, D., Mura, C., Sorkine-Hornung, O., and Pajarola, R. 2014. Object detection and classification from large-scale cluttered indoor scans. CGF EG.
30. Mitra, N. J., Flory, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. SGP, 173–182.
31. Monszpart, A., Mellado, N., Brostow, G., and Mitra, N. 2015. RAPter: Rebuilding man-made scenes with regular arrangements of planes. ACM SIGGRAPH.
32. Müller, M., McMillan, L., Dorsey, J., and Jagnow, R. 2001. Real-time simulation of deformation and fracture of stiff materials. Eurogr. Worksh. on Comp. Anim. and Sim., 113–124.
33. Nan, L., Xie, K., and Sharf, A. 2012. A search-classify approach for cluttered indoor scene understanding. ACM SIGGRAPH Asia 31, 6, 137:1–137:10.
34. Newcombe, R. A., Fox, D., and Seitz, S. M. 2015. Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. IEEE CVPR.
35. Popovič, J., Seitz, S. M., Erdmann, M., Popović, Z., and Witkin, A. 2000. Interactive manipulation of rigid body simulations. ACM SIGGRAPH, 209–217.
36. Salzmann, M., and Urtasun, R. 2011. Physically-based motion models for 3D tracking: A convex formulation. IEEE ICCV, 2064–2071.
37. Schlecht, J., and Barnard, K. 2009. Learning models of object structure. NIPS.
38. Schroeder, W., Martin, K., and Lorensen, B. 2006. The visualization toolkit (4th ed.).
39. Shao, T., Xu, W., Zhou, K., Wang, J., Li, D., and Guo, B. 2012. An interactive approach to semantic modeling of indoor scenes with an RGBD camera. ACM SIGGRAPH Asia 31, 6.
40. Shao*, T., Monszpart*, A., Zheng, Y., Koo, B., Xu, W., Zhou, K., and Mitra, N. 2014. Imagining the unseen: Stability-based cuboid arrangements for scene understanding. ACM SIGGRAPH Asia. * Joint first authors.
41. Silberman, N., Hoiem, D., Kohli, P., and Fergus, R. 2012. Indoor segmentation and support inference from RGBD images. ECCV.
42. Smith, B., Kaufman, D. M., Vouga, E., Tamstorf, R., and Grinspun, E. 2012. Reflections on simultaneous impact. ACM TOG 31, 4, 106:1–106:12.
43. Su, J., Schroeder, C., and Fedkiw, R. 2009. Energy stability and fracture for frame rate rigid body simulations. SCA, 155–164.
44. Tang, D., Ngo, J. T., and Marks, J. 1995. N-body spacetime constraints. JVCA 6, 3, 143–154. Cross Ref
45. Teh, C.-H., and Chin, R. T. 1989. On the detection of dominant points on digital curves. IEEE PAMI 11, 8, 859–872.
46. Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. ACM SIGGRAPH 21, 4.
47. Twigg, C. D., and James, D. L. 2007. Many-worlds browsing for control of multibody dynamics. ACM TOG 26, 3, 14.
48. Wang, H., Liao, M., Zhang, Q., Yang, R., and Turk, G. 2009. Physically Guided Liquid Surface Modeling from Videos. ACM SIGGRAPH 28, 3, article 90.
49. Witkin, A., and Kass, M. 1988. Spacetime constraints. ACM SIGGRAPH 22, 4, 159–168.
50. Xiong, X., and Huber, D. 2010. Using context to create semantic 3d models of indoor environments. BMVC, 1–11.
51. Zheng, B., Zhao, Y., Yu, J. C., Ikeuchi, K., and Zhu, S.-C. 2013. Beyond point clouds: Scene understanding by reasoning geometry and physics. IEEE CVPR.
52. Zivkovic, Z. 2004. Improved adaptive gaussian mixture model for background subtraction. IEEE ICPR 2, 28–31.
