“Interactive mechanism modeling from multi-view images” by Xu, Li, Xu, Deng, Yang, et al. …
Conference:
Type(s):
Title:
- Interactive mechanism modeling from multi-view images
Session/Category Title: Data Driven Modeling
Presenter(s)/Author(s):
Abstract:
In this paper, we present an interactive system for mechanism modeling from multi-view images. Its key feature is that the generated 3D mechanism models contain not only geometric shapes but also internal motion structures: they can be directly animated through kinematic simulation. Our system consists of two steps: interactive 3D modeling and stochastic motion parameter estimation. At the 3D modeling step, our system is designed to integrate the sparse 3D points reconstructed from multi-view images and a sketching interface to achieve accurate 3D modeling of a mechanism. To recover the motion parameters, we record a video clip of the mechanism motion and adopt stochastic optimization to recover its motion parameters by edge matching. Experimental results show that our system can achieve the 3D modeling of a range of mechanisms from simple mechanical toys to complex mechanism objects.
References:
1. Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., and Szeliski, R. 2011. Building rome in a day. Commun. ACM 54, 10 (Oct.), 105–112.
2. Ahmadabadian, A. H., Robson, S., Boehm, J., Shortis, M., Wenzel, K., and Fritsch, D. 2013. A comparison of dense matching algorithms for scaled surface reconstruction using stereo camera rigs. ISPRS Journal of Photogrammetry and Remote Sensing 78, 0, 157 — 167. Cross Ref
3. Bächer, M., Bickel, B., James, D. L., and Pfister, H. 2012. Fabricating articulated characters from skinned meshes. ACM Trans. Graph. 31, 4 (July), 47:1–47:9.
4. Bey, A., Chaine, R., and Raphael, M. 2011. Reconstruction of consistent 3d cad models from point cloud data using a priori cad models. In Proceedings of ISPRS, vol. 12.
5. Canny, J. 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 6, 679–698.
6. Cao, Y.-P., Ju, T., Fu, Z., and Hu, S.-M. 2014. Interactive image-guided modeling of extruded shapes. Comput. Graph. Forum 33, 7, 101–110.
7. Ceylan, D., Li, W., Mitra, N. J., Agrawala, M., and Pauly, M. 2013. Designing and fabricating mechanical automata from mocap sequences. ACM Trans. Graph. 32, 6, 186.
8. Chen, X., Kang, S. B., Xu, Y.-Q., Dorsey, J., and Shum, H.-Y. 2008. Sketching reality: Realistic interpretation of architectural designs. ACM Trans. Graph. 27, 2 (May), 11:1–11:15.
9. Chen, T., Zhu, Z., Shamir, A., Hu, S.-M., and Cohen-Or, D. 2013. 3-sweep: extracting editable objects from a single photo. ACM Trans. Graph. 32, 6, 195.
10. Chin, R. T., and Dyer, C. R. 1986. Model-based recognition in robot vision. ACM Comput. Surv. 18, 1, 67–108.
11. Coros, S., Thomaszewski, B., Noris, G., Sueda, S., Forberg, M., Sumner, R. W., Matusik, W., and Bickel, B. 2013. Computational design of mechanical characters. ACM Trans. Graph. 32, 4 (July), 83:1–83:12.
12. Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM Trans. Graph. 31, 4 (July), 44:1–44:10.
13. Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., and Alexa, M. 2012. Sketch-based shape retrieval. ACM Trans. Graph. 31, 4 (July), 31:1–31:10.
14. Furukawa, Y., and Ponce, J. 2007. Accurate, dense, and robust multi-view stereopsis. In CVPR ’07, 1–8.
15. Gal, R., Sorkine, O., Mitra, N. J., and Cohen-Or, D. 2009. iwires: An analyze-and-edit approach to shape manipulation. ACM Trans. Graph. 28, 3 (July), 33:1–33:10.
16. Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision, second ed. Cambridge University Press, ISBN: 0521540518.
17. Huang, J., and Menq, C.-H. 2002. Automatic cad model reconstruction from multiple point clouds for reverse engineering. J. Comput. Inf. Sci. Eng. 2, 3, 160–170. Cross Ref
18. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: A sketching interface for 3d freeform design. In Proc. of SIGGRAPH’99, 409–416.
19. Jacquet, B., Angst, R., and Pollefeys, M. 2013. Articulated and restricted motion subspaces and their signatures. In CVPR, IEEE Computer Society, 1506–1513.
20. Jorge, N., and Stephen, W. 2006. Numerical Optimization. Springer-Verlag New York.
21. Kalogerakis, E., Chaudhuri, S., Koller, D., and Koltun, V. 2012. A probabilistic model for component-based shape synthesis. ACM Trans. Graph. 31, 4 (July), 55:1–55:11.
22. Kraevoy, V., Sheffer, A., Shamir, A., and Cohen-Or, D. 2008. Non-homogeneous resizing of complex models. ACM Trans. Graph. 27, 5 (Dec.), 111:1–111:9.
23. Kragic, D., and Christensen, H. I. 2003. Confluence of parameters in model based tracking. In ICRA, IEEE, 3485–3490.
24. Li, W., Agrawala, M., Curless, B., and Salesin, D. 2008. Automated generation of interactive 3d exploded view diagrams. ACM Trans. Graph. 27, 3.
25. Li, Y., Wu, X., Chrysathou, Y., Sharf, A., Cohen-Or, D., and Mitra, N. J. 2011. Globfit: Consistently fitting primitives by discovering global relations. ACM Trans. Graph. 30, 4 (July), 52:1–52:12.
26. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91–110.
27. Mitra, N., Wand, M., Zhang, H. R., Cohen-Or, D., Kim, V., and Huang, Q.-X. 2013. Structure-aware shape processing. In SIGGRAPH Asia 2013 Courses, SA ’13, 1:1–1:20.
28. Mitra, N. J., Yang, Y.-L., Yan, D.-M., Li, W., and Agrawala, M. 2013. Illustrating how mechanical assemblies work. Commun. ACM 56, 1 (Jan.), 106–114.
29. Olsen, L., Samavati, F. F., Sousa, M. C., and Jorge, J. A. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1, 85–103.
30. Owada, S., Nielsen, F., Okabe, M., and Igarashi, T. 2004. Volumetric illustration: Designing 3d models with internal textures. ACM Trans. Graph. 23, 3 (Aug.), 322–328.
31. Owada, S., Nielsen, F., Nakazawa, K., and Igarashi, T. 2007. A sketching interface for modeling the internal structures of 3d shapes. In ACM SIGGRAPH 2007 Courses, SIGGRAPH ’07.
32. Schmidt, R., Wyvill, B., Sousa, M. C., and Jorge, J. A. 2007. Shapeshop: Sketch-based solid modeling with blobtrees. In ACM SIGGRAPH 2007 Courses, SIGGRAPH ’07.
33. Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR’06, vol. 1, 519–528.
34. Shao, T., Li, W., Zhou, K., Xu, W., Guo, B., and Mitra, N. J. 2013. Interpreting concept sketches. ACM Trans. Graph. 32, 4, 56:1–56:10.
35. Shen, C.-H., Fu, H., Chen, K., and Hu, S.-M. 2012. Structure recovery by part assembly. ACM Trans. Graph. 31, 6 (Nov.), 180:1–180:11.
36. Sinha, S. N., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. 2008. Interactive 3d architectural modeling from unordered photo collections. ACM Trans. Graph. 27, 5 (Dec.), 159:1–159:10.
37. Slater, N., 2011. Mechanisms and mechanical devices source-book. McGraw-Hill.
38. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3d. ACM Trans. Graph. 25, 3, 835–846.
39. Song, L., Wu, W., Guo, J., and Li, X. 2013. Survey on camera calibration technique. In Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2013 5th International Conference on, vol. 2, 389–392.
40. Sun, D., Roth, S., and Black, M. J. 2010. Secrets of optical flow estimation and their principles. In IEEE CVPR, 2432–2439.
41. Thomaszewski, B., Coros, S., Gauge, D., Megaro, V., Grinspun, E., and Gross, M. 2014. Computational design of linkage-based characters. ACM Trans. Graph. 33, 4 (July), 64:1–64:9.
42. Valgaerts, L., Bruhn, A., Mainberger, M., and Weickert, J. 2012. Dense versus sparse approaches for estimating the fundamental matrix. International Journal of Computer Vision 96, 2, 212–234.
43. Wang, Y., Xu, K., Li, J., Zhang, H., Shamir, A., Liu, L., Cheng, Z., and Xiong, Y. 2011. Symmetry hierarchy of man-made objects. Computer Graphics Forum 30, 2, 287–296. Cross Ref
44. Wikipedia, 2015. Involute. http://en.wikipedia.org/wiki/Involute/.
45. Wu, C., 2015. Visualsfm: A visual structure from motion system. http://ccwu.me/vsfm/.
46. Xu, W., Wang, J., Yin, K., Zhou, K., van de Panne, M., Chen, F., and Guo, B. 2009. Joint-aware manipulation of deformable models. ACM Trans. Graph. 28, 3.
47. Xu, K., Zheng, H., Zhang, H., Cohen-Or, D., Liu, L., and Xiong, Y. 2011. Photo-inspired model-driven 3d object modeling. ACM Trans. Graph. 30, 4 (July), 80:1–80:10.
48. Xu, K., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Fit and diverse: set evolution for inspiring 3d shape galleries. ACM Trans. Graph. 31, 4, 57.
49. Xu, B., Chang, W., Sheffer, A., Bousseau, A., McCrae, J., and Singh, K. 2014. True2form: 3d curve networks from 2d sketches via selective regularization. ACM Trans. Graph. 33, 4 (July), 131:1–131:13.
50. Ycer, K., Wang, O., Sorkine-Hornung, A., and Sorkine-Hornung, O. 2015. Reconstruction of articulated objects from a moving camera. In IEEE ICCV, 823–831.
51. Yin, K., Huang, H., Zhang, H., Gong, M., Cohen-Or, D., and Chen, B. 2014. Morfit: Interactive surface reconstruction from incomplete point clouds with curve-driven topology and geometry control. ACM Transactions on Graphics (Special Issue of SIGGRAPH Asia) 33, 6, 202:1–202:13.
52. Zhang, Z. 2000. A flexible new technique for camera calibration. IEEE TPAMI 22, 11, 1330–1334.
53. Zheng, Y., Fu, H., Cohen-Or, D., Au, O. K.-C., and Tai, C.-L. 2011. Component-wise controllers for structure-preserving shape manipulation. Comput. Graph. Forum 30, 2, 563–572. Cross Ref
54. Zheng, Y., Chen, X., Cheng, M.-M., Zhou, K., Hu, S.-M., and Mitra, N. J. 2012. Interactive images: Cuboid proxies for smart image manipulation. ACM Trans. Graph. 31, 4 (July), 99:1–99:11.
55. Zhu, L., Xu, W., Snyder, J., Liu, Y., Wang, G., and Guo, B. 2012. Motion-guided mechanical toy modeling. ACM Trans. Graph. 31, 6 (Nov.), 127:1–127:10.
