“Photo-inspired model-driven 3D object modeling” by Xu, Zheng, Zhang, Cohen-Or, Liu, et al. …

We introduce an algorithm for 3D object modeling where the user draws creative inspiration from an object captured in a single photograph. Our method leverages the rich source of photographs for creative 3D modeling. However, with only a photo as a guide, creating a 3D model from scratch is a daunting task. We support the modeling process by utilizing an available set of 3D candidate models. Specifically, the user creates a digital 3D model as a geometric variation from a 3D candidate. Our modeling technique consists of two major steps. The first step is a user-guided image-space object segmentation to reveal the structure of the photographed object. The core step is the second one, in which a 3D candidate is automatically deformed to fit the photographed target under the guidance of silhouette correspondence. The set of candidate models have been pre-analyzed to possess useful high-level structural information, which is heavily utilized in both steps to compensate for the ill-posedness of the analysis and modeling problems based only on content in a single image. Equally important, the structural information is preserved by the geometric variation so that the final product is coherent with its inherited structural information readily usable for subsequent model refinement or processing.

1. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. on Graphics (Proc. SIGGRAPH) 22, 3, 587–594. Google ScholarDigital Library
2. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: shape completion and animation of people. ACM Trans. on Graphics (Proc. SIGGRAPH) 24, 3, 408–416. Google ScholarDigital Library
3. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH, 187–194. Google Scholar
4. Chaudhuri, S., and Koltun, V. 2010. Data-driven suggestions for creativity support in 3D modeling. ACM Trans. on Graphics (Proc. SIGGRAPH ASIA) 29, 6, 183:1–9. Google Scholar
5. Delong, A., Osokin, A., Isack, H., and Boykov, Y. 2010. Fast approximate energy minimization with label costs. In Proc. IEEE Conf. on CVPR, 2173–2180.Google Scholar
6. Duchenne, O., Bach, F., Kweon, I., and Ponce, J. 2009. A tensor-based algorithm for high-order graph matching. In Proc. IEEE Conf. on CVPR, 1980–1987.Google Scholar
7. Funkhouser, T., Kazhdan, M., Shilane, P., Min, P., Kiefer, W., Tal, A., Rusinkiewicz, S., and Dobkin, D. 2004. Modeling by example. ACM Trans. on Graphics (Proc. SIGGRAPH) 23, 3, 652–663. Google ScholarDigital Library
8. Gal, R., Sorkine, O., Mitra, N. J., and Cohen-Or, D. 2009. iWIRES: an analyze-and-edit approach to shape manipulation. ACM Trans. on Graphics (Proc. SIGGRAPH) 28, 3, 33:1–10. Google ScholarDigital Library
9. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: A sketching interface for 3D freeform design. In Proc. SIGGRAPH, 409–416. Google Scholar
10. Karpenko, O. A., and Hughes, J. F. 2006. SmoothSketch: 3D free-form shapes from complex sketches. ACM Trans. on Graphics (Proc. SIGGRAPH) 25, 3, 589–598. Google ScholarDigital Library
11. Kavan, L., Collins, S., Žára, J., and O’Sullivan, C. 2007. Skinning with dual quaternions. In Proc. of Symposium on Interactive 3D Graphics and Games, 39–46. Google Scholar
12. Kraevoy, V., Julius, D., and Sheffer, A. 2007. Model composition from interchangeable components. In Proc. Pacific Graphics, 129–138. Google Scholar
13. Kraevoy, V., van de Panne, M., and Sheffer, A. 2009. Modeling from contour drawings. In Proc. EUROGRAPHICS Symposium on Sketch-Based Interfaces and Modeling, 37–44. Google Scholar
14. Lau, M., Saul, G., Mitani, J., and Igarashi, T. 2010. Modeling-in-context: User design of complementary objects with a single photo. In Proc. EUROGRAPHICS Symposium on Sketch-Based Interfaces and Modeling, 17–24. Google ScholarDigital Library
15. Lee, J., and Funkhouser, T. 2008. Sketch-based search and composition of 3D models. In Proc. of EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling. Google Scholar
16. Müller, M., Heidelberger, B., Teschner, M., and Gross, M. 2005. Meshless deformations based on shape matching. ACM Trans. on Graphics (Proc. SIGGRAPH) 24, 3, 471–478. Google ScholarDigital Library
17. Nealen, A., Sorkine, O., Alexa, M., and Cohen-Or, D. 2005. A sketch-based interface for detail-preserving mesh editing. ACM Trans. on Graphics (Proc. SIGGRAPH) 24, 3, 1142–1147. Google ScholarDigital Library
18. Rivers, A., Durand, F., and Igarashi., T. 2010. 3D modeling with silhouettes. ACM Trans. on Graphics (Proc. SIGGRAPH) 29, 3, 109:1–8. Google ScholarDigital Library
19. Rother, C., Kolmogorov, V., and Blake, A. 2004. GrabCut: Interactive foreground extraction using iterated graph cuts. ACM Trans. on Graphics (Proc. SIGGRAPH) 23, 3, 309–314. Google ScholarDigital Library
20. Shimshoni, I., Moses, Y., and Lindenbaum, M. 2000. Shape reconstruction of 3D bilaterally symmetric surfaces. Int. J. of Comp. Vis. (IJCV) 39, 2, 97–110. Google ScholarCross Ref
21. Shin, H., and Igarashi, T. 2007. Magic canvas: interactive design of a 3D scene prototype from freehand sketches. In Proc. of Graphics Interface, 63–70. Google Scholar
22. Sorkine, O., and Botsch, M. 2009. Interactive shape modeling and deformation. In Proc. Eurographics Tutorial.Google Scholar
23. Talton, J. O., Gibson, D., Yang, L., Hanrahan, P., and Koltun, V. 2009. Exploratory modeling with collaborative design spaces. ACM Trans. on Graphics (Proc. SIGGRAPH ASIA) 28, 5, 167:1–10. Google Scholar
24. Tan, G., Chen, W., and Liu, L. 2010. Image driven shape deformation using styles. Journal of Zhejiang University (SCIENCE C) 11, 1, 27–35.Google ScholarCross Ref
25. Xi, P., Lee, W.-S., and Shu, C. 2007. A data-driven approach to human-body cloning using a segmented body database. In Proc. Pacific Graphics, 139–147. Google Scholar
26. Xu, K., Li, H., Zhang, H., Cohen-Or, D., Xiong, Y., and Cheng, Z. 2010. Style-content separation by anisotropic part scales. ACM Trans. on Graphics (Proc. SIGGRAPH ASIA) 29, 6, 184:1–9. Google Scholar
27. Zhang, D., and Lu, G. 2002. An integrated approach to shape based image retrieval. In Proc. Asian Conference on Computer Vision, 652–657.Google Scholar
28. Zheng, Y., Fu, H., Cohen-Or, D., Au, O. K.-C., and Tai, C.-L. 2011. Component-wise controllers for structure-preserving shape manipulation. Computer Graphics Forum (Proc. EUROGRAPHICS) 30, 2, 563–572.Google ScholarCross Ref