“Image-based reconstruction of wire art” by Liu, Ceylan, Lin, Wang and Mitra
Conference:
Type:
Title:
- Image-based reconstruction of wire art
Session/Category Title: Fabricating Curves, Surfaces & Volumes
Presenter(s)/Author(s):
Abstract:
Objects created by connecting and bending wires are common in furniture design, metal sculpting, wire jewelry, etc. Reconstructing such objects with traditional depth and image based methods is extremely difficult due to their unique characteristics such as lack of features, thin elements, and severe self-occlusions. We present a novel image-based method that reconstructs a set of continuous 3D wires used to create such an object, where each wire is composed of an ordered set of 3D curve segments. Our method exploits two main observations: simplicity – wire objects are often created using only a small number of wires, and smoothness – each wire is primarily smoothly bent with sharp features appearing only at joints or isolated points. In light of these observations, we tackle the challenging image correspondence problem across featureless wires by first generating multiple candidate 3D curve segments and then solving a global selection problem that balances between image and smoothness cues to identify the correct 3D curves. Next, we recover a decomposition of such curves into a set of distinct and continuous wires by formulating a multiple traveling salesman problem, which finds smooth paths, i.e., wires, connecting the curves. We demonstrate our method on a wide set of real examples with varying complexity and present high-fidelity results using only 3 images for each object. We provide the source code and data for our work in the project website.
References:
1. Caroline Baillard, Cordelia Schmid, Andrew Zisserman, and Andrew Fitzgibbon. 1999. Automatic Line Matching and 3D Reconstruction of Buildings from Multiple Views. In ISPRS Conf. on Automatic Extraction of GIS Objects from Digital Imagery. 69–80.Google Scholar
2. Matthew Berger, Andrea Tagliasacchi, Lee M. Seversky, Pierre Alliez, Joshua A. Levine, Andrei Sharf, and Claudio T. Silva. 2014. State of the Art in Surface Reconstruction from Point Clouds. In CGF Eurographics – State of the Art Reports. Google ScholarCross Ref
3. Rikard Berthilsson, Kalle Astrom, and Anders Heyden. 2001. Reconstruction of General Curves, Using Factorization and Bundle Adjustment. IJCV 41, 3 (Feb. 2001), 171–182. Google ScholarDigital Library
4. Tao Chen, Zhe Zhu, Ariel Shamir, Shi-Min Hu, and Daniel Cohen-Or. 2013. 3-Sweep: Extracting Editable Objects from a Single Photo. In ACM SIGGRAPH Asia. ACM, New York, NY, USA, Article 195, 10 pages. Google ScholarDigital Library
5. Charlotte Delmas, Marie-Odile Berger, Erwan Kerrien, Cyril Riddell, Yves Trousset, René Anxionnat, and Serge Bracard. 2015. Three-dimensional curvilinear device reconstruction from two fluoroscopic views. In SPIE, Medical Imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling, Vol. 9415. San Diego, CA, France, 94150F. Google ScholarCross Ref
6. Ricardo Fabbri and Benjamin Kimia. 2010. 3D curve sketch: Flexible curve-based stereo reconstruction and calibration. In IEEE CVPR. 1538–1545. Google ScholarCross Ref
7. Ricardo Fabbri and Benjamin B. Kimia. 2016. Multiview Differential Geometry of Curves. IJCV 120, 3 (Dec. 2016), 324–346. Google ScholarDigital Library
8. Xinyi Fan, Linguang Zhang, Benedict Brown, and Szymon Rusinkiewicz. 2016. Automated View and Path Planning for Scalable Multi-object 3D Scanning. In ACM SIGGRAPH Asia. ACM, New York, NY, USA, Article 239, 13 pages. Google ScholarDigital Library
9. Yasutaka Furukawa and Jean Ponce. 2010. Accurate, Dense, and Robust Multiview Stereopsis. IEEE PAMI 32, 8 (Aug. 2010), 1362–1376. Google ScholarDigital Library
10. Ran Gal, Olga Sorkine, Niloy J. Mitra, and Daniel Cohen-Or. 2009. iWIRES: An Analyze-and-edit Approach to Shape Manipulation. ACM Trans. Graph. 28, 3, Article 33 (July 2009), 10 pages. Google ScholarDigital Library
11. Michael Goesele, Noah Snavely, Brian Curless, Hugues Hoppe, and Steven M. Seitz. 2007. Multi-View Stereo for Community Photo Collections. In IEEE ICCV. 1–8. Google ScholarCross Ref
12. Inc. Gurobi Optimization. 2016. Gurobi Optimizer Reference Manual. (2016). http://www.gurobi.comGoogle Scholar
13. Richard Hartley and Andrew Zisserman. 2004. Multiple View Geometry in Computer Vision (second ed.). Cambridge University Press, ISBN: 0521540518. Google ScholarCross Ref
14. Manuel Hofer, Michael Maurer, and Horst Bischof. 2014. Improving Sparse 3D Models for Man-Made Environments Using Line-Based 3D Reconstruction. In International Conference on 3D Vision (3DV).Google ScholarDigital Library
15. Manuel Hofer, Michael Maurer, and Horst Bischof. 2016. Efficient 3D scene abstraction using line segments. Computer Vision and Image Understanding (2016). Google ScholarDigital Library
16. Manuel Hofer, Andreas Wendel, and Horst Bischof. 2013. Line-based 3D reconstruction of wiry objects. In Computer Vision Winter Workshop (CVWW).Google Scholar
17. Wei Hong, Yi Ma, and Yizhou Yu. 2004. Reconstruction of 3-D Symmetric Curves from Perspective Images without Discrete Features. In ECCV. Springer Berlin Heidelberg, Berlin, Heidelberg, 533–545. Google ScholarCross Ref
18. Hui Huang, Shihao Wu, Daniel Cohen-Or, Minglun Gong, Hao Zhang, Guiqing Li, and Baoquan Chen. 2013. L1-medial Skeleton of Point Cloud. In ACM SIGGRAPH. ACM, New York, NY, USA, Article 65, 8 pages. Google ScholarDigital Library
19. Emmanuel Iarussi, Wilmot Li, and Adrien Bousseau. 2015. WrapIt: Computer-assisted Crafting of Wire Wrapped Jewelry. In ACM SIGGRAPH Asia. ACM, New York, NY, USA, Article 221, 8 pages. Google ScholarDigital Library
20. Arjun Jain, Christian Kurz, Thorsten Thormählen, and Hans-Peter Seidel. 2010. Exploiting Global Connectivity Constraints for Reconstruction of 3D Line Segment from Images. In IEEE CVPR. San Francisco, CA.Google Scholar
21. Fredrik Kahl and Jonas August. 2003. Multiview reconstruction of space curves. In IEEE ICCV. 1017–1024 vol.2. Google ScholarCross Ref
22. Imdat Kara and Tolga Bektas. 2006. Integer linear programming formulations of multiple salesman problems and its variations. European Journal of Operational Research 174, 3 (2006), 1449 — 1458. Google ScholarCross Ref
23. Peter Kovesi. 2017. Peter’s Functions for Computer Vision. (2017). http://www.peterkovesi.com/matlabfns/Google Scholar
24. Guo Li, Ligang Liu, Hanlin Zheng, and Niloy J. Mitra. 2010. Analysis, Reconstruction and Manipulation Using Arterial Snakes. In ACM SIGGRAPH Asia (SIGGRAPH ASIA ’10). ACM, New York, NY, USA, Article 152, 10 pages. Google ScholarDigital Library
25. Tobias Martin, Juan Montes, Jean-Charles Bazin, and Tiberiu Popa. 2014. Topology-aware Reconstruction of Thin Tubular Structures. In SIGGRAPH Asia 2014 Technical Briefs (SA ’14). ACM, New York, NY, USA, Article 12, 4 pages. Google ScholarDigital Library
26. Eder Miguel, Mathias Lepoutre, and Bernd Bickel. 2016. Computational Design of Stable Planar-Rod Structures. ACM SIGGRAPH 35, 4 (2016). Google ScholarDigital Library
27. Irina Nurutdinova and Andrew Fitzgibbon. 2015. Towards Pointless Structure from Motion: 3D Reconstruction and Camera Parameters from General 3D Curves. In IEEE ICCV. IEEE Computer Society, Washington, DC, USA, 2363–2371. Google ScholarDigital Library
28. Jack M. Ogden. 1991. Classical gold wire. Some aspects of its manufacture and use. Jewellery Studies (1991), 95–105, Abb.Google Scholar
29. Dushyant Rao, Soon-Jo Chung, and Seth Hutchinson. 2012. CurveSLAM: An approach for vision-based navigation without point features. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. 4198–4204. Google ScholarCross Ref
30. Luc Robert and Olivier D. Faugeras. 1991. Curve-based stereo: figural continuity and curvature. In IEEE CVPR. 57–62. Google ScholarCross Ref
31. Noah Snavely, Steven M. Seitz, and Richard Szeliski. 2006. Photo Tourism: Exploring Photo Collections in 3D. In ACM SIGGRAPH (SIGGRAPH ’06). ACM, New York, NY, USA, 835–846. Google ScholarDigital Library
32. Damien Teney and Justus Piater. 2012. Sampling-Based Multiview Reconstruction without Correspondences for 3D Edges. In Int. Conf. on 3D Imaging, Modeling, Processing, Visualization Transmission. 160–167. Google ScholarDigital Library
33. Inc. The MathWorks. 2016. Optimization Toolbox User’s Guide. (2016). http://www.mathworks.com/access/helpdesk/help/toolbox/optim/Google Scholar
34. Anil Usumezbas, Ricardo Fabbri, and Benjamin B. Kimia. 2016. From Multiview Image Curves to 3D Drawings. In ECCV. 70–87. Google ScholarCross Ref
35. Changchang Wu. 2011. VisualSFM: A Visual Structure from Motion System. (2011). http://ccwu.me/vsfm/Google Scholar
36. Changchang Wu, Sameer Agarwal, Brian Curless, and Steven M. Seitz. 2011. Multicore bundle adjustment. In IEEE CVPR. 3057–3064. Google ScholarDigital Library
37. Shihao Wu, Wei Sun, Pinxin Long, Hui Huang, Daniel Cohen-Or, Minglun Gong, Oliver Deussen, and Baoquan Chen. 2014. Quality-driven Poisson-guided Autoscanning. In ACM SIGGRAPH Asia. ACM, New York, NY, USA, Article 203, 12 pages. Google ScholarDigital Library
38. Yi Jun Xiao and Youfu Li. 2005. Optimized stereo reconstruction of free-form space curves based on a nonuniform rational B-spline model. J. Opt. Soc. Am. A 22, 9 (Sep 2005), 1746–1762. Google ScholarCross Ref
39. Feilong Yan, Andrei Sharf, Wenzhen Lin, Hui Huang, and Baoquan Chen. 2014. Proactive 3D Scanning of Inaccessible Parts. In ACM SIGGRAPH Asia. ACM, New York, NY, USA, Article 157, 8 pages. Google ScholarDigital Library
40. Kangxue Yin, Hui Huang, Hao Zhang, Minglun Gong, Daniel Cohen-Or, and Baoquan Chen. 2014. Morfit: Interactive Surface Reconstruction from Incomplete Point Clouds with Curve-driven Topology and Geometry Control. In ACM SIGGRAPH Asia. ACM, New York, NY, USA, Article 202, 12 pages. Google ScholarDigital Library
