“Dynamic furniture modeling through assembly instructions” by Shao, Li, Rong, Zheng and Zhou
Conference:
Type(s):
Title:
- Dynamic furniture modeling through assembly instructions
Session/Category Title: Indoor Scene Modeling
Presenter(s)/Author(s):
Abstract:
We present a technique for parsing widely used furniture assembly instructions, and reconstructing the 3D models of furniture components and their dynamic assembly process. Our technique takes as input a multi-step assembly instruction in a vector graphic format and starts to group the vector graphic primitives into semantic elements representing individual furniture parts, mechanical connectors (e.g., screws, bolts and hinges), arrows, visual highlights, and numbers. To reconstruct the dynamic assembly process depicted over multiple steps, our system identifies previously built 3D furniture components when parsing a new step, and uses them to address the challenge of occlusions while generating new 3D components incrementally. With a wide range of examples covering a variety of furniture types, we demonstrate the use of our system to animate the 3D furniture assembly process and, beyond that, the semantic-aware furniture editing as well as the fabrication of personalized furnitures.
References:
1. Agrawala, M., Phan, D., Heiser, J., Haymaker, J., Klingner, J., Hanrahan, P., and Tversky, B. 2003. Designing effective step-by-step assembly instructions. ACM Trans. Graph. 22, 3 (July), 828–837.
2. Bae, S.-H., Balakrishnan, R., and Singh, K. 2008. Ilovesketch: As-natural-as-possible sketching system for creating 3d curve models. In Proceedings of UIST, ACM, UIST ’08, 151–160.
3. Berthouzoz, F., Garg, A., Kaufman, D. M., Grinspun, E., and Agrawala, M. 2013. Parsing sewing patterns into 3d garments. ACM Trans. Graph. 32, 4 (July), 85:1–85:12.
4. Besl, P. J., and McKay, N. D. 1992. A method for registration of 3-d shapes. IEEE TPAMI 14, 2, 239–256.
5. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE TPAMI 23, 11, 1222–1239.
6. Cao, Y., Ju, T., Fu, Z., and Hu, S. 2014. Interactive image-guided modeling of extruded shapes. Comput. Graph. Forum 33, 7, 101–110.
7. Chen, X., Kang, S. B., Xu, Y.-Q., Dorsey, J., and Shum, H.-Y. 2008. Sketching reality: Realistic interpretation of architectural designs. ACM Trans. Graph. 27, 2 (May), 11:1–11:15.
8. Chen, T., Zhu, Z., Shamir, A., Hu, S.-M., and Cohen-Or, D. 2013. 3sweep: Extracting editable objects from a single photo. ACM Trans. Graph. 32, 6 (Nov.), 195:1–195:10.
9. De Paoli, C., and Singh, K. 2015. Secondskin: Sketch-based construction of layered 3d models. ACM Trans. Graph. 34, 4 (July), 126:1–126:10.
10. Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM Trans. Graph. 31, 4 (July), 44:1–44:10.
11. Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., and Alexa, M. 2012. Sketch-based shape retrieval. ACM Trans. Graph. 31, 4 (July), 31:1–31:10.
12. Fanelli, G., Gall, J., and Gool, L. J. V. 2011. Real time head pose estimation with random regression forests. In CVPR 2011, Colorado Springs, CO, USA, 20-25 June 2011, 617–624.
13. Fu, H., Zhou, S., Liu, L., and Mitra, N. J. 2011. Animated construction of line drawings. ACM Trans. Graph. 30, 6 (Dec.), 133:1–133:10.
14. Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., and Jacobs, D. 2003. A search engine for 3d models. ACM Trans. Graph. 22, 1 (Jan.), 83–105.
15. Gennari, L., Kara, L. B., Stahovich, T. F., and Shimada, K. 2005. Combining geometry and domain knowledge to interpret hand-drawn diagrams. Computers & Graphics 29, 4, 547–562.
16. Gingold, Y., Igarashi, T., and Zorin, D. 2009. Structured annotations for 2d-to-3d modeling. ACM Trans. Graph. 28, 5 (Dec.), 148:1–148:9.
17. Gupta, A., Fox, D., Curless, B., and Cohen, M. 2012. Duplotrack: A real-time system for authoring and guiding duplo block assembly. In Proceedings of UIST ’12, 389–402.
18. Haralick, R. M., and Queeney, D. 1982. Understanding engineering drawings. Computer Graphics and Image Processing 19, 1, 90. Cross Ref
19. Harltey, A., and Zisserman, A. 2006. Multiple view geometry in computer vision (2. ed.). Cambridge University Press.
20. Heiser, J., Phan, D., Agrawala, M., Tversky, B., and Hanrahan, P. 2004. Identification and validation of cognitive design principles for automated generation of assembly instructions. In Proceedings of the working conference on Advanced Visual Interfaces, ACM, 311–319.
21. Huang, Z., Fu, H., and Lau, R. W. H. 2014. Data-driven segmentation and labeling of freehand sketches. ACM Trans. Graph. 33, 6 (Nov.), 175:1–175:10.
22. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: A sketching interface for 3d freeform design. In Proceedings of ACM SIGGRAPH ’99, 409–416.
23. 2 Jr., J. J. L., and Zeleznik, R. C. 2004. Mathpad2: a system for the creation and exploration of mathematical sketches. ACM Trans. Graph. 23, 3, 432–440.
24. Karpenko, O. A., and Hughes, J. F. 2006. Smoothsketch: 3d free-form shapes from complex sketches. ACM Trans. Graph. 25, 3 (July), 589–598.
25. Koo, B., Li, W., Yao, J., Agrawala, M., and Mitra, N. J. 2014. Creating works-like prototypes of mechanical objects. ACM Trans. Graph. 33, 6 (Nov.), 217:1–217:9.
26. Lau, M., Ohgawara, A., Mitani, J., and Igarashi, T. 2011. Converting 3d furniture models to fabricatable parts and connectors. ACM Trans. Graph. 30, 4 (July), 85:1–85:6.
27. LaViola, J., DAVIS, R., and IGARASHI, T. 2006. An introduction to sketch-based interfaces. ACM SIGGRAPH Course Notes.
28. Li, W., Agrawala, M., and Salesin, D. 2004. Interactive image-based exploded view diagrams. In Proceedings of Graphics Interface 2004, GI ’04, 203–212.
29. Li, H., Hu, R., Alhashim, I., and Zhang, H. 2015. Foldabilizing furniture. ACM Trans. Graph. 34, 4 (July), 90:1–90:12.
30. Lowe, D. G. 1999. Object recognition from local scale-invariant features. In ICCV, 1150–1157.
31. Mena, J. B. 2003. State of the art on automatic road extraction for GIS update: a novel classification. Pattern Recognition Letters 24, 16, 3037–3058.
32. Mitra, N. J., Yang, Y.-L., Yan, D.-M., Li, W., and Agrawala, M. 2010. Illustrating how mechanical assemblies work. ACM Trans. Graph. 29, 4, 58.
33. Mohr, P., Kerbl, B., Donoser, M., Schmalstieg, D., and Kalkofen, D. 2015. Retargeting technical documentation to augmented reality. In Proceedings of CHI ’15, ACM, New York, NY, USA, 3337–3346.
34. Ouyang, T. Y., and Davis, R. 2011. Chemink: a natural real-time recognition system for chemical drawings. In Proceedings of IUI, 2011, 267–276.
35. Rong, Y., Zheng, Y., Shao, T., Yang, Y., and Zhou, K. 2016. An interactive approach for functional prototype recovery from a single rgbd image. Computational Visual Media 2, 1, 87–96. Cross Ref
36. Saul, G., Lau, M., Mitani, J., and Igarashi, T. 2011. SketchChair: An All-in-one Chair Design System for End Users. In Proceedings of TEI, ACM, TEI ’11.
37. Schmidt, R., Khan, A., Singh, K., and Kurtenbach, G. 2009. Analytic drawing of 3d scaffolds. ACM Trans. Graph. 28, 5 (Dec.), 149:1–149:10.
38. Schulz, A., Shamir, A., Levin, D. I. W., Sitthi-amorn, P., and Matusik, W. 2014. Design and fabrication by example. ACM Trans. Graph. 33, 4 (July), 62:1–62:11.
39. Shao, T., Xu, W., Yin, K., Wang, J., Zhou, K., and Guo, B. 2011. Discriminative sketch-based 3d model retrieval via robust shape matching. Comput. Graph. Forum 30, 7, 2011–2020. Cross Ref
40. Shao, T., Li, W., Zhou, K., Xu, W., Guo, B., and Mitra, N. J. 2013. Interpreting concept sketches. ACM Trans. Graph. 32, 4 (July), 56:1–56:10.
41. Shotton, J., Johnson, M., and Cipolla, R. 2008. Semantic texton forests for image categorization and segmentation. In CVPR 2008, Anchorage, Alaska, USA.
42. Shtof, A., Agathos, A., Gingold, Y. I., Shamir, A., and Cohen-Or, D. 2013. Geosemantic snapping for sketch-based modeling. Comput. Graph. Forum 32, 2, 245–253. Cross Ref
43. Sun, Z., Wang, C., Zhang, L., and Zhang, L. 2012. Free hand-drawn sketch segmentation. In ECCV 2012, 626–639.
44. Tombre, K. 1998. Graphics Recognition Algorithms and Systems: Second International Workshop, GREC’ 97 Nancy, France, August 22–23, 1997 Selected Papers. ch. Analysis of engineering drawings: State of the art and challenges, 257–264.
45. Tversky, B., Zacks, J., Lee, P., and Heiser, J. 2000. Lines, blobs, crosses and arrows: Diagrammatic communication with schematic figures. In Theory and application of diagrams. Springer, 221–230.
46. Tversky, B., Y, J. B. M., and Betrancourt, M. 2002. Animation: Can it facilitate. International Journal of Human-Computer Studies 57, 247–262.
47. Umetani, N., Igarashi, T., and Mitra, N. J. 2012. Guided exploration of physically valid shapes for furniture design. ACM Trans. Graph. 31, 4 (July), 86:1–86:11.
48. Xu, K., Chen, K., Fu, H., Sun, W.-L., and Hu, S.-M. 2013. Sketch2scene: Sketch-based co-retrieval and co-placement of 3d models. ACM Trans. Graph. 32, 4 (July), 123:1–123:15.
49. Xu, B., Chang, W., Sheffer, A., Bousseau, A., McCrae, J., and Singh, K. 2014. True2form: 3d curve networks from 2d sketches via selective regularization. ACM Trans. Graph. 33, 4 (July), 131:1–131:13.
50. Zauner, J., Haller, M., Brandl, A., and Hartmann, W. 2003. Authoring of a mixed reality furniture assembly instructor. In ACM SIGGRAPH 2003 Sketches & Applications.
51. Zeleznik, R. C., Herndon, K. P., and Hughes, J. F. 2006. Sketch: An interface for sketching 3d scenes. In ACM SIGGRAPH 2006 Courses, ACM, New York, NY, USA, SIGGRAPH ’06.
52. Zheng, Y., Chen, X., Cheng, M.-M., Zhou, K., Hu, S.-M., and Mitra, N. J. 2012. Interactive images: Cuboid proxies for smart image manipulation. ACM Trans. Graph. 31, 4 (July).
