“SurfaceBrush: from virtual reality drawings to manifold surfaces” by Rosales, Rodriguez and Sheffer
Conference:
Type(s):
Title:
- SurfaceBrush: from virtual reality drawings to manifold surfaces
Session/Category Title: VR and AR
Presenter(s)/Author(s):
Abstract:
Popular Virtual Reality (VR) tools allow users to draw varying-width, ribbonlike 3D brush strokes by moving a hand-held controller in 3D space. Artists frequently use dense collections of such strokes to draw virtual 3D shapes. We propose SurfaceBrush, a surfacing method that converts such VR drawings into user-intended manifold free-form 3D surfaces, providing a novel approach for modeling 3D shapes. The inputs to our method consist of dense collections of artist-drawn stroke ribbons described by the positions and normals of their central polylines, and ribbon widths. These inputs are highly distinct from those handled by existing surfacing frameworks and exhibit different sparsity and error patterns, necessitating a novel surfacing approach. We surface the input stroke drawings by identifying and leveraging local coherence between nearby artist strokes. In particular, we observe that strokes intended to be adjacent on the artist imagined surface often have similar tangent directions along their respective polylines. We leverage this local stroke direction consistency by casting the computation of the user-intended manifold surface as a constrained matching problem on stroke polyline vertices and edges. We first detect and smoothly connect adjacent similarly-directed sequences of stroke edges producing one or more manifold partial surfaces. We then complete the surfacing process by identifying and connecting adjacent similarly directed edges along the borders of these partial surfaces. We confirm the usability of the SurfaceBrush interface and the validity of our drawing analysis via an observational study. We validate our stroke surfacing algorithm by demonstrating an array of manifold surfaces computed by our framework starting from a range of inputs of varying complexity, and by comparing our outputs to reconstructions computed using alternative means.
References:
1. Fatemeh Abbasinejad, Pushkar Joshi, and Nina Amenta. 2012. Surface Patches from Unorganized Space Curves. In Proc. Symposium on Computational Geometry. 417–418. Google ScholarDigital Library
2. Judith Amores and Jaron Lanier. 2017. HoloARt: Painting with Holograms in Mixed Reality. In Proc. Human Factors in Computing Systems. 421–424. Google ScholarDigital Library
3. Haim Avron, Andrei Sharf, Chen Greif, and Daniel Cohen-Or. 2010. ℓ1 Sparsee Reconstruction of Sharp Point Set Surfaces. ACM Trans. Graph. (2010), 135:1–135:12. Google ScholarDigital Library
4. Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: As-natural-as-possible Sketching System for Creating 3D Curve Models. In Proc. Symposium on User Interface Software and Technology. 151–160. Google ScholarDigital Library
5. Nikhil Bansal, Avrim Blum, and Shuchi Chawla. 2004. Correlation clustering. Machine Learning 56, 1–3 (2004), 89–113. Google ScholarDigital Library
6. Gavin Barill, Neil G. Dickson, Ryan Schmidt, David I. W. Levin, and Alec Jacobson. 2018. Fast Winding Numbers for Soups and Clouds. ACM Trans. Graph. (2018), 43:1–43:12. Google ScholarDigital Library
7. Matthew Berger, Andrea Tagliasacchi, Lee M. Seversky, Pierre Alliez, Gaël Guennebaud, Joshua A. Levine, Andrei Sharf, and Claudio T. Silva. 2017. A Survey of Surface Reconstruction from Point Clouds. Comput. Graph. Forum 36, 1 (2017), 301–329. Google ScholarDigital Library
8. F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin. 1999. The ball-pivoting algorithm for surface reconstruction. IEEE Transactions on Visualization and Computer Graphics 5, 4 (1999), 349–359. Google ScholarDigital Library
9. Mikhail Bessmeltsev, Caoyu Wang, Alla Sheffer, and Karan Singh. 2012. Design-Driven Quadrangulation of Closed 3D Curves. ACM Trans.Grap. 31, 5 (2012). Google ScholarDigital Library
10. CGAL. 2018. CGAL The Computational Geometry Algorithms Library. https://www.cgal.org/Google Scholar
11. DesignSpace. 2018. DesignSpace VR. http://www.designspacevr.org/Google Scholar
12. Tamal K. Dey and Samrat Goswami. 2003. Tight Cocone: A Water-tight Surface Reconstructor. J. Comput. Inf. Sci. Eng. 3, 4 (2003), 302–307.Google ScholarCross Ref
13. Holger Diehl, Franz Müller, and Udo Lindemann. 2004. From raw 3D-Sketches to exact CAD product models Concept for an assistant-system. In Sketch Based Interfaces and Modeling. Google ScholarDigital Library
14. Herbert Edelsbrunner and Ernst P. Mücke. 1994. Three-dimensional Alpha Shapes. ACM Trans. Graph. 13, 1 (1994), 43–72. Google ScholarDigital Library
15. Mark Finch and Hugues Hoppe. 2011. Freeform Vector Graphics with Controlled Thin-Plate Splines. ACM Trans. Graph. 30, 6 (2011). Google ScholarDigital Library
16. Michele Fiorentino, Raffaele de Amicis, Giuseppe Monno, and Andre Stork. 2002. Spacedesign: A Mixed Reality Workspace for Aesthetic Industrial Design. In Proc. Symposium on Mixed and Augmented Reality. Google ScholarDigital Library
17. Hongbo Fu, Shizhe Zhou, Ligang Liu, and Niloy J. Mitra. 2011. Animated Construction of Line Drawings. ACM Trans. Graph. (2011), 133:1–133:10. Google ScholarDigital Library
18. Simon Fuhrmann and Michael Goesele. 2014. Floating Scale Surface Reconstruction. ACM Transactions on Graphics 33 (2014), 1–11. Google ScholarDigital Library
19. Kun Gao and Alyn Rockwood. 2005. Multi-sided attribute based modeling. Mathematics of Surfaces XI (2005), 219–232. Google ScholarDigital Library
20. GoogleBlocks. 2018. Google Blocks. https://vr.google.com/blocks/Google Scholar
21. GravitySketch. 2018. Gravity Sketch. https://www.gravitysketch.com/Google Scholar
22. Cindy Grimm and Pushkar Joshi. 2012. Just DrawIt: A 3D Sketching System. In Proc. Symposium on Sketch-Based Interfaces and Modeling. 121–130. Google ScholarDigital Library
23. Tovi Grossman, Ravin Balakrishnan, Gordon Kurtenbach, George Fitzmaurice, Azam Khan, and Bill Buxton. 2002. Creating Principal 3D Curves with Digital Tape Drawing. In Proc. Human Factors in Computing Systems. 121–128. Google ScholarDigital Library
24. Zhiyang Huang, Ming Zou, Nathan Carr, and Tao Ju. 2017. Topology-controlled Reconstruction of Multi-labelled Domains from Cross-sections. ACM Trans. Graph. 36, 4 (2017), 76:1–76:12. Google ScholarDigital Library
25. J.H. Israel, E. Wiese, M. Mateescu, C. Zöllner, and R. Stark. 2009. Investigating three-dimensional sketching for early conceptual design—Results from expert discussions and user studies. Computers and Graphics (2009), 462 — 473. Google ScholarDigital Library
26. B. Jackson and D. F. Keefe. 2016. Lift-Off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR. IEEE Trans. on Visualization and Computer Graphics 22, 4 (2016), 1442–1451. Google ScholarDigital Library
27. Tao Ju. 2004. Robust Repair of Polygonal Models. ACM Trans. Graph. (2004), 888–895. Google ScholarDigital Library
28. R. Karp. 1972. Reducibility among combinatorial problems. In Complexity of Computer Computations. Plenum Press, 85–103.Google Scholar
29. Michael Kazhdan and Hugues Hoppe. 2013. Screened Poisson Surface Reconstruction. ACM Trans. Graph. 32, 3 (2013), 29:1–29:13. Google ScholarDigital Library
30. D. Keefe, R. Zeleznik, and D. Laidlaw. 2007. Drawing on Air: Input Techniques for Controlled 3D Line Illustration. IEEE Transactions on Visualization and Computer Graphics 13, 5 (2007), 1067–1081. Google ScholarDigital Library
31. Daniel F. Keefe, Daniel Acevedo Feliz, Tomer Moscovich, David H. Laidlaw, and Joseph J. LaViola, Jr. 2001. CavePainting: A Fully Immersive 3D Artistic Medium and Interactive Experience. In Proc. Symposium on Interactive 3D Graphics. 85–93. Google ScholarDigital Library
32. Margret Keuper, Evgeny Levinkov, Nicolas Bonneel, Guillaume Lavoué, Thomas Brox, and Bjorn Andres. 2015. Efficient decomposition of image and mesh graphs by lifted multicuts. In Proc. ICCV. 1751–1759. Google ScholarDigital Library
33. Yongkwan Kim, Sang-Gyun An, Joon Hyub Lee, and Seok-Hyung Bae. 2018. Agile 3D Sketching with Air Scaffolding. In Proc. Human Factors in Computing Systems. 238:1–238:12. Google ScholarDigital Library
34. Kodon. 2018. TenkLabs Kodon. https://www.tenklabs.com/kodonGoogle Scholar
35. Ravikrishna Kolluri, Jonathan Richard Shewchuk, and James F. O’Brien. 2004. Spectral Surface Reconstruction from Noisy Point Clouds. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (SGP ’04). 11–21. Google ScholarDigital Library
36. Jung-hoon Kwon, Han-wool Choi, Jeong-in Lee, and Young-Ho Chai. 2005. Free-Hand Stroke Based NURBS Surface for Sketching and Deforming 3D Contents. In Proc. Pacific-Rim Conference on Advances in Multimedia Information Processing. 315–326. Google ScholarDigital Library
37. Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2017. BendSketch: Modeling Freeform Surfaces Through 2D Sketching. ACM Trans. Graph. (2017), 125:1–125:14. Google ScholarDigital Library
38. Chenxi Liu, Enrique Rosales, and Alla Sheffer. 2018. StrokeAggregator: Consolidating Raw Sketches into Artist-Intended Curve Drawings. ACM Trans. Graph. 37, 4 (2018). Google ScholarDigital Library
39. Xueting Liu, Tien-Tsin Wong, and Pheng-Ann Heng. 2015. Closure-aware Sketch Simplification. ACM Trans. Graph. (2015), 168:1–168:10. Google ScholarDigital Library
40. A. Nasri, M. Sabin, and Z. Yasseen. 2009. Filling N-Sided Regions by Quad Meshes for Subdivision Surfaces. Computer Graphics Forum 28, 6 (2009), 1644–1658.Google ScholarCross Ref
41. Andrew Nealen, Takeo Igarashi, Olga Sorkine, and Marc Alexa. 2007. FiberMesh: Designing Freeform Surfaces with 3D Curves. ACM Trans. Graph. 26, 3 (2007). Google ScholarDigital Library
42. G. Noris, D. Sýkora, A. Shamir, S. Coros, B. Whited, M. Simmons, A. Hornung, M. Gross, and R. Sumner. 2012. Smart Scribbles for Sketch Segmentation. Comput. Graph. Forum (2012), 2516–2527. Google ScholarDigital Library
43. OculusMedium. 2016. OculusMedium. https://oculus.com/medium/Google Scholar
44. L. Olsen, F.F. Samavati, M.C. Sousa, and J. Jorge. 2009. Sketch-Based Modeling: A Survey. Computers & Graphics 33 (2009). Issue 1. Google ScholarDigital Library
45. A. C. Öztireli, G. Guennebaud, and M. Gross. 2009. Feature Preserving Point Set Surfaces based on Non-Linear Kernel Regression. Computer Graphics Forum (2009).Google Scholar
46. PaintLab. 2018. PaintLab VR. http://paintlabvr.com/Google Scholar
47. Hao Pan, Yang Liu, Alla Sheffer, Nicholas Vining, Chang-Jian Li, and Wenping Wang. 2015. Flow Aligned Surfacing of Curve Networks. ACM Trans. Graph. 34, 4 (2015), 127:1–127:10. Google ScholarDigital Library
48. Poly. 2018. Tilt Brush sketches. https://poly.google.com/tiltbrushGoogle Scholar
49. E. Sachs, A. Roberts, and D. Stoops. 1991. 3-Draw: a tool for designing 3D shapes. IEEE Computer Graphics and Applications 11, 6 (1991), 18–26. Google ScholarDigital Library
50. S. Schaefer, J. Warren, and D. Zorin. 2004. Lofting curve networks using subdivision surfaces. Proc. Symposium on Geometry Processing (2004), 103. Google ScholarDigital Library
51. Steven Schkolne, Michael Pruett, and Peter Schröder. 2001. Surface Drawing: Creating Organic 3D Shapes with the Hand and Tangible Tools. In Proc. Human Factors in Computing Systems. 261–268. Google ScholarDigital Library
52. S. Schkolne and P. Schroeder. 1999. Surface Drawing. Caltech Department of Computer Science Technical Report CS-TR-99-03. Google ScholarDigital Library
53. Ryan Schmidt, Azam Khan, Karan Singh, and Gord Kurtenbach. 2009. Analytic Drawing of 3D Scaffolds. ACM Trans. Graph. 28, 5 (2009), 149:1–149:10. Google ScholarDigital Library
54. Cloud Shao, Adrien Bousseau, Alla Sheffer, and Karan Singh. 2012. CrossShade: Shading Concept Sketches Using Cross-Section Curves. ACM Trans. Graph. 31, 4 (2012). Google ScholarDigital Library
55. ShapeLab. 2018. ShapeLab. https://store.steampowered.com/app/571890/ShapeLab/Google Scholar
56. Ojaswa Sharma and Nidhi Agarwal. 2016. 3D Surface Reconstruction from Unorganized Sparse Cross Sections. In Proc. Graphics Interface. 33–40. Google ScholarDigital Library
57. Chen Shen, James F. O’Brien, and Jonathan R. Shewchuk. 2004. Interpolating and Approximating Implicit Surfaces from Polygon Soup. ACM Trans. Graph. (2004), 896–904. Google ScholarDigital Library
58. Sketchfab. 2018. Tilt Brush. https://sketchfab.com/tags/tiltbrushGoogle Scholar
59. Thomas F. Stahovich, Eric J. Peterson, and Hanlung Lin. 2014. An efficient, classification-based approach for grouping pen strokes into objects. Computers & Graphics (2014), 14 — 30.Google Scholar
60. Shun’ichi Tano, T. Kodera, Takashi Nakashima, I. Kawano, K. Nakanishi, G. Hamagishi, M. Inoue, A. Watanabe, T. Okamoto, K. Kawagoe, K. Kaneko, T. Hotta, and M. Tatsuoka. 2003. Godzilla: Seamless 2D and 3D Sketch Environment for Reflective and Creative Design Work. In INTERACT.Google Scholar
61. Shun’ichi Tano, Shinya Yamamoto, Junko Ichino, Tomonori Hashiyama, and Mitsuru Iwata. 2013. Truly Useful 3D Drawing System for Professional Designer by “Life-Sized and Operable” Feature and New Interaction. In Human-Computer Interaction – INTERACT 2013. 37–55.Google Scholar
62. TiltBrush. 2018. Google TiltBrush. https://tiltbrush.com/Google Scholar
63. Anil Usumezbas, Ricardo Fabbri, and Benjamin B. Kimia. 2017. The Surfacing of Multiview 3D Drawings via Lofting and Occlusion Reasoning. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 4560–4569.Google ScholarCross Ref
64. T. Várady, Alyn Rockwood, and P. Salvi. 2011. Transfinite surface interpolation over irregular n-sided domains. Computer-Aided Design iv (2011). Google ScholarDigital Library
65. A. Viterbi. 1967. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory 13, 2 (1967), 260–269. Google ScholarDigital Library
66. Ruimin Wang, Ligang Liu, Zhouwang Yang, Kang Wang, Wen Shan, Jiansong Deng, and Falai Chen. 2016. Construction of Manifolds via Compatible Sparse Representations. ACM Trans. Graph. (2016), 14:1–14:10. Google ScholarDigital Library
67. Gerold Wesche and Hans-Peter Seidel. 2001. FreeDrawer: A Free-form Sketching System on the Responsive Workbench. In Proc. Virtual Reality Software and Technology. 167–174. Google ScholarDigital Library
68. Jun Xing, Hsiang-Ting Chen, and Li-Yi Wei. 2014. Autocomplete Painting Repetitions. ACM Trans. Graph. (2014), 172:1–172:11. Google ScholarDigital Library
69. Shiyao Xiong, Juyong Zhang, Jianmin Zheng, Jianfei Cai, and Ligang Liu. 2014. Robust Surface Reconstruction via Dictionary Learning. ACM Trans. Graph. (2014), 201:1–201:12. Google ScholarDigital Library
70. Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014. True2Form: 3D Curve Networks from 2D Sketches via Selective Regularization. ACM Trans. Graph. 33, 4 (2014), 131:1–131:13. Google ScholarDigital Library
71. Ming Zou, Michelle Holloway, Nathan Carr, and Tao Ju. 2015. Topology-constrained Surface Reconstruction from Cross-sections. ACM Trans. Graph. 34, 4 (2015), 128:1–128:10. Google ScholarDigital Library
