“AdaptiBrush: adaptive general and predictable VR ribbon brush” by Rosales, Araújo, Rodriguez, Vining, Yoon, et al. …
Conference:
Type(s):
Title:
- AdaptiBrush: adaptive general and predictable VR ribbon brush
Session/Category Title: NPR and Digital Art
Presenter(s)/Author(s):
Abstract:
Virtual reality drawing applications let users draw 3D shapes using brushes that form ribbon shaped, or ruled-surface, strokes. Each ribbon is uniquely defined by its user-specified ruling length, path, and the ruling directions at each point along this path. Existing brushes use the trajectory of a handheld controller in 3D space as the ribbon path, and compute the ruling directions using a fixed mapping from a specific controller coordinate-frame axis. This fixed mapping forces users to rotate the controller and thus their wrists to change ribbon normal or ruling directions, and requires substantial physical effort to draw even medium complexity ribbons. Since human ability to rotate their wrists continuously is heavily restricted, the space of ribbon geometries users can comfortably draw using these brushes is limited. These brushes can be unpredictable, producing ribbons with unexpectedly varying width or flipped and wobbly normals in response to seemingly natural hand gestures. Our AdaptiBrush ribbon brush system dramatically extends the space of ribbon geometries users can comfortably draw while enabling them to accurately predict the ribbon shape that a given hand motion produces. We achieve this by introducing a novel adaptive ruling direction computation method, enabling users to easily change ribbon ruling and normal orientation using predominantly translational controller, and thus wrist, motion. We facilitate ease-of-use by computing predictable ruling directions that smoothly change in both world and controller coordinate systems, and facilitate ease-of-learning by prioritizing ruling directions which are well-aligned with one of the controller coordinate system axes. Our comparative user studies confirm that our more general and predictable ruling computation leads to significant improvements in brush usability and effectiveness compared to all prior brushes; in a head to head comparison users preferred AdaptiBrush over the next-best brush by a margin of 2 to 1.
References:
1. Judith Amores and Jaron Lanier. 2017. HoloARt: Painting with Holograms in Mixed Reality. In Proc. Human Factors in Computing Systems. 421–424.
2. Rahul Arora, Rubaiat Habib Kazi, Fraser Anderson, Tovi Grossman, Karan Singh, and George Fitzmaurice. 2017. Experimental Evaluation of Sketching on Surfaces in VR. In Proc. Human Factors in Computing Systems. 5643–5654.
3. Rahul Arora and Karan Singh. 2021. Mid-Air Drawing of Curves on 3D Surfaces in Virtual Reality. ACM Trans. Graph. 40, – (2021), 17.
4. Ilya Baran, Jaakko Lehtinen, and Jovan Popović. 2010. Sketching clothoid splines using shortest paths. In Computer Graphics Forum, Vol. 29. 655–664.
5. Mayra Donaji Barrera Machuca, Wolfgang Stuerzlinger, and Paul Asente. 2019. The Effect of Spatial Ability on Immersive 3D Drawing. In Proc. Creativity and Cognition. 173–186.
6. Sukanya Bhattacharjee and Parag Chaudhuri. 2020. A Survey on Sketch Based Content Creation: from the Desktop to Virtual and Augmented Reality. Computer Graphics Forum (2020).
7. Holger Diehl, Franz Müller, and Udo Lindemann. 2004. From raw 3D-Sketches to exact CAD product models Concept for an assistant-system. In Sketch Based Interfaces and Modeling.
8. M.P. do Carmo. 2016. Differential Geometry of Curves and Surfaces. Dover Publications.
9. Davis A Forman, Garrick N Forman, Maddalena Mugnosso, Jacopo Zenzeri, Bernadette Murphy, and Michael WR Holmes. 2020. Sustained isometric wrist flexion and extension maximal voluntary contractions similarly impair hand-tracking accuracy in young adults using a wrist robot. Frontiers in Sports and Active Living 2 (2020).
10. GravitySketch. 2019. Gravity Sketch. (2019). https://www.gravitysketch.com/
11. Tovi Grossman, Ravin Balakrishnan, Gordon Kurtenbach, George Fitzmaurice, Azam Khan, and Bill Buxton. 2002. Creating Principal 3D Curves with Digital Tape Drawing. In Proc. Human Factors in Computing Systems. 121–128.
12. Tovi Grossman, Ravin Balakrishnan, and Karan Singh. 2003. An Interface for Creating and Manipulating Curves Using a High Degree-of-freedom Curve Input Device. In Proc. CHI Conference on Human Factors in Computing Systems. 185–192.
13. LLC Gurobi Optimization. 2020. Gurobi Optimizer Reference Manual. (2020). http://www.gurobi.com
14. Juan David Hincapié-Ramos, Xiang Guo, Paymahn Moghadasian, and Pourang Irani. 2014. Consumed Endurance: A Metric to Quantify Arm Fatigue of Mid-air Interactions. In Proc. CHI Conference on Human Factors in Computing Systems. 1063–1072.
15. Zhiyang Huang, Nathan Carr, and Tao Ju. 2019. Variational Implicit Point Set Surfaces. ACM Trans. Graph. 38, 4, Article 124 (2019).
16. J.H. Israel, E. Wiese, M. Mateescu, C. Zöllner, and R. Stark. 2009. Investigating three-dimensional sketching for early conceptual design-Results from expert discussions and user studies. Computers and Graphics (2009), 462 — 473.
17. B. Jackson and D. F. Keefe. 2016. Lift-Off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR. IEEE Trans. on Visualization and Computer Graphics (2016), 1442–1451.
18. Sujin Jang, Wolfgang Stuerzlinger, Satyajit Ambike, and Karthik Ramani. 2017. Modeling Cumulative Arm Fatigue in Mid-Air Interaction Based on Perceived Exertion and Kinetics of Arm Motion. In Proc. CHI Conference on Human Factors in Computing Systems. 3328–3339.
19. D. Keefe, R. Zeleznik, and D. Laidlaw. 2007. Drawing on Air: Input Techniques for Controlled 3D Line Illustration. IEEE TVCG 13, 5 (2007), 1067–1081.
20. Daniel F. Keefe, Daniel Acevedo Feliz, Tomer Moscovich, David H. Laidlaw, and Joseph J. LaViola, Jr. 2001. CavePainting: A Fully Immersive 3D Artistic Medium and Interactive Experience. In Proc. I3D. 85–93.
21. Viktor Kelkkanen, Markus Fiedler, and David Lindero. 2020. Bitrate Requirements of Non-Panoramic VR Remote Rendering. In Proc. 28th ACM International Conference on Multimedia. 3624–3631.
22. Yongkwan Kim, Sang-Gyun An, Joon Hyub Lee, and Seok-Hyung Bae. 2018. Agile 3D Sketching with Air Scaffolding. In Proc. Human Factors in Computing Systems. 238:1–238:12.
23. Robert I Kumar, Garrick N Forman, Davis A Forman, Maddalena Mugnosso, Jacopo Zenzeri, Duane C Button, and Michael WR Holmes. 2020. Dynamic wrist flexion and extension fatigue induced via submaximal contractions similarly impairs hand tracking accuracy in young adult males and females. Frontiers in Sports and Active Living 2 (2020), 135.
24. Joseph J LaViola, Ernst Kruijff, Ryan P McMahan, Doug Bowman, and Ivan P Poupyrev. 2017. 3D user interfaces: theory and practice. Addison-Wesley Professional.
25. Roman Luks and Fotis Liarokapis. 2019. Investigating Motion Sickness Techniques for Immersive Virtual Environments. In Proc. 12th ACM International Conference on Pervasive Technologies Related to Assistive Environments. 280–288.
26. James McCrae and Karan Singh. 2009. Sketching piecewise clothoid curves. Computers & Graphics 33, 4 (2009), 452–461.
27. Mozilla. 2021. A-Painter. (2021). https://blog.mozvr.com/a-painter/
28. Peng Nan, Amnad Tongtib, and Theeraphong Wongratanaphisan. 2019. Evaluation of Upper Limb Joint’s Range of Motion Data by Kinect Sensor for Rehabilitation Exercise Game. In Proc. Medical and Health Informatics. 92–98.
29. Oculus. 2019. Quill. (2019). https://quill.fb.com/
30. Oculus. 2021. Oculus VR best practices guide. “https://developer.oculus.com/learn/”. (2021).
31. PaintLab. 2019. PaintLab VR. (2019). http://paintlabvr.com/
32. Andrew K. Palmer, Frederick W. Werner, Dennis Murphy, and Richard Glisson. 1985. Functional wrist motion: A biomechanical study. The Journal of Hand Surgery 10, 1 (1985), 39–46.
33. Robert S Porter and Justin L Kaplan. 2011. The Merck manual of diagnosis and therapy. Merck Sharp & Dohme Corp.
34. Helmut Pottmann and Johannes Wallner. 2009. Computational line geometry. Springer Science & Business Media.
35. Dominik Rausch, Ingo Assenmacher, and Torsten Kuhlen. 2010. 3D Sketch Recognition for Interaction in Virtual Environments. In Workshop in Virtual Reality Interactions and Physical Simulation. The Eurographics Association.
36. Enrique Rosales, Jafet Rodriguez, and Alla Sheffer. 2019. SurfaceBrush: From Virtual Reality Drawings to Manifold Surfaces. ACM Transaction on Graphics 38, 4 (2019).
37. S. Schkolne and P. Schroeder. 1999. Surface Drawing. Caltech Department of Computer Science Technical Report CS-TR-99-03.
38. Shun’ichi Tano, T. Kodera, Takashi Nakashima, I. Kawano, K. Nakanishi, G. Hamagishi, M. Inoue, A. Watanabe, T. Okamoto, K. Kawagoe, K. Kaneko, T. Hotta, and M. Tatsuoka. 2003. Godzilla: Seamless 2D and 3D Sketch Environment for Reflective and Creative Design Work. In INTERACT.
39. TiltBrush. 2019. Google TiltBrush. (2019). https://tiltbrush.com/
40. Unity. 2019. SteamVR Plugin. (2019). https://assetstore.unity.com/packages/tools/integration/steamvr-plugin-32647
41. E. Wiese, J. H. Israel, A. Meyer, and S. Bongartz. 2010. Investigating the Learnability of Immersive Free-hand Sketching. In Proc. SBIM. 135–142.
42. Emilie Yu, Rahul Arora, Tibor Stanko, J. Andreas Bærentzen, Karan Singh, and Adrien Bousseau. 2021. CASSIE: Curve and Surface Sketching in Immersive Environments. In Proc. CHI. 1–14.
43. Shumin Zhai. 1998. User Performance in Relation to 3D Input Device Design. SIGGRAPH Comput. Graph. 32, 4 (Nov. 1998), 50–54.
