“Perception-aware modeling and fabrication of digital drawing tools” by Piovarči, Levin, Kaufman and Didyk
Conference:
Type:
Entry Number: 123
Title:
- Perception-aware modeling and fabrication of digital drawing tools
Session/Category Title: Perception & Haptics
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Digital drawing is becoming a favorite technique for many artists. It allows for quick swaps between different materials, reverting changes, and applying selective modifications to finished artwork. These features enable artists to be more efficient and creative. A significant disadvantage of digital drawing is poor haptic feedback. Artists are usually limited to one surface and a few different stylus nibs, and while they try to find a combination that suits their needs, this is typically challenging. In this work, we address this problem and propose a method for designing, evaluating, and optimizing different stylus designs. We begin with collecting a representative set of traditional drawing tools. We measure their physical properties and conduct a user experiment to build a perceptual space that encodes perceptually-relevant attributes of drawing materials. The space is optimized to both explain our experimental data and correlate it with measurable physical properties. To embed new drawing tool designs into the space without conducting additional experiments and measurements, we propose a new, data-driven simulation technique for characterizing stylus-surface interaction. We finally leverage the perceptual space, our simulation, and recent advancements in multi-material 3D printing to demonstrate the application of our system in the design of new digital drawing tools that mimic traditional drawing materials.
References:
1. Michelle Annett, Fraser Anderson, Walter F Bischof, and Anoop Gupta. 2014. The pen is mightier: understanding stylus behaviour while inking on tablets. In Proceedings of Graphics Interface 2014. Canadian Information Processing Society, 193–200. Google ScholarDigital Library
2. Atakan Arasan, Cagatay Basdogan, and Tevfik Metin Sezgin. 2013. Haptic stylus with inertial and vibro-tactile feedback. In 2013 World Haptics Conference (WHC). 425–430.Google ScholarCross Ref
3. John F. Archard. 1953. Contact and Rubbing of Flat Surfaces. Journal of Applied Physics 24, 8 (1953), 981–988.Google ScholarCross Ref
4. Brian Armstrong-Hélouvry. 1991. Control of Machines with Friction. Kluwer Academic Publishers, Boston, Massachusetts.Google Scholar
5. Mordecai Avriel. 2003. Nonlinear Programming: Analysis and Methods. Dover Publications.Google Scholar
6. Olivier Bau, Ivan Poupyrev, Ali Israr, and Chris Harrison. 2010. TeslaTouch: Electro-vibration for Touch Surfaces. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (UIST ’10). ACM, New York, NY, USA, 283–292. Google ScholarDigital Library
7. Sliman Bensmaïa, Mark Hollins, and Jeffrey Yau. 2005. Vibrotactile intensity and frequency information in the Pacinian system: A psychophysical model. Attention, Perception, & Psychophysics 67, 5 (2005), 828–841.Google ScholarCross Ref
8. Wouter M Bergmann Tiest and Astrid ML Kappers. 2006. Analysis of haptic perception of materials by multidimensional scaling and physical measurements of roughness and compressibility. Acta psychologica 121, 1 (2006), 1–20.Google Scholar
9. Florence Bertails-Descoubes, Florent Cadoux, Gilles Daviet, and Vincent Acary. 2011. A nonsmooth Newton solver for capturing exact Coulomb friction in fiber assemblies. ACM Transactions on Graphics 30, 1 (Feb. 2011), 6:1–6:14. Google ScholarDigital Library
10. Brian Blood. 2009. Physics of musical instruments. (2009).Google Scholar
11. Carlos Canudas de Wit, Henrik Olsson, Karl J. Astrom, and Pablo Lischinsky. 1995. A new model for control of systems with friction. IEEE Trans. Automat. Control 40, 3 (Mar 1995), 419–425.Google Scholar
12. Desai Chen, David I. W. Levin, Wojciech Matusik, and Danny M. Kaufman. 2017. Dynamics-aware Numerical Coarsening for Fabrication Design. ACM Trans. Graph. 36, 4, Article 84 (July 2017), 15 pages. Google ScholarDigital Library
13. Youngjun Cho, Andrea Bianchi, Nicolai Marquardt, and Nadia Bianchi-Berthouze. 2016. RealPen: Providing Realism in Handwriting Tasks on Touch Surfaces Using Auditory-Tactile Feedback. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST ’16). ACM, New York, NY, USA, 195–205. Google ScholarDigital Library
14. Seungmoon Choi and Hong Z. Tan. 2005. Toward Realistic Haptic Rendering of Surface Textures. In ACM SIGGRAPH 2005 Courses (SIGGRAPH ’05). ACM, New York, NY, USA, Article 125. Google ScholarDigital Library
15. Vasilios G Chouvardas, Amalia N Miliou, and Miltiadis K Hatalis. 2008. Tactile displays: Overview and recent advances. Displays 29, 3 (2008), 185–194.Google ScholarCross Ref
16. Vasilios G Chouvardas, Amalia N Miliou, Miltiadis K Hatalis, et al. 2005. Tactile displays: a short overview and recent developments. In Proceedings of the 5th International Conference on Technology and Automation (ICTA’05). 246–251.Google Scholar
17. Theresa Cooke, Sebastian Kannengiesser, Christian Wallraven, and Heinrich H Bülthoff.Google Scholar
18. 2006. Object feature validation using visual and haptic similarity ratings. ACM Transactions on Applied Perception (TAP) 3, 3 (2006), 239–261. Google ScholarDigital Library
19. Theresa Cooke, Christian Wallraven, and Heinrich H Bülthoff. 2010. Multidimensional scaling analysis of haptic exploratory procedures. ACM Transactions on Applied Perception (TAP) 7, 1 (2010), 7. Google ScholarDigital Library
20. Philip R. Dahl. 1976. Solid Friction Damping of Mechanical Vibrations. AIAA Journal 14, 12 (1976), 1675–1682.Google ScholarCross Ref
21. Jérémy Danna and Jean-Luc Velay. 2015. Basic and supplementary sensory feedback in handwriting. Frontiers in Psychology 6 (2015), 169.Google ScholarCross Ref
22. Nicholas Fellion, Thomas Pietrzak, and Audrey Girouard. 2017. FlexStylus: leveraging flexion input for pen interaction. In UIST 2017 (UIST 2017). Québec, Canada, 2482 — 2489. Google ScholarDigital Library
23. Claire Watson Garcia. 2003. Drawing for the Absolute and Utter Beginner. Watson-Guptill.Google Scholar
24. Ioannis Gkioulekas, Bei Xiao, Shuang Zhao, Edward H Adelson, Todd Zickler, and Kavita Bala. 2013. Understanding the role of phase function in translucent appearance. ACM Transactions on Graphics (TOG) 32, 5 (2013), 147. Google ScholarDigital Library
25. Avraham Harnoy, Bernard Friedland, and Simon Cohn. 2008. Modeling and measuring friction effects. IEEE Control Systems 28, 6 (Dec 2008), 82–91.Google Scholar
26. Vincent Hayward and M Cruz-Hernandez. 2000. Tactile display device using distributed lateral skin stretch. In Proceedings of the haptic interfaces for virtual environment and teleoperator systems symposium, Vol. 69. ASME, 1309–1314.Google ScholarCross Ref
27. Richard Helps and Clarissa Helps. 2016. Measuring Stylus and Tablet Performance for Usability in Sketching. In Proceedings of the 5th Annual Conference on Research in Information Technology. ACM, 19–24. Google ScholarDigital Library
28. Marlis Hochbruck and Alexander Ostermann. 2010. Exponential integrators. Acta Numerica 19 (2010), 209–286.Google ScholarCross Ref
29. Mark Hollins, Sliman Bensmaïa, Kristie Karlof, and Forrest Young. 2000. Individual differences in perceptual space for tactile textures: Evidence from multidimensional scaling. Perception & Psychophysics 62, 8 (2000), 1534–1544.Google ScholarCross Ref
30. Mark Hollins, Florian Lorenz, and Daniel Harper. 2006. Somatosensory Coding of Roughness: The Effect of Texture Adaptation in Direct and Indirect Touch. Journal of Neuroscience 26, 20 (2006), 5582–5588.Google ScholarCross Ref
31. Ali Israr, Seungmoon Choi, and Hong Z. Tan. 2006a. Detection Threshold and Mechanical Impedance of the Hand in a Pen-Hold Posture. In 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems. 472–477.Google Scholar
32. Ali Israr, Hong Z. Tan, and Charlotte M. Reed. 2006b. Frequency and amplitude discrimination along the kinesthetic-cutaneous continuum in the presence of masking stimuli. The Journal of the Acoustical Society of America 120, 5 (2006), 2789–2800.Google ScholarCross Ref
33. Lynette A Jones and Ian W Hunter. 1990. A perceptual analysis of stiffness. Experimental Brain Research 79, 1 (1990), 150–156.Google ScholarCross Ref
34. Dean Karnopp. 1985. Computer Simulation of Stick-Slip Friction in Mechanical Dynamic Systems. ASME Journal of dynamic Systems, Measurement and Control 107, 1 (Mar 1985), 100–103.Google Scholar
35. Danny M. Kaufman, Timothy Edmunds, and Dinesh K. Pai. 2005. Fast Frictional Dynamics for Rigid Bodies. ACM Trans. Graph. 24, 3 (July 2005), 946–956. Google ScholarDigital Library
36. Danny M. Kaufman, Shinjiro Sueda, Doug L. James, and Dinesh K. Pai. 2008. Staggered Projections for Frictional Contact in Multibody Systems. ACM Trans. Graph. 27, 5, Article 164 (Dec. 2008), 11 pages. Google ScholarDigital Library
37. Seung-Chan Kim, Ali Israr, and Ivan Poupyrev. 2013. Tactile Rendering of 3D Features on Touch Surfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ’13). ACM, New York, NY, USA, 531–538. Google ScholarDigital Library
38. Roberta L. Klatzky and Susan J. Lederman. 2002. Perceiving texture through a probe. Touch in virtual environments (2002), 180–193.Google Scholar
39. Roberta L. Klatzky and Susan J. Lederman. 2008. Perceiving object properties through a rigid link. Haptic Rendering: Algorithms and Applications 1 (2008), 7–19.Google ScholarCross Ref
40. Roberta L. Klatzky, Susan J. Lederman, Cheryl Hamilton, Molly Grindley, and Robert H. Swendsen. 2003. Feeling textures through a probe: Effects of probe and surface geometry and exploratory factors. Attention, Perception, & Psychophysics 65, 4 (2003), 613–631.Google ScholarCross Ref
41. Johnny C. Lee, Paul H. Dietz, Darren Leigh, William S. Yerazunis, and Scott E. Hudson. 2004. Haptic Pen: A Tactile Feedback Stylus for Touch Screens. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST ’04). ACM, New York, NY, USA, 291–294. Google ScholarDigital Library
42. Dingzeyu Li, David I. W. Levin, Wojciech Matusik, and Changxi Zheng. 2016. Acoustic Voxels: Computational Optimization of Modular Acoustic Filters. ACM Trans. Graph. 35, 4, Article 88 (July 2016), 12 pages. Google ScholarDigital Library
43. Thomas H. Massie, J. Kenneth Salisbury, et al. 1994. The phantom haptic interface: A device for probing virtual objects. In Proceedings of the ASME winter annual meeting, symposium on haptic interfaces for virtual environment and teleoperator systems, Vol. 55. Chicago, IL, 295–300.Google Scholar
44. Ralph Mayer. 1991. Artist’s Handbook of Materials and Techniques: Revised and Updated. Viking Books.Google Scholar
45. Dominik L. Michels, Gerrit A. Sobottka, and Andreas G. Weber. 2014. Exponential Integrators for Stiff Elastodynamic Problems. ACM Trans. Graph. 33, 1, Article 7 (Feb. 2014), 20 pages. Google ScholarDigital Library
46. Brian Mirtich and John Canny. 1995. Impulse-based Simulation of Rigid Bodies. In Proceedings of the 1995 Symposium on Interactive 3D Graphics (I3D ’95). ACM, New York, NY, USA, 181–ff. Google ScholarDigital Library
47. Ilana Nisky, Assaf Pressman, Carla M. Pugh, Ferdinando A. Mussa-Ivaldi, and Amir Karniel. 2011. Perception and Action in Teleoperated Needle Insertion. EEE Trans. Haptics 4, 3 (July 2011), 155–166. Google ScholarDigital Library
48. Miguel A. Otaduy and Ming C. Lin. 2004. A Perceptually-inspired Force Model for Haptic Texture Rendering. In Proceedings of the 1st Symposium on Applied Perception in Graphics and Visualization (APGV ’04). ACM, New York, NY, USA, 123–126. Google ScholarDigital Library
49. Fabio Pellacini, James A. Ferwerda, and Donald P. Greenberg. 2000. Toward a psychophysically-based light reflection model for image synthesis. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., 55–64. Google ScholarDigital Library
50. Alvaro G. Perez, Daniel Lobo, Francesco Chinello, Gabriel Cirio, Monica Malvezzi, Jose San Martin, Domenico Prattichizzo, and Miguel A. Otaduy. 2017. Optimization-based wearable tactile rendering. IEEE transactions on haptics (2017).Google Scholar
51. Henry Petroski. 1992. The pencil: A history of design and circumstance. Alfred a Knopf Incorporated.Google Scholar
52. Michal Piovarči, David I.W. Levin, Jason Rebello, Desai Chen, Roman Ďurikovič, Hanspeter Pfister, Wojciech Matusik, and Piotr Didyk. 2016. An Interaction-Aware, Perceptual Model For Non-Linear Elastic Objects. ACM Transactions on Graphics (Proc. SIGGRAPH) 35, 4 (2016). Google ScholarDigital Library
53. Helena Pongrac. 2008. Vibrotactile perception: examining the coding of vibrations and the just noticeable difference under various conditions. Multimedia Systems 13, 4 (2008), 297–307. Google ScholarDigital Library
54. Ivan Poupyrev, Makoto Okabe, and Shigeaki Maruyama. 2004. Haptic Feedback for Pen Computing: Directions and Strategies. In CHI ’04 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’04). ACM, New York, NY, USA, 1309–1312. Google ScholarDigital Library
55. Joseph M. Romano and Katherine J. Kuchenbecker. 2012. Creating Realistic Virtual Textures from Contact Acceleration Data. IEEE Transactions on Haptics 5, 2 (April 2012), 109–119. Google ScholarDigital Library
56. Leonid Sigal, Moshe Mahler, Spencer Diaz, Kyna McIntosh, Elizabeth Carter, Timothy Richards, and Jessica Hodgins. 2015. A Perceptual Control Space for Garment Simulation. ACM Trans. Graph. 34, 4, Article 117 (July 2015), 10 pages. Google ScholarDigital Library
57. D. Amnon Silverstein and Joyce E. Farrell. 2001. Efficient method for paired comparison. Journal of Electronic Imaging 10, 2 (2001), 394–398.Google ScholarCross Ref
58. Hyunyoung Song, Hrvoje Benko, Francois Guimbretiere, Shahram Izadi, Xiang Cao, and Ken Hinckley. 2011. Grips and Gestures on a Multi-Touch Pen. 1323–1332. https://www.microsoft.com/en-us/research/publication/grips-and-gestures-on-a-multi-touch-pen/ Google ScholarDigital Library
59. Gwidon W Stachowiak. 2006. Wear: materials, mechanisms and practice. John Wiley & Sons.Google Scholar
60. Luis L. Thurstone. 1927. A law of comparative judgment. Psychological Review 34, 4 (1927).Google ScholarCross Ref
61. Wouter M Bergmann Tiest and Astrid ML Kappers. 2009. Cues for haptic perception of compliance. Haptics, IEEE Transactions on 2, 4 (2009), 189–199. Google ScholarDigital Library
62. Stephen Van Dulken. 2002. Inventing the 20th Century: 100 Inventions that Shaped the World from the Airplane to the Zipper. NYU Press.Google Scholar
63. Hector Vazquez-Leal, Roberto Castaneda-Sheissa, Uriel Filobello-Nino, Arturo Sarmiento-Reyes, and Jesus Sanchez Orea. 2012. High Accurate Simple Approximation of Normal Distribution Integral. Mathematical Problems in Engineering 2012 (2012).Google Scholar
64. Qinglong Wang, Xiangshi Ren, Sayan Sarcar, and Xiaoying Sun. 2016. EV-Pen: Leveraging Electrovibration Haptic Feedback in Pen Interaction. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS ’16). ACM, New York, NY, USA, 57–66. Google ScholarDigital Library
65. Rhiannon Williams. 2015. Jony Ive interview: The story of the Apple Pencil. http://www.telegraph.co.uk/technology/apple/11988396/Jony-Ive-interview-The-story-of-the-Apple-Pencil.html. (2015). accessed on: 8 January 2018.Google Scholar
66. Josh Wills, Sameer Agarwal, David Kriegman, and Serge Belongie. 2009. Toward a perceptual space for gloss. ACM Transactions on Graphics 28, 4 (2009), 1–15. Google ScholarDigital Library
67. Takashi Yoshioka, Sliman Bensmaïa, James C. Craig, and Shen-Fu. Hsiao. 2007. Texture perception through direct and indirect touch: An analysis of perceptual space for tactile textures in two modes of exploration. Somatosensory & Motor Research 24, 1–2 (2007), 53–70.Google ScholarCross Ref
68. Takashi Yoshioka and Julia Zhou. 2009. Factors Involved in Tactile Texture Perception Through Probes. Advanced Robotics 23, 6 (2009), 747–766.Google ScholarCross Ref
