“NICER: A New and Improved Consumed Endurance and Recovery Metric to Quantify Muscle Fatigue of Mid-air Interactions”
Conference:
Type(s):
Title:
- NICER: A New and Improved Consumed Endurance and Recovery Metric to Quantify Muscle Fatigue of Mid-air Interactions
Presenter(s)/Author(s):
Abstract:
Natural gestures are crucial for mid-air interactions, but predicting and managing muscle fatigue is challenging. We present a new computational model, NICER, which combines a correction term for 3D interaction and a recovery factor for a rest period to predict fatigue across a variety of gesture-based interactive applications.
References:
[1]
Ashrant Aryal, Ali Ghahramani, and Burcin Becerik-Gerber. 2017. Monitoring fatigue in construction workers using physiological measurements. Automation in Construction 82 (2017), 154–165.
[2]
Myroslav Bachynskyi, Antti Oulasvirta, Gregorio Palmas, and Tino Weinkauf. 2014. Is motion capture-based biomechanical simulation valid for hci studies? study and implications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 3215–3224.
[3]
Myroslav Bachynskyi, Gregorio Palmas, Antti Oulasvirta, and Tino Weinkauf. 2015. Informing the design of novel input methods with muscle coactivation clustering. ACM Transactions on Computer-Human Interaction (TOCHI) 21, 6 (2015), 1–25.
[4]
Kayne Barclay, Danny Wei, Christof Lutteroth, and Robert Sheehan. 2011. A quantitative quality model for gesture based user interfaces. In Proceedings of the 23rd Australian Computer-Human Interaction Conference. 31–39.
[5]
Gunnar Borg. 1998. Borg’s perceived exertion and pain scales. Human kinetics.
[6]
Doug A. Bowman, Ryan P. McMahan, and Eric D. Ragan. 2012. Questioning Naturalism in 3D User Interfaces. Commun. ACM 55, 9 (sep 2012), 78–88.
[7]
Don B Chaffin, Gunnar BJ Andersson, and Bernard J Martin. 2006. Occupational biomechanics. John wiley & sons.
[8]
Noshaba Cheema, Laura A Frey-Law, Kourosh Naderi, Jaakko Lehtinen, Philipp Slusallek, and Perttu H?m?l?inen. 2020. Predicting mid-air interaction movements and fatigue using deep reinforcement learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[9]
Noshaba Cheema, Rui Xu, Nam Hee Kim, Perttu H?m?l?inen, Vladislav Golyanik, Marc Habermann, Christian Theobalt, and Philipp Slusallek. 2023. Discovering Fatigued Movements for Virtual Character Animation. In SIGGRAPH Asia 2023 Conference Papers. 1–12.
[10]
Shaozhang Dai, Yi Li, Tim Dwyer, Barrett Ens, and Lonni Besan?on. 2024. Magic Portal Interaction to Support Precise Embodied Mid-air and Haptic Selection-at-a-distance. (2024).
[11]
Niklas Elmqvist, Andrew Vande Moere, Hans-Christian Jetter, Daniel Cernea, Harald Reiterer, and T J Jankun-Kelly. 2011. Fluid interaction for information visualization. Information Visualization 10 (2011), 327–340. Issue 4.
[12]
Jo?o Marcelo Evangelista Belo, Anna Maria Feit, Tiare Feuchtner, and Kaj Gr?nb?k. 2021. XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 290, 11 pages.
[13]
Mohd Farooq and Abid Ali Khan. 2012. Effect of elbow flexion, forearm rotation and upper arm abduction on MVC grip and grip endurance time. International Journal of Occupational Safety and Ergonomics 18, 4 (2012), 487–498.
[14]
Sue A Ferguson, W Gary Allread, Peter Le, Joseph D Rose, and William S Marras. 2011. Shoulder muscle oxygenation during repetitive tasks. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 55. SAGE Publications Sage CA: Los Angeles, CA, 1039–1041.
[15]
Lincoln E Ford, Alvin J Detterline, Kevin K Ho, and Wenyuan Cao. 2000. Gender-and height-related limits of muscle strength in world weightlifting champions. Journal of Applied Physiology 89, 3 (2000), 1061–1064.
[16]
Laura A Frey Law and Keith G Avin. 2010. Endurance time is joint-specific: a modelling and meta-analysis investigation. Ergonomics 53, 1 (2010), 109–129.
[17]
Laura A Frey-Law, John M Looft, and Jesse Heitsman. 2012. A three-compartment muscle fatigue model accurately predicts joint-specific maximum endurance times for sustained isometric tasks. Journal of biomechanics 45, 10 (2012), 1803–1808.
[18]
Laura A Frey-Law, Mitchell Schaffer, and Frank K Urban III. 2021. Muscle fatigue modelling: Solving for fatigue and recovery parameter values using fewer maximum effort assessments. International journal of industrial ergonomics 82 (2021), 103104.
[19]
Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
[20]
Kimberley Hayes, Judie R Walton, Zoltan L Szomor, and George AC Murrell. 2002. Reliability of 3 methods for assessing shoulder strength. Journal of Shoulder and Elbow surgery 11, 1 (2002), 33–39.
[21]
Juan David Hincapi?-Ramos, Xiang Guo, Paymahn Moghadasian, and Pourang Irani. 2014. Consumed Endurance: A Metric to Quantify Arm Fatigue of Mid-Air Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 1063–1072.
[22]
Sujin Jang, Wolfgang Stuerzlinger, Satyajit Ambike, and Karthik Ramani. 2017. Modeling cumulative arm fatigue in mid-air interaction based on perceived exertion and kinetics of arm motion. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 3328–3339.
[23]
Haiyan Jiang, Dongdong Weng, Xiaonuo Dongye, Le Luo, and Zhenliang Zhang. 2023. Commonsense Knowledge-Driven Joint Reasoning Approach for Object Retrieval in Virtual Reality. ACM Transactions on Graphics (TOG) 42, 6 (2023), 1–18.
[24]
Thomas Kosch, Jakob Karolus, Johannes Zagermann, Harald Reiterer, Albrecht Schmidt, and Pawe? W. Wo?niak. 2023. A Survey on Measuring Cognitive Workload in Human-Computer Interaction. Comput. Surveys (1 2023).
[25]
Bongshin Lee, Arjun Srinivasan, Petra Isenberg, and John Stasko. 2021. Post-WIMP Interaction for Information Visualization. Foundations and Trends? in Human-Computer Interaction 14 (2021), 1–95. Issue 1.
[26]
Yi Li, Robert Crowther, Jim Smiley, Tim Dwyer, Benjamin Tag, Pourang Irani, and Barrett Ens. 2023. Revisiting Consumed Endurance: A NICE Way to Quantify Shoulder Fatigue in Virtual Reality. In Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology. 1–10.
[27]
Jaime Lien, Nicholas Gillian, M Emre Karagozler, Patrick Amihood, Carsten Schwesig, Erik Olson, Hakim Raja, and Ivan Poupyrev. 2016. Soli: Ubiquitous gesture sensing with millimeter wave radar. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–19.
[28]
RS Lindle, EJ Metter, NA Lynch, J vL Fleg, JL Fozard, J Tobin, TA Roy, and BF Hurley. 1997. Age and gender comparisons of muscle strength in 654 women and men aged 20–93 yr. Journal of applied physiology 83, 5 (1997), 1581–1587.
[29]
Jing Z Liu, Robert W Brown, and Guang H Yue. 2002. A dynamical model of muscle activation, fatigue, and recovery. Biophysical journal 82, 5 (2002), 2344–2359.
[30]
Zhe Liu, Daniel Vogel, and James R Wallace. 2018. Applying the cumulative fatigue model to interaction on large, multi-touch displays. In Proceedings of the 7th ACM International Symposium on Pervasive Displays. 1–9.
[31]
Benjamin Long, Sue Ann Seah, Tom Carter, and Sriram Subramanian. 2014. Rendering volumetric haptic shapes in mid-air using ultrasound. ACM Transactions on Graphics (TOG) 33, 6 (2014), 1–10.
[32]
John M Looft and Laura A Frey-Law. 2020. Adapting a fatigue model for shoulder flexion fatigue: Enhancing recovery rate during intermittent rest intervals. Journal of biomechanics 106 (2020), 109762.
[33]
John M Looft, Nicole Herkert, and Laura Frey-Law. 2018. Modification of a three-compartment muscle fatigue model to predict peak torque decline during intermittent tasks. Journal of biomechanics 77 (2018), 16–25.
[34]
Liang Ma, Damien Chablat, Fouad Bennis, and Wei Zhang. 2009. A new simple dynamic muscle fatigue model and its validation. International journal of industrial ergonomics 39, 1 (2009), 211–220.
[35]
Liang Ma, Damien Chablat, Fouad Bennis, Wei Zhang, and Fran?ois Guillaume. 2010. A new muscle fatigue and recovery model and its ergonomics application in human simulation. Virtual and Physical Prototyping 5, 3 (2010), 123–137.
[36]
Ganesh R Naik. 2014. Applications, challenges, and advancements in electromyography signal processing. IGI Global.
[37]
Donald A. Norman. 1993. Things That Make Us Smart: Defending Human Attributes in the Age of the Machine. Addison-Wesley Longman Publishing Co., Inc., USA.
[38]
Antti Oulasvirta, Niraj Ramesh Dayama, Morteza Shiripour, Maximilian John, and Andreas Karrenbauer. 2020. Combinatorial optimization of graphical user interface designs. Proc. IEEE 108, 3 (2020), 434–464.
[39]
DM Pincivero, MK Timmons, and D Elsing. 2010. RPE angle effects in young and middle-aged adults. International journal of sports medicine (2010), 257–260.
[40]
Pierre Plantard, Edouard Auvinet, Anne-Sophie Le Pierres, and Franck Multon. 2015. Pose estimation with a kinect for ergonomic studies: Evaluation of the accuracy using a virtual mannequin. Sensors 15, 1 (2015), 1785–1803.
[41]
Ivan Poupyrev, Mark Billinghurst, Suzanne Weghorst, and Tadao Ichikawa. 1996. The go-go interaction technique: non-linear mapping for direct manipulation in VR. In Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology (Seattle, Washington, USA) (UIST ’96). Association for Computing Machinery, New York, NY, USA, 79–80.
[42]
Inmaculada Rodriguez, R Boulic, and D Meziat. 2002. A joint-level model of fatigue for the postural control of virtual humans. CONF (2002).
[43]
Harrison Jesse Smith and Michael Neff. 2018. Communication Behavior in Embodied Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12.
[44]
Benjamin Tag, Andrew W. Vargo, Aman Gupta, George Chernyshov, Kai Kunze, and Tilman Dingler. 2019. Continuous alertness assessments: Using EOG glasses to unobtrusively monitor fatigue levels in-the-wild. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, USA.
[45]
Jonathan Taylor, Lucas Bordeaux, Thomas Cashman, Bob Corish, Cem Keskin, Toby Sharp, Eduardo Soto, David Sweeney, Julien Valentin, Benjamin Luff, Arran Topalian, Erroll Wood, Sameh Khamis, Pushmeet Kohli, Shahram Izadi, Richard Banks, Andrew Fitzgibbon, and Jamie Shotton. 2016. Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences. ACM Trans. Graph. 35, 4, Article 143 (jul 2016), 12 pages.
[46]
Ana Villanueva, Sujin Jang, Wolfgang Stuerzlinger, Satyajit Ambike, and Karthik Ramani. 2023. Advanced modeling method for quantifying cumulative subjective fatigue in mid-air interaction. International Journal of Human-Computer Studies 169 (2023), 102931.
[47]
Junhong Wang, Shaoming Sun, and Yining Sun. 2021. A muscle fatigue classification model based on LSTM and improved wavelet packet threshold. Sensors 21, 19 (2021), 6369.
[48]
James Wickham, Tania Pizzari, Katie Stansfeld, Amanda Burnside, and Lyn Watson. 2010. Quantifying ‘normal’shoulder muscle activity during abduction. Journal of Electromyography and Kinesiology 20, 2 (2010), 212–222.
[49]
Ting Xia and Laura A Frey Law. 2008. A theoretical approach for modeling peripheral muscle fatigue and recovery. Journal of biomechanics 41, 14 (2008), 3046–3052.
[50]
Hayder A Yousif, Ammar Zakaria, Norasmadi Abdul Rahim, Ahmad Faizal Bin Salleh, Mustafa Mahmood, Khudhur A Alfarhan, Latifah Munirah Kamarudin, Syed Muhammad Mamduh, Ali Majid Hasan, and Moaid K Hussain. 2019. Assessment of muscles fatigue based on surface EMG signals using machine learning and statistical approaches: A review. In IOP conference series: materials science and engineering, Vol. 705. IOP Publishing, 012010.
[51]
Difeng Yu, Hai-Ning Liang, Xueshi Lu, Kaixuan Fan, and Barrett Ens. 2019. Modeling endpoint distribution of pointing selection tasks in virtual reality environments. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–13.
[52]
Yunxiang Zhang, Kenneth Chen, and Qi Sun. 2023. Toward Optimized VR/AR Ergonomics: Modeling and Predicting User Neck Muscle Contraction. In ACM SIGGRAPH 2023 Conference Proceedings. 1–12.
