“FaceDrive: Facial Expression Driven Operation to Control Virtual Supernumerary Robotic Arms” by Fukuoka, Verhulst, Nakamura, Takizawa, Masai, et al. …
Conference:
Experience Type(s):
Title:
- FaceDrive: Facial Expression Driven Operation to Control Virtual Supernumerary Robotic Arms
Presenter(s):
Description:
Supernumerary Robotic Arms (SRAs) can make physical activities easier, but require cooperation with the operator. To improve cooperation, we predict the operator’s intentions by using his/her Facial Expressions (FEs). We propose to map FEs to SRAs commands (e.g. grab, release). To measure FEs, we used a optical sensor-based approach (here inside a HMD), the sensors data are fed to a SVM classifying them in FEs. The SRAs can then carry out commands by predicting the operator’s FEs (and arguably, the operator’s intention). We made a Virtual reality Environment (VE) with SRAs and synchronizable avatar to investigate the most suitable mapping between FEs and SRAs. In SIGGRAPH Asia 2019, the user can manipulate virtual SRAs using his/her FEs.
References:
[1] M.Niwa, H.Iizuka, H.Ando, and T.Maeda. 2012. Tumori Control : Manipulate Robot with detection and transmission of an Archetype of Behavioral Intention. Transactions of the Virtual Reality Society of Japan 17, 1 (2012), 3–10. https://doi.org/10.18974/tvrsj.17.1_3
[2] K. Suzuki, F. Nakamura, J. Otsuka, K. Masai, Y. Itoh, Y. Sugiura, and M. Sugimoto. 2017. Recognition and mapping of facial expressions to avatar by embedded photo reflective sensors in head mounted display. In 2017 IEEE Virtual Reality (VR). 177–185. https://doi.org/10.1109/VR.2017.7892245
