“Holographic Sign Language Interpreters” by Yang, Mousas and Adamo-Villani

  • ©Fu-Chia Yang, Christos Mousas, and Nicoletta Adamo-Villani

Conference:


Type:


Entry Number: 08

Title:

    Holographic Sign Language Interpreters

Presenter(s)/Author(s):


Abstract:


    We describe the implementation of a prototype system of 3D holographic sign language interpreters. The signing avatars, observed through wearable Mixed Reality (MR) smartglasses (e.g., Microsoft HoloLens), translate speech to Signed Exact English (SEE) in real-time. Such a system can be used by deaf and hard of hearing students in the classroom or other contexts to remove current accessibility barriers.

References:


    1. N. Adamo-Villani and S. Anasingaraju. 2017. Holographic signing avatars for deaf education. In E-learning, E-education, and online training. 54–61.
    2. V. Constantinou, A. Ioannou, I. Klironomos, M. Antona, and C. Stephanidis. 2020. Technology support for the inclusion of deaf students in mainstream schools: a summary of research from 2007 to 2017. Universal Access in the Information Society 19, 1 (2020), 195–200.
    3. A. Dachis. 2016. HoloHear is like Google translate for deaf people.
    4. J. M. De Martino, I. R. Silva, C. Z. Bolognini, P. D. P. Costa, K. M. O. Kumada, L. C. Coradine, P. H. da S. Brito, W. M. do Amaral, Â. B. Benetti, E. T. Poeta, L. M. G. Angare, C. M. Ferreira, and D. F. De Conti. 2017. Signing avatars: making education more inclusive. Universal Access in the Information Society 16, 3 (2017), 793–808.
    5. S. Eden. 2020. Technology makes things possible: Improving the abilities of deaf and hard-of-hearing children with advanced technologies. The Oxford Handbook of deaf studies in learning and cognition (2020), 407–425.
    6. A. Miller, J. Malasig, B. Castro, V. L. Hanson, H. Nicolau, and A. Brandão. 2017. The use of smart glasses for lecture comprehension by deaf and hard of hearing students. In ACM CHI Conference Extended Abstracts on Human Factors in Computing Systems. 1909–1915.
    7. B. S. Parton. 2017. Glass vision 3D: digital discovery for the deaf. TechTrends 61, 2 (2017), 141–146.
    8. Lorna C. Quandt, Athena Willis, Melody Schwenk, Kaitlyn Weeks, and Ruthie Ferster. 2022. Attitudes Toward Signing Avatars Vary Depending on Hearing Status, Age of Signed Language Acquisition, and Avatar Type. Frontiers in Psychology 13 (2022).

ACM Digital Library Publication:


Overview Page: