“Sign Language Subtitling” by Adamo-Villani and Beni
Conference:
Type(s):
Title:
- Sign Language Subtitling
Presenter(s)/Author(s):
Abstract:
The object of this paper is the development of a new method of sign language subtitling for motion pictures aimed at deaf children who cannot read English yet and can communicate only via signs. The method is based on the recently introduced concept of “semantroid” (an avatar limited to head and hands) and on the implementation of a new scrolling technique which allows for concurrent display of 4 subtitling windows at the bottom of the screen. To maximize the readability of the semantroid’s face and hand configurations, we have created a new 3D model and we have shaded it with a combination of 3D and toon shaders. In addition, we have followed recent results in color perception to optimize the visibility of those body parts directly involved in the signing motion. The signing semantroid, optimized for maximum readability, can be scaled to fit in a very small area, and thus it is possible to display four captioning windows simultaneously. The concurrent display of several progressive animated signed sentences allows for review of the information, a feature not present in any sign language subtitling method presented so far. As an example of application, we have used the subtitling scrolling semantroid in a video of a chemistry experiment and we have tested its effectiveness with a group of hearing and non-hearing signers.
Acknowledgements:
This research is partially supported by the School of Technology at Purdue University (I3 grant – Proposal #00006585 – http://www.tech.purdue.edu/cgt/I3/), by the Envision Center for Data Perceptualization, and by gh, LLC Inc., IN. We thank Professor Ronnie Wilbur from the Department of Audiology and Speech Sciences at Purdue University for her valuable contributions to the project. We also thank Marie Nadolske for performing the signs and for her continuous help and feedback on the project. In addition, we are grateful to all the signers who have participated in the evaluation of the method and to the Indiana School for the Deaf (ISD) for providing the testing ground for a thorough evaluation of the ASL subtitling technique.
