“Head-mounted photometric stereo for performance capture” by Jones, Fyffe, Yu, Ma, Busch, et al. …
Conference:
Type(s):
Title:
- Head-mounted photometric stereo for performance capture
Session/Category Title: Interaction Omelette
Presenter(s)/Author(s):
Abstract:
Head-mounted cameras are an increasingly important tool for capturing an actor’s facial performance. Such cameras provide a fixed, unoccluded view of the face. The resulting imagery is useful for observing motion capture dots or as input to existing video analysis techniques. Unfortunately current systems are typically affected by ambient light and generally fail to record subtle 3D shape changes between expressions Artistic interventions is often required to cleanup and map the captured performance onto a virtual character. We have developed a system that augments a head-mounted camera with LED-based photometric stereo. The system allows observation of the face independent of the ambient light and records per-pixel surface normal. Our data can be used to generate dynamic 3D geometry, for facial relighting, or as input of machine learning algorithms to accurately control an animated face.
References:
1. Cootes, T. F., Edwards, G. J., and Taylor, C. J. 2001. Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 6, 681–685.
2. Malzbender, T., Wilburn, B., Gelb, D., and Ambrisco, B. 2006. Surface enhancement using real-time photometric stereo and reflectance transformation. In Rendering Techniques 2006: 17th Eurographics Workshop on Rendering, 245–250.
3. Woodham, R. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering 19, 1, 139–144.
