“Head-mounted Photometric Stereo for Performance Capture” by Jones, Fyffe, Yu, Ma, Busch, et al. …

  • ©Andrew Jones, Graham Fyffe, Xueming Yu, Wan-Chun Alex Ma, Jay Busch, Mark T. Bolas, and Paul E. Debevec

Title:


    Head-mounted Photometric Stereo for Performance Capture

Presenter(s):


Entry Number: 14


Description:


    Head-mounted cameras are an increasingly important tool for capturing an actor’s facial performance. Such cameras provide a fixed, unoccluded view of the face. The resulting imagery is useful for observing motion capture dots or as input to existing video analysis techniques. Unfortunately current systems are typically affected by ambient light and generally fail to record subtle 3D shape changes between expressions Artistic interventions is often required to cleanup and map the captured performance onto a virtual character. We have developed a system that augments a headmounted camera with LED-based photometric stereo. The system allows observation of the face independent of the ambient light and records per-pixel surface normal. Our data can be used to generate dynamic 3D geometry, for facial relighting, or as input of machine learning algorithms to accurately control an animated face.


Other Information:


    References

    COOTES, T. F., EDWARDS, G. J., AND TAYLOR, C. J. 2001. Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 6, 681–685.

    MALZBENDER, T., WILBURN, B., GELB, D., AND AMBRISCO, B. 2006. Surface enhancement using real-time photometric stereo and reflectance transformation. In Rendering Techniques 2006: 17th Eurographics Workshop on Rendering, 245–250.

    WOODHAM, R. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering 19, 1, 139–144.


Additional Images:

©Andrew Jones, Graham Fyffe, Xueming Yu, Wan-Chun Alex Ma, Jay Busch, Mark T. Bolas, and Paul E. Debevec ©Andrew Jones, Graham Fyffe, Xueming Yu, Wan-Chun Alex Ma, Jay Busch, Mark T. Bolas, and Paul E. Debevec ©Andrew Jones, Graham Fyffe, Xueming Yu, Wan-Chun Alex Ma, Jay Busch, Mark T. Bolas, and Paul E. Debevec ©Andrew Jones, Graham Fyffe, Xueming Yu, Wan-Chun Alex Ma, Jay Busch, Mark T. Bolas, and Paul E. Debevec ©Andrew Jones, Graham Fyffe, Xueming Yu, Wan-Chun Alex Ma, Jay Busch, Mark T. Bolas, and Paul E. Debevec

PDF:



Conference:


Overview Page:


Type: