“DeforMe: Projection-Based Visualization of Deformable Surfaces Using Invisible Textures” by Punpongsanon, Iwai and Sato
Conference:
Experience Type(s):
Title:
- DeforMe: Projection-Based Visualization of Deformable Surfaces Using Invisible Textures
Organizer(s)/Presenter(s):
Description:
In this paper, we present DeforMe, a new projection-based mixed reality technique for augmenting deformable surfaces with deformation rendering graphics. DeforMe combines a geometry tracking method with a deformation reconstruction model to estimate the tangential deformation of a surface. The motions of the feature points are measured between two successive frames. Moreover, the system estimates the surface deformation on the basis of the moving least squares algorithm, and interpolates the deformation estimation result to the projected graphics. Users can interact in real time with the deformable object, while the realistic projected graphics deform according to the deformation of the surface. We aim to integrate our technique with various types of design-support and interactive applications in spatial augmented reality.
The goal of our project is to realize interactive deformation visualization of a deformable object through enhancing the limitation of current projection-based mixed reality (MR) technologies, which support only rigid objects. In particular, we focus on the real-time estimation of “planar deformation”, which is the tangential deformation of the surface such as a human skin, clay, and flubber, as realistically as possible. Once it is achieved, a user can interact with superimposed graphics directly through deforming the object, which is projected onto the surface to visualize the deformation. We also try to estimate subtle surface deformations so that we can visualize the self-dilation of a deformable object caused by the gravity force. Since deformable surfaces ubiquitously exist in our daily spaces, the proposed technique has a potential to be applied to various application fields, such as interactive clay toy, scientific visualization, user interface on user’s body, product design, and so on. For example, we can provide an interactive clay toy application where a user can create a figure by clay on which dynamic graphics is projected to enhance the appearance of the clay and interactive. As an example of the scientific visualization, we can support hydromechanics engineers to assess the deformation of a material from different aspects by interactively visualizing various physical properties one by one with projected imagery. Furthermore, we can create a novel ubiquitous user interface in which a user can manipulate a computer by touching and deforming his/her skin. In summary, our contribution is to expand the possibility of projection-based MR so that we can interact with digital information on immediate surfaces in our daily spaces.
References:
[1] Follmer, S., Leithinger, D., Olwal, A., Cheng, N., and Ishii, H. 2012. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In Proceedings of ACM symposium on User interface software and technology, ACM, New York, NY, USA, 519–528.
[2] Kakehi, Y., Jo, K., Sato, K., Minamizawa, K., Nii, H., Kawakami, N., Naemura, T., and Tachi, S. 2008. Forcetile: tabletop tangible interface with vision-based force distribution sensing. In Proceedings of ACM SIGGRAPH 2008 new tech demos, ACM, New York, NY, USA, 17: 1.
[3] Kamiyama, K., Kajimoto, H., Vlack, K., Kawakami, N., Mizota, T., and Tachi, S. 2004. Gelforce. In Proceedings of ACM SIGGRAPH 2004 Emerging technologies, ACM, New York, NY, USA, 5.
[4] Piper, B., Ratti, C., and Ishii, H. 2002. Illuminating clay: a 3-d tangible interface for landscape analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 355–362.
[5] Saakes, D., Inami, M., Igarashi, T., Koizumi, N., and Raskar, R. 2012. Shader printer. In Proceedings of ACM SIGGRAPH 2012 Emerging Technologies, ACM, New York, NY, USA, 18: 1.
[6] Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. ACM Transactions on Graphics 25, 3, 533–540.
[7] Steimle, J., Jordt, A., and Maes, P. 2013. Flexpad: highly flexible bending interactions for projected handheld displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 237–246.
