“Dynamic Projection Mapping for Thin Plants using a Robust Tracking Method against Occlusion” by Sueyoshi and Morimoto
Conference:
Type(s):
Title:
- Dynamic Projection Mapping for Thin Plants using a Robust Tracking Method against Occlusion
Presenter(s):
Description:
We propose a system to generate an interactive projection mapping onto thin plants automatically. Recently, various objects have become targets for projection mapping (PM). For example, in the artwork “Projections in the Forest” [van Schoor and Mawad 2014], plants and creatures are the projection targets. In this video artwork, the bioluminescent-like plants and creatures in the forest were created by PM and then filmed and edited. However, this work required considerable time (six weeks) to register effect animations (effects) manually for targets in the forest. It is thus very time-consuming to create PM works, particularly for natural objects. Our system therefore registers projection areas on targets semi-automatically. When plants, our projection targets, move and deformed by wind or are touched by a hand, the system can track the targets. Therefore, this system enables dynamic PM (DPM) for thin plants such as leaves and flowers. Previous works [Bermano et al. 2017; Narita et al. 2015] applied pre-input 3D models of the targets and/or markers attached to the targets in advance. A previous DPM generation method for unknown shapes without such measures [Miyashita et al. 2018] required expensive hardware. Therefore, our system generates DPM semi-automatically using low-cost hardware with reduced prior input. The system enables DPM creation on plants by considering the user’s contact with the plants. The system calculates occlusion of the projection target. This calculation is robust against interaction because the occluded areas are recalculated by applying a rigid deformation. In addition, the projected effects can be changed interactively by using the occluded position with the contact time obtained using a capacitive sensor. The concept of our effects is to make plants themselves appear luminous, e.g., as if they were bioluminescent. The system generates effects automatically using the shape of the projection target (Fig. 1).
Acknowledgements:
This work was supported by JSPS KAKENHI Grant Number 18K11956. This work was also supported by HAYAO NAKAYAMA Foundation for Science & Technology and Culture.
