“StyleTune: Interactive Style Transfer Enhancement on Mobile Devices” by Buchheim, Reimann, Pasewaldt, Döllner and Trapp
Conference:
- SIGGRAPH 2021
-
More from SIGGRAPH 2021:
Title:
- StyleTune: Interactive Style Transfer Enhancement on Mobile Devices
Developer(s):
Description:
We present StyleTune, a mobile app for interactive style transfer enhancement that enables global and spatial control over stroke elements and can generate high fidelity outputs. The app uses adjustable neural style transfer (NST) networks to enable art-direction of stroke size and orientation in the output image. The implemented approach enables continuous and seamless edits through a unified stroke-size representation in the feature space of the style transfer network. StyleTune introduces a three-stage user interface, that enables users to first explore global stroke parametrizations for a chosen NST. They can then interactively locally retouch the stroke size and orientation using brush metaphors. Finally, high resolution outputs of 20 Megapixels and more can be obtained using a patch- based upsampling and local detail transfer approach, that transfers small-scale details such as paint-bristles and canvas structure. The app uses Apple’s CoreML and Metal APIs for efficient on-device processing.
Acknowledgements:
This work was partially funded by the German Federal Ministry of Education and Research (BMBF) through grants 01IS18092 (“md- ViPro”) and 01IS19006 (“KI-Labor ITSE”)
