“A Framework for content-adaptive photo manipulation macros: Application to face, landscape, and global manipulations” by Berthouzoz, Li, Dontcheva and Agrawala

We present a framework for generating content-adaptive macros that can transfer complex photo manipulations to new target images. We demonstrate applications of our framework to face, landscape, and global manipulations. To create a content-adaptive macro, we make use of multiple training demonstrations. Specifically, we use automated image labeling and machine learning techniques to learn the dependencies between image features and the parameters of each selection, brush stroke, and image processing operation in the macro. Although our approach is limited to learning manipulations where there is a direct dependency between image features and operation parameters, we show that our framework is able to learn a large class of the most commonly used manipulations using as few as 20 training demonstrations. Our framework also provides interactive controls to help macro authors and users generate training demonstrations and correct errors due to incorrect labeling or poor parameter estimation. We ask viewers to compare images generated using our content-adaptive macros with and without corrections to manually generated ground-truth images and find that they consistently rate both our automatic and corrected results as close in appearance to the ground truth. We also evaluate the utility of our proposed macro generation workflow via a small informal lab study with professional photographers. The study suggests that our workflow is effective and practical in the context of real-world photo editing.

Amini, A., Curwen, R., and Gore, J. 1996. Snakes and splines for tracking non-rigid heart motion. In Proceedings of ECCV. 249–261. Google ScholarDigital Library
Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. In Proceedings of ACM Trans. Graph. 25, 3, 637–645. Google ScholarDigital Library
Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., and Nayar, S. 2008. Face swapping: Automatically replacing faces in photographs. Trans. graph. 27, 3. Google ScholarDigital Library
Bolin, M., Webber, M., Rha, P., Wilson, T., and Miller, R. C. 2005. Automation and customization of rendered web pages. In Proceedings of the UIST Symposium. 163–172. Google ScholarDigital Library
Cypher, A. and Halbert, D. 1993. Watch What I Do: Programming by Demonstration. MIT Press. Google ScholarDigital Library
Dewdney, A. 1989. A potpourri of programmed prose and prosody. Scientific Amer.Google Scholar
Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Example-based style synthesis. In In Proceedings of the Conference on Computer Vision and Pattern Recognition. 143–150.Google Scholar
Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. 2004. Least angle regression. In Annals of Statistics, 407–451.Google Scholar
Efros, A. and Freeman, W. 2001. Image quilting for texture synthesis and transfer. In Proceedings of the SIGGRAPH Conference. 341–346. Google ScholarDigital Library
Felzenszwalb, P., McAllester, D., and Ramanan, D. 2008. A discriminatively trained, multiscale, deformable part model. In Proceedings of the CVPR Conference.Google Scholar
Grabler, F., Agrawala, M., Li, W., Dontcheva, M., and Igarashi, T. 2009. Generating photo manipulation tutorials by demonstration. ACM Trans. Graph. 28, 3, 66. Google ScholarDigital Library
Guo, D. and Sim, T. 2009. Digital face makeup by example. In Proceedings of the Computer Vision and Pattern Recognition Conference. IEEE Computer Society, 73–79.Google Scholar
Hasinoff, S., Józwiak, M., Durand, F., and Freeman, W. 2010. Search-and-replace editing for personal photo collections. In Proceedings of the ICCP. 2. 8.Google Scholar
Hertzmann, A., Jacobs, C., Oliver, N., Curless, B., and Salesin, D. 2001. Image analogies. In Proceedings of the SIGGRAPH Conference. 327–340. Google ScholarDigital Library
Hertzmann, A., Oliver, N., Curless, B., and Seitz, S. 2002. Curve analogies. In Proceedings of the Eurographics Workshop on Rendering. 233–246. Google ScholarDigital Library
Hoiem, D., Efros, A., and Hebert, M. 2005. Geometric context from a single image. In Proceedings of the ICCV. 654–661. Google ScholarDigital Library
Huggins, B. 2005. Photoshop: Retouching Cookbook for Digital Photographers. O’Reilly. Google ScholarDigital Library
Jones, M. and Rehg, J. 2002. Statistical color models with application to skin detection. Int. J. Comput. Vision 46, 1, 81–96. Google ScholarDigital Library
Kalnins, R., Markosian, L., Meier, B., Kowalski, M., Lee, J., Davidson, P., Webb, M., Hughes, J., and Finkelstein, A. 2002. WYSIWYG NPR: Drawing strokes directly on 3D models. ACM Trans. Graph. 21, 3, 755–762. Google ScholarDigital Library
Kang, S., Kapoor, A., and Lischinski, D. 2010. Personalization of image enhancement. In Proceedings of the CVPR.Google Scholar
Kass, M., Witkin, A., and Terzopoulos, D. 1988. Snakes: Active contour models. Int. J. comput. Vis. 1, 4, 321–331.Google Scholar
Kelby, S. 2007. The Adobe Photoshop CS3 Book for Digital Photographers. Voices That Matter. Google ScholarDigital Library
Kurlander, D. and Feiner, S. 1992. A history-based macro by example system. In Proceedings of the UIST Symposium. 99–106. Google ScholarDigital Library
Lau, T., Bergman, L., Castelli, V., and Oblinger, D. 2004. Sheepdog: Learning procedures for technical support. In Proceedings of the IUI Conference. 109–116. Google ScholarDigital Library
Lewis, D. 1998. Naive (Bayes) at forty: The independence assumption in information retrieval. In Proceedings of the ECML Conference. 8, 4–15. Google ScholarDigital Library
Lieberman, H. 1993. Mondrian: A teachable graphical editor. In Watch What I Do: Programming by Demonstration, 341–358. Google ScholarDigital Library
Lieberman, H. 2001. Your Wish is My Command: Giving Users the Power to Instruct their Software. Morgan Kaufmann.Google Scholar
Little, G., Lau, T., Cypher, A., Lin, J., Haber, E., and Kandogan, E. 2007. Koala: Capture, share, automate, personalize business processes on the web. In Proceedings of the CHI. 943–946. Google ScholarDigital Library
Liu, Z., Shan, Y., and Zhang, Z. 2001. Expressive expression mapping with ratio images. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 276. Google ScholarDigital Library
Modugno, F. and Myers, B. 1994. Pursuit: Graphically representing programs in a demonstrational visual shell. In Proceedings of the CHI. 455–456. Google ScholarDigital Library
Nguyen, M., Lalonde, J., Efros, A., and De la Torre, F. 2008. Image-based shaving. Comput. Graph. Forum. 27, 627–635.Google ScholarCross Ref
Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 34–41. Google ScholarDigital Library
Schwarz, D. 2005. Current research in concatenative sound synthesis. In Proceedings of the ICMC. 9–12.Google Scholar
Simhon, S. and Dudek, G. 2003. Curve Synthesis from Learned Refinement Models. http://www.clm.mcgill.ca/saol/pubs/eq03.pdf.Google Scholar
Zhou, Y., Gu, L., and Zhang, H. 2003. Bayesian tangent shape model: Estimating shape and pose parameters via bayesian inference. In Proceedings of the CVPR Conference. 109–116. Google ScholarDigital Library