“A Framework for content-adaptive photo manipulation macros: Application to face, landscape, and global manipulations” by Berthouzoz, Li, Dontcheva and Agrawala

  • ©Floraine Berthouzoz, Wilmot Li, Mira Dontcheva, and Maneesh Agrawala




    A Framework for content-adaptive photo manipulation macros: Application to face, landscape, and global manipulations



    We present a framework for generating content-adaptive macros that can transfer complex photo manipulations to new target images. We demonstrate applications of our framework to face, landscape, and global manipulations. To create a content-adaptive macro, we make use of multiple training demonstrations. Specifically, we use automated image labeling and machine learning techniques to learn the dependencies between image features and the parameters of each selection, brush stroke, and image processing operation in the macro. Although our approach is limited to learning manipulations where there is a direct dependency between image features and operation parameters, we show that our framework is able to learn a large class of the most commonly used manipulations using as few as 20 training demonstrations. Our framework also provides interactive controls to help macro authors and users generate training demonstrations and correct errors due to incorrect labeling or poor parameter estimation. We ask viewers to compare images generated using our content-adaptive macros with and without corrections to manually generated ground-truth images and find that they consistently rate both our automatic and corrected results as close in appearance to the ground truth. We also evaluate the utility of our proposed macro generation workflow via a small informal lab study with professional photographers. The study suggests that our workflow is effective and practical in the context of real-world photo editing.


    Amini, A., Curwen, R., and Gore, J. 1996. Snakes and splines for tracking non-rigid heart motion. In Proceedings of ECCV. 249–261. Google ScholarDigital Library
    Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. In Proceedings of ACM Trans. Graph. 25, 3, 637–645. Google ScholarDigital Library
    Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., and Nayar, S. 2008. Face swapping: Automatically replacing faces in photographs. Trans. graph. 27, 3. Google ScholarDigital Library
    Bolin, M., Webber, M., Rha, P., Wilson, T., and Miller, R. C. 2005. Automation and customization of rendered web pages. In Proceedings of the UIST Symposium. 163–172. Google ScholarDigital Library
    Cypher, A. and Halbert, D. 1993. Watch What I Do: Programming by Demonstration. MIT Press. Google ScholarDigital Library
    Dewdney, A. 1989. A potpourri of programmed prose and prosody. Scientific Amer.Google Scholar
    Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Example-based style synthesis. In In Proceedings of the Conference on Computer Vision and Pattern Recognition. 143–150.Google Scholar
    Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. 2004. Least angle regression. In Annals of Statistics, 407–451.Google Scholar
    Efros, A. and Freeman, W. 2001. Image quilting for texture synthesis and transfer. In Proceedings of the SIGGRAPH Conference. 341–346. Google ScholarDigital Library
    Felzenszwalb, P., McAllester, D., and Ramanan, D. 2008. A discriminatively trained, multiscale, deformable part model. In Proceedings of the CVPR Conference.Google Scholar
    Grabler, F., Agrawala, M., Li, W., Dontcheva, M., and Igarashi, T. 2009. Generating photo manipulation tutorials by demonstration. ACM Trans. Graph. 28, 3, 66. Google ScholarDigital Library
    Guo, D. and Sim, T. 2009. Digital face makeup by example. In Proceedings of the Computer Vision and Pattern Recognition Conference. IEEE Computer Society, 73–79.Google Scholar
    Hasinoff, S., Józwiak, M., Durand, F., and Freeman, W. 2010. Search-and-replace editing for personal photo collections. In Proceedings of the ICCP. 2. 8.Google Scholar
    Hertzmann, A., Jacobs, C., Oliver, N., Curless, B., and Salesin, D. 2001. Image analogies. In Proceedings of the SIGGRAPH Conference. 327–340. Google ScholarDigital Library
    Hertzmann, A., Oliver, N., Curless, B., and Seitz, S. 2002. Curve analogies. In Proceedings of the Eurographics Workshop on Rendering. 233–246. Google ScholarDigital Library
    Hoiem, D., Efros, A., and Hebert, M. 2005. Geometric context from a single image. In Proceedings of the ICCV. 654–661. Google ScholarDigital Library
    Huggins, B. 2005. Photoshop: Retouching Cookbook for Digital Photographers. O’Reilly. Google ScholarDigital Library
    Jones, M. and Rehg, J. 2002. Statistical color models with application to skin detection. Int. J. Comput. Vision 46, 1, 81–96. Google ScholarDigital Library
    Kalnins, R., Markosian, L., Meier, B., Kowalski, M., Lee, J., Davidson, P., Webb, M., Hughes, J., and Finkelstein, A. 2002. WYSIWYG NPR: Drawing strokes directly on 3D models. ACM Trans. Graph. 21, 3, 755–762. Google ScholarDigital Library
    Kang, S., Kapoor, A., and Lischinski, D. 2010. Personalization of image enhancement. In Proceedings of the CVPR.Google Scholar
    Kass, M., Witkin, A., and Terzopoulos, D. 1988. Snakes: Active contour models. Int. J. comput. Vis. 1, 4, 321–331.Google Scholar
    Kelby, S. 2007. The Adobe Photoshop CS3 Book for Digital Photographers. Voices That Matter. Google ScholarDigital Library
    Kurlander, D. and Feiner, S. 1992. A history-based macro by example system. In Proceedings of the UIST Symposium. 99–106. Google ScholarDigital Library
    Lau, T., Bergman, L., Castelli, V., and Oblinger, D. 2004. Sheepdog: Learning procedures for technical support. In Proceedings of the IUI Conference. 109–116. Google ScholarDigital Library
    Lewis, D. 1998. Naive (Bayes) at forty: The independence assumption in information retrieval. In Proceedings of the ECML Conference. 8, 4–15. Google ScholarDigital Library
    Lieberman, H. 1993. Mondrian: A teachable graphical editor. In Watch What I Do: Programming by Demonstration, 341–358. Google ScholarDigital Library
    Lieberman, H. 2001. Your Wish is My Command: Giving Users the Power to Instruct their Software. Morgan Kaufmann.Google Scholar
    Little, G., Lau, T., Cypher, A., Lin, J., Haber, E., and Kandogan, E. 2007. Koala: Capture, share, automate, personalize business processes on the web. In Proceedings of the CHI. 943–946. Google ScholarDigital Library
    Liu, Z., Shan, Y., and Zhang, Z. 2001. Expressive expression mapping with ratio images. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 276. Google ScholarDigital Library
    Modugno, F. and Myers, B. 1994. Pursuit: Graphically representing programs in a demonstrational visual shell. In Proceedings of the CHI. 455–456. Google ScholarDigital Library
    Nguyen, M., Lalonde, J., Efros, A., and De la Torre, F. 2008. Image-based shaving. Comput. Graph. Forum. 27, 627–635.Google ScholarCross Ref
    Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 34–41. Google ScholarDigital Library
    Schwarz, D. 2005. Current research in concatenative sound synthesis. In Proceedings of the ICMC. 9–12.Google Scholar
    Simhon, S. and Dudek, G. 2003. Curve Synthesis from Learned Refinement Models. http://www.clm.mcgill.ca/saol/pubs/eq03.pdf.Google Scholar
    Zhou, Y., Gu, L., and Zhang, H. 2003. Bayesian tangent shape model: Estimating shape and pose parameters via bayesian inference. In Proceedings of the CVPR Conference. 109–116. Google ScholarDigital Library

ACM Digital Library Publication: