“Probabilistic color-by-numbers: suggesting pattern colorizations using factor graphs” by Lin, Ritchie, Fisher and Hanrahan

  • ©Sharon Lin, Daniel Ritchie, Matthew Fisher, and Patrick (Pat) Hanrahan

Conference:


Type:


Title:

    Probabilistic color-by-numbers: suggesting pattern colorizations using factor graphs

Session/Category Title: Color & Compositing

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We present a probabilistic factor graph model for automatically coloring 2D patterns. The model is trained on example patterns to statistically capture their stylistic properties. It incorporates terms for enforcing both color compatibility and spatial arrangements of colors that are consistent with the training examples. Using Markov Chain Monte Carlo, the model can be sampled to generate a diverse set of new colorings for a target pattern. This general probabilistic framework allows users to guide the generated suggestions via conditional inference or additional soft constraints. We demonstrate results on a variety of coloring tasks, and we evaluate the model through a perceptual study in which participants judged sampled colorings to be significantly preferable to other automatic baselines.

References:


    1. Albers, J. 1963. The interaction of color. Art news 62, 1.Google Scholar
    2. Bribiesca, E. 1997. Measuring 2-d shape compactness using the contact perimeter. Computers & Mathematics with Applications 33, 11.Google ScholarCross Ref
    3. Carbonell, J., and Goldstein, J. 1998. The use of mmr, diversity-based reranking for reordering documents and producing summaries. In Proc. ACM SIGIR 1998. Google ScholarDigital Library
    4. Charpiat, G., Hofmann, M., and Schölkopf, B. 2008. Automatic image colorization via multimodal predictions. In Proc. ECCV 2008. Google ScholarDigital Library
    5. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. In Proc. SIGGRAPH 2006. Google ScholarDigital Library
    6. Datta, R., Joshi, D., Li, J., and Wang, J. Z. 2006. Studying aesthetics in photographic images using a computational approach. In Proc. ECCV 2006. Google ScholarDigital Library
    7. Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? In Proc. SIGGRAPH 2012. Google ScholarDigital Library
    8. Geyer, C. 1991. Markov chain monte carlo maximum likelihood. In Proc. of the 23rd Symposium on the Inteface: Computing Science and Statistics, 156–163.Google Scholar
    9. Hastings, W. K. 1970. Monte carlo sampling methods using markov chains and their applications. Biometrika 57, 1.Google ScholarCross Ref
    10. Heer, J., and Stone, M. 2012. Color naming models for color selection, image editing and palette design. In Proc. ACM CHI 2012. Google ScholarDigital Library
    11. Hinton, G. E. 2002. Training products of experts by minimizing contrastive divergence. Neural Computation. Google ScholarDigital Library
    12. Itten, J. 1974. The Art of Color: The Subjective Experience and Objective Rationale of Color. Wiley.Google Scholar
    13. Jain, A., Thormählen, T., Ritschel, T., and Seidel, H.-P. 2012. Material memex: automatic material suggestions for 3d objects. In Proc. SIGGRAPH Asia 2012. Google ScholarDigital Library
    14. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. In Proc. SIGGRAPH 2004. Google ScholarDigital Library
    15. Lin, S., and Hanrahan, P. 2013. Modeling how people extract color themes from images. In Proc. of CHI 2013, ACM, New York, NY, USA, CHI ’13. Google ScholarDigital Library
    16. Lübbe, E. 2010. Colours in the Mind – Colour Systems in Reality. Books on Demand.Google Scholar
    17. Mccallum, A., Schultz, K., and Singh, S. 2009. Factorie: Probabilistic programming via imperatively defined factor graphs. In Proc. NIPS 2009.Google Scholar
    18. Meier, B., Spalter, A., and Karelitz, D. 2004. Interactive color palette tools. Computer Graphics and Applications 24, 3. Google ScholarDigital Library
    19. Merrell, P., Schkufza, E., Li, Z., Agrawala, M., and Koltun, V. 2011. Interactive furniture layout using interior design guidelines. In Proc. SIGGRAPH 2011. Google ScholarDigital Library
    20. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E. 1953. Equation of State Calculations by Fast Computing Machines. The Journal of Chemical Physics 21, 6.Google ScholarCross Ref
    21. Munsell, A., and Birren, F. 1969. A Grammar of Color: A Basic Treatise on the Color System of Albert H. Munsell. Van Nostrand Reinhold.Google Scholar
    22. Neal, R. M. 2010. Mcmc using hamiltonian dynamics. In Handbook of Markov Chain Monte-Carlo (Steve Brooks, Andrew Gelman, Gailin Jones and Xiao-Li Meng, Eds.Google Scholar
    23. Nguyen, C. H., Ritschel, T., Myszkowski, K., Eisemann, E., and Seidel, H.-P. 2012. 3d material style transfer. Computer Graphics Forum. Google ScholarDigital Library
    24. O’Donovan, P., Agarwala, A., and Hertzmann, A. 2011. Color Compatibility From Large Datasets. ACM Transactions on Graphics 30, 4. Google ScholarDigital Library
    25. Palmer, S. E., and Schloss, K. B. 2010. An ecological valence theory of human color preference. Proceedings of the National Academy of Sciences 107, 19, 8877–8882.Google ScholarCross Ref
    26. Robert, C. P. 1995. Simulation of truncated normal variables. Statistics and Computing 5.Google Scholar
    27. Sauvaget, C., Manuel, S., Vittaut, J.-N., Suarez, J., and Boyer, V. 2010. Segmented images colorization using harmony. In Proc. of Signal-Image Technology and Internet-Based Systems (SITIS) 2010, vol. 1. Google ScholarDigital Library
    28. Sharma, G., Wu, W., and Dalal, E. N. 2005. The CIEDE2000 color-difference formula: implementation notes, supplementary test data, and mathematical observations. Color research and application 30, 1.Google Scholar
    29. Sutton, T., and Whelan, B. 2004. The Complete Color Harmony: Expert Color Information for Professional Color Results. Color Harmony Series. Quayside Publishing Group.Google Scholar
    30. Wang, B., Yu, Y., Wong, T.-T., Chen, C., and Xu, Y.-Q. 2010. Data-driven image color theme enhancement. In Proc. SIGGRAPH Asia 2010. Google ScholarDigital Library
    31. Welsh, T., Ashikhmin, M., and Mueller, K. 2002. Transferring color to greyscale images. In Proc. SIGGRAPH 2002. Google ScholarDigital Library
    32. Yeh, Y.-T., Breeden, K., Yang, L., Fisher, M., and Hanrahan, P. 2012. Synthesis of tiled patterns using factor graphs. ACM Transactions on Graphics. Google ScholarDigital Library
    33. Yeh, Y.-T., Yang, L., Watson, M., Goodman, N. D., and Hanrahan, P. 2012. Synthesizing open worlds with constraints using locally annealed reversible jump mcmc. In Proc. SIGGRAPH 2012. Google ScholarDigital Library
    34. Yu, L.-F., Yeung, S.-K., Terzopoulos, D., and Chan, T. F. 2012. Dressup!: outfit synthesis through automatic optimization. In Proc. SIGGRAPH Asia 2012. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: