“Look over here: attention-directing composition of manga elements” by Cao, Lau and Chan

  • ©Ying Cao, Rynson W. H. Lau, and Antoni B. Chan




    Look over here: attention-directing composition of manga elements

Session/Category Title: Typography & Illustration




    Picture subjects and text balloons are basic elements in comics, working together to propel the story forward. Japanese comics artists often leverage a carefully designed composition of subjects and balloons (generally referred to as panel elements) to provide a continuous and fluid reading experience. However, such a composition is hard to produce for people without the required experience and knowledge. In this paper, we propose an approach for novices to synthesize a composition of panel elements that can effectively guide the reader’s attention to convey the story. Our primary contribution is a probabilistic graphical model that describes the relationships among the artist’s guiding path, the panel elements, and the viewer attention, which can be effectively learned from a small set of existing manga pages. We show that the proposed approach can measurably improve the readability, visual appeal, and communication of the story of the resulting pages, as compared to an existing method. We also demonstrate that the proposed approach enables novice users to create higher-quality compositions with less time, compared with commercially available programs.


    1. Barnes, C., Goldman, D., Shechtman, E., and Finkelstein, A. 2010. Video tapestries with continuous temporal zoom. In ACM SIGGRAPH’10. Google ScholarDigital Library
    2. Bishop, C. 2006. Pattern Recognition and Machine Learning. Springer. Google Scholar
    3. Cao, Y., Chan, A., and Lau, R. 2012. Automatic stylistic manga layout. In ACM SIGGRAPH Asia’12. Google ScholarDigital Library
    4. Christensen, J., Marks, J., and Shieber, S. 1994. Placing text labels on maps and diagrams. Academic Press.Google Scholar
    5. Christensen, J., Marks, J., and Shieber, S. 1995. An empirical study of algorithms for point-feature label placement. ACM Trans. on Graphics 14, 203–232. Google ScholarDigital Library
    6. Chun, B., Ryu, D., Hwang, W., and Cho, H. 2006. An automated procedure for word balloon placement in cinema comics. LNCS 4292, 576–585. Google ScholarDigital Library
    7. ComiPo, 2012. http://www.comipo.com/en.Google Scholar
    8. Correa, C., and Ma, K.-L. 2010. Dynamic video narratives. In ACM SIGGRAPH’10. Google ScholarDigital Library
    9. DeCarlo, D., and Santella, A. 2002. Stylization and abstraction of photograph. In ACM SIGGRAPH’02. Google ScholarDigital Library
    10. Edmondson, S., Christensen, J., Marks, J., and Shieber, S. 1996. A general cartographic labelling algorithm. Computers & Geosciences 33, 13–24.Google Scholar
    11. Folse, S., 2010. http://hoodedutilitarian.com/visual-languages-of-manga-and-comics/.Google Scholar
    12. Friedman, N., and Nachman, D. 2000. Gaussian process networks. In Conf. on Uncertainty in Artificial Intelligence’00. Google ScholarDigital Library
    13. Huang, H., Zhang, L., and Zhang, H. 2011. Arcimboldo-like collage using internet images. In ACM SIGGRAPH Asia’11. Google ScholarDigital Library
    14. Jain, E., Sheikh, Y., and Hodgins, J. 2012. Inferring artistic intention in comic art through viewer gaze. In ACM SAP’12. Google ScholarDigital Library
    15. Jain, E. 2012. Attention-guided Algorithms to Retarget and Augment Animations, Stills and Videos. PhD thesis, Carnegie Mellon University. Google ScholarDigital Library
    16. Judd, T., Ehinger, K., Durand, F., and Torralba, A. 2009. Learning to predict where humans look. In ICCV’09.Google Scholar
    17. Kesting, M., 2004. Basic thoughts about visual composition. http://www.hippasus.com/resources/viscomp/index.html.Google Scholar
    18. Kim, J., and Pellacini, F. 2002. Jigsaw image mosaics. In ACM SIGGRAPH’02. Google ScholarDigital Library
    19. Kurlander, D., Skelly, T., and Salesin, D. 1996. Comic chat. In ACM SIGGRAPH’96. Google ScholarDigital Library
    20. MangaStudio, 2011. http://manga.smithmicro.com/.Google Scholar
    21. McCloud, S. 1994. Understanding Comics: The Invisible Art. William Morrow.Google Scholar
    22. McCloud, S. 2006. Making Comics: Storytelling Secrets of Comics, Manga and Graphic Novels. William Morrow.Google Scholar
    23. Murphy, K., 1998. A brief introduction to graphical models and bayesian networks. http://www.cs.ubc.ca/~murphyk/Bayes/bayes.html#reading.Google Scholar
    24. Murphy, K., 2002. Bayes net toolbox for matlab. https://code.google.com/p/bnt/.Google Scholar
    25. Omori, T., Igaki, T., Ishii, T., Kurata, K., and Masuda, N. 2004. Eye catchers in comics: Controlling eye movements in reading pictorial and textual media. In Int’l Congress of Psychology’04.Google Scholar
    26. Qu, Y., Pang, W., Wong, T., and Heng, P. 2006. Richness-preserving manga screening. In ACM SIGGRAPH’06. Google ScholarDigital Library
    27. Qu, Y., Wong, T., and Heng, P. 2008. Manga colorization. In ACM SIGGRAPH Asia’08. Google ScholarDigital Library
    28. Ramanathan, S., Yanulevskaya, V., and Sebe, N. 2011. Can computers learn from humans to see better?: inferring scene semantics from viewers’ eye movements. In ACM MM’11. Google ScholarDigital Library
    29. Rasmussen, C., and Williams, C., 2013. Gaussian processes for machine learning matlab code. http://www.gaussianprocess.org/gpml/code/matlab/doc/.Google Scholar
    30. Rother, C., Bordeaux, L., Hamadi, Y., and Blake, A. 2006. Autocollage. In ACM SIGGRAPH’06. Google ScholarDigital Library
    31. Sakoe, H., and Chiba, S. 1978. Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. ASSP 26, 43–49.Google ScholarCross Ref
    32. SamuraiMangaWorkshop, 2011. Manga artist interview – shin shinmoto – 1/2. https://www.youtube.com/watch?v=fxwcHnYcgLo.Google Scholar
    33. Scott-Baron, H. 2006. Manga Clip Art: Everything You Need to Create Your Own Professional-Looking Manga Artwork. Andrews McMeel Publishing.Google Scholar
    34. Toyoura, M., Sawada, T., Kunihiro, M., and Mao, X. 2012. Using eye-tracking data for automatic film comic creation. In Symp. on Eye Tracking Research and Applications’12. Google ScholarDigital Library
    35. Yuille, A., 2006. The EM algorithm. http://www.stat.ucla.edu/~yuille/courses/stat153/emtutorial.pdf.Google Scholar

ACM Digital Library Publication:

Overview Page: