“Learning to group discrete graphical patterns”
Conference:
Type(s):
Title:
- Learning to group discrete graphical patterns
Session/Category Title: Learning Geometry
Presenter(s)/Author(s):
Abstract:
We introduce a deep learning approach for grouping discrete patterns common in graphical designs. Our approach is based on a convolutional neural network architecture that learns a grouping measure defined over a pair of pattern elements. Motivated by perceptual grouping principles, the key feature of our network is the encoding of element shape, context, symmetries, and structural arrangements. These element properties are all jointly considered and appropriately weighted in our grouping measure. To better align our measure with human perceptions for grouping, we train our network on a large, human-annotated dataset of pattern groupings consisting of patterns at varying granularity levels, with rich element relations and varieties, and tempered with noise and other data imperfections. Experimental results demonstrate that our deep-learned measure leads to robust grouping results.
References:
1. Sawsan AlHalawani, Yongliang Yang, Han Liu, and Niloy J. Mitra. 2013. Interactive Facades: Analysis and Synthesis of Semi-Regular Facades. Computer Graphics Forum (Eurographics) (2013), to appear.
2. Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. 2011. Contour detection and hierarchical image segmentation. IEEE transactions on pattern analysis and machine intelligence 33, 5 (2011), 898–916.
3. Fan Bao, Michael Schwarz, and Peter Wonka. 2013. Procedural facade variations from a single layout. ACM Trans. on Graph 32, 1 (2013).
4. Connelly Barnes, Eli Shechtman, Dan B. Goldman, and Adam Finkelstein. 2010. The Generalized Patchmatch Correspondence Algorithm. In Proc. of ECCV.
5. M. Bokeloh, M. Wand, and H.-P. Seidel. 2010. A Connection between Partial Symmetry and Inverse Procedural Modeling. ACM Trans. on Graph 29, 4 (2010), 104:1–104:10.
6. Yilan Chen, Hongbo Fu, and Kin Chung Au. 2016. A Multi-level Sketch-based Interface for Decorative Pattern Exploration. In SIGGRAPH ASIA 2016 Technical Briefs (SA ’16). Article 26, 4 pages.
7. John H. Conway, Heidi Burgiel, and Chaim Goodman-Strauss. 2008. The Symmetries of Things. A K Peters/CRC Press.
8. Agné Desolneux, Lionel Moisan, and Jean-Michel Morel. 2002. Gestalt theory and computer vision. Springer.
9. Jacob Feldman. 2003. Perceptual grouping by selection of a logically minimal model. Proc. of ICCV 55 (2003), 5–25.
10. Brendan J. Frey and Delbert Dueck. 2007. Clustering by passing messages between data points. Science 315 (2007), 2007. Cross Ref
11. John H. Graham, Shmuel Raz, Hagit Hel-Or, and Eviatar Nevo. 2010. Fluctuating Asymmetry: Methods, Theory, and Applications. Symmetry 2, 2 (2010), 466–540.
12. Klaus Greff, Antti Rasmus, Mathias Berglund, Tele Hao, Harri Valpola, and Juergen Schmidhuber. 2016. Tagger: Deep Unsupervised Perceptual Grouping. In Advances in Neural Information Processing Systems. 4484–4492.
13. Paul Guerrero, Gilbert Bernstein, Wilmot Li, and Niloy J. Mitra. 2016. PATEX: Exploring Pattern Variations. ACM Trans. on Graph 35, 4, Article 48 (July 2016), 48:1–48:13 pages.
14. Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006. Dimensionality Reduction by Learning an Invariant Mapping. In Proc. CVPR.
15. Xufeng Han, T. Leung, Y. Jia, R. Sukthankar, and A. C. Berg. 2015. MatchNet: Unifying feature and metric learning for patch-based matching. In Proc. of CVPR.
16. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2001. The Elements of Statistical Learning. Springer New York Inc., New York, NY, USA.
17. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. CVPR (2017).
18. Phillip Isola, Daniel Zoran, Dilip Krishnan, and Edward H Adelson. 2016. Learning visual groups from co-occurrences in space and time. International Conference on Learning Representations, Workshop paper (2016).
19. Gaetano Kanizsa. 1980. Grammatica del Vedere. Il Mulino.
20. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014).
21. W. Köhler. 1929. Gestalt Psychology. Liveright, New York.
22. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proc. NIPS.
23. Michael Kubovy and Martin van den Berg. 2008. The whole is equal to the sum of its parts: A probabilistic model of grouping by proximity and similarity in regular patterns. Psychological Review 115, 1 (2008), 131–154. Cross Ref
24. Yanxi Liu, Hagit Hel-Or, Craig S. Kaplan, and Luc J. Van Gool. 2010. Computational Symmetry in Computer Vision and Computer Graphics. Foundations and Trends in Computer Graphics and Vision, Vol. 5. 1–195. Cross Ref
25. Michal Lukac, Daniel Sykora, Kalyan Sunkavalli, Eli Shechtman, Ondrej Jamriska, Nathan Carr, and Tomas Pajdla. 2017. Nautilus: Recovering Regional Symmetry Transformations for Image Editing. ACM Trans. on Graph to appear (2017).
26. Michael Maire, Takuya Narihira, and Stella X Yu. 2016. Affinity CNN: Learning pixel-centric pairwise relations for figure/ground embedding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 174–182. Cross Ref
27. Kevis-Kokitsi Maninis, Jordi Pont-Tuset, Pablo Arbeláez, and Luc Van Gool. 2016. Convolutional oriented boundaries. In European Conference on Computer Vision. 580–596. Cross Ref
28. Niloy J. Mitra, Mark Pauly, Michael Wand, and Duygu Ceylan. 2012. Symmetry in 3D Geometry: Extraction and Applications. In Proc. of Eurographics STAR Report.
29. Liangliang Nan, Andrei Sharf, Ke Xie, Tien-Tsin Wong, Oliver Deussen, Daniel Cohen-Or, and Baoquan Chen. 2011. Conjoining Gestalt Rules for Abstraction of Architectural Drawings. ACM Trans. on Graph 30, 6 (2011). 185:1–185:10.
30. S. Palmer. 1992. Common region: a new principle of perceptual grouping. Cognitive Psychology 24 (1992), 436–447. Cross Ref
31. Stephen E. Palmer. 1977. Hierarchical structure in perceptual representation. Cognitive Psychology 9, 4 (1977), 441–474. Cross Ref
32. Joshua Podolak, Philip Shilane, Aleksey Golovinskiy, Szymon Rusinkiewicz, and Thomas Funkhouser. 2006. A Planar-reflective Symmetry Transform for 3D Shapes. ACM Trans. on Graph 25, 3 (2006).
33. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) 115, 3 (2015), 211–252.
34. Jianbo Shi and Jitendra Malik. 2000. Normalized Cuts and Image Segmentation. IEEE Trans. Pat. Ana. & Mach. Int. (2000).
35. Patricio Simari, Evangelos Kalogerakis, and Karan Singh. 2006. Folding meshes: hierarchical mesh segmentation based on planar symmetry. Symp. on Geom. Proc. (2006), 111–119.
36. Edgar Simo-Serra, Eduard Trulls, Luis Ferraz, Iasonas Kokkinos, Pascal Fua, and Francesc Moreno-Noguer. 2015. Discriminative Learning of Deep Convolutional Feature Point Descriptors. In Proc. of ICCV.
37. O. Stava, B. Benes, R. Mech, D. Aliga, and P. Kristof. 2010. Inverse Procedural Modeling by Automatic Generation of L-systems. Computer Graphics Forum 29, 2 (2010), 665–674.
38. Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik G. Learned-Miller. 2015. Multi-view Convolutional Neural Networks for 3D Shape Recognition. In Proc. ICCV.
39. Oliver van Kaick, Kai Xu, Hao Zhang, Yanzhen Wang, Shuyang Sun, Ariel Shamir, and Daniel Cohen-Or. 2013. Co-Hierarchical Analysis of Shape Structures. ACM Trans. on Graph 32, 4 (2013), Article 69.
40. Kaijun Wang. 2010. Fast Affinity Propagation Clustering under Given Number of Clusters. (2010). https://www.mathworks.com/matlabcentral/fileexchange/25722-fast-affinity-propagation-clustering-under-given-number-of-clusters
41. Yanzhen Wang, Kai Xu, Jun Li, Hao Zhang, Ariel Shamir, Ligang Liu, Zhiquan Cheng, and Yueshan Xiong. 2011. Symmetry Hierarchy of Man-Made Objects. Computer Graphics Forum (Eurographics) 30, 2 (2011), 287–296. Cross Ref
42. M. Wertheimer. 1938. Laws of organization in perceptual forms. 71–88.
43. H. Weyl. 1952. Symmetry. Princeton University Press.
44. Fuzhang Wu, Dong-Ming Yan, Weiming Dong, Xiaopeng Zhang, and Peter Wonka. 2014. Inverse procedural modeling of facade layouts. ACM Trans. on Graph 33, 4 (2014), 121:1–121:10.
45. Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised Deep Embedding for Clustering Analysis. In Proc. of ICML.
46. Pengfei Xu, Hongbo Fu, Chiew-Lan Tai, and Takeo Igarashi. 2015. GACA: Group-Aware Command-based Arrangement of Graphic Elements. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015, Seoul, Republic of Korea, April 18-23, 2015. 2787–2795.
47. Jianwei Yang, Devi Parikh, and Dhruv Batra. 2016. Joint Unsupervised Learning of Deep Representations and Image Clusters. In Proc. CVPR. Cross Ref
48. Yi-Ting Yeh, Katherine Breeden, Lingfeng Yang, Matthew Fisher, and Pat Hanrahan. 2013. Synthesis of Tiled Patterns Using Factor Graphs. ACM Trans. on Graph 32, 1, Article 3 (Feb. 2013), 3:1–3:13 pages.
49. Kwang Moo Yi, Eduard Trulls, Vincent Lepetit, and Pascal Fua. 2016. LIFT: Learned Invariant Feature Transform. In IEEE ECCV.
50. S. Zagoruyko and N. Komodakis. 2015. Learning to compare image patches via convolutional neural networks. In Proc. of CVPR. 4353–4361.
51. Hao Zhang, Kai Xu, Wei Jiang, Jinjie Lin, Daniel Cohen-Or, and Baoquan Chen. 2013. Layered Analysis of Irregular Facades via Symmetry Maximization. ACM Trans. on Graph 32, 4 (2013), Article 121.
52. Arthur Zimek, Erich Schubert, and Hans-Peter Kriegel. 2012. A Survey on Unsupervised Outlier Detection in High-dimensional Numerical Data. Stat. Anal. Data Min. 5, 5 (2012).
53. C. Lawrence Zitnick and Piotr Dollár. 2014. Edge Boxes: Locating Object Proposals from Edges. In Proc. of ECCV.
