“Hierarchical and view-invariant light field segmentation by maximizing entropy rate on 4D ray graphs” by Li and Heidrich
Conference:
Type(s):
Title:
- Hierarchical and view-invariant light field segmentation by maximizing entropy rate on 4D ray graphs
Session/Category Title: Photography in the Field
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Image segmentation is an important first step of many image processing, computer graphics, and computer vision pipelines. Unfortunately, it remains difficult to automatically and robustly segment cluttered scenes, or scenes in which multiple objects have similar color and texture. In these scenarios, light fields offer much richer cues that can be used efficiently to drastically improve the quality and robustness of segmentations.In this paper we introduce a new light field segmentation method that respects texture appearance, depth consistency, as well as occlusion, and creates well-shaped segments that are robust under view point changes. Furthermore, our segmentation is hierarchical, i.e. with a single optimization, a whole hierarchy of segmentations with different numbers of regions is available. All this is achieved with a submodular objective function that allows for efficient greedy optimization. Finally, we introduce a new tree-array type data structure, i.e. a disjoint tree, to efficiently perform submodular optimization on very large graphs. This approach is of interest beyond our specific application of light field segmentation.We demonstrate the efficacy of our method on a number of synthetic and real data sets, and show how the obtained segmentations can be used for applications in image processing and graphics.
References:
1. Radhakrishna Achanta, Pablo Marquez Neila, Pascal Fua, and Sabine Süsstrunk. 2018. Scale-Adaptive Superpixels. Color and Imaging Conference Final Program and Proceedings (2018).Google Scholar
2. Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk. 2012. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods. PAMI (2012).Google Scholar
3. Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. 2011. Contour Detection and Hierarchical Image Segmentation. PAMI (2011).Google Scholar
4. Alper Ayvaci and Stefano Soatto. 2012. Detachable Object Detection: Segmentation and Depth Ordering from Short-Baseline Video. PAMI (2012).Google Scholar
5. Sai Bi, Xiaoguang Han, and Yizhou Yu. 2015. An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition. TOG (2015).Google Scholar
6. Jiansheng Chen, Zhengqin Li, and Bo Huang. 2017. Linear Spectral Clustering Superpixel. TIP (2017).Google Scholar
7. Jingchun Cheng, Yi-Hsuan Tsai, Shengjin Wang, and Ming-Hsuan Yang. 2017. SegFlow: Joint Learning for Video Object Segmentation and Optical Flow. In ICCV.Google Scholar
8. Thomas M. Cover and Joy A. Thomas. 2006. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing).Google ScholarDigital Library
9. Donald G. Dansereau, Bernd Girod, and Gordon Wetzstein. 2019. LiFF: Light Field Features in Scale and Depth. In CVPR.Google Scholar
10. Donald G. Dansereau, Ian Mahon, Oscar Pizarro, and Stefan B Williams. 2011. Plenoptic flow: Closed-form visual odometry for light field cameras. In IROS.Google Scholar
11. Piotr Dollár and C Lawrence Zitnick. 2015. Fast edge detection using structured forests. PAMI (2015).Google Scholar
12. Pedro F. Felzenszwalb and Daniel P. Huttenlocher. 2004. Efficient Graph-Based Image Segmentation. IJCV (2004).Google Scholar
13. Brian Fulkerson, Andrea Vedaldi, and Stefano Soatto. 2009. Class segmentation and object localization with neighborhoods. In ICCV. 670–677.Google Scholar
14. Matthias Grundmann, Vivek Kwatra, Mei Han, and Irfan Essa. 2010. Efficient Hierarchical Graph Based Video Segmentation. In CVPR.Google Scholar
15. Matthieu Hog, Neus Sabater, and Christine Guillemot. 2016. Light Field Segmentation Using a Ray-Based Graph Structure. In ECCV.Google Scholar
16. Matthieu Hog, Neus Sabater, and Christine Guillemot. 2017. Super-rays for Efficient Light Field Processing. IEEE Journal of Selected Topics in Signal Processing (2017).Google Scholar
17. Katrin Honauer, Ole Johannsen, Daniel Kondermann, and Bastian Goldlücke. 2016. A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields. In ACCV.Google Scholar
18. Aaron Isaksen, Leonard McMillan, and Steven J Gortler. 2000. Dynamically reparameterized light fields. In SIGGRAPH.Google Scholar
19. Naeemullah Khan, Marei Algarni, Anthony Yezzi, and Ganesh Sundaramoorthi. 2015. Shape-tailored local descriptors and their application to segmentation and tracking. In CVPR.Google Scholar
20. Naeemullah Khan, Byung-Woo Hong, Anthony Yezzi, and Ganesh Sundaramoorthi. 2017. Coarse-to-Fine Segmentation with Shape-Tailored Continuum Scale Spaces. In CVPR.Google Scholar
21. Andreas Krause and Daniel Golovin. 2014. Submodular function maximization.Google Scholar
22. Jure Leskovec, Andreas Krause, Carlos Guestrin, Christos Faloutsos, Jeanne Van-Briesen, and Natalie Glance. 2007. Cost-effective outbreak detection in networks. In SIGKDD.Google Scholar
23. Marc Levoy and Pat Hanrahan. 1996. Light field rendering. In SIGGRAPH.Google Scholar
24. Nianyi Li, Jinwei Ye, Yu Ji, Haibin Ling, and Jingyi Yu. 2014. Saliency detection on light field. In CVPR.Google Scholar
25. Ming-Yu Liu, Oncel Tuzel, Srikumar Ramalingam, and Rama Chellappa. 2011. Entropy rate superpixel segmentation. In CVPR.Google Scholar
26. Ming-Yu Liu, Oncel Tuzel, Srikumar Ramalingam, and Rama Chellappa. 2014. Entropyrate clustering: Cluster analysis via maximizing a submodular function subject to a matroid constraint. PAMI (2014).Google Scholar
27. Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In CVPR.Google Scholar
28. Ehsan Miandji, Saghi Hajisharif, and Jonas Unger. 2019. A Unified Framework for Compression and Compressed Sensing of Light Fields and Light Field Videos. TOG (2019).Google Scholar
29. Hajime Mihara, Takuya Funatomi, Kenichiro Tanaka, Hiroyuki Kubo, Yasuhiro Mukaigawa, and Hajime Nagahara. 2016. 4D light field segmentation with spatial and angular consistencies. In ICCP.Google Scholar
30. Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz, and Pat Hanrahan. 2005. Light field photography with a hand-held plenoptic camera. (2005).Google Scholar
31. Sylvain Paris and Frédo Durand. 2007. A Topological Approach to Hierarchical Segmentation using Mean Shift. In ICCV.Google Scholar
32. Zhao Pei, Yanning Zhang, Tao Yang, Xiuwei Zhang, and Yee-Hong Yang. 2012. A novel multi-object detection method in complex scene using synthetic aperture imaging. Pattern Recognition (2012).Google Scholar
33. David Stutz, Alexander Hermans, and Bastian Leibe. 2018. Superpixels: An evaluation of the state-of-the-art. CVIU (2018).Google Scholar
34. Michael W Tao, Sunil Hadap, Jitendra Malik, and Ravi Ramamoorthi. 2013. Depth from combining defocus and correspondence using light-field cameras. In ICCV.Google Scholar
35. Michael Van den Bergh, Xavier Boix, Gemma Roig, Benjamin de Capitani, and Luc Van Gool. 2012. Seeds: Superpixels extracted via energy-driven sampling. In ECCV.Google Scholar
36. Kartik Venkataraman, Dan Lelescu, Jacques Duparré, Andrew McMahon, Gabriel Molina, Priyam Chatterjee, Robert Mullis, and Shree Nayar. 2013. Picam: An ultra-thin high performance monolithic camera array. TOG (2013).Google ScholarDigital Library
37. Ting-Chun Wang, Alexei A Efros, and Ravi Ramamoorthi. 2015a. Occlusion-Aware Depth Estimation Using Light-Field Cameras. In ICCV.Google Scholar
38. Wenguan Wang, Jianbing Shen, and F. Porikli. 2015b. Saliency-aware geodesic video object segmentation. In CVPR.Google Scholar
39. Sven Wanner, Christoph Straehle, and Bastian Goldluecke. 2013. Globally Consistent Multi-label Assignment on the Ray Space of 4D Light Fields. In CVPR.Google Scholar
40. Bennett Wilburn, Neel Joshi, Vaibhav Vaish, Eino-Ville Talvala, Emilio Antunez, Adam Barth, Andrew Adams, Mark Horowitz, and Marc Levoy. 2005. High performance imaging using large camera arrays. TOG (2005).Google Scholar
41. Gaochang Wu, Belen Masia, Adrian Jarabo, Yuchen Zhang, Liangyong Wang, Qionghai Dai, Tianyou Chai, and Yebin Liu. 2017. Light Field Image Processing: An Overview. IEEE Journal of Selected Topics in Signal Processing (2017).Google Scholar
42. Yichao Xu, Hajime Nagahara, Atsushi Shimada, and Rin-ichiro Taniguchi. 2015. TransCut: Transparent Object Segmentation From a Light-Field Image. In ICCV.Google Scholar
43. Kaan Yücer, Alexander Sorkine-Hornung, Oliver Wang, and Olga Sorkine-Hornung. 2016. Efficient 3D Object Segmentation from Densely Sampled Light Fields with Applications to 3D Reconstruction. TOG (2016).Google Scholar
44. Guangming Zang, Mohamed Aly, Ramzi Idoughi, Peter Wonka, and Wolfgang Heidrich. 2018a. Super-Resolution and Sparse View CT Reconstruction. In ECCV.Google Scholar
45. Guangming Zang, Ramzi Idouchi, Ran Tao, Gilles Lubineau, Peter Wonka, and Wolfgang Heidrich. 2018b. Space-time tomography for continuously deforming objects. TOG (2018).Google Scholar
46. Guangming Zang, Ramzi Idoughi, Ran Tao, Gilles Lubineau, Peter Wonka, and Wolfgang Heidrich. 2019. Warp-and-project Tomography for Rapidly Deforming Objects. TOG (2019).Google Scholar
47. Hao Zhu, Qi Zhang, and Qing Wang. 2017. 4D Light Field Superpixel and Segmentation. In CVPR.Google Scholar
ACM Digital Library Publication:
- Hierarchical and view-invariant light field segmentation by maximizing entropy rate on 4D ray graphs
