“Imagining the unseen: stability-based cuboid arrangements for scene understanding” by Shao, Monszpart, Zheng, Koo, Xu, et al. …
Conference:
Type(s):
Title:
- Imagining the unseen: stability-based cuboid arrangements for scene understanding
Session/Category Title: Scenes, Syntax, Statistics and Semantics
Presenter(s)/Author(s):
Abstract:
Missing data due to occlusion is a key challenge in 3D acquisition, particularly in cluttered man-made scenes. Such partial information about the scenes limits our ability to analyze and understand them. In this work we abstract such environments as collections of cuboids and hallucinate geometry in the occluded regions by globally analyzing the physical stability of the resultant arrangements of the cuboids. Our algorithm extrapolates the cuboids into the un-seen regions to infer both their corresponding geometric attributes (e.g., size, orientation) and how the cuboids topologically interact with each other (e.g., touch or fixed). The resultant arrangement provides an abstraction for the underlying structure of the scene that can then be used for a range of common geometry processing tasks. We evaluate our algorithm on a large number of test scenes with varying complexity, validate the results on existing benchmark datasets, and demonstrate the use of the recovered cuboid-based structures towards object retrieval, scene completion, etc.
References:
1. Anand, A., Koppula, H. S., Joachims, T., and Saxena, A. 2011. Contextually guided semantic labeling and search for 3d point clouds. CoRR abs/1111.5358.
2. Arikan, M., Schwärzler, M., Flöry, S., Wimmer, M., and Maierhofer, S. 2013. O-snap: Optimization-based snapping for modeling architecture. ACM TOG 32, 1, 6:1–6:15.
3. Bao, Y., Chandraker, M., Lin, Y., and Savarese, S. 2013. Dense object reconstruction using semantic priors. In IEEE CVPR.
4. Blum, M., Griffith, A., and Neumann, B., 1970. A stability test for configurations of blocks. TR-AI Memo.
5. Davis, J., Marschner, S. R., Garr, M., and Levoy, M. 2002. Filling holes in complex surfaces using volumetric diffusion. In Proc. 3DPVT.
6. Fidler, S., Dickinson, S., and Urtasun, R. 2012. 3d object detection and viewpoint estimation with a deformable 3d cuboid model. In NIPS, 620–628.
7. Gibson, J. 1986. The Ecological Approach To Visual Perception {Paperback}. Pschology Press.
8. Guo, R., and Hoiem, D. 2013. Support surface prediction in indoor scenes. IEEE ICCV.
9. Gupta, A., Efros, A. A., and Hebert, M. 2010. Blocks world revisited: Image understanding using qualitative geometry and mechanics. In ECCV.
10. Gupta, A., Satkin, S., Efros, A. A., and Hebert, M. 2011. From 3D scene geometry to human workspace. In CVPR, 1961–1968.
11. Harary, G., Tal, A., and Grinspun, E. 2013. Context-based coherent surface completion. ACM TOG.
12. Hartley, E., Kermgard, B., Fried, D., Bowdish, J., Pero, L. D., and Barnard, K. 2012. Bayesian geometric modeling of indoor scenes. IEEE CVPR, 2719–2726.
13. Hedau, V., Hoiem, D., and Forsyth, D. 2010. Thinking inside the box: using appearance models and context based on room geometry. In ECCV, 224–237.
14. Hedau, V. 2012. Recovering free space of indoor scenes from a single image. In IEEE CVPR, 2807–2814.
15. Jia, Z., Gallagher, A., Saxena, A., and Chen, T. 2013. 3d-based reasoning with blocks, support, and stability. In IEEE CVPR, 1–8.
16. Jiang, H., and Xiao, J. 2013. A linear approach to matching cuboids in rgbd images. In IEEE CVPR.
17. Kim, Y. M., Mitra, N. J., Yan, D. M., and Guibas, L. 2012. Acquiring 3d indoor environments with variability and repetition. ACM SIGGRAPH Asia 31, 6, 138:1–138:11.
18. Kim, B.-S., Kohli, P., and Savarese, S. 2013. 3D scene understanding by Voxel-CRF. In Proceedings of the International Conference on Computer Vision.
19. Kim, Y. M., Mitra, N. J., Huang, Q., and Guibas, L. 2013. Guided real-time scanning of indoor objects. Computer Graphics Forum (Proc. Pacific Graphics) 32, 177–186.Cross Ref
20. Koppula, H., Anand, A., Joachims, T., and Saxena, A. 2011. Semantic labeling of 3d point clouds for indoor scenes. In NIPS.
21. Lafarge, F., and Alliez, P. 2013. Surface reconstruction through point set structuring. CGF.
22. Lee, D. C., Gupta, A., Hebert, M., and Kanade, T. 2010. Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces. In NIPS, vol. 24.
23. Leordeanu, M., and Hebert, M. 2005. A spectral technique for correspondence problems using pairwise constraints. In IEEE ICCV, 1482–1489.
24. Li, Y., Wu, X., Chrysanthou, Y., Sharf, A., Cohen-Or, D., and Mitra, N. J. 2011. Globfit: Consistently fitting primitives by discovering global relations. ACM SIGGRAPH 30, 4.
25. Mattausch, O., Panozzo, D., Mura, C., Sorkine-Hornung, O., and Pajarola, R. 2014. Object detection and classification from large-scale cluttered indoor scans. CGF Eurographics.
26. Nan, L., Xie, K., and Sharf, A. 2012. A search-classify approach for cluttered indoor scene understanding. ACM SIGGRAPH Asia 31, 6, 137:1–137:10.
27. Pauly, M., Mitra, N. J., Giesen, J., Gross, M., and Guibas, L. 2005. Example-based 3d scan completion. In SGP, 23–32.
28. Rother, C., Kolmogorov, V., and Blake, A. 2004. “grab-cut”: interactive foreground extraction using iterated graph cuts. ACM SIGGRAPH 23, 3, 309–314.
29. Schlecht, J., and Barnard, K. 2009. Learning models of object structure. In NIPS.
30. Schnabel, R., Wahl, R., and Klein, R. 2007. Efficient ransac for point-cloud shape detection. CGF 26, 2, 214–226.Cross Ref
31. Shao, T., Xu, W., Zhou, K., Wang, J., Li, D., and Guo, B. 2012. An interactive approach to semantic modeling of indoor scenes with an rgbd camera. ACM SIGGRAPH Asia 31, 6, 136:1–136:11.
32. Sharf, A., Alexa, M., and Cohen-Or, D. 2004. Context-based surface completion. In ACM SIGGRAPH, 878–887.
33. Shen, C.-H., Fu, H., Chen, K., and Hu, S.-M. 2012. Structure recovery by part assembly. ACM SIGGRAPH Asia 31, 6, 180:1–180:11.
34. Silberman, N., and Fergus, R. 2011. Indoor scene segmentation using a structured light sensor. In Proc. ICCV – Workshop on 3D Representation and Recognition, 601–608.
35. Silberman, N., Hoiem, D., Kohli, P., and Fergus, R. 2012. Indoor segmentation and support inference from rgbd images. In ECCV.
36. Umetani, N., Igarashi, T., and Mitra, N. J. 2012. Guided exploration of physically valid shapes for furniture design. ACM SIGGRAPH 31, 4, 86:1–86:11.
37. Whiting, E., Ochsendorf, J., and Durand, F. 2009. Procedural modeling of structurally-sound masonry buildings. ACM Trans. Graph. 28, 5 (Dec.), 112:1–112:9.
38. Xiao, J., Russell, B., and Torralba, A. 2012. Localizing 3D cuboids in single-view images. In NIPS. 755–763.
39. Xiong, X., and Huber, D. 2010. Using context to create semantic 3d models of indoor environments. In BMVC, 1–11.
40. Zheng, Q., Sharf, A., Wan, G., Li, Y., Mitra, N. J., Cohen-Or, D., and Chen, B. 2010. Non-local scan consolidation for 3d urban scenes. ACM SIGGRAPH 29, 4, 94:1–94:9.
41. Zheng, Y., Chen, X., Cheng, M.-M., Zhou, K., Hu, S.-M., and Mitra, N. J. 2012. Interactive images: Cuboid proxies for smart image manipulation. ACM SIGGRAPH 31, 4, 99:1–99:11.
42. Zheng, B., Zhao, Y., Yu, J. C., Ikeuchi, K., and Zhu, S.-C. 2013. Beyond point clouds: Scene understanding by reasoning geometry and physics. In IEEE CVPR.
