“Scalable real-time volumetric surface reconstruction” by Chen, Bautembach and Izadi
Conference:
Type:
Title:
- Scalable real-time volumetric surface reconstruction
Session/Category Title: Surface Reconstruction
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We address the fundamental challenge of scalability for real-time volumetric surface reconstruction methods. We design a memory efficient, hierarchical data structure for commodity graphics hardware, which supports live reconstruction of large-scale scenes with fine geometric details. Our sparse data structure fuses overlapping depth maps from a moving depth camera into a single volumetric representation, from which detailed surface models are extracted. Our hierarchy losslessly streams data bidirectionally between GPU and host, allowing for unbounded reconstructions. Our pipeline, comprised of depth map post-processing, camera pose estimation, volumetric fusion, surface extraction, and streaming, runs entirely in real-time. We experimentally demonstrate that a shallow hierarchy with relatively large branching factors yields the best memory/speed tradeoff, consuming an order of magnitude less memory than a regular grid. We compare an implementation of our data structure to existing methods and demonstrate higher-quality reconstructions on a variety of large-scale scenes, all captured in real-time.
References:
1. Alliez, P., Cohen-Steiner, D., Tong, Y., and Desbrun, M. 2007. Voronoi-based variational reconstruction of unoriented point sets. In Proc. SGP 2007, vol. 257, Eurographics, 39–48. Google ScholarDigital Library
2. Amanatides, J., and Woo, A. 1987. A fast voxel traversal algorithm for ray tracing. In Proc. Eurographics, vol. 87, 3–10.Google Scholar
3. Besl, P., and McKay, N. 1992. A method for registration of 3-D shapes. IEEE Trans. PAMI 14, 2, 239–256. Google ScholarDigital Library
4. Bolitho, M., Kazhdan, M., Burns, R., and Hoppe, H. 2007. Multilevel streaming for out-of-core surface reconstruction. In Proc. SGP 2007, vol. 257, Eurographics, 69–78. Google ScholarDigital Library
5. Chang, C., Chatterjee, S., and Kube, P. R. 1994. A quantization error analysis for convergent stereo. In Proc. ICIP 94, vol. 2, IEEE, 735–739.Google Scholar
6. Chen, Y., and Medioni, G. 1992. Object modelling by registration of multiple range images. Image and vision computing 10, 3, 145–155. Google ScholarDigital Library
7. Chien, C., Sim, Y., and Aggarwal, J. 1988. Generation of volume/surface octree from range data. In Proc. CVPR 98, IEEE, 254–260.Google Scholar
8. Connolly, C. 1984. Cumulative generation of octree models from range data. In Proc. ICRA 84, vol. 1, IEEE, 25–32.Google ScholarCross Ref
9. Crassin, C., Neyret, F., Lefebvre, S., and Eisemann, E. 2009. GigaVoxels: Ray-guided streaming for efficient and detailed voxel rendering. In Proc. I3D 2009, ACM, 15–22. Google ScholarDigital Library
10. Cui, Y., Schuon, S., Chan, D., Thrun, S., and Theobalt, C. 2010. 3D shape scanning with a time-of-flight camera. In Proc. CVPR 2010, IEEE, 1173–1180.Google Scholar
11. Curless, B., and Levoy, M. 1996. A volumetric method for building complex models from range images. In Proceedings of SIGGRAPH 96, Annual Conference Series, 303–312. Google ScholarDigital Library
12. Frisken, S., Perry, R., Rockwood, A., and Jones, T. 2000. Adaptively sampled distance fields: a general representation of shape for computer graphics. In Proceedings of SIGGRAPH 2000, Annual Conference Series, 249–254. Google ScholarDigital Library
13. Fuhrmann, S., and Goesele, M. 2011. Fusion of depth maps with multiple scales. ACM Trans. Graph. 30, 6 (December), 148:1–148:8. Google ScholarDigital Library
14. Greene, N., Kass, M., Miller, G., et al. 1993. Hierarchical z-buffer visibility. In Proceedings of SIGGRAPH 93, Annual Conference Series, 231–238. Google ScholarDigital Library
15. Henry, P., Krainin, M., Herbst, E., Ren, X., and Fox, D. 2010. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. In Proc. ISER, vol. 20, 22–25.Google Scholar
16. Higuchi, K., Hebert, M., and Ikeuchi, K. 1995. Building 3-d models from unregistered range images. Graphical models and image processing 57, 4, 315–333. Google ScholarDigital Library
17. Hilton, A., Stoddart, A., Illingworth, J., and Windeatt, T. 1996. Reliable surface reconstruction from multiple range images. Computer Vision (Proc. ECCV 96), 117–126. Google ScholarDigital Library
18. Hilton, A., Stoddart, A. J., Illingworth, J., and Windeatt, T. 1998. Implicit surface-based geometric fusion. Computer Vision and Image Understanding 69, 3, 273–291. Google ScholarDigital Library
19. Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., and Stuetzle, W. 1992. Surface reconstruction from unorganized points. In Computer Graphics, vol. 26, 71–78. Google ScholarDigital Library
20. Hornung, A., Wurm, K. M., Bennewitz, M., Stachniss, C., and Burgard, W. 2013. OctoMap: An efficient probabilistic 3D mapping framework based on octrees. Autonomous Robots 34, 3, 189–206. Google ScholarDigital Library
21. Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. 2011. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In Proc. UIST 2011, ACM, 559–568. Google ScholarDigital Library
22. Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In Proc. SGP 2006, vol. 256, Eurographics, 61–70. Google ScholarDigital Library
23. Laine, S., and Karras, T. 2010. Efficient sparse voxel octrees. In Proc. I3D 2010, ACM, 55–63. Google ScholarDigital Library
24. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., et al. 2000. The digital michelangelo project: 3D scanning of large statues. In Proceedings of SIGGRAPH 2000, Annual Conference Series, 131–144. Google ScholarDigital Library
25. Lorensen, W., and Cline, H. 1987. Marching cubes: A high resolution 3d surface construction algorithm. In Computer Graphics, vol. 21, 163–169. Google ScholarDigital Library
26. Musialski, P., Wonka, P., Aliaga, D., Wimmer, M., van Gool, L., Purgathofer, W., Mitra, N., Pauly, M., Wand, M., Ceylan, D., et al. 2012. A survey of urban reconstruction. In Proc. Eurographics 2012 STARs, Eurographics, 1–28.Google Scholar
27. Newcombe, R., and Davison, A. J. 2010. Live dense reconstruction with a single moving camera. In Proc. CVPR 2010, IEEE, 1498–1505.Google Scholar
28. Newcombe, R., Lovegrove, S. J., and Davison, A. J. 2011. DTAM: Dense tracking and mapping in real-time. In Proc. ICCV 2011, IEEE, 2320–2327. Google ScholarDigital Library
29. Newcombe, R. A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohli, P., Shotton, J., Hodges, S., and Fitzgibbon, A. 2011. KinectFusion: Real-time dense surface mapping and tracking. In Proc. ISMAR, IEEE, 127–136. Google ScholarDigital Library
30. Nguyen, C., Izadi, S., and Lovell, D. 2012. Modeling Kinect sensor noise for improved 3D reconstruction and tracking. In Proc. 3DIMPVT 2012, IEEE, 524–530. Google ScholarDigital Library
31. Osher, S., and Fedkiw, R. P. 2003. Level set methods and dynamic implicit surfaces. Applied mathematical science. Springer, New York, N.Y.Google Scholar
32. Parker, S., Shirley, P., Livnat, Y., Hansen, C., and Sloan, P.-P. 1998. Interactive ray tracing for isosurface rendering. In Proc. Visualization 98, IEEE, 233–238. Google ScholarDigital Library
33. Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., and Koch, R. 2004. Visual modeling with a hand-held camera. IJCV 2004 59, 3, 207–232. Google ScholarDigital Library
34. Pollefeys, M., Nistér, D., Frahm, J., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S., Merrell, P., et al. 2008. Detailed real-time urban 3D reconstruction from video. IJCV 2008 78, 2, 143–167. Google ScholarDigital Library
35. Potmesil, M. 1987. Generating octree models of 3d objects from their silhouettes in a sequence of images. Computer Vision, Graphics, and Image Processing 40, 1, 1–29. Google ScholarDigital Library
36. Roth, H., and Vona, M. 2012. Moving volume KinectFusion. In Proc. BMVC 2012, BMVA Press, 112.1–112.11.Google Scholar
37. Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3D model acquisition. ACM Trans. Graph. 21, 3 (July), 438–446. Google ScholarDigital Library
38. Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proc. CVPR 2006, vol. 1, IEEE, 519–528. Google ScholarDigital Library
39. Stückler, J., and Behnke, S. 2012. Robust real-time registration of RGB-D images using multi-resolution surfel representations. In Proc. ROBOTIK 2012, VDE, 1–4.Google Scholar
40. Szeliski, R. 1993. Rapid octree construction from image sequences. CVGIP Image Understanding 58, 23–23. Google ScholarDigital Library
41. Thrun, S., Burgard, W., Fox, D., et al. 2005. Probabilistic Robotics. MIT Press, Cambridge, MA.Google Scholar
42. Turk, G., and Levoy, M. 1994. Zippered polygon meshes from range images. In Proceedings of SIGGRAPH 94, Annual Conference Series, 311–318. Google ScholarDigital Library
43. Weise, T., Wismer, T., Leibe, B., and Van Gool, L. 2009. In-hand scanning with online loop closure. In Proc. ICCV 2009 Workshops, IEEE, 1630–1637.Google ScholarCross Ref
44. Wheeler, M., Sato, Y., and Ikeuchi, K. 1998. Consensus surfaces for modeling 3D objects from multiple range images. In Proc. ICCV 98, IEEE, 917–924. Google ScholarDigital Library
45. Whelan, T., Kaess, M., Fallon, M., Johannsson, H., Leonard, J., and McDonald, J. 2012. Kintinuous: Spatially extended KinectFusion. In Proc. RSS Workshop on RGB-D: Advanced Reasoning with Depth Cameras.Google Scholar
46. Zach, C., Pock, T., and Bischof, H. 2007. A globally optimal algorithm for robust TV-L1 range image integration. In Proc. ICCV 2007, IEEE, 1–8.Google Scholar
47. Zeng, M., Zhao, F., Zheng, J., and Liu, X. 2013. Octree-based fusion for realtime 3d reconstruction. Graphical Models 75, 3 (May), 126–136. Google ScholarDigital Library
48. Zhou, K., Gong, M., Huang, X., and Guo, B. 2011. Data-parallel octrees for surface reconstruction. IEEE Trans. Visualization and Computer Graphics 17, 5, 669–681. Google ScholarDigital Library
