“Real-time 3D reconstruction at scale using voxel hashing” by Nießner, Zollhöfer, Izadi and Stamminger
Conference:
Type(s):
Title:
- Real-time 3D reconstruction at scale using voxel hashing
Session/Category Title: Geometry Ops
Presenter(s)/Author(s):
Abstract:
Online 3D reconstruction is gaining newfound interest due to the availability of real-time consumer depth cameras. The basic problem takes live overlapping depth maps as input and incrementally fuses these into a single 3D model. This is challenging particularly when real-time performance is desired without trading quality or scale. We contribute an online system for large and fine scale volumetric reconstruction based on a memory and speed efficient data structure. Our system uses a simple spatial hashing scheme that compresses space, and allows for real-time access and updates of implicit surface data, without the need for a regular or hierarchical grid data structure. Surface data is only stored densely where measurements are observed. Additionally, data can be streamed efficiently in or out of the hash table, allowing for further scalability during sensor motion. We show interactive reconstructions of a variety of scenes, reconstructing both fine-grained details and large scale environments. We illustrate how all parts of our pipeline from depth map pre-processing, camera pose estimation, depth map fusion, and surface rendering are performed at real-time rates on commodity graphics hardware. We conclude with a comparison to current state-of-the-art online systems, illustrating improved performance and reconstruction quality.
References:
1. Alcantara, D. A., Sharf, A., Abbasinejad, F., Sengupta, S., Mitzenmacher, M., Owens, J. D., and Amenta, N. 2009. Real-time parallel hashing on the GPU. ACM Transactions on Graphics (TOG) 28, 5, 154.
2. Amanatides, J., and Woo, A. 1987. A fast voxel traversal algorithm for ray tracing. In Proc. Eurographics, vol. 87, 3–10.
3. Bastos, T., and Celes, W. 2008. Gpu-accelerated adaptively sampled distance fields. In Shape Modeling and Applications, 2008. SMI 2008. IEEE International Conference on, IEEE, 171–178.
4. Besl, P., and McKay, N. 1992. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. and Mach. Intell. 14, 2, 239–256.
5. Chang, C., Chatterjee, S., and Kube, P. R. 1994. A quantization error analysis for convergent stereo. In Proc. ICIP 94, vol. 2, IEEE, 735–739.
6. Chen, Y., and Medioni, G. 1992. Object modelling by registration of multiple range images. Image and Vision Computing 10, 3, 145–155.
7. Chen, J., Bautembach, D., and Izadi, S. 2013. Scalable real-time volumetric surface reconstruction. ACM Transactions on Graphics (TOG) 32, 4, 113.
8. Crassin, C., Neyret, F., Lefebvre, S., and Eisemann, E. 2009. Gigavoxels: Ray-guided streaming for efficient and detailed voxel rendering. In Proc. Symp. Interactive 3D Graphics and Games, ACM, 15–22.
9. Curless, B., and Levoy, M. 1996. A volumetric method for building complex models from range images. In In Proc. Computer graphics and interactive techniques, ACM, 303–312.
10. Gallup, D., Pollefeys, M., and Frahm, J.-M. 2010. 3D reconstruction using an n-layer heightmap. In Pattern Recognition. Springer, 1–10.
11. García, I., Lefebvre, S., Hornus, S., and Lasram, A. 2011. Coherent parallel hashing. ACM Transactions on Graphics (TOG) 30, 6, 161.
12. Gross, M., and Pfister, H. 2007. Point-based graphics. Morgan Kaufmann.
13. Hadwiger, M., Beyer, J., Jeong, W.-K., and Pfister, H. 2012. Interactive volume exploration of petascale microscopy data streams using a visualization-driven virtual memory approach. Visualization and Computer Graphics, IEEE Transactions on 18, 12, 2285–2294.
14. Harris, M., Sengupta, S., and Owens, J. D. 2007. Parallel prefix sum (scan) with cuda. GPU gems 3, 39, 851–876.
15. Henry, P., Krainin, M., Herbst, E., Ren, X., and Fox, D. 2012. RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments. Int. J. Robotics Research 31 (Apr.), 647–663.
16. Higuchi, K., Hebert, M., and Ikeuchi, K. 1995. Building 3-D models from unregistered range images. Graphical Models and Image Processing 57, 4, 315–333.
17. Hilton, A., Stoddart, A., Illingworth, J., and Windeatt, T. 1996. Reliable surface reconstruction from multiple range images. J. Computer Vision (Proc. ECCV), 117–126.
18. Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., and Stuetzle, W. 1992. Surface reconstruction from unorganized points. ACM SIGGRAPH Computer Graphics 26, 2, 71–78.
19. Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., and Fitzgibbon, A. 2011. KinectFusion: Real-time 3D reconstruction and interaction using a moving depth camera. In Proc. ACM Symp. User Interface Software and Technology, 559–568.
20. Johnson, A. E., and Bing Kang, S. 1999. Registration and integration of textured 3d data. Image and vision computing 17, 2, 135–147.
21. Kämpe, V., Sintorn, E., and Assarsson, U. 2013. High resolution sparse voxel dags. ACM Transactions on Graphics (TOG) 32, 4, 101.
22. Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In Proc. EG Symp. Geometry Processing.
23. Keller, M., Lefloch, D., Lambers, M., Izadi, S., Weyrich, T., and Kolb, A. 2013. Real-time 3d reconstruction in dynamic scenes using point-based fusion. In Proc. of Joint 3DIM/3DPVT Conference (3DV), IEEE, 1–8.
24. Kraus, M., and Ertl, T. 2002. Adaptive texture maps. In Proc. of the ACM SIGGRAPH/EUROGRAPHICS conference on Graphics hardware, Eurographics Association, 7–15.
25. Laine, S., and Karras, T. 2011. Efficient sparse voxel octrees. Visualization and Computer Graphics, IEEE Transactions on 17, 8, 1048–1059.
26. Lefebvre, S., and Hoppe, H. 2006. Perfect spatial hashing. ACM Transactions on Graphics (TOG) 25, 3, 579–588.
27. Lefebvre, S., Hornus, S., and Neyret, F., 2005. Gpu gems 2. chapter 37: Octree textures on the gpu.
28. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., et al. 2000. The digital michelangelo project: 3D scanning of large statues. In In Proc. Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., 131–144.
29. Lorensen, W., and Cline, H. 1987. Marching cubes: A high resolution 3D surface construction algorithm. Computer Graphics 21, 4, 163–169.
30. Low, K.-L. 2004. Linear least-squares optimization for point-to-plane icp surface registration. Tech. rep., Chapel Hill, University of North Carolina.
31. Newcombe, R. A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohli, P., Shotton, J., Hodges, S., and Fitzgibbon, A. 2011. KinectFusion: Real-time dense surface mapping and tracking. In Proc. IEEE Int. Symp. Mixed and Augmented Reality, 127–136.
32. Nguyen, C., Izadi, S., and Lovell, D. 2012. Modeling Kinect sensor noise for improved 3D reconstruction and tracking. In Proc. Int. Conf. 3D Imaging, Modeling, Processing, Visualization and Transmission, 524–530.
33. Pan, J., and Manocha, D. 2011. Fast gpu-based locality sensitive hashing for k-nearest neighbor computation. In Proc. of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM, 211–220.
34. Pollefeys, M., Nistér, D., Frahm, J., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S., Merrell, P., et al. 2008. Detailed real-time urban 3D reconstruction from video. Int. J. Comp. Vision 78, 2, 143–167.
35. Reichl, F., Chajdas, M. G., Bürger, K., and Westermann, R. 2012. Hybrid sample-based surface rendering. In Vision, Modeling & Visualization, The Eurographics Association, 47–54.
36. Roth, H., and Vona, M. 2012. Moving volume KinectFusion. In British Machine Vision Conf.
37. Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3D model acquisition. ACM Transactions on Graphics (TOG) 21, 3, 438–446.
38. Stückler, J., and Behnke, S. 2012. Integrating depth and color cues for dense multi-resolution scene mapping using rgb-d cameras. In Proc. of the IEEE Int. Conf. on Multisensor Fusion and Information Integration (MFI),(Hamburg, Germany).
39. Teschner, M., Heidelberger, B., Müller, M., Pomeranets, D., and Gross, M. 2003. Optimized spatial hashing for collision detection of deformable objects. In Proc. of Vision, Modeling, Visualization VMV03, 47–54.
40. Turk, G., and Levoy, M. 1994. Zippered polygon meshes from range images. In In Proc. Computer graphics and interactive techniques, 311–318.
41. Weise, T., Wismer, T., Leibe, B., and Van Gool, L. 2009. In-hand scanning with online loop closure. In Proc. IEEE Int. Conf. Computer Vision Workshops, 1630–1637.
42. Wheeler, M., Sato, Y., and Ikeuchi, K. 1998. Consensus surfaces for modeling 3D objects from multiple range images. In Proc. IEEE Int. Conf. Computer Vision, 917–924.
43. Whelan, T., Johannsson, H., Kaess, M., Leonard, J., and McDonald, J. 2012. Robust tracking for real-time dense rgb-d mapping with kintinuous. Tech. rep. Query date: 2012-10-25.
44. Zeng, M., Zhao, F., Zheng, J., and Liu, X. 2012. Octree-based fusion for realtime 3D reconstruction. Graphical Models.
45. Zhou, Q.-Y., and Koltun, V. 2013. Dense scene reconstruction with points of interest. ACM Transactions on Graphics (TOG) 32, 4, 112.
46. Zhou, K., Gong, M., Huang, X., and Guo, B. 2011. Data-parallel octrees for surface reconstruction. IEEE Trans. Vis. and Comp. Graph. 17, 5, 669–681.
