“A Framework for Modeling 3D Scenes Using Pose-Free Equations” by Aliaga, Zhang and Boutin

Many applications in computer graphics require detailed 3D digital models of real-world environments. The automatic and semi-automatic modeling of such spaces presents several fundamental challenges. In this work, we present an easy and robust camera-based acquisition approach for the modeling of 3D scenes which is a significant departure from current methods. Our approach uses a novel pose-free formulation for 3D reconstruction. Unlike self-calibration, omitting pose parameters from the acquisition process implies no external calibration data must be computed or provided. This serves to significantly simplify acquisition, to fundamentally improve the robustness and accuracy of the geometric reconstruction given noise in the measurements or error in the initial estimates, and to allow using uncalibrated active correspondence methods to obtain robust data. Aside from freely taking pictures and moving an uncalibrated digital projector, scene acquisition and scene point reconstruction is automatic and requires pictures from only a few viewpoints. We demonstrate how the combination of these benefits has enabled us to acquire several large and detailed models ranging from 0.28 to 2.5 million texture-mapped triangles.

1. Aliaga, D. and Carlbom, I. 2001. Plenoptic stitching: A method for reconstructing interactive walkthroughs. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 443–450. 
2. Aliaga, D., Zhang, J., and Boutin, M. 2007. Simplifying the reconstruction of 3d models using parameter elimination. In Proceedings of the Workshop on Visual Representations and Modeling of Large-Scale Environments, IEEE International Conference on Computer Vision.
3. Besl, P. and Mckay, N. 1992. A method for registration of 3-d shapes. IEEE Trans. Patt. Anal. Mach. Intell. 14, 2, 239–256. 
4. Buehler, C., Boose, M., Mcmillan, L., Gortler ,S., and Cohen, M. 2001. Unstructured lumigraph rendering. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 425–432. 
5. Diebel, J. and Thrun, S. 2005. An application of Markov random fields to range sensing. In Proceedings of the Conference on Neural Information Processing Systems. 291–298.
6. Favaro, P. and Soatto, S. 2005. A geometric approach to shape from defocus. IEEE Trans. Patt. Anal. Mach. Intell. 27, 3, 406–417. 
7. Fels, M. and Olver, P. 1998. Moving coframes: A practical algorithm. Acta Appl. Math. 51, 161–213.
8. Fermüller, C. and Aloimonos, Y. 2000. Observability of 3D motion. Int. J. Comput. Vis., 43–62. 
9. Furukawa, R. and Kawasaki, H. 2005. Uncalibrated multiple image stereo system with arbitrarily movable camera and projector for wide range scanning. In Proceedings of the Conference on 3D Digital Imaging and Modeling. 302–309. 
10. Gortler, S., Grzeszczuk, R., Szeliski, R., and Cohen, M. 1996. The lumigraph. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 43–54. 
11. Hemayed, E. 2003. A survey of camera self-calibration. In Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance. 351–357. 
12. Huber, D. and Hebert, M. 2003. Fully automatic registration of multiple 3D datasets. Image Vis. Comput. 21, 7, 637–650.
13. Johnson, A.E and Hebert, M. 1999. Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Trans. Patt. Anal. Mach. Intell. 21, 5, 433–449. 
14. Levoy, M. and Hanrahan, P. 1996. Light field rendering. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 31–42. 
15. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., et al. 2000. The digital Michelangelo project: 3D scanning of large statues. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 131–144. 
16. Li, Y. and Lu, R. 2004. Uncalibrated Euclidean 3D recon using an active vision system. IEEE Trans. Robotics Autom. 20, 1, 15–25.
17. Lourakis, M. and Argyos, A. 2004. The design and implementation of a generic sparse bundle adjustment software package based on the Levenberg-Marquardt algorithm. Tech. rep. 340, Institute of Computer Science — FORTH.
18. Lu, Y., Zhang, J., Wu, J., and Li, Z. 2004. A survey of motion-parallax-based 3-d reconstruction algorithms. IEEE Trans. Syst. Man Cybernet. 34, 4, 532–548. 
19. Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J. M., Yang, R., Nistèr, D., and Pollefeys, M. 2007. Real-Time visibility-based fusion of depth maps. In Proceedings of the IEEE International Conference on Computer Vision.
20. Nister, D. 2003. Preemptive RANSAC for live structure and motion estimation. In Proceedings of the IEEE International Conference on Computer Vision. 199–206. 
21. Pollefeys, M., van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., and Koch, R. 2004. Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59, 3, 207–232. 
22. Ribo, M. and Brandner, M. 2005. State of the art on vision-based structured light systems for 3D measurements. In Proceedings of the IEEE International Workshop on Robotic Sensors: Robotic and Sensor Environments. 2–7.
23. Rusinkiewicz, S. and Levoy, M. 2001. Efficient variants of the ICP algorithm. In Proceedings of the Conference on 3D Digital Imaging and Modeling.
24. Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-Time 3D model acquisition. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 438–446. 
25. Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., and Tarantola, S. 2008. Global Sensitivity Analysis: The Primer. Wiley-Interscience.
26. Salvi, J., Pages, J., and Batlle, J. 2004. Pattern codification strategies in structured light systems. Pattern Recogn. 37, 827–849.
27. Scharstein, D. and Szeliski, R. 2003. High-Accuracy stereo depth maps using structured light. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 195–202. 
28. Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 519–526. 
29. Shum, H. and He, L. 1999. Concentric mosaics. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 299–306. 
30. Sturm, P. 2002. Critical motion sequences for the self-calibration of cameras and stereo systems with variable focal length. Image Vis. Comput. 20, 5-6, 415–426.
31. Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. Int. J. Comput, Vis. 9, 2, 137–154. 
32. Tomasi, C. 1994. Pictures and trails: A new framework for the computation of shape and motion from perspective image sequences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 913–918.
33. Triggs, B., McLauchlan, P., Hartley, R., and Fitzgibbon, A. 2000. Bundle adjustment – A modern synthesis. In Vision Algorithms: Theory and Practice. Springer. 
34. Werman, M. and Shashua, A. 1995. The study of 3d-from-2d using elimination. In Proceedings of the IEEE International Conference on Computer Vision. 473–479. 
35. Williams, N., Hantak, C., Low, K. L., Thomas, J., Keller, K., Nyland, L., Luebke, D., and Lastra, A. 2003. Monticello through the window. In Proceedings of the Symposium on VR, Archaeology and Intelligent Cultural Heritage. 
36. Zhang, J., Aliaga, D., Boutin, M., and Insley, R. 2006. Angle independent bundle adjustment refinement. In Proceedings of the 3DPVT Conference. 25–32. 
37. Zhang, L. and Nayar, S. 2006. Projection defocus analysis for scene capture and image display. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 907–915. 
38. Zhu, J., Wang, L., Yang, R., and Davis, J. 2008. Fusion of time-of-flight depth and stereo for high accuracy depth maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.