“Automatic photo pop-up” by Hoiem, Efros and Hebert

This paper presents a fully automatic method for creating a 3D model from a single photograph. The model is made up of several texture-mapped planar billboards and has the complexity of a typical children’s pop-up book illustration. Our main insight is that instead of attempting to recover precise geometry, we statistically model geometric classes defined by their orientations in the scene. Our algorithm labels regions of the input image into coarse categories: “ground”, “sky”, and “vertical”. These labels are then used to “cut and fold” the image into a pop-up model using a set of simple assumptions. Because of the inherent ambiguity of the problem and the statistical nature of the approach, the algorithm is not expected to work on every image. However. it performs surprisingly well for a wide range of scenes taken from a typical person’s photo album.

1. Chen, E. 1995. QuickTime VR – an image-based approach to virtual environment navigation. In ACM SIGGRAPH 95, 29–38. Google ScholarDigital Library
2. Cipolla, R., Robertson, D., and Boyer, E. 1999. Photobuilder – 3d models of architectural scenes from uncalibrated images. In IEEE Int. Conf. on Multimedia Computing and Systems, vol. I. 25–31. Google ScholarDigital Library
3. Collins, M., Schapire, R., and Singer, Y. 2002. Logistic regression, adaboost and bregman distances. Machine Learning 48, 1–3, 253–285. Google ScholarDigital Library
4. Criminisi, A., Reid, I., and Zisserman, A. 2000. Single view metrology. Int. Journal of Computer Vision 40, 2, 123–148. Google ScholarDigital Library
5. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In ACM SIGGRAPH 96, 11–20. Google ScholarDigital Library
6. Duda, R., and Hart, P. 1972. Use of the hough transformation to detect lines and curves in pictures. Communications of the ACM 15, 1, 11–15. Google ScholarDigital Library
7. Duda, R., Hart, P., and Stork, D. 2000. Pattern Classification. Wiley-Interscience Publication. Google ScholarDigital Library
8. Everingham, M. R., Thomas, B. T., and Troscianko, T. 1999. Head-mounted mobility aid for low vision using scene classification techniques. The Intl J. of Virtual Reality 3, 4, 3–12.Google Scholar
9. Felzenszwalb, P., and Huttenlocher, D. 2004. Efficient graph-based image segmentation. Int. Journal of Computer Vision 59, 2, 167–181. Google ScholarDigital Library
10. Friedman, J., Hastie, T., and Tibshirani, R. 2000. Additive logistic regression: a statistical view of boosting. Annals of Statistics 28, 2, 337–407.Google ScholarCross Ref
11. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The Lumigraph. In ACM SIGGRAPH 96, 43–54. Google ScholarDigital Library
12. Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision, 2nd ed. Cambridge University Press. Google ScholarDigital Library
13. Horry, Y., Anjyo, K.-I., and Arai, K. 1997. Tour into the picture: using a spidery mesh interface to make animation from a single image. In ACM SIGGRAPH 97, 225–232. Google ScholarDigital Library
14. Kang, H., Pyo, S., Anjyo, K., and Shin, S. 2001. Tour into the picture using a vanishing line and its extension to panoramic images. In Proc. Eurographics, 132–141.Google Scholar
15. Konishi, S., and Yuille, A. 2000. Statistical cues for domain specific image segmentation with performance analysis. In Computer Vision and Pattern Recognition, 1125–1132.Google Scholar
16. Kosecka, J., and Zhang, W. 2002. Video compass. In European Conf. on Computer Vision, Springer-Verlag, 476–490 Google ScholarDigital Library
17. Levoy, M., and Hanrahan, P. 1996. Light field rendering. In ACM SIGGRAPH 96, 31–42. Google ScholarDigital Library
18. Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Trans. on Graphics 23, 3, 303–308. Google ScholarDigital Library
19. Liebowitz, D., Criminisi, A., and Zisserman, A. 1999. Creating architectural models from images. In Proc. Eurographics, vol. 18, 39–50.Google ScholarCross Ref
20. Martin, D., Fowlkes, C., Tal, D., and Malik, J. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Int. Conf. on Computer Vision, vol. 2, 416–423.Google ScholarCross Ref
21. Nistér, D. 2001. Automatic dense reconstruction from uncalibrated video sequences. PhD thesis, Royal Institute of Technology KTH.Google Scholar
22. Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In ACM SIGGRAPH 2001, ACM Press, 433–442. Google ScholarDigital Library
23. Pollefeys, M., Gool, L. V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., and Koch, R. 2004. Visual modeling with a hand-held camera. Int. J. of Computer Vision 59, 3, 207–232. Google ScholarDigital Library
24. Quinlan, J. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc. Google ScholarDigital Library
25. Ren, X., and Malik, J. 2003. Learning a classification model for segmentation, In Int. Conf. on Computer Vision, 10–17. Google ScholarDigital Library
26. Singhal, A., Luo, J., and Zhu, W. 2003. Probabilistic spatial context models for scene content understanding. In Computer Vision and Pattern Recognition, 235–241. Google ScholarDigital Library
27. Tao, H., Sawhney, H. S., and Kumar, R. 2001. A global matching framework for stereo computation. In Int. Conf. on Computer Vision, 532–539.Google Scholar
28. Zhang, L., Dugas-Phocion, G., Samson, J., and Seitz, S. 2001. Single view modeling of free-form scenes. In Computer Vision and Pattern Recognition, 990–997.Google Scholar
29. Ziegler, R., Matusik, W., Pfister, H., and McMillan, L. 2003. 3d reconstruction using labeled image regions. In Eurographics Symposium on Geometry Processing, 248–259. Google ScholarDigital Library