“Panoramic video textures” by Agarwala, Zheng, Pal, Agrawala, Cohen, et al. …

This paper describes a mostly automatic method for taking the output of a single panning video camera and creating a panoramic video texture (PVT): a video that has been stitched into a single, wide field of view and that appears to play continuously and indefinitely. The key problem in creating a PVT is that although only a portion of the scene has been imaged at any given time, the output must simultaneously portray motion throughout the scene. Like previous work in video textures, our method employs min-cut optimization to select fragments of video that can be stitched together both spatially and temporally. However, it differs from earlier work in that the optimization must take place over a much larger set of data. Thus, to create PVTs, we introduce a dynamic programming step, followed by a novel hierarchical min-cut optimization algorithm. We also use gradient-domain compositing to further smooth boundaries between video fragments. We demonstrate our results with an interactive viewer in which users can interactively pan and zoom on high-resolution PVTs.

1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics 23, 3, 294–302.]] Google ScholarDigital Library
2. Bergen, J. R., Anandan, P., Hanna, K. J., and Hingorani, R. 1992. Hierarchical model-based motion estimation. In European Conference on Computer Vision, 237–252.]] Google ScholarDigital Library
3. Brown, M., and Lowe, D. 2003. Recognising panoramas. In Proceedings of ICCV 03, 1218–1225.]] Google ScholarDigital Library
4. Chen, S. E. 1995. Quicktime VR – an image-based approach to virtual environment navigation. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, 29–38.]] Google ScholarDigital Library
5. Finkelstein, A., Jacobs, C. E., and Salesin, D. H. 1996. Multiresolution video. In Proceedings of SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 281–290.]] Google ScholarDigital Library
6. Fitzgibbon, A. W. 2001. Stochastic rigidity: Image registration for nowhere-static scenes. In Proceedings of ICCV 2001, 662–670.]]Google ScholarCross Ref
7. Freeman, W. T., Pasztor, E. C., and Carmichael, O. T. 2000. Learning low-level vision. International Journal of Computer Vision 40, 1, 25–47.]] Google ScholarDigital Library
8. Irani, M., and Anandan, P. 1998. Video indexing based on mosaic representation. Proceedings of IEEE 86, 5, 905–921.]]Google ScholarCross Ref
9. Kimber, D., Foote, J., and Lertsithichai, S. 2001. Fly-about: spatially indexed panoramic video. In Proceedings of ACM MULTIMEDIA ’01, 339–347.]] Google ScholarDigital Library
10. Kolmogorov, V., and Zabih, R. 2002. What energy functions can be minimized via graph cuts? In European Conference on Computer Vision (ECCV), 65–81.]] Google ScholarDigital Library
11. Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics 22, 3, 277–286.]] Google ScholarDigital Library
12. Nayar, S. K. 1997. Catadioptric omnidirectional camera. In Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR ’97), 482.]] Google ScholarDigital Library
13. Neumann, U., Pintaric, T., and Rizzo, A. 2000. Immersive panoramic video. In Proceedings of MULTIMEDIA ’00, 493–494.]] Google ScholarDigital Library
14. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics 22, 3, 313–318.]] Google ScholarDigital Library
15. Point Grey Research, 2005. http://ptgrey.com.]]Google Scholar
16. Rav-Acha, A., Pritch, Y., Lischinski, D., and Peleg, S. 2005. Dynamosaics: Video mosaics with non-chronological time. In Proceedings of CVPR 2005, To appear.]] Google ScholarDigital Library
17. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In Proceedings of SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 489–498.]] Google ScholarDigital Library
18. Szeliski, R., and Shum, H.-Y. 1997. Creating full view panoramic mosaics and environment maps. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 251–258.]] Google ScholarDigital Library
19. Triggs, B., Mclauchlan, P. F., Hartley, R. I., and Fitzgibbon, A. W. 2000. Bundle adjustment – a modern synthesis. In ICCV ’99: Proceedings of the International Workshop on Vision Algorithms, Springer-Verlag, 298–372.]] Google ScholarDigital Library
20. Uyttendaele, M., Criminisi, A., Kang, S. B., Winder, S. A. J., Hartley, R., and Szeliski, R. 2004. High-quality image-based interactive exploration of real-world environments. IEEE Computer Graphics and Applications 24, 3, 52–63.]] Google ScholarDigital Library
21. Wang, H., Raskar, R., and Ahuja, N. 2004. Seamless video editing. In Proceedings of the International Conference on Pattern Recognition (ICPR), 858–861.]] Google ScholarDigital Library