“Gigapixel Panorama Video Loops” by He, Liao, Sander and Hoppe

  • ©Mingming He, Jing Liao, Pedro V. Sander, and Hugues Hoppe

Conference:


Type:


Title:

    Gigapixel Panorama Video Loops

Session/Category Title: Computational Photos and Videos

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We present the first technique to create wide-angle, high-resolution looping panoramic videos. Starting with a 2D grid of registered videos acquired on a robotic mount, we formulate a combinatorial optimization to determine for each output pixel the source video and looping parameters that jointly maximize spatiotemporal consistency. This optimization is accelerated by reducing the set of source labels using a graph-coloring scheme. We parallelize the computation and implement it out-of-core by partitioning the domain along low-importance paths. The merged panorama is assembled using gradient-domain blending and stored as a hierarchy of video tiles. Finally, an interactive viewer adaptively preloads these tiles for responsive browsing and allows the user to interactively edit and improve local regions. We demonstrate these techniques on gigapixel-sized looping panoramas.

References:


    1. Aseem Agarwala. 2007. Efficient gradient-domain compositing using quadtrees. ACM Trans. Graph. 26, 3 (2007), 94. Google ScholarDigital Library
    2. A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen. 2004. Interactive Digital Photomontage. ACM Trans. Graph. 23, 3 (2004). Google ScholarDigital Library
    3. Aseem Agarwala, Ke Colin Zheng, Chris Pal, Maneesh Agrawala, Michael Cohen, Brian Curless, David Salesin, and Richard Szeliski. 2005. Panoramic video textures. ACM Trans. Graph. 24, 3 (July 2005), 7. Google ScholarDigital Library
    4. Jiamin Bai, Aseem Agarwala, Maneesh Agrawala, and Ravi Ramamoorthi. 2012. Selectively De-Animating Video. ACM Trans. Graph. 31, 4 (2012). Google ScholarDigital Library
    5. Jiamin Bai, Aseem Agarwala, Maneesh Agrawala, and Ravi Ramamoorthi. 2013. Automatic cinemagraph portraits. Computer Graphics Forum 32, 4 (2013), 17–25. Google ScholarDigital Library
    6. Jamie Beck and Kevin Burg. 2012. Cinemagraphs. http://cinemagraphs.com/. (2012).Google Scholar
    7. Jean-Yves Bouguet. 2001. Pyramidal implementation of the affine Lucas-Kanade feature tracker: Description of the algorithm. Intel Corporation 5, 1–10 (2001), 4.Google Scholar
    8. Matthew Brown and David G. Lowe. 2007. Automatic panoramic image stitching using invariant features. IJCV 74, 1 (2007), 59–73. Google ScholarDigital Library
    9. V. Couture, M. Langer, and S. Roy. 2011. Panoramic stereo video textures. ICCV (2011), 1251–1258. Google ScholarDigital Library
    10. Vincent Couture, Michael S. Langer, and Sébastien Roy. 2012. Perception of blending in stereo motion panoramas. ACM Trans. Applied Perception 9, 3 (2012), 15. Google ScholarDigital Library
    11. Dan B. Goldman. 2010. Vignette and exposure calibration and compensation. IEEE transactions on pattern analysis and machine intelligence 32, 12 (2010), 2276–2288. Google ScholarDigital Library
    12. João F. Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. 2015. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 3 (2015), 583–596.Google ScholarDigital Library
    13. Chris Hermans, Cedric Vanaken, Tom Mertens, Frank Van Reeth, and Philippe Bekaert. 2008. Augmented panoramic video. In Computer Graphics Forum, Vol. 27.Google ScholarCross Ref
    14. Cheuk Yiu Ip and Amitabh Varshney. 2011. Saliency-assisted navigation of very large landscape images. IEEE TVCG 17, 12 (2011). Google ScholarDigital Library
    15. Neel Joshi, Sisil Mehta, Steven Drucker, Eric Stollnitz, Hugues Hoppe, Matt Uyttendaele, and Michael Cohen. 2012. Cliplets: Juxtaposing still and dynamic imagery. Proc. of UIST (2012). Google ScholarDigital Library
    16. George Karypis. 2003. Multi-constraint mesh partitioning for contact/impact computations. In Proc. of ACM/IEEE conference on Supercomputing. ACM, 56. Google ScholarDigital Library
    17. M. Kazhdan and H. Hoppe. 2008. Streaming Multigrid for Gradient-Domain Operations on Large Images. ACM Trans. Graph. 27, 3 (2008). Google ScholarDigital Library
    18. Michael Kazhdan, Dinoj Surendran, and Hugues Hoppe. 2010. Distributed gradient-domain processing of planar and spherical images. ACM Trans. Graph. 29, 2 (2010). Google ScholarDigital Library
    19. V. Kolmogorov and R. Zabih. 2004. What Energy Functions can be Minimized via Graph Cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26, 2 (2004). Google ScholarDigital Library
    20. Johannes Kopf, Matt Uyttendaele, Oliver Deussen, and Michael F. Cohen. 2007. Capturing and viewing gigapixel images. ACM Trans. Graph. 26, 3 (2007), 93. Google ScholarDigital Library
    21. Vivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk, and Aaron Bobick. 2003. Graphcut textures: image and video synthesis using graph cuts. In ACM Trans. Graph., Vol. 22. 277–286. Google ScholarDigital Library
    22. Jing Liao, Mark Finch, and Hugues Hoppe. 2015. Fast computation of seamless video loops. ACM Trans. Graph. 34, 6 (2015), 197. Google ScholarDigital Library
    23. Jing Liao, Neel Joshi, and Hugues Hoppe. 2013. Automated video looping with progressive dynamism. ACM Trans. Graph. 32, 4 (2013). Google ScholarDigital Library
    24. Jiangyu Liu and Jian Sun. 2010. Parallel graph-cuts by adaptive bottom-up merging. In Proc. CVPR. IEEE, 2181–2188.Google Scholar
    25. Qing Luan, Steven M. Drucker, Johannes Kopf, Ying-Qing Xu, and Michael F. Cohen. 2008. Annotating gigapixel images. In Proc. ACM UIST. 33–36. Google ScholarDigital Library
    26. Microsoft Research. 2016. Image Composite Editor. http://research.microsoft.com/en-us/um/redmond/groups/ivm/ICE/. (2016).Google Scholar
    27. Federico Perazzi, Alexander Sorkine-Hornung, Henning Zimmer, Peter Kaufmann, Oliver Wang, S. Watson, and Markus H. Gross. 2015. Panoramic video from unstructured camera arrays. Computer Graphics Forum 34, 2 (2015), 57–68. Google ScholarDigital Library
    28. Patrick Pérez, Michel Gangnet, and Andrew Blake. 2003. Poisson image editing. ACM Trans. Graph. 22, 3 (2003). Google ScholarDigital Library
    29. Sujin Philip, Brian Summa, Valerio Pascucci, and P.-T. Bremer. 2011. Hybrid CPU-GPU solver for gradient domain processing of massive images. In Intl. Conf. on Parallel and Distributed Systems. Google ScholarDigital Library
    30. Sujin Philip, Brian Summa, Julien Tierny, Peer-Timo Bremer, and Valerio Pascucci. 2015. Distributed Seams for Gigapixel Panoramas. IEEE TVCG 21, 3 (2015), 350–362.Google Scholar
    31. Sören Pirk, Michael F. Cohen, Oliver Deussen, Matt Uyttendaele, and Johannes Kopf. 2012. Video enhanced gigapixel panoramas. SIGGRAPH Asia 2012 Technical Briefs (2012). Google ScholarDigital Library
    32. Alex Rav-Acha, Yael Pritch, Dani Lischinski, and Shmuel Peleg. 2007. Dynamosaicing: Mosaicing of Dynamic Scenes. IEEE Trans. on Pattern Anal. Mach. Intell. 29, 10 (2007). Google ScholarDigital Library
    33. Arno Schödl, Richard Szeliski, David H. Salesin, and Irfan Essa. 2000. Video textures. In Proc. SIGGRAPH. 489–498. Google ScholarDigital Library
    34. Laura Sevilla-Lara, Jonas Wulff, Kalyan Sunkavalli, and Eli Shechtman. 2015. Smooth Loops from Unconstrained Video. Computer Graphics Forum 34, 4 (2015), 99–107. Google ScholarDigital Library
    35. Petter Strandmark and Fredrik Kahl. 2010. Parallel and distributed graph cuts by dual decomposition. In Proc. CVPR. IEEE, 2085–2092.Google ScholarCross Ref
    36. Brian Summa, Giorgio Scorzelli, Ming Jiang, Peer-Timo Bremer, and Valerio Pascucci. 2011. Interactive editing of massive imagery made simple: Turning Atlanta into Atlantis. ACM Transactions on Graphics (TOG) 30, 2 (2011), 7. Google ScholarDigital Library
    37. Brian Summa, Julien Tierny, and Valerio Pascucci. 2012. Panorama weaving: Fast and flexible seam processing. ACM Trans. Graph. 31, 4 (2012). Google ScholarDigital Library
    38. Richard Szeliski. 2006. Image alignment and stitching: A tutorial. Foundations and Trends in Computer Graphics and Vision 2, 1 (2006). Google ScholarDigital Library
    39. Richard Szeliski and Heung-Yeung Shum. 1997. Creating full view panoramic image mosaics and environment maps. In Proc. ACM SIGGRAPH. Google ScholarDigital Library
    40. James Tompkin, Fabrizio Pece, Rajvi Shah, Shahram Izadi, Jan Kautz, and Christian Theobalt. 2013. Video collections in panoramic contexts. In Proc. ACM UIST. Google ScholarDigital Library
    41. James Tompkin, Fabrizio Pece, Kartic Subr, and Jan Kautz. 2011. Towards Moment Images: Automatic Cinemagraphs. In Proc. of the 8th European Conference on Visual Media Production (CVMP 2011). Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: