“Robust patch-based hdr reconstruction of dynamic scenes”
Conference:
Type(s):
Title:
- Robust patch-based hdr reconstruction of dynamic scenes
Session/Category Title: Color and Photos
Presenter(s)/Author(s):
Abstract:
High dynamic range (HDR) imaging from a set of sequential exposures is an easy way to capture high-quality images of static scenes, but suffers from artifacts for scenes with significant motion. In this paper, we propose a new approach to HDR reconstruction that draws information from all the exposures but is more robust to camera/scene motion than previous techniques. Our algorithm is based on a novel patch-based energy-minimization formulation that integrates alignment and reconstruction in a joint optimization through an equation we call the HDR image synthesis equation. This allows us to produce an HDR result that is aligned to one of the exposures yet contains information from all of them. We present results that show considerable improvement over previous approaches.
References:
1. Akyüz, A. O. 2011. Photographically guided alignment for HDR images. In Eurographics – Areas Papers, Eurographics Association, Llandudno, UK, 73–74.
2. Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., and Szeliski, R. 2011. A database and evaluation methodology for optical flow. Int. J. Comput. Vision 92, 1 (Mar.), 1–31.
3. Banterle, F., Artusi, A., Debattista, K., and Chalmers, A. 2011. Advanced High Dynamic Range Imaging: Theory and Practice. AK Peters (CRC Press), Natick, MA, USA.
4. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28 (July), 24:1–24:11.
5. Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. 2010. The generalized PatchMatch correspondence algorithm. In ECCV 2010, 29–43.
6. Bogoni, L. 2000. Extending dynamic range of monochrome and color images through fusion. In ICPR 2000, 3007–3016.
7. Brown, L. G. 1992. A survey of image registration techniques. ACM Comput. Surv. 24, 4 (Dec.), 325–376.
8. Brox, T., and Malik, J. 2011. Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Trans. on Pattern Analysis and Machine Intelligence 33, 3 (Mar.), 500–513.
9. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In ECCV 2004, 25–36.
10. Bruhn, A., Weickert, J., and Schnörr, C. 2005. Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. Int. J. Comput. Vision 61, 3 (Feb.), 211–231.
11. Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of ACM SIGGRAPH 1997, 369–378.
12. Eden, A., Uyttendaele, M., and Szeliski, R. 2006. Seamless image stitching of scenes with large motions and exposure differences. In CVPR 2006, vol. 2, 2498–2505.
13. Gallo, O., Gelfand, N., Chen, W., Tico, M., and Pulli, K. 2009. Artifact-free high dynamic range imaging. In ICCP 2009, 1–7.
14. Grosch, T. 2006. Fast and robust high dynamic range image generation with camera and object movement. In Vision, Modeling and Visualization, 277–284.
15. HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM Trans. Graph. 30, 4 (July), 70:1–70:10.
16. Heo, Y. S., Lee, K. M., Lee, S. U., Moon, Y., and Cha, J. 2010. Ghost-free high dynamic range imaging. In ACCV 2010, vol. 4, 486–500.
17. Jacobs, K., Loscos, C., and Ward, G. 2008. Automatic high-dynamic range image generation for dynamic scenes. IEEE Computer Graphics and Applications 28, 2 (Mar.-Apr.), 84–93.
18. Jinno, T., and Okuda, M. 2008. Motion blur free HDR image acquisition using multiple exposures. In ICIP 2008, 1304–1307.
19. Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2003. High dynamic range video. ACM Trans. Graph. 22, 3 (July), 319–325.
20. Khan, E., Akyüz, A., and Reinhard, E. 2006. Ghost removal in high dynamic range images. In ICIP 2006, 2005–2008.
21. Lin, H.-Y., and Chang, W.-Z. 2009. High dynamic range imaging for stereoscopic scene representation. In ICIP 2009, 4249–4252.
22. Lin, S., and Zhang, L. 2005. Determining the radiometric response function from a single grayscale image. In CVPR 2005, vol. 2, 66–73.
23. Lin, S., Gu, J., Yamazaki, S., and Shum, H.-Y. 2004. Radiometric calibration from a single image. In CVPR 2004, vol. 2, 938–945.
24. Liu, X., and El Gamal, A. 2003. Synthesis of high dynamic range motion blur free image from multiple captures. IEEE Trans. on Circuits and Systems I: Fundamental Theory and Applications 50, 4 (Apr.), 530–539.
25. Liu, C. 2009. Beyond Pixels: Exploring New Representations and Applications for Motion Analysis. Doctoral thesis, Massachusetts Institute of Technology.
26. Lucas, B. D., and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In IJCAI 1981, 674–679.
27. Mangiat, S., and Gibson, J. 2010. High dynamic range video with ghost removal. In Proc. SPIE 7798, no. 779812, 1–8.
28. Mann, S., and Picard, R. W. 1995. On being ‘undigital’ with digital cameras: Extending dynamic range by combining differently exposed pictures. In Proc. of Society for Imaging Science and Technology, 442–448.
29. Min, T.-H., Park, R.-H., and Chang, S. 2009. Histogram based ghost removal in high dynamic range images. In ICME 2009, 530–533.
30. Mitsunaga, T., and Nayar, S. 1999. Radiometric self calibration. In CVPR 1999, vol. 1, 374–380.
31. Nayar, S., and Mitsunaga, T. 2000. High dynamic range imaging: spatially varying pixel exposures. In CVPR 2000, 472–479.
32. Pece, F., and Kautz, J. 2010. Bitmap movement detection: HDR for dynamic scenes. In Conference on Visual Media Production (CVMP) 2010, 1–8.
33. Photomatix, 2012. Commercially-available HDR processing software. http://www.hdrsoft.com/.
34. Raman, S., and Chaudhuri, S. 2011. Reconstruction of high contrast images for dynamic scenes. The Visual Computer 27, 12 (Dec.), 1099–1114.
35. Ratcliff, T., 2012. Stuck in Customs HDR Photography. http://www.stuckincustoms.com.
36. Reinhard, E., Heidrich, W., Debevec, P., Pattanaik, S., Ward, G., and Myszkowski, K. 2010. High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting, second ed. Morgan Kaufmann.
37. Shechtman, E., Rav-Acha, A., Irani, M., and Seitz, S. 2010. Regenerative morphing. In CVPR 2010, 615–622.
38. Sidibe, D., Puech, W., and Strauss, O. 2009. Ghost detection and removal in high dynamic range images. In Proc. of EUSIPCO 2009, 2240–2244.
39. Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In CVPR 2008, 1–8.
40. Tocci, M. D., Kiser, C., Tocci, N., and Sen, P. 2011. A versatile HDR video production system. ACM Trans. Graph. 30, 4 (July), 41:1–41:10.
41. Tomaszewska, A., and Mantiuk, R. 2007. Image registration for multi-exposure high dynamic range image acquisition. In Intl. Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG) 2007.
42. Ward, G. 2003. Fast, robust image registration for compositing high dynamic range photographs from hand-held exposures. journal of graphics, gpu, and game tools 8, 2, 17–30.
43. Wexler, Y., Shechtman, E., and Irani, M. 2007. Space-time completion of video. IEEE Trans. on Pattern Analysis and Machine Intelligence 29, 3 (Mar.), 463–476.
44. Wu, S., Xie, S., Rahardja, S., and Li, Z. 2010. A robust and fast anti-ghosting algorithm for high dynamic range imaging. In ICIP 2010, 397–400.
45. Xu, L., Jia, J., and Matsushita, Y. 2010. Motion detail preserving optical flow estimation. In CVPR 2010, 1293–1300.
46. Yao, S. 2011. Robust image registration for multiple exposure high dynamic range image synthesis. In Proc. of SPIE 7870, no. 78700Q, 1–9.
47. Zhang, W., and Cham, W.-K. 2012. Gradient-directed multi-exposure composition. IEEE Trans. on Image Processing 21, 4 (Apr.), 2318–2323.
48. Zimmer, H., Bruhn, A., and Weickert, J. 2011. Freehand HDR imaging of moving scenes with simultaneous resolution enhancement. Computer Graphics Forum 30, 2 (Apr.), 405–414.
49. Zitová, B., and Flusser, J. 2003. Image registration methods: a survey. Image and Vision Computing 21, 11 (Oct.), 977–1000.
