“Fast burst images denoising” by Liu, Yuan, Tang, Uyttendaele and Sun
Conference:
Type(s):
Title:
- Fast burst images denoising
Session/Category Title: Digital Photography
Presenter(s)/Author(s):
Abstract:
This paper presents a fast denoising method that produces a clean image from a burst of noisy images. We accelerate alignment of the images by introducing a lightweight camera motion representation called homography flow. The aligned images are then fused to create a denoised output with rapid per-pixel operations in temporal and spatial domains. To handle scene motion during the capture, a mechanism of selecting consistent pixels for temporal fusion is proposed to “synthesize” a clean, ghost-free image, which can largely reduce the computation of tracking motion between frames. Combined with these efficient solutions, our method runs several orders of magnitude faster than previous work, while the denoising quality is comparable. A smartphone prototype demonstrates that our method is practical and works well on a large variety of real examples.
References:
1. Adams, A., Gelfand, N., Dolson, J., and Levoy, M. 2009. Gaussian kd-trees for fast high-dimensional filtering. ACM Trans. Graph. (Proc. of SIGGRAPH) 28, 3.
2. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. Patchmatch: A randomized correspondence algorithm for structural image editing. SIGGRAPH 28, 3.
3. Bennett, E. P., and McMillan, L. 2005. Video enhancement using per-pixel virtual exposures. ACM Trans. Graph. (Proc. of SIGGRAPH) 24, 3, 845–852.
4. Bronshtein, I. N., and Semendyayev, K. A. 1997. Handbook of Mathematics. Springer-Verlag, New York, NY, USA.
5. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In Proc. ECCV.
6. Buades, A., Coll, B., and Morel, J.-M. 2005. A non-local algorithm for image denoising. In Proc. CVPR.
7. Buades, A., Lou, Y., Morel, J.-M., and Tang, Z. 2009. A note on multi-image denoising. In In Proceedings of the International Workshop on Local and Non-Local Approximation (LNLA) in Image Processing.
8. Buades, A., Lou, Y., Morel, J.-M., and Tang, Z. 2010. Multi image noise estimation and denoising. In HAL.
9. Cai, J. F., Ji, H., Liu, C., and Shen, Z. 2009. Blind motion deblurring using multiple images. J. Comput. Physics 228, 14, 5057–5071.
10. Calonder, M., Lepetit, V., Strecha, C., and Fua, P. 2010. Brief: binary robust independent elementary features. In Proc. ECCV.
11. Chatterjee, P., Joshi, N., Kang, S. B., and Matsushita, Y. 2011. Noise suppression in low-light images through joint denoising and demosaicing. In Proc. CVPR.
12. Chen, J., and Tang, C.-K. 2007. Spatio-temporal markov random field for video denoising. In Proc. CVPR.
13. Chen, J., Tang, C.-K., and Wang, J. 2009. Noise brush: Interactive high quality image-noise separation. ACM Trans. Graph. (Proc. of SIGGRAPH ASIA) 28, 5.
14. Chen, X., Kang, S. B., Yang, J., and Yu, J. 2013. Fast patch-based denoising using approximated patch geodesic paths. In Proc. CVPR.
15. Cho, S., Wang, J., and Lee, S. 2012. Vdeo deblurring for hand-held cameras using patch-based synthesis. Proc. ACM SIGGRAPH 31, 4, 64:1–64:9.
16. Dabov, K., Foi, A., and Egiazarian, K. 2007. Video denoising by sparse 3d transform-domain collaborative filtering. In Proc. European Signal Process. Conf., EUSIPCO.
17. Dabov, K., Foi, A., Egiazarian, K., and Egiazarian, K. 2007. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. on Image Processing 16, 8, 2080–2095.
18. Farsiu, S., Robinson, M. D., Elad, M., and Milanfar, P. 2004. Fast and robust multiframe super resolution. IEEE Trans. on Image Processing 13, 10, 1327–1344.
19. Gallo, O., Gelfand, N., Chen, W., Tico, M., and Pulli, K. 2009. Artifact-free high dynamic range imaging.
20. Gonzalez, R. C., and Woods, R. E. 2007. Digital Image Processing. Prentice Hall, 3rd edition.
21. Granados, M., Kim, K. I., Tompkin, J., and Theobalt, C. 2013. Automatic noise modeling for ghost-free hdr reconstruction. ACM Trans. Graph. (Proc. of SIGGRAPH ASIA) 32, 6, 1–10.
22. Grundmann, M., Kwatra, V., Castro, D., and Essa, I. 2012. Calibration-free rolling shutter removal. In Proc. ICCP.
23. Harris, C., and Stephens, M. 1988. A combined corner and edge detector. In In Proc. of Fourth Alvey Vision Conference.
24. Hartley, R., and Zisserman, A. 2003. Multiple View Geometry in Computer Vision, 2 ed. Cambridge University Press, New York, NY, USA.
25. Jacobs, D. E., Baek, J., and Levoy, M. 2012. Focal stack compositing for depth of field control. In Stanford Computer Graphics Laboratory Technical Report.
26. Joshi, N., and Cohen, M. F. 2010. Seeing mt. rainier: lucky imaging for multi-image denoising, sharpening, and haze removal. In Proc. ICCP.
27. Kalantari, N. K., Shechtman, E., Barnes, C., Darabi, S., Goldman, D. B., and Sen, P. 2013. Patch-based high dynamic range video. ACM Trans. Graph. (Proc. of SIGGRAPH ASIA) 32, 6, 202:1–202:8.
28. Levin, A., and Nadler, B. 2011. Natural image denoising: Optimality and inherent bounds. In Proc. CVPR, 2833–2840.
29. Liu, C., and Freeman, W. T. 2010. A high-quality video denoising algorithm based on reliable motion estimation. Proc. ECCV, 706–719.
30. Liu, C., Szeliski, R., Kang, S. B., Zitnick, C. L., and Freeman, W. T. 2008. Automatic estimation and removal of noise from a single image.
31. Liu, S., Yuan, L., Tan, P., and Sun, J. 2013. Bundled camera paths for video stabilization. ACM Trans. Graph. (Proc. of SIGGRAPH) 32, 4, 78:1–78:10.
32. Liu, C. 2009. Beyond Pixels: Exploring New Representations and Applications for Motion Analysis. PhD thesis, Massachusetts Institute of Technology.
33. Maggioni, M., Katkovnik, V., Egiazarian, K., and Foi, A. 2013. A nonlocal transform-domain filter for volumetric data denoising and reconstruction. IEEE Trans. on Image Processing, 1, 119–133.
34. Martin, D., Fowlkes, C., Tal, D., and Malik, J. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. ICCV.
35. Paris, S., and Durand, F. 2009. A fast approximation of the bilateral filter using a signal processing approach. International Journal of Computer Vision 81, 24–52.
36. Portilla, J., Strela, V., Wainwright, M. J., and Simoncelli, E. P. 2003. Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Trans. on Image Processing 12, 11, 1338–1351.
37. Reinhard, E., Ward, G., Pattanaik, S. N., Debevec, P. E., and Heidrich, W. 2010. High Dynamic Range Imaging – Acquisition, Display, and Image-Based Lighting (2. ed.). Academic Press.
38. Roth, S., and Black, M. J. 2005. Fields of experts: a framework for learning image priors. In Proc. CVPR.
39. Sen, P., Kalantari, N. K., Yaesoubi, M., Darabi, S., Goldman, D. B., and Shechtman, E. 2012. Robust patch-based hdr reconstruction of dynamic scenes. ACM Trans. Graph. (Proc. of SIGGRAPH) 31, 6, 203:1–203:11.
40. Tico, M. 2008. Multiframe image denoising and stabilization. In EUSIPCO.
41. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839–846.
42. Viola, P., and Jones, M. 2001. Robust real-time object detection. In International Journal of Computer Vision.
43. Zhang, M., and Gunturk, B. K. 2008. Multiresolution bilateral filtering for image denoising. IEEE Trans. on Image Processing 17, 12, 2324–2333.
44. Zhang, L., and Wu, X. 2005. Color demosaicking via directional linear minimum mean square-error estimation. TIP 14, 12, 2167–2178.
45. Zhang, L., Vaddadi, S., Jin, H., and Nayar, S. K. 2009. Multiple view image denoising. In Proc. CVPR, 1542–1549.
46. Zontak, M., Mosseri, I., and Irani, M. 2013. Separating signal from noise using patch recurrence across scales. In Proc. CVPR.
