“Personal Photo Enhancement Using Example Images” by Joshi, Matusik, Adelson and Kriegman

We describe a framework for improving the quality of personal photos by using a person’s favorite photographs as examples. We observe that the majority of a person’s photographs include the faces of a photographer’s family and friends and often the errors in these photographs are the most disconcerting. We focus on correcting these types of images and use common faces across images to automatically perform both global and face-specific corrections. Our system achieves this by using face detection to align faces between “good” and “bad” photos such that properties of the good examples can be used to correct a bad photo. These “personal” photos provide strong guidance for a number of operations and, as a result, enable a number of high-quality image processing operations. We illustrate the power and generality of our approach by presenting a novel deblurring algorithm, and we show corrections that perform sharpening, superresolution, in-painting of over- and underexposured regions, and white-balancing.

1. Agarwala, A., Dontcheva, M., Agarwala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Trans. Graph. 23, 3, 294–302. 
2. Agrawal, A., Raskar, R., Nayar, S. K., and Li, Y. 2005. Removing photography artifacts using gradient projection and flash-exposure sampling. ACM Trans. Graph. 24, 3, 828–835. 
3. Apostoloff, N. and Fitzgibbon, A. 2004. Bayesian video matting using learnt image priors. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’04). Vol. 01. 407–414.
4. Bae, S., Paris, S., and Durand, F. 2006. Two-Scale tone management for photographic look. ACM Trans. Graph. 25, 3, 637–645. 
5. Baker, S. and Kanade, T. 2000. Hallucinating faces. In Proceedings of the 4th International Conference on Automatic Face and Gesture Recognition. 
6. Barrow, H. and Tenenbaum, J. 1978. Recovering intrinsic scene characteristics from images. Comput. Vision Syst., 3–26.
7. Bascle, B., Blake, A., and Zisserman, A. 1996. Motion deblurring and super-resolution from an image sequence. In Proceedings of the 4th European Conference on Computer Vision-Volume II (ECCV’96). Springer, 573–582. 
8. Blanz, V. and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In Proceedings of SIGGRAPH 99. 187–194. 
9. Eisemann, E. and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. 23, 3, 673–678. 
10. Elad, M. and Aharon, M. 2006. Image denoising via learned dictionaries and sparse representation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06). IEEE Computer Society, 895–900. 
11. Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., and Freeman, W. T. 2006. Removing camera shake from a single photograph. ACM Trans. Graph. 25, 787–794. 
12. Finlayson, G. D., Drew, M. S., and Lu, C. 2004. Intrinsic images by entropy minimization. In Proceedings of the IEEE International Conference on Computer Vision (ECCV). 582–595.
13. Fitzgibbon, A., Wexler, Y., and Zisserman, A. 2005. Image-based rendering using image-based priors. Int. J. Comput. Vision 63, 2, 141–151. 
14. Freeman, W. T., Jones, T. R., and Pasztor, E. C. 2002. Example-based super-resolution. IEEE Comput. Graph. Appl. 22, 2, 56–65. 
15. Gross, R., Matthews, I., and Baker, S. 2005. Generic vs. person specific active appearance models. Image Vision Comput. 23, 11, 1080–1093. 
16. Hays, J. and Efros, A. A. 2007. Scene completion using millions of photographs. In ACM SIGGRAPH Papers (SIGGRAPH’07). ACM, New York, 4. 
17. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’01). ACM, New York, 327–340. 
18. Land, E. H. and McCann, J. J. 1971. Lightness and retinex theory. J. Optical Soc. Amer. 61, 1–11.
19. Levin, A., Fergus, R., Durand, F., and Freeman, W. T. 2007. Image and depth from a conventional camera with a coded aperture. In ACM SIGGRAPH Papers (SIGGRAPH’07). ACM Press, New York, 70. 
20. Levin, A., Lischinski, D., and Weiss, Y. 2006. A closed form solution to natural image matting. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06). IEEE Computer Society, 61–68. 
21. Levin, A., Zomat, A., Peleg, S., and Weiss, Y. 2004. Seamless image stitching in the gradient domain. In IEEE International Conference on Computer Vision (ECCV’04).
22. Levin, A., Zomet, A., and Weiss, Y. 2003. Learning how to inpaint from global image statistics. In Proceedings of the 9th IEEE International Conference on Computer Vision (ICCV’03). 305. 
23. Leyvand, T., Cohen-Or, D., Dror, G., and Lischinski, D. 2006. Digital face beautification. In ACM SIGGRAPH Sketches (SIGGRAPH’06). 169. 
24. Liu, C., Freeman, W. T., Szeliski, R., and Kang, S. B. 2006. Noise estimation from a single image. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06). IEEE Computer Society, 901–908. 
25. Liu, C., Shum, H.-Y., and Freeman, W. T. 2007. Face hallucination: Theory and practice. Int. J. Comput. Vision 75, 1, 115–134. 
26. Liu, C., Shum, H.-Y., and Zhang, C. 2001. A two-step approach to hallucinating faces: Global parametric model and local nonparametric model. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’01). 192–198.
27. Lucy, L. 1974. Bayesian-Based iterative method of image restoration. J. Astro. 79, 745–754.
28. Penev, P. S. and Sirovich, L. 2000. The global dimensionality of face space. In Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition 2000 (FG’00). IEEE Computer Society, 264. 
29. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. Graph. 22, 3, 313–318. 
30. Petschnigg, G., Szeliski, R., Agarwala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23, 3, 664–672. 
31. Rav-Acha, A. and Peleg, S. 2005. Two motion-blurred images are better than one. Pattern Recogn. Lett. 26, 3, 311–317. 
32. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 21, 5, 34–41. 
33. Richardson, W. 1972. Bayesian-Based iterative method of image restoration. J. Optical Soc. Amer. A 62, 55–59.
34. Roth, S. and Black, M. J. 2005. Fields of experts: A framework for learning image priors. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’05). 860–867. 
35. Rother, C., Bordeaux, L., Hamadi, Y., and Blake, A. 2006. Autocollage. In ACM SIGGRAPH Papers (SIGGRAPH’06). ACM Press, New York, 847–852. 
36. Tappen, M. F., Adelson, E. H., and Freeman, W. T. 2006. Estimating intrinsic component images using non-linear regression. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’06). 1992–1999. 
37. Turk, M. A. and Pentland, A. P. 1991. Face recognition using eigenfaces. In Proceedings of the Computer Vision and Pattern Recognition (CVPR’91). 586–591.
38. van de Weijer, J., Gevers, T., and Gijsenij, A. 2007. Edge-Based color constancy. In Proceedings of the IEEE Conference on IEEE Trans. Image Process. 16, 9, 2207–2214. 
39. Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’61). 609–615.
40. Weiss, Y. 2001. Deriving intrinsic images from image sequences. http://www.ai.mit.edu/courses/6.899/papers/13-02.PDF
41. Yuan, L., Sun, J., Quan, L., and Shum, H.-Y. 2007. Image deblurring with blurred/noisy image pairs. In ACM SIGGRAPH Papers (SIGGRAPH’07). ACM, New York, 1.