“Distortion-free wide-angle portraits on camera phones” by Shih, Lai and Liang
Conference:
Type:
Title:
- Distortion-free wide-angle portraits on camera phones
Session/Category Title: Photo Science
Presenter(s)/Author(s):
Abstract:
Photographers take wide-angle shots to enjoy expanding views, group portraits that never miss anyone, or composite subjects with spectacular scenery background. In spite of the rapid proliferation of wide-angle cameras on mobile phones, a wider field-of-view (FOV) introduces a stronger perspective distortion. Most notably, faces are stretched, squished, and skewed, to look vastly different from real-life. Correcting such distortions requires professional editing skills, as trivial manipulations can introduce other kinds of distortions. This paper introduces a new algorithm to undistort faces without affecting other parts of the photo. Given a portrait as an input, we formulate an optimization problem to create a content-aware warping mesh which locally adapts to the stereographic projection on facial regions, and seamlessly evolves to the perspective projection over the background. Our new energy function performs effectively and reliably for a large group of subjects in the photo. The proposed algorithm is fully automatic and operates at an interactive rate on the mobile platform. We demonstrate promising results on a wide range of FOVs from 70° to 120°.
References:
1. Michael D Abràmoff, Paulo J Magalhàes, and Sunanda J Ram. 2004. Image processing with ImageJ. Biophotonics international 11, 7 (2004), 36–42.Google Scholar
2. Sameer Agarwal, Keir Mierle, and Others. 2012. Ceres Solver. http://ceres-solver.org.Google Scholar
3. Aseem Agarwala, Maneesh Agrawala, Michael Cohen, David Salesin, and Richard Szeliski. 2006. Photographing long scenes with multi-viewpoint panoramas. In ACM TOG, Vol 25. 853–861. Google ScholarDigital Library
4. Dimitri P Bertsekas. 1999. Nonlinear programming.Google Scholar
5. Duane C Brown. 1966. Decentering distortion of lenses. Photogrammetric Engineering and Remote Sensing (1966).Google Scholar
6. Robert Carroll, Aseem Agarwala, and Maneesh Agrawala. 2010. Image warps for artistic perspective manipulation. In ACM TOG, Vol. 29. 127. Google ScholarDigital Library
7. Robert Carroll, Maneesh Agrawal, and Aseem Agarwala. 2009. Optimizing content-preserving projections for wide-angle images. In ACM TOG, Vol. 28. 43. Google ScholarDigital Library
8. Che-Han Chang and Yung-Yu Chuang. 2012. A line-structure-preserving approach to image resizing. In Proc. CVPR. 1075–1082. Google ScholarDigital Library
9. Che-Han Chang, Chia-Kai Liang, and Yung-Yu Chuang. 2011. Content-aware display adaptation and interactive editing for stereoscopic images. IEEE TMM 13, 4 (2011), 589–601. Google ScholarDigital Library
10. Che-Han Chang, Yoichi Sato, and Yung-Yu Chuang. 2014. Shape-preserving half-projective warps for image stitching. In Proc. CVPR. 3254–3261. Google ScholarDigital Library
11. Yu-Sheng Chen and Yung-Yu Chuang. 2016. Natural image stitching with the global similarity prior. In Proc. ECCV. 186–201.Google ScholarCross Ref
12. Song-Pei Du, Shi-Min Hu, and Ralph R Martin. 2013. Changing perspective in stereoscopic images. IEEE TVCG 19, 8 (2013), 1288–1297. Google ScholarDigital Library
13. Claude E Duchon. 1979. Lanczos filtering in one and two dimensions. Journal of applied meteorology 18, 8 (1979), 1016–1022.Google ScholarCross Ref
14. DxO. 2018. Correcting volume deformation with DxO ViewPoint. https://www.dxo.com/dxo-viewpoint/features/.Google Scholar
15. Ohad Fried, Eli Shechtman, Dan B Goldman, and Adam Finkelstein. 2016. Perspective-aware manipulation of portrait photos. ACM TOG 35, 4 (2016), 128. Google ScholarDigital Library
16. Ran Gal, Olga Sorkine, and Daniel Cohen-Or. 2006. Feature-aware texturing. Rendering Techniques 2006, 17 (2006), 2. Google ScholarDigital Library
17. GoPro. 2017. HERO6 Black Field of View (FOV) Information. https://gopro.com/help/articles/question_answer/hero6-black-field-of-view-fov-information.Google Scholar
18. Heng Guo, Shuaicheng Liu, Tong He, Shuyuan Zhu, Bing Zeng, and Moncef Gabbouj. 2016. Joint video stitching and stabilization from moving cameras. IEEE TIP 25, 11, 5491–5503.Google Scholar
19. Kaiming He, Huiwen Chang, and Jian Sun. 2013. Rectangling panoramic images via warping. ACM TOG 32, 4 (2013), 79. Google ScholarDigital Library
20. Chintan Intwala and Aseem Agarwala. 2015. Perspective warp. US Patent 9,117,253.Google Scholar
21. Hamid Izadinia, Bryan C Russell, Ali Farhadi, Matthew D Hoffman, and Aaron Hertzmann. 2015. Deep classifiers from image tags in the wild. In Proc. 2015 Workshop on Community-Organized Multimodal Mining: Opportunities for Novel Solutions. 13–18. Google ScholarDigital Library
22. Yoshihiro Kanamori, Nguyen Huu Cuong, and Tomoyuki Nishita. 2011. Local optimization of distortions in wide-angle images using moving least-squares. In Proceedings of the 27th Spring Conference on Computer Graphics. ACM, 51–56. Google ScholarDigital Library
23. Johannes Kopf, Dani Lischinski, Oliver Deussen, Daniel Cohen-Or, and Michael Cohen. 2009. Locally adapted projections to reduce panorama distortions. In CGF, Vol. 28. 1083–1089. Google ScholarDigital Library
24. Hyunjoon Lee, Eli Shechtman, Jue Wang, and Seungyong Lee. 2012. Automatic upright adjustment of photographs. In Proc. CVPR. 877–884. Google ScholarDigital Library
25. Tommer Leyvand, Daniel Cohen-Or, Gideon Dror, and Dani Lischinski. 2008. Data-driven enhancement of facial attractiveness. ACM TOG 27, 3 (2008), 38. Google ScholarDigital Library
26. Dongping Li, Kaiming He, Jian Sun, and Kun Zhou. 2015. A geodesic-preserving method for image warping. In Proc. CVPR. 213–221.Google Scholar
27. Feng Liu, Michael Gleicher, Hailin Jin, and Aseem Agarwala. 2009. Content-preserving warps for 3D video stabilization. In ACM TOG, Vol. 28. ACM, 44. Google ScholarDigital Library
28. Feng Liu, Michael Gleicher, Jue Wang, Hailin Jin, and Aseem Agarwala. 2011. Subspace video stabilization. ACM TOG 30, 1 (2011), 4. Google ScholarDigital Library
29. Darko Pavić, Volker Schönefeld, and Leif Kobbelt. 2006. Interactive image completion with perspective correction. The Visual Computer 22, 9–11 (2006), 671–681. Google ScholarDigital Library
30. Samsung. 2017. GalaxyS7: About Shape Correction. www.samsung.com/nz/support/mobile-devices/galaxy-s7-about-shape-correction/.Google Scholar
31. Thomas K Sharpless, Bruno Postle, and Daniel M German. 2010. Pannini: a new projection for rendering wide angle perspective images. In Proc. 6th international conference on Computational Aesthetics in Graphics, Visualization and Imaging. 9–16. Google ScholarDigital Library
32. John Parr Snyder. 1987. Map projections-A working manual. Vol. 1395. US Government Printing Office.Google Scholar
33. Mahdi Abbaspour Tehrani, Aditi Majumder, and M Gopi. 2016. Correcting perceived perspective distortions using object specific planar transformations. In Proc. ICCP.Google ScholarCross Ref
34. TensorFlow. 2018. TensorFlow-Mobile/Lite. https://www.tensorflow.org/mobile/.Google Scholar
35. Kim H Veltman. 1986. Perspective, anamorphosis and vision. Marburger Jahrbuch für (1986).Google Scholar
36. Dhanraj Vishwanath, Ahna R Girshick, and Martin S Banks. 2005. Why pictures look right when viewed from the wrong place. Nature neuroscience 8, 10 (2005), 1401.Google Scholar
37. Neal Wadhwa, Rahul Garg, David Jacobs, Bryan Feldman, Nori Kanazawa, Robert Carroll, Yair Movshovitz-Attias, Jonathan Barron, Yael Pritch, and Marc Levoy. 2018. Synthetic depth-of-field with a single-camera mobile phone. ACM TOG 37, 4 (2018). Google ScholarDigital Library
38. Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkine, and Tong-Yee Lee. 2008. Optimized scale-and-stretch for image resizing. In ACM TOG, Vol. 27. 118. Google ScholarDigital Library
39. Jin Wei, Chen-Feng Li, Shi-Min Hu, Ralph R Martin, and Chiew-Lan Tai. 2012. Fisheye video correction. IEEE TVCG 18, 10 (2012), 1771–1783. Google ScholarDigital Library
40. Lior Wolf, Moshe Guttmann, and Daniel Cohen-Or. 2007. Non-homogeneous content-driven video-retargeting. (2007).Google Scholar
41. Lihi Zelnik-Manor, Gabriele Peters, and Pietro Perona. 2005. Squaring the circle in panoramas. In Proc. ICCV, Vol. 2. 1292–1299. Google ScholarDigital Library
42. David Zhang, Qijun Zhao, and Fangmei Chen. 2011. Quantitative analysis of human facial beauty using geometric features. Pattern Recognition 44, 4 (2011), 940–950. Google ScholarDigital Library
43. Fang-Lue Zhang, Xian Wu, Hao-Tian Zhang, Jue Wang, and Shi-Min Hu. 2016. Robust background identification for dynamic video editing. ACM TOG 35, 6 (2016), 197. Google ScholarDigital Library
44. Zhengyou Zhang. 2000. A flexible new technique for camera calibration. IEEE Trans. PAMI 22 (2000). Google ScholarDigital Library
45. Denis Zorin and Alan H Barr. 1995. Correction of geometric perceptual distortions in pictures. In SIGGRAPH. 257–264. Google ScholarDigital Library
