“Enabling warping on stereoscopic images”
Conference:
Type(s):
Title:
- Enabling warping on stereoscopic images
Session/Category Title: Stereo and Displays
Presenter(s)/Author(s):
Abstract:
Warping is one of the basic image processing techniques. Directly applying existing monocular image warping techniques to stereoscopic images is problematic as it often introduces vertical disparities and damages the original disparity distribution. In this paper, we show that these problems can be solved by appropriately warping both the disparity map and the two images of a stereoscopic image. We accordingly develop a technique for extending existing image warping algorithms to stereoscopic images. This technique divides stereoscopic image warping into three steps. Our method first applies the user-specified warping to one of the two images. Our method then computes the target disparity map according to the user specified warping. The target disparity map is optimized to preserve the perceived 3D shape of image content after image warping. Our method finally warps the other image using a spatially-varying warping method guided by the target disparity map. Our experiments show that our technique enables existing warping methods to be effectively applied to stereoscopic images, ranging from parametric global warping to non-parametric spatially-varying warping.
References:
1. Basha, T., Moses, Y., and Avidan, S. 2011. Geometrically consistent stereo seam carving. In IEEE International Conference on Computer Vision.
2. Chang, C.-H., Liang, C.-K., and Chuang, Y.-Y. 2011. Content-aware display adaptation and interactive editing for stereoscopic images. IEEE Trans. on Multimedia 13, 589–601.
3. Cutting, J. E., and Vishton, P. M. 1995. Perceiving layout and knowing distances: the integration, relative potency and contextual use of different information about depth. In Handbook of perception and Cognition., W. Epstein and S. Rogers, Eds., vol. 5: Perception of Space and Motion. 69–117.
4. Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., and Seidel, H.-P. 2011. A perceptual model for disparity. ACM Trans. Graph. 30, 4, 96:1–96:10.
5. Gal, R., Sorkine, O., and Cohen-Or, D. 2006. Feature-aware texturing. In Proceedings of EUROGRAPHICS Symposium on Rendering, 297–303.
6. Gomes, J., Darsa, L., Costa, B., and Velho, L. 1999. Warping and Morphing of Graphical Objects. Morgan Kaufmann Publishers.
7. Guttmann, M., Wolf, L., and Cohen-Or, D. 2009. Semi-automatic stereo extraction from video footage. In IEEE International Conference on Computer Vision, 136–142.
8. Harel, J., Koch, C., and Perona, P. 2007. Graph-based visual saliency. In NIPS, vol. 19, 545–552.
9. Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision. Cambridge University Press.
10. Heckbert, P. S. 1989. Fundamentals of texture mapping and image warping. Tech. rep., UC Berkeley.
11. Heinzle, S., Greisen, P., Gallup, D., Chen, C., Saner, D., Smolic, A., Burg, A., Matusik, W., and Gross, M. 2011. Computational stereo camera system with programmable control loop. ACM Trans. Graph. 30, 94:1–94:10.
12. Hirschmuller, H., and Scharstein, D. 2009. Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1582–1599.
13. Hoffman, D., Girshick, A., Akeley, K., and Banks, M. 2008. Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. Journal of Vision 8, 3, 1–30.
14. Howard, I. P., and Rogers, B. J. 2002. Seeing in Depth. Oxford University Press.
15. Jones, G., Lee, D., Holliman, N., and Ezra, D. 2001. Controlling perceived depth in stereoscopic images. In Stereoscopic Displays and Virtual Reality Systems VIII, 42–53.
16. Kim, C., Hornung, A., Heinzle, S., Matusik, W., and Gross, M. 2011. Multi-perspective stereoscopy from light fields. ACM Trans. Graph. 30, 6, 190:1–190:10.
17. Knorr, S., and Sikora, T. 2007. An image-based rendering (ibr) approach for realistic stereo view synthesis of tv broadcast based on structure from motion. In IEEE International Conference on Image Processing, 572–575.
18. Koppal, S., Zitnick, C., Cohen, M., Kang, S. B., Ressler, B., and Colburn, A. 2011. A viewer-centric editor for 3d movies. IEEE Computer Graphics and Applications 31, 20–35.
19. Krähenbühl, P., Lang, M., Hornung, A., and Gross, M. 2009. A system for retargeting of streaming video. ACM Trans. Graph. 28, 5, 126:1–126:10.
20. Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., and Gross, M. 2010. Nonlinear disparity mapping for stereoscopic 3d. ACM Transactions on Graphics 29, 4.
21. Lee, K.-Y., Chung, C.-D., and Chuang, Y.-Y. 2012. Scene warping: Layer-based stereoscopic image resizing. In IEEE Conf. on Computer Vision and Pattern Recognition, 49–56.
22. Liu, F., Gleicher, M., Jin, H., and Agarwala, A. 2009. Content-preserving warps for 3d video stabilization. ACM Transactions on Graphics 28, 3, 44.
23. Liu, C.-W., Huang, T.-H., Chang, M.-H., Lee, K.-Y., Liang, C.-K., and Chuang, Y.-Y. 2011. 3d cinematography principles and their applications to stereoscopic media processing. In ACM international conference on Multimedia, 253–262.
24. Lo, W.-Y., van Baar, J., Knaus, C., Zwicker, M., and Gross, M. 2010. Stereoscopic 3d copy & paste. ACM Transactions on Graphics 29, 6, 147:1–147:10.
25. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91–110.
26. Mendiburu, B. 2009. 3D Movie Making: Stereoscopic Digital Cinema from Script to Screen. Focal Press.
27. Moustakas, K., Tzovaras, D., and Strintzis, M. 2008. Stereoscopic video generation based on efficient layered structure and motion estimation from a monoscopic image sequence. IEEE Trans. Circuits Syst. Video Technol. 15, 8, 1065–1073.
28. Mueller, R., Ward, C., and Hušák, M. 2008. A systematized wysiwyg pipeline for digital stereoscopic 3d filmmaking. In Proceedings of SPIE, vol. 6803.
29. Oskam, T., Hornung, A., Bowles, H., Mitchell, K., and Gross, M. 2011. Oscam – optimized stereoscopic camera control for interactive 3d. ACM Trans. Graph. 30, 6, 189:1–189:8.
30. Saxena, A., Sun, M., and Ng, A. 2009. Make3d: Learning 3d scene structure from a single still image. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 5, 824–840.
31. Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47 (April), 7–42.
32. Smolic, A., Poulakos, S., Heinzle, S., Greisen, P., Lang, M., Hornung, A., Farre, M., Stefanoski, N., Wang, O., Schnyder, L., Monroy, R., and Gross, M. 2011. Disparity-aware stereo 3d production tools. In European Conference on Visual Media Production.
33. Sun, D., Roth, S., and Black, M. J. 2010. Secrets of optical flow estimation and their principles. In IEEE Conference on Computer Vision and Pattern Recognition, 2432–2439.
34. Szeliski, R. 2010. Computer Vision: Algorithms and Applications. Springer.
35. Wang, C., and Sawchuk, A. A. 2008. Disparity manipulation for stereo images and video. In Proc. SPIE, Vol. 6803.
36. Wang, L., Jin, H., Yang, R., and Gong, M. 2008. Stereoscopic inpainting: Joint color and depth completion from stereo images. In IEEE Conf. on Comput. Vision and Pattern Recogn.
37. Wang, Y.-S., Tai, C.-L., Sorkine, O., and Lee, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM Transactions on Graphics 27, 5.
38. Wolberg, G. 1990. Digital image warping. Wiley-IEEE Computer Society Press.
39. Zilly, F., Muller, M., Eisert, P., and Kauff, P. 2010. The stereoscopic analyzer-an image-based assistance tool for stereo shooting and 3d production. In IEEE International Conference on Image Processing, 4029–4032.
