“Stereoscopizing cel animations”
Conference:
Type(s):
Title:
- Stereoscopizing cel animations
Session/Category Title: Eyes Wide Open
Presenter(s)/Author(s):
Abstract:
While hand-drawn cel animation is a world-wide popular form of art and entertainment, introducing stereoscopic effect into it remains difficult and costly, due to the lack of physical clues. In this paper, we propose a method to synthesize convincing stereoscopic cel animations from ordinary 2D inputs, without labor-intensive manual depth assignment nor 3D geometry reconstruction. It is mainly automatic due to the need of producing lengthy animation sequences, but with the option of allowing users to adjust or constrain all intermediate results. The system fits nicely into the existing production flow of cel animation. By utilizing the T-junction cue available in cartoons, we first infer the initial, but not reliable, ordering of regions. One of our major contributions is to resolve the temporal inconsistency of ordering by formulating it as a graph-cut problem. However, the resultant ordering remains insufficient for generating convincing stereoscopic effect, as ordering cannot be directly used for depth assignment due to its discontinuous nature. We further propose to synthesize the depth through an optimization process with the ordering formulated as constraints. This is our second major contribution. The optimized result is the spatiotemporally smooth depth for synthesizing stereoscopic effect. Our method has been evaluated on a wide range of cel animations and convincing stereoscopic effect is obtained in all cases.
References:
1. Amer, M., Raich, R., and Todorovic, S. 2010. Monocular extraction of 2.1 d sketch. In Proc. of the International Conference on Image Processing.
2. Apostoloff, N., and Fitzgibbon, A. 2005. Learning spatiotemporal T-junctions for occlusion detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 2, IEEE, 553–559.
3. Ashikhmin, M. 2001. Synthesizing natural textures. In Proceedings of the 2001 symposium on Interactive 3D graphics, ACM, 217–226.
4. Assa, J., and Wolf, L. 2007. Diorama construction from a single image. In Computer Graphics Forum, vol. 26, Wiley Online Library, 599–608.
5. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. Pattern Analysis and Machine Intelligence, IEEE Transactions on 23, 11, 1222–1239.
6. Bruce, V., Green, P. R., and Georgeson, M. A. 2003. Visual Perception: Physiology, Psychology, & Ecology. Psychology Press.
7. Criminisi, A., Pérez, P., and Toyama, K. 2004. Region filling and object removal by exemplar-based image inpainting. Image Processing, IEEE Transactions on 13, 9, 1200–1212.
8. Dimiccoli, M., and Salembier, P. 2009. Exploiting t-junctions for depth segregation in single images. In Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on, IEEE, 1229–1232.
9. Dimiccoli, M., and Salembier, P. 2009. Hierarchical region-based representation for segmentation and filtering with depth in single images. In Image Processing (ICIP), 2009 16th IEEE International Conference on, IEEE, 3533–3536.
10. Forsyth, D. 2001. Shape from texture and integrability. In Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, vol. 2, IEEE, 447–452.
11. Gill, P., Murray, W., Saunders, M., and Wright, M. 1984. Procedures for optimization problems with a mixture of bounds and general linear constraints. ACM Transactions on Mathematical Software (TOMS) 10, 3, 282–298.
12. Gingold, Y., Igarashi, T., and Zorin, D. 2009. Structured annotations for 2d-to-3d modeling. In ACM Transactions on Graphics (TOG), vol. 28, ACM, 148.
13. Goldberg, E. 2009. Medial axis techniques for stereoscopic extraction. In SIGGRAPH 2009: Talks, ACM, 74.
14. Guzmán, A. 1968. Decomposition of a visual scene into three dimensional bodies. In Proceedings of the December 9-11, 1968, fall joint computer conference, part I, AFIPS ’68 (Fall, part I), 291–304.
15. He, K., Sun, J., and Tang, X. 2009. Single image haze removal using dark channel prior. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, IEEE, 1956–1963.
16. Horn, B. 1990. Height and gradient from shading. International journal of computer vision 5, 1, 37–75.
17. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: a sketching interface for 3d freeform design. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., 409–416.
18. Jia, Z., Gallagher, A., Chang, Y., and Chen, T. 2012. A learning-based framework for depth ordering. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, IEEE, 294–301.
19. Joshi, P., and Carr, N. 2008. Repoussé: Automatic inflation of 2D artwork. In Eurographics Workshop on Sketch-Based Interfaces and Modeling, The Eurographics Association, 49–55.
20. Kahn, A. 1962. Topological sorting of large networks. Communications of the ACM 5, 11, 558–562.
21. Kang, S., and Szeliski, R. 2004. Extracting view-dependent depth maps from a collection of images. International Journal of Computer Vision 58, 2, 139–163.
22. Karpenko, O., and Hughes, J. 2006. SmoothSketch: 3D freeform shapes from complex sketches. In ACM Transactions on Graphics (TOG), vol. 25, ACM, 589–598.
23. Kim, Y., Winnemöller, H., and Lee, S. 2013. Wysiwyg stereo painting. In Proceedings of ACM Symposium on Interactive 3D Graphics and Games 2013.
24. Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., and Gross, M. 2010. Nonlinear disparity mapping for stereoscopic 3d. ACM Transactions on Graphics (TOG) 29, 4, 75.
25. Lee, S., Feng, D., and Gooch, B. 2008. Automatic construction of 3d models from architectural line drawings. In Proceedings of the 2008 symposium on Interactive 3D graphics and games, ACM, 123–130.
26. Lipson, H., and Shpitalni, M. 1996. Optimization-based reconstruction of a 3d object from a single freehand line drawing. Computer-Aided Design 28, 8, 651–663.
27. Metzger, W. 1936. Gesetze des sehens. W. Kramer.
28. Nayar, S., and Nakagawa, Y. 1990. Shape from focus: An effective approach for rough surfaces. In Robotics and Automation, 1990. Proceedings., 1990 IEEE International Conference on, IEEE, 218–225.
29. Nealen, A., Igarashi, T., Sorkine, O., and Alexa, M. 2007. Fibermesh: designing freeform surfaces with 3D curves. In ACM Transactions on Graphics (TOG), vol. 26, ACM, 41.
30. Noris, G., Hornung, A., Sumner, R. W., Simmons, M., and Gross, M. 2013. Topology-driven vectorization of clean line drawings. ACM Transactions on Graphics (TOG) 32, 1, 4.
31. Production I. G., Sanzigen Animation Studio, and Ishimori Productions, 2012. Cyborg 009.
32. Production I.G., 2011. Ghost in the shell: Stand alone complex: Solid state society.
33. Rademacher, P. 1999. View-dependent geometry. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, 439–446.
34. Rivers, A., Igarashi, T., and Durand, F. 2010. 2-5D cartoon models. ACM Transactions on Graphics (TOG) 29, 4, 59.
35. Schar, S., Bieri, H., Killer, T., and Jiang, X. 2008. Introducing stereo effects into cel animations. In 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video, 2008, IEEE, 353–356.
36. Sun, J., Yuan, L., Jia, J., and Shum, H. 2005. Image completion with structure propagation. ACM Transactions on Graphics (ToG) 24, 3, 861–868.
37. Super, B., and Bovik, A. 1995. Shape from texture using local spectral moments. Pattern Analysis and Machine Intelligence, IEEE Transactions on 17, 4, 333–343.
38. Sýkora, D., Sedlacek, D., Jinchao, S., Dingliana, J., and Collins, S. 2010. Adding depth to cartoons using sparse depth (in) equalities. In Computer Graphics Forum, vol. 29, Wiley Online Library, 615–623.
39. Tokyo Movie Shinsha, 1977. Ie naki ko.
40. Varley, P., and Martin, R. 2002. Estimating depth from line drawing. In Proceedings of the seventh ACM symposium on Solid modeling and applications, ACM, 180–191.
41. Ventura, J., Diverdi, S., and Höllerer, T. 2009. A sketch-based interface for photo pop-up. In Proceedings of the 6th Eurographics Symposium on Sketch-Based Interfaces and Modeling, ACM, 21–28.
42. Wang, O., Lang, M., Frei, M., Hornung, A., Smolic, A., and Gross, M. 2011. Stereobrush: interactive 2d to 3d conversion using discontinuous warps. In Proceedings of the Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling, ACM, 47–54.
43. Ward, B., Kang, S. B., and Bennett, E. P. 2011. Depth director: A system for adding depth to movies. Computer Graphics and Applications, IEEE 31, 1, 36–48.
44. Wu, T., Sun, J., Tang, C., and Shum, H. 2008. Interactive normal reconstruction from a single image. ACM Transactions on Graphics (TOG) 27, 5, 119.
45. Zhang, L., Dugas-Phocion, G., Samson, J., and Seitz, S. 2002. Single-view modeling of free-form scenes. The Journal of Visualization and Computer Animation 13, 4, 225–235.
46. Zhang, G., Jia, J., Wong, T., and Bao, H. 2008. Recovering consistent video depth maps via bundle optimization. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, IEEE, 1–8.
47. Zhang, S., Chen, T., Zhang, Y., Hu, S., and Martin, R. 2009. Vectorizing cartoon animations. Visualization and Computer Graphics, IEEE Transactions on 15, 4, 618–629.
