“Joint view expansion and filtering for automultiscopic 3D displays”
Conference:
Type(s):
Title:
- Joint view expansion and filtering for automultiscopic 3D displays
Session/Category Title: Eyes Wide Open
Presenter(s)/Author(s):
Abstract:
Multi-view autostereoscopic displays provide an immersive, glasses-free 3D viewing experience, but they require correctly filtered content from multiple viewpoints. This, however, cannot be easily obtained with current stereoscopic production pipelines. We provide a practical solution that takes a stereoscopic video as an input and converts it to multi-view and filtered video streams that can be used to drive multi-view autostereoscopic displays. The method combines a phase-based video magnification and an interperspective antialiasing into a single filtering process. The whole algorithm is simple and can be efficiently implemented on current GPUs to yield a near real-time performance. Furthermore, the ability to retarget disparity is naturally supported. Our method is robust and works well for challenging video scenes with defocus blur, motion blur, transparent materials, and specularities. We show that our results are superior when compared to the state-of-the-art depth-based rendering methods. Finally, we showcase the method in the context of a real-time 3D videoconferencing system that requires only two cameras.
References:
1. Akeley, K., Watt, S. J., Girshick, A. R., and Banks, M. S. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graph. 23, 3, 804–813.
2. Balogh, T. 2006. The holovizio system. In Electronic Imaging 2006, 60550U-60550U.
3. Brown, M. Z., Burschka, D., and Hager, G. D. 2003. Advances in computational stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 8, 993–1008.
4. Chai, J.-X., Tong, X., Chan, S.-C., and Shum, H.-Y. 2000. Plenoptic sampling. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., 307–318.
5. Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., and Seidel, H.-P. 2010. Adaptive image-based stereo view synthesis. In Proc. VMV.
6. Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., Seidel, H.-P., and Matusik, W. 2012. A luminance-contrast-aware disparity model and applications. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 31, 6, 184:1–184:10.
7. Farre, M., Wang, O., Lang, M., Stefanoski, N., Hornung, A., and Smolic, A. 2011. Automatic content creation for multiview autostereoscopic displays using image domain warping. In IEEE International Conference on Multimedia and Expo.
8. Holliman, N. S., Dodgson, N. A., Favalora, G. E., and Pockett, L. 2011. Three-dimensional displays: a review and applications analysis. IEEE Transactions on Broadcasting 57, 2, 362–371.
9. Hosni, A., Rhemann, C., Bleyer, M., Rother, C., and Gelautz, M. 2013. Fast cost-volume filtering for visual correspondence and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 2, 504–511.
10. Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically reparameterized light fields. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing Co., 297–306.
11. Lambooij, M., IJsselsteijn, W., Fortuin, M., and Heynderickx, I. 2009. Visual discomfort and visual fatigue of stereoscopic displays: a review. J Imaging Science and Technology 53, 030201–14.
12. Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., and Gross, M. 2010. Nonlinear disparity mapping for stereoscopic 3D. ACM Trans. Graph. 29, 4, 75:1–75:10.
13. Levoy, M., and Hanrahan, P. 1996. Light field rendering. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, ACM, 31–42.
14. Lipton, L., and Feldman, M. H. 2002. New autostereoscopic display technology: the synthagram. In Electronic Imaging 2002, International Society for Optics and Photonics, 229–235.
15. Liu, C., Torralba, A., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Trans. Graph. 24, 3, 519–526.
16. Matusik, W., and Pfister, H. 2004. 3d tv: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Trans. Graph. 23, 3, 814–824.
17. Richardt, C., Stoll, C., Dodgson, N. A., Seidel, H.-P., and Theobalt, C. 2012. Coherent spatiotemporal filtering, upsampling and rendering of RGBZ videos. Computer Graphics Forum (Proc. Eurographics) 31, 2, 247–256.
18. Schmidt, A., and Grasnick, A. 2002. Multiviewpoint autostereoscopic dispays from 4d-vision gmbh. In Electronic Imaging 2002, 212–221.
19. Simoncelli, E. P., and Freeman, W. T. 1995. The steerable pyramid: A flexible architecture for multi-scale derivative computation. In IEEE International Conference on Image Processing, vol. 3, 444–447.
20. Simoncelli, E. P., Freeman, W. T., Adelson, E. H., and Heeger, D. J. 1992. Shiftable multiscale transforms. IEEE Transactions on Information Theory 38, 2, 587–607.
21. Smolic, A., Muller, K., Dix, K., Merkle, P., Kauff, P., and Wiegand, T. 2008. Intermediate view interpolation based on multiview video plus depth for advanced 3d video systems. In IEEE International Conference on Image Processing, 2448–2451.
22. Stewart, J., Yu, J., Gortler, S. J., and McMillan, L. 2003. A new reconstruction filter for undersampled light fields. In Proceedings of the 14th Eurographics workshop on Rendering, Eurographics Association, 150–156.
23. Takaki, Y., and Nago, N. 2010. Multi-projection of lenticular displays to construct a 256-view super multi-view display. Optics Express 18, 9, 8824–8835.
24. Wadhwa, N., Rubinstein, M., Guttag, J., Durand, F., and Freeman, W. T. 2013. Phase-based video motion processing. ACM Trans. Graph. (Proc. SIGGRAPH) 32, 4, 80:1–80:10.
25. Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4, 600–612.
26. Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4, 1–11.
27. Wilburn, B. S., Smulski, M., Lee, H.-H. K., and Horowitz, M. A. 2001. Light field video camera. In Electronic Imaging 2002, International Society for Optics and Photonics, 29–36.
28. Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., and Freeman, W. T. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4, 65:1–65:8.
29. Zwicker, M., Matusik, W., Durand, F., and Pfister, H. 2006. Antialiasing for automultiscopic 3d displays. In Proceedings of the 17th Eurographics conference on Rendering Techniques, Eurographics Association, 73–82.
