“How do people edit light fields?” by Jarabo, Masia, Bousseau, Pellacini and Gutierrez
Conference:
Type:
Title:
- How do people edit light fields?
Session/Category Title: Changing Your Perception
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a thorough study to evaluate different light field editing interfaces, tools and workflows from a user perspective. This is of special relevance given the multidimensional nature of light fields, which may make common image editing tasks become complex in light field space. We additionally investigate the potential benefits of using depth information when editing, and the limitations imposed by imperfect depth reconstruction using current techniques. We perform two different experiments, collecting both objective and subjective data from a varied number of editing tasks of increasing complexity based on local point-and-click tools. In the first experiment, we rely on perfect depth from synthetic light fields, and focus on simple edits. This allows us to gain basic insight on light field editing, and to design a more advanced editing interface. This is then used in the second experiment, employing real light fields with imperfect reconstructed depth, and covering more advanced editing tasks. Our study shows that users can edit light fields with our tested interface and tools, even in the presence of imperfect depth. They follow different workflows depending on the task at hand, mostly relying on a combination of different depth cues. Last, we confirm our findings by asking a set of artists to freely edit both real and synthetic light fields.
References:
1. Chen, B., Ofek, E., Shum, H.-Y., and Levoy, M. 2005. Interactive deformation of light fields. In Proc. of I3D’05, 139–146. Google ScholarDigital Library
2. Colbert, M., Pattanaik, S., and Křivánek, J. 2006. BRDF-shop: Creating physically correct bidirectional reflectance distribution functions. IEEE Comput. Graph. Appl. 26, 1, 30–36. Google ScholarDigital Library
3. Cunningham, D., and Wallraven, C. 2011. Experimental Design: From User Studies to Psychophysics. AK Peters. Google ScholarDigital Library
4. Davis, A., Levoy, M., and Durand, F. 2012. Unstructured light fields. Comp. Graph. Forum 31, 2pt1, 305–314. Google ScholarDigital Library
5. Fleming, R. W., Dror, R. O., and Adelson, E. H. 2003. Real-world illumination and the perception of surface reflectance properties. Journal of Vision 3, 5, 347–368.Google ScholarCross Ref
6. Goldman, D. B., Gonterman, C., Curless, B., Salesin, D., and Seitz, S. M. 2008. Video object annotation, navigation, and composition. In Proc. of UIST’08, 3–12. Google ScholarDigital Library
7. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The Lumigraph. In Proc. of SIGGRAPH’96, 43–54. Google ScholarDigital Library
8. HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM Trans. Graph. 30, 4, 70:1–70:10. Google ScholarDigital Library
9. Haeberli, P. 1990. Paint by numbers: Abstract image representations. SIGGRAPH Comput. Graph. 24, 4, 207–214. Google ScholarDigital Library
10. Hasinoff, S., Jozwiak, M., Durand, F., and Freeman, W. 2010. Search-and-replace editing for personal photo collections. In Proc of ICCP’10, 1–8.Google Scholar
11. Hoaglin, D. C., and Iglewicz, B. 1987. Fine-tuning some resistant rules for outlier labeling. Journal of the American Statistical Association 82, 400, 1147–1149.Google ScholarCross Ref
12. Horn, D. R., and Chen, B. 2007. Lightshop: interactive light field manipulation and rendering. In Proc. of I3D’07, 121–128. Google ScholarDigital Library
13. Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically reparameterized light fields. In Proc. of SIGGRAPH’00, 297–306. Google ScholarDigital Library
14. Jarabo, A., Masia, B., and Gutierrez, D. 2011. Efficient propagation of light field edits. In Proc. of SIACG’11, 75–80.Google Scholar
15. Kerr, W. B., and Pellacini, F. 2009. Toward evaluating lighting design interface paradigms for novice users. ACM Trans. Graph. 28, 3, 26:1–26:9. Google ScholarDigital Library
16. Kerr, W. B., and Pellacini, F. 2010. Toward evaluating material design interface paradigms for novice users. ACM Trans. Graph. 29, 4, 35:1–35:10. Google ScholarDigital Library
17. Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A., and Gross, M. 2013. Scene reconstruction from high spatio-angular resolution light fields. ACM Trans. Graph. 32, 4, 73:1–73:12. Google ScholarDigital Library
18. Kim, Y., Winnemöller, H., and Lee, S. 2013. WYSIWYG stereo painting. In Proc. of I3D’13, 169–176. Google ScholarDigital Library
19. Levoy, M., and Hanrahan, P. 1996. Light field rendering. In Proc. of SIGGRAPH’96, 31–42. Google ScholarDigital Library
20. Lo, W.-Y., van Baar, J., Knaus, C., Zwicker, M., and Gross, M. H. 2010. Stereoscopic 3D copy & paste. ACM Trans. Graph. 29, 6, 147:1–147:10. Google ScholarDigital Library
21. Lytro Inc., 2013. The Lytro camera. http://www.lytro.com.Google Scholar
22. Marton, F., Agus, M., Gobbetti, E., Pintore, G., and Rodriguez, M. B. 2012. Natural exploration of 3D massive models on large-scale light field displays using the fox proximal navigation technique. Computers & Graphics 36, 8, 893–903. Google ScholarDigital Library
23. Masia, B., Wetzstein, G., Didyk, P., and Gutierrez, D. 2013. A survey on computational displays: Pushing the boundaries of optics, computation and perception. Computers & Graphics 37, 1012–1038. Google ScholarDigital Library
24. Ou, J., Karlík, O., Křivánek, J., and Pellacini, F. 2012. Toward evaluating progressive rendering methods in appearance design tasks. IEEE Comput. Graph. Appl. 33. Google ScholarDigital Library
25. Pharr, M., and Humphreys, G. 2010. Physically based rendering: From theory to implementation. Morgan Kaufmann. Google ScholarDigital Library
26. Price, B. L., and Cohen, S. 2011. Stereocut: Consistent interactive object selection in stereo image pairs. In Proc. of ICCV’11, 1148–1155. Google ScholarDigital Library
27. Rubinstein, M., Gutierrez, D., Sorkine, O., and Shamir, A. 2010. A comparative study of image retargeting. ACM Trans. Graph. 29, 5, 160:1–160:10. Google ScholarDigital Library
28. Santosa, S., Chevalier, F., Balakrishnan, R., and Singh, K. 2013. Direct space-time trajectory control for visual media editing. In Proc. of CHI’13, 1149–1158. Google ScholarDigital Library
29. Seitz, S. M., and Kutulakos, K. N. 2002. Plenoptic image editing. Int. Journal of Computer Vision 48, 2, 115–129. Google ScholarDigital Library
30. Shum, H.-Y., Sun, J., Yamazaki, S., Li, Y., and Tang, C.-K. 2004. Pop-up light field: An interactive image-based modeling and rendering system. ACM Trans. Graph. 23, 2, 143–162. Google ScholarDigital Library
31. Stavrakis, E., and Gelautz, M. 2004. Image-based stereoscopic painterly rendering. In Proc. of EGSR’04, 53–60. Google ScholarDigital Library
32. Templin, K., Didyk, P., Ritschel, T., Myszkowski, K., and Seidel, H.-P. 2012. Highlight microdisparity for improved gloss depiction. ACM Trans. Graph. 31, 4, 92:1–92:5. Google ScholarDigital Library
33. Tompkin, J., Muff, S., Jakuschevskij, S., McCann, J., Kautz, J., Alexa, M., and Matusik, W. 2012. Interactive light field painting. In ACM SIGGRAPH 2012 Emerging Technologies, 12:1–12:1. Google ScholarDigital Library
34. Vaish, V., Wilburn, B., Joshi, N., and Levoy, M. 2004. Using plane + parallax for calibrating dense camera arrays. In Proc. of CVPR’04, 2–9.Google Scholar
35. Venkataraman, K., Lelescu, D., Duparré, J., McMahon, A., Molina, G., Chatterjee, P., Mullis, R., and Nayar, S. 2013. PiCam: An ultra-thin high performance monolithic camera array. ACM Trans. Graph. 32, 6, 166:1–166:13. Google ScholarDigital Library
36. Vertical Horizon, 2013. Lightfield Iris. http://www.verticalhorizon-software.com/LFI/index.html.Google Scholar
37. Wang, L., Lin, S., Lee, S., Guo, B., and Shum, H.-Y. 2005. Light field morphing using 2D features. IEEE Trans. Vis. Comput. Graph. 11, 1, 25–34. Google ScholarDigital Library
38. Wanner, S., and Goldluecke, B. 2012. Globally consistent depth labeling of 4d light fields. In Proc. of CVPR’12, 41–48. Google ScholarDigital Library
39. Wetzstein, G., Lanman, D., Gutierrez, D., and Hirsch, M. 2012. Computational displays: combining optical fabrication, computational processing, and perceptual tricks to build the displays of the future. In ACM SIGGRAPH 2012 Courses. Google ScholarDigital Library
40. Yücer, K., Jacobson, A., Hornung, A., and Sorkine, O. 2012. Transfusive image manipulation. ACM Trans. Graph. 31, 6, 176:1–176:9. Google ScholarDigital Library
41. Zhang, Z., Wang, L., Guo, B., and Shum, H.-Y. 2002. Feature-based light field morphing. ACM Trans. Graph. 21, 3, 457–464. Google ScholarDigital Library
