“How do people edit light fields?” by Jarabo, Masia, Bousseau, Pellacini and Gutierrez

  • ©Adrian Jarabo, Belen Masia, Adrien Bousseau, Fabio Pellacini, and Diego Gutierrez

Conference:


Type:


Title:

    How do people edit light fields?

Session/Category Title: Changing Your Perception


Presenter(s)/Author(s):


Moderator(s):



Abstract:


    We present a thorough study to evaluate different light field editing interfaces, tools and workflows from a user perspective. This is of special relevance given the multidimensional nature of light fields, which may make common image editing tasks become complex in light field space. We additionally investigate the potential benefits of using depth information when editing, and the limitations imposed by imperfect depth reconstruction using current techniques. We perform two different experiments, collecting both objective and subjective data from a varied number of editing tasks of increasing complexity based on local point-and-click tools. In the first experiment, we rely on perfect depth from synthetic light fields, and focus on simple edits. This allows us to gain basic insight on light field editing, and to design a more advanced editing interface. This is then used in the second experiment, employing real light fields with imperfect reconstructed depth, and covering more advanced editing tasks. Our study shows that users can edit light fields with our tested interface and tools, even in the presence of imperfect depth. They follow different workflows depending on the task at hand, mostly relying on a combination of different depth cues. Last, we confirm our findings by asking a set of artists to freely edit both real and synthetic light fields.

References:


    1. Chen, B., Ofek, E., Shum, H.-Y., and Levoy, M. 2005. Interactive deformation of light fields. In Proc. of I3D’05, 139–146. Google ScholarDigital Library
    2. Colbert, M., Pattanaik, S., and Křivánek, J. 2006. BRDF-shop: Creating physically correct bidirectional reflectance distribution functions. IEEE Comput. Graph. Appl. 26, 1, 30–36. Google ScholarDigital Library
    3. Cunningham, D., and Wallraven, C. 2011. Experimental Design: From User Studies to Psychophysics. AK Peters. Google ScholarDigital Library
    4. Davis, A., Levoy, M., and Durand, F. 2012. Unstructured light fields. Comp. Graph. Forum 31, 2pt1, 305–314. Google ScholarDigital Library
    5. Fleming, R. W., Dror, R. O., and Adelson, E. H. 2003. Real-world illumination and the perception of surface reflectance properties. Journal of Vision 3, 5, 347–368.Google ScholarCross Ref
    6. Goldman, D. B., Gonterman, C., Curless, B., Salesin, D., and Seitz, S. M. 2008. Video object annotation, navigation, and composition. In Proc. of UIST’08, 3–12. Google ScholarDigital Library
    7. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The Lumigraph. In Proc. of SIGGRAPH’96, 43–54. Google ScholarDigital Library
    8. HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM Trans. Graph. 30, 4, 70:1–70:10. Google ScholarDigital Library
    9. Haeberli, P. 1990. Paint by numbers: Abstract image representations. SIGGRAPH Comput. Graph. 24, 4, 207–214. Google ScholarDigital Library
    10. Hasinoff, S., Jozwiak, M., Durand, F., and Freeman, W. 2010. Search-and-replace editing for personal photo collections. In Proc of ICCP’10, 1–8.Google Scholar
    11. Hoaglin, D. C., and Iglewicz, B. 1987. Fine-tuning some resistant rules for outlier labeling. Journal of the American Statistical Association 82, 400, 1147–1149.Google ScholarCross Ref
    12. Horn, D. R., and Chen, B. 2007. Lightshop: interactive light field manipulation and rendering. In Proc. of I3D’07, 121–128. Google ScholarDigital Library
    13. Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically reparameterized light fields. In Proc. of SIGGRAPH’00, 297–306. Google ScholarDigital Library
    14. Jarabo, A., Masia, B., and Gutierrez, D. 2011. Efficient propagation of light field edits. In Proc. of SIACG’11, 75–80.Google Scholar
    15. Kerr, W. B., and Pellacini, F. 2009. Toward evaluating lighting design interface paradigms for novice users. ACM Trans. Graph. 28, 3, 26:1–26:9. Google ScholarDigital Library
    16. Kerr, W. B., and Pellacini, F. 2010. Toward evaluating material design interface paradigms for novice users. ACM Trans. Graph. 29, 4, 35:1–35:10. Google ScholarDigital Library
    17. Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A., and Gross, M. 2013. Scene reconstruction from high spatio-angular resolution light fields. ACM Trans. Graph. 32, 4, 73:1–73:12. Google ScholarDigital Library
    18. Kim, Y., Winnemöller, H., and Lee, S. 2013. WYSIWYG stereo painting. In Proc. of I3D’13, 169–176. Google ScholarDigital Library
    19. Levoy, M., and Hanrahan, P. 1996. Light field rendering. In Proc. of SIGGRAPH’96, 31–42. Google ScholarDigital Library
    20. Lo, W.-Y., van Baar, J., Knaus, C., Zwicker, M., and Gross, M. H. 2010. Stereoscopic 3D copy & paste. ACM Trans. Graph. 29, 6, 147:1–147:10. Google ScholarDigital Library
    21. Lytro Inc., 2013. The Lytro camera. http://www.lytro.com.Google Scholar
    22. Marton, F., Agus, M., Gobbetti, E., Pintore, G., and Rodriguez, M. B. 2012. Natural exploration of 3D massive models on large-scale light field displays using the fox proximal navigation technique. Computers & Graphics 36, 8, 893–903. Google ScholarDigital Library
    23. Masia, B., Wetzstein, G., Didyk, P., and Gutierrez, D. 2013. A survey on computational displays: Pushing the boundaries of optics, computation and perception. Computers & Graphics 37, 1012–1038. Google ScholarDigital Library
    24. Ou, J., Karlík, O., Křivánek, J., and Pellacini, F. 2012. Toward evaluating progressive rendering methods in appearance design tasks. IEEE Comput. Graph. Appl. 33. Google ScholarDigital Library
    25. Pharr, M., and Humphreys, G. 2010. Physically based rendering: From theory to implementation. Morgan Kaufmann. Google ScholarDigital Library
    26. Price, B. L., and Cohen, S. 2011. Stereocut: Consistent interactive object selection in stereo image pairs. In Proc. of ICCV’11, 1148–1155. Google ScholarDigital Library
    27. Rubinstein, M., Gutierrez, D., Sorkine, O., and Shamir, A. 2010. A comparative study of image retargeting. ACM Trans. Graph. 29, 5, 160:1–160:10. Google ScholarDigital Library
    28. Santosa, S., Chevalier, F., Balakrishnan, R., and Singh, K. 2013. Direct space-time trajectory control for visual media editing. In Proc. of CHI’13, 1149–1158. Google ScholarDigital Library
    29. Seitz, S. M., and Kutulakos, K. N. 2002. Plenoptic image editing. Int. Journal of Computer Vision 48, 2, 115–129. Google ScholarDigital Library
    30. Shum, H.-Y., Sun, J., Yamazaki, S., Li, Y., and Tang, C.-K. 2004. Pop-up light field: An interactive image-based modeling and rendering system. ACM Trans. Graph. 23, 2, 143–162. Google ScholarDigital Library
    31. Stavrakis, E., and Gelautz, M. 2004. Image-based stereoscopic painterly rendering. In Proc. of EGSR’04, 53–60. Google ScholarDigital Library
    32. Templin, K., Didyk, P., Ritschel, T., Myszkowski, K., and Seidel, H.-P. 2012. Highlight microdisparity for improved gloss depiction. ACM Trans. Graph. 31, 4, 92:1–92:5. Google ScholarDigital Library
    33. Tompkin, J., Muff, S., Jakuschevskij, S., McCann, J., Kautz, J., Alexa, M., and Matusik, W. 2012. Interactive light field painting. In ACM SIGGRAPH 2012 Emerging Technologies, 12:1–12:1. Google ScholarDigital Library
    34. Vaish, V., Wilburn, B., Joshi, N., and Levoy, M. 2004. Using plane + parallax for calibrating dense camera arrays. In Proc. of CVPR’04, 2–9.Google Scholar
    35. Venkataraman, K., Lelescu, D., Duparré, J., McMahon, A., Molina, G., Chatterjee, P., Mullis, R., and Nayar, S. 2013. PiCam: An ultra-thin high performance monolithic camera array. ACM Trans. Graph. 32, 6, 166:1–166:13. Google ScholarDigital Library
    36. Vertical Horizon, 2013. Lightfield Iris. http://www.verticalhorizon-software.com/LFI/index.html.Google Scholar
    37. Wang, L., Lin, S., Lee, S., Guo, B., and Shum, H.-Y. 2005. Light field morphing using 2D features. IEEE Trans. Vis. Comput. Graph. 11, 1, 25–34. Google ScholarDigital Library
    38. Wanner, S., and Goldluecke, B. 2012. Globally consistent depth labeling of 4d light fields. In Proc. of CVPR’12, 41–48. Google ScholarDigital Library
    39. Wetzstein, G., Lanman, D., Gutierrez, D., and Hirsch, M. 2012. Computational displays: combining optical fabrication, computational processing, and perceptual tricks to build the displays of the future. In ACM SIGGRAPH 2012 Courses. Google ScholarDigital Library
    40. Yücer, K., Jacobson, A., Hornung, A., and Sorkine, O. 2012. Transfusive image manipulation. ACM Trans. Graph. 31, 6, 176:1–176:9. Google ScholarDigital Library
    41. Zhang, Z., Wang, L., Guo, B., and Shum, H.-Y. 2002. Feature-based light field morphing. ACM Trans. Graph. 21, 3, 457–464. Google ScholarDigital Library


ACM Digital Library Publication: