“Surface flows for image-based shading design” by Vergne, Barla, Fleming and Granier

  • ©Romain Vergne, Pascal Barla, Roland W. Fleming, and Xavier Granier




    Surface flows for image-based shading design



    We present a novel method for producing convincing pictures of shaded objects based entirely on 2D image operations. This approach, which we call image-based shading design, offers direct artistic control in the picture plane by deforming image primitives so that they appear to conform to specific 3D shapes. Using a differential analysis of reflected radiance, we identify the two types of surface flows involved in the depiction of shaded objects, which are consistent with recent perceptual studies. We then introduce two novel deformation operators that closely mimic surface flows while providing direct artistic controls in real-time.


    1. Anderson, B. L., and Kim, J. 2009. Image statistics do not explain the perception of gloss and lightness. J. Vis. 9, 11, 1–17.Google ScholarCross Ref
    2. Anjyo, K., Wemler, S., and Baxter, W. 2006. Tweakable light and shade for cartoon animation. In Proc. symposium on Non-photorealistic animation and rendering, ACM, 133–139. Google ScholarDigital Library
    3. Bigün, J., and Granlund, G. 1987. Optimal orientation detection of linear symmetry. In Proc. IEEE First International Conference on Computer Vision, 433–438.Google Scholar
    4. Breton, P., and Zucker, S. W. 1996. Shadows and shading flow fields. In IEEE Conf. Computer Vision and Pattern Recognition, 782–789. Google ScholarDigital Library
    5. Colbert, M., Pattanaik, S., and Krivanek, J. 2006. BRDF-Shop: Creating Physically Correct Bidirectional Reflectance Distribution Functions. IEEE Comput. Graph. Appl. 26, 30–36. Google ScholarDigital Library
    6. DelPozo, A., and Savarese, S. 2007. Detecting specular surfaces on natural images. In IEEE Conf. Computer Vision and Pattern Recognition, 18–23.Google Scholar
    7. Fang, H., and Hart, J. C. 2004. Textureshop: texture synthesis as a photograph editing tool. ACM Trans. Graph. 23, 354–359. Google ScholarDigital Library
    8. Fleming, R. W., Torralba, A., and Adelson, E. H. 2004. Specular reflections and the perception of shape. J. Vis. 4, 9, 798–820.Google ScholarCross Ref
    9. Fleming, R. W., Torralba, A., and Adelson, E. H. 2009. Shape from sheen. Tech. rep., MIT-CSAIL-TR-2009-051.Google Scholar
    10. Fleming, R. W., Holtmann-Rice, D., and Bülthoff, H. H. 2011. Estimation of 3D shape from image orientations. Proc. National Academy of Sciences 108, 51, 20438–20443.Google ScholarCross Ref
    11. Fleming, R., Jäkel, F., and Maloney, L. 2011. Visual perception of thick transparent materials. Psychological Science 22, 6, 812–820.Google ScholarCross Ref
    12. Gibson, J. J. 1950. The Perception of the Visual World. Houghton Mifflin.Google Scholar
    13. Huggins, P. S., and Zucker, S. W. 2001. Folds and cuts: How shading flows into edges. In Proc. IEEE Int. Conf. Computer Vision, 153–158.Google Scholar
    14. Jeschke, S., Cline, D., and Wonka, P. 2011. Estimating color and texture parameters for vector graphics. Computer Graphics Forum 30, 2, 523–532.Google ScholarCross Ref
    15. Johnston, S. F. 2002. Lumo: illumination for cel animation. In Proc. symposium on Non-photorealistic animation and rendering, ACM, 45–ff. Google ScholarDigital Library
    16. Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. ACM Trans. Graph. 25, 654–663. Google ScholarDigital Library
    17. Kim, J., and Anderson, B. L. 2010. Image statistics and the perception of surface gloss and lightness. J. Vis. 10, 9, 3.Google ScholarCross Ref
    18. Kim, J., Marlow, P., and Anderson, B. 2011. The perception of gloss depends on highlight congruence with surface shading. J Vis 11, 9.Google ScholarCross Ref
    19. Koenderink, J. J., and Van Doorn, A. J. 1980. Photometric invariants related to solid shape. Optica Acta 27, 7, 981–996.Google ScholarCross Ref
    20. Li, A., and Zaidi, Q. 2000. Perception of three-dimensional shape from texture is based on patterns of oriented energy. Vision Research 40, 2, 217–42.Google ScholarCross Ref
    21. McCann, J., and Pollard, N. S. 2008. Real-time gradient-domain painting. ACM Trans. Graph. 27, 3, 93:1–93:7. Google ScholarDigital Library
    22. Motoyoshi, I., Nishida, S., and Adelson, E. H. 2005. Luminance re-mapping for the control of apparent material. In Proc. Symposium on Applied Perception in Graphics and Visualization, ACM, 165–165. Google ScholarDigital Library
    23. Motoyoshi, I. 2010. Highlight-shading relationship as a cue for the perception of translucent and transparent materials. J Vis 10, 9, 6.Google ScholarCross Ref
    24. Nicodemus, F. E., Richmond, J. C., Hsia, J. J., Ginsberg, I. W., and Limperis, T. 1977. Geometric Considerations and Nomenclature for Reflectance. National Bureau of Standards.Google Scholar
    25. Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In ACM Trans. Graph., 433–442. Google ScholarDigital Library
    26. Okabe, M., Matsushita, Y., Shen, L., and Igarashi, T. 2007. Illumination Brush: Interactive Design of All-Frequency Lighting. In Pacific Conference on Computer Graphics and Applications, 171–180. Google ScholarDigital Library
    27. Orzan, A., Bousseau, A., Winnemöller, H., Barla, P., Thollot, J., and Salesin, D. 2008. Diffusion curves: a vector representation for smooth-shaded images. ACM Trans. Graph. 27, 3, 92:1–92:8. Google ScholarDigital Library
    28. Pacanowski, R., Granier, X., Schlick, C., and Poulin, P. 2008. Sketch and Paint-based Interface for Highlight Modeling. In Eurographics workshop on Sketch-Based Interfaces and Modeling, 17–23. Google ScholarDigital Library
    29. Pellacini, F., Battaglia, F., Morley, R. K., and Finkelstein, A. 2007. Lighting with paint. ACM Trans. Graph. 26, 2. Google ScholarDigital Library
    30. Pellacini, F. 2010. envylight: an interface for editing natural illumination. ACM Trans. Graph. 29, 34:1–34:8. Google ScholarDigital Library
    31. Pharr, M., and Humphreys, G. 2010. Physically Based Rendering, Second Edition: From Theory To Implementation, 2nd ed. Morgan Kaufmann Publishers Inc. Google ScholarDigital Library
    32. Quixel, 2011. ndo2. http://quixel.se/.Google Scholar
    33. Ramamoorthi, R., Mahajan, D., and Belhumeur, P. 2007. A first-order analysis of lighting, shading, and shadows. ACM Trans. Graph. 26, 1. Google ScholarDigital Library
    34. Ramanarayanan, G., Ferwerda, J., Walter, B., and Bala, K. 2007. Visual equivalence: towards a new standard for image fidelity. ACM Trans. Graph. 26, 3. Google ScholarDigital Library
    35. Ritschel, T., Okabe, M., Thormählen, T., and Seidel, H.-P. 2009. Interactive reflection editing. ACM Trans. Graph. 28, 5, 129:1–129:7. Google ScholarDigital Library
    36. Ritschel, T., Thormählen, T., Dachsbacher, C., Kautz, J., and Seidel, H.-P. 2010. Interactive on-surface signal deformation. ACM Trans. Graph. 29, 4. Google ScholarDigital Library
    37. Schlick, C. 1994. Graphics Gems V. Morgan Kauffman, ch. A Fast Alternative to Phong’s Specular Model, 385–384. Google ScholarDigital Library
    38. Shao, C., Bousseau, A., Sheffer, A., and Singh, K. 2012. Crossshade: Shading concept sketching using cross-section curves. ACM Transactions on Graphics (Proc. SIGGRAPH) 31, 4 (Aug.). Google ScholarDigital Library
    39. Shepard, D. 1968. A two-dimensional interpolation function for irregularly-spaced data. In Proc. 1968 23rd ACM national conference, 517–524. Google ScholarDigital Library
    40. Sloan, P.-P. J., Martin, W., Gooch, A., and Gooch, B. 2001. The lit sphere: a model for capturing npr shading from art. In Graphics interface, 143–150. Google ScholarDigital Library
    41. Sun, J., Liang, L., Wen, F., and Shum, H.-Y. 2007. Image vectorization using optimized gradient meshes. ACM Trans. Graph. 26, 3, 11. Google ScholarDigital Library
    42. Tappen, M. F. 2011. Recovering shape from a single image of a mirrored surface from curvature constraints. In IEEE Conf. Computer Vision and Pattern Recognition, 2545–2552. Google ScholarDigital Library
    43. Todd, J. T., and Akerstrom, R. A. 1987. Perception of three-dimensional form from patterns of optical texture. J. experimental psychology. Human perception and performance 13, 2, 242–255.Google Scholar
    44. Todd, J. T., and Mingolla, E. 1983. Perception of Surface Curvature and Direction of Illuminant from Patterns of Shading. J. Experimental Psychology: Human Perception and Performance 9, 583–595.Google ScholarCross Ref
    45. Toler-Franklin, C., Finkelstein, A., and Rusinkiewicz, S. 2007. Illustration of complex real-world objects using images with normals. In Proc. symposium on Non-photorealistic animation and rendering, ACM, 111–119. Google ScholarDigital Library
    46. Vasilyev, Y., Zickler, T., Gortler, S. J., and BenShahar, O. 2011. Shape from specular flow: Is one flow enough? In IEEE Conf. Computer Vision and Pattern Recognition, 2561–2568. Google ScholarDigital Library
    47. Vergne, R., Pacanowski, R., Barla, P., Granier, X., and Schlick, C. 2011. Improving Shape Depiction under Arbitrary Rendering. IEEE Trans. Visualization and Computer Graphics 17, 8, 1071–1081. Google ScholarDigital Library
    48. Winnemöller, H., Orzan, A., Boissieux, L., and Thollot, J. 2009. Texture Design and Draping in 2D Images. Computer Graphics Forum 28, 4, 1091–1099. Google ScholarDigital Library
    49. Wu, T.-P., Tang, C.-K., Brown, M. S., and Shum, H.-Y. 2007. ShapePalettes: interactive normal transfer via sketching. ACM Trans. Graph. 26, 3, 44. Google ScholarDigital Library
    50. Yeung, S. K., Tang, C.-K., Brown, M. S., and Kang, S. B. 2011. Matting and compositing of transparent and refractive objects. ACM Trans. Graph. 30, 1, 2:1–2:13. Google ScholarDigital Library

ACM Digital Library Publication: