“Measurement-based editing of diffuse albedo with consistent interreflections” by Dong, Dong, Tong and Peers

We present a novel measurement-based method for editing the albedo of diffuse surfaces with consistent interreflections in a photograph of a scene under natural lighting. Key to our method is a novel technique for decomposing a photograph of a scene in several images that encode how much of the observed radiance has interacted a specified number of times with the target diffuse surface. Altering the albedo of the target area is then simply a weighted sum of the decomposed components. We estimate the interaction components by recursively applying the light transport operator and formulate the resulting radiance in each recursion as a linear expression in terms of the relevant interaction components. Our method only requires a camera-projector pair, and the number of required measurements per scene is linearly proportional to the decomposition degree for a single target area. Our method does not impose restrictions on the lighting or on the material properties in the unaltered part of the scene. Furthermore, we extend our method to accommodate editing of the albedo in multiple target areas with consistent interreflections and we introduce a prediction model for reducing the acquisition cost. We demonstrate our method on a variety of scenes and validate the accuracy on both synthetic and real examples.

1. Bai, J., Chandraker, M., Ng, T.-T., and Ramamoorthi, R. 2010. A dual theory of inverse and forward light transport. In ECCV, 294–307. Google ScholarDigital Library
2. Barrow, H. G., and Tenenbaum, J. M. 1978. Recovering Intrinsic Scene Characteristics from Images. In Computer Vision Systems, vol. 27, 3–26.Google Scholar
3. Beigpour, S., and van de Weijer, J. 2011. Object recoloring based on intrinsic image estimation. In IEEE ICCV, 327–334. Google ScholarDigital Library
4. Ben-Artzi, A., Egan, K., Durand, F., and Ramamoorthi, R. 2008. A precomputed polynomial representation for interactive brdf editing with global illumination. ACM Trans. Graph. 27, 2 (May). Google ScholarDigital Library
5. Carroll, R., Ramamoorthi, R., and Agrawala, M. 2011. Illumination decomposition for material recoloring with consistent interreflections. ACM Trans. Graph. 30, 4 (July). Google ScholarDigital Library
6. Hašan, M., and Ramamoorthi, R. 2013. Interactive albedo editing in path-traced volumetric materials. ACM Trans. Graph. 32, 2 (Apr.). Google ScholarDigital Library
7. Inokuchi, S., Sato, K., and Matsuda, F. 1984. Range imaging system for 3-D object recognition. In International Conference on Pattern Recognition, 806–808.Google Scholar
8. Nayar, S. K., Krishnan, G., Grossberg, M. D., and Raskar, R. 2006. Fast separation of direct and global components of a scene using high frequency illumination. ACM Trans. Graph. 25, 3 (July), 935–944. Google ScholarDigital Library
9. Ng, T.-T., Pahwa, R. S., Bai, J., Tan, K.-H., and Ramamoorthi, R. 2012. From the rendering equation to stratified light transport inversion. Int. J. Comput. Vision 96, 2 (Jan.), 235–251. Google ScholarDigital Library
10. O’Toole, M., and Kutulakos, K. N. 2010. Optical computing for fast light transport analysis. ACM Trans. Graph. 29, 6 (Dec.), 164:1–164:12. Google ScholarDigital Library
11. O’Toole, M., Raskar, R., and Kutulakos, K. N. 2012. Primal-dual coding to probe light transport. ACM Trans. Graph. 31, 4 (July), 39:1–39:11. Google ScholarDigital Library
12. O’Toole, M., Heide, F., Xiao, L., Hullin, M. B., Heidrich, W., and Kutulakos, K. N. 2014. Temporal frequency probing for 5d transient analysis of global light transport. ACM Trans. Graph. 33, 4 (July), 87:1–87:11. Google ScholarDigital Library
13. Pharr, M., and Humphreys, G. 2010. Physically Based Rendering, Second Edition: From Theory To Implementation, 2nd ed. Morgan Kaufmann Publishers Inc. Google ScholarDigital Library
14. Reddy, D., Ramamoorthi, R., and Curless, B. 2012. Frequency-space decomposition and acquisition of light transport under spatially varying illumination. In ECCV, 596–610. Google ScholarDigital Library
15. Seitz, S. M., Matsushita, Y., and Kutulakos, K. N. 2005. A theory of inverse light transport. In IEEE ICCV, 1440–1447. Google ScholarDigital Library
16. Sun, X., Zhou, K., Chen, Y., Lin, S., Shi, J., and Guo, B. 2007. Interactive relighting with dynamic brdfs. ACM Trans. Graph. 26, 3. Google ScholarDigital Library
17. Wu, D., Velten, A., O’toole, M., Masia, B., Agrawal, A., Dai, Q., and Raskar, R. 2014. Decomposing global light transport using time of flight imaging. Int. J. Comput. Vision 107, 2 (Apr.), 123–138. Google ScholarDigital Library
18. Yu, Y., Debevec, P., Malik, J., and Hawkins, T. 1999. Inverse global illumination: Recovering reflectance models of real scenes from photographs. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’99, 215–224. Google ScholarDigital Library