“Optical computing for fast light transport analysis”
Conference:
Type(s):
Title:
- Optical computing for fast light transport analysis
Session/Category Title: Imaging hardware
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a general framework for analyzing the transport matrix of a real-world scene at full resolution, without capturing many photos. The key idea is to use projectors and cameras to directly acquire eigenvectors and the Krylov subspace of the unknown transport matrix. To do this, we implement Krylov subspace methods partially in optics, by treating the scene as a “black box subroutine” that enables optical computation of arbitrary matrix-vector products. We describe two methods—optical Arnoldi to acquire a low-rank approximation of the transport matrix for relighting; and optical GMRES to invert light transport. Our experiments suggest that good quality relighting and transport inversion are possible from a few dozen low-dynamic range photos, even for scenes with complex shadows, caustics, and other challenging lighting effects.
References:
1. Amano, T., and Kato, H. 2008. Real world dynamic appearance enhancement with procam feedback. Proc. IEEE PRO-CAMS. Google ScholarDigital Library
2. Ambs, P. 2009. A short history of optical computing: rise, decline, and evolution. Proc. SPIE 7388.Google ScholarCross Ref
3. Athale, R. A., and Collins, W. C., 1982. Optical matrixmatrix multiplier based on outer product decomposition.Google Scholar
4. Bai, J., Chandraker, M., Ng, T.-T., and Ramamoorthi, R. 2010. A dual theory of inverse and forward light transport. Proc. ECCV. Google ScholarDigital Library
5. Basri, R., and Jacobs, D. 2003. Lambertian reflectance and linear subspaces. IEEE T-PAMI 25, 2, 218–233. Google ScholarDigital Library
6. Casasent, D. 1981. Hybrid processors. Optical Information Processing Fundamentals, 181–233.Google Scholar
7. Cederquist, J., and Lee, S. 1979. The use of feedback in optical information processing. Applied Physics A: Materials Science & Processing.Google Scholar
8. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. Proc. SIGGRAPH. Google ScholarDigital Library
9. Fuchs, M., Blanz, V., Lensch, H., and Seidel, H.-P. 2007. Adaptive sampling of reflectance fields. ACM TOG 26, 2 (Jun). Google ScholarDigital Library
10. Garg, G., Talvala, E., Levoy, M., and Lensch, H. 2006. Symmetric photography: Exploiting data-sparseness in reflectance fields. Proc. Eurographics Symp. Rendering. Google ScholarDigital Library
11. Goodman, J. W. 2005. Introduction to fourier optics, 3rd edition. Roberts & Company Publishers.Google Scholar
12. Greenbaum, A., and Trefethen, L. N. 1994. Gmres/cr and arnoldi/lanczos as matrix approximation problems. SIAM J. Sci. Comput. 15, 2, 359–368. Google ScholarDigital Library
13. Grossberg, M., Peri, H., Nayar, S., and Belhumeur, P. 2004. Making one object look like another: controlling appearance using a projector-camera system. Proc. CVPR, 452–459.Google Scholar
14. Guilfoyle, P., and Stone, R. 1991. Digital optical computer ii. Proc. SPIE 1563, 214–222.Google Scholar
15. Gutknecht, M. H. 2007. Block krylov space methods for linear systems with multiple right-hand sides: an introduction. In Modern Mathematical Models, Methods and Algorithms for Real World Systems. 420–447.Google Scholar
16. Koenderink, J. J., and van Doorn, A. J. 1983. Geometrical modes as a general method to treat diffuse interreflections in radiometry. J. Opt. Soc. Am 73, 6, 843–850.Google ScholarCross Ref
17. Kumar, B. V. K. V., and Casasent, D. 1981. Eigenvector determination by iterative optical methods. Applied Optics 20, 21, 3707–3710.Google ScholarCross Ref
18. Langer, M. 1999. When shadows become interreflections. Int. J. Computer Vision 34, 2, 193–204. Google ScholarDigital Library
19. Larsen, R. M. http://soi.stanford.edu/ rmunk/propack/.Google Scholar
20. Leith, E. 2000. The evolution of information optics. IEEE J. Select Topics in Quantum Electronics 6, 6, 1297–1304.Google ScholarCross Ref
21. Liesen, J., and Tichy, P. 2004. Convergence analysis of krylov subspace methods. GAMM Mitt. Ges. Angew. Math. Mech. 27, 2, 153–173 (2005).Google ScholarCross Ref
22. Mahajan, D., Shlizerman, I., Ramamoorthi, R., and Belhumeur, P. 2007. A theory of locally low dimensional light transport. Proc. SIGGRAPH. Google ScholarDigital Library
23. Matusik, W., Pfister, H., Ziegler, R., and Ngan, A. 2002. Acquisition and rendering of transparent and refractive objects. Proc. Eurographics Symp. on Rendering. Google ScholarDigital Library
24. Ng, R., Ramamoorthi, R., and Hanrahan, P. 2003. All-frequency shadows using non-linear wavelet lighting approximation. Proc. SIGGRAPH. Google ScholarDigital Library
25. Ng, T., Pahwa, R., Bai, J., Quek, T., and Tan, K. 2009. Radiometric compensation using stratified inverses. Proc. ICCV.Google Scholar
26. Peers, P., and Dutré, P. 2003. Wavelet environment matting. Proc. Eurographics Symp. on Rendering. Google ScholarDigital Library
27. Peers, P., and Dutré, P. 2005. Inferring reflectance functions from wavelet noise. Proc. Eurographics Symp. Rendering. Google ScholarDigital Library
28. Peers, P., Mahajan, D., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., and Debevec, P. 2009. Compressive light transport sensing. ACM TOG 28, 1. Google ScholarDigital Library
29. Psaltis, D., and Athale, R. A. 1986. High accuracy computation with linear analog optical systems: a cricitcal study. Applied Optics 25, 18, 3071–3077.Google ScholarCross Ref
30. Rajbenbach, H., Fainman, Y., and Lee, S. H. 1987. Optical implementation of an interative algorithm for matrix-inversion. Applied Optics 26, 6, 1024–1031.Google ScholarCross Ref
31. Saad, Y. 2003. Iterative methods for sparse linear systems. Google ScholarDigital Library
32. Salvi, J., Pages, J., and Batlle, J. 2004. Pattern codification strategies in structured light systems. Pattern Recogn 37, 4, 827–849.Google ScholarCross Ref
33. Schechner, Y., Nayar, S., and Belhumeur, P. 2007. Multiplexing for optimal lighting. IEEE T-PAMI 29, 8, 1339–1354. Google ScholarDigital Library
34. Seitz, S., Matsushita, Y., and Kutulakos, K. 2005. A theory of inverse light transport. Proc. ICCV, 1440–1447. Google ScholarDigital Library
35. Sen, P., and Darabi, S. 2009. Compressive dual photography. Proc. Eurographics.Google Scholar
36. Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., and Lensch, H. 2005. Dual photography. Proc. SIGGRAPH. Google ScholarDigital Library
37. Simon, H. D., and Zha, H. 2000. Low-rank matrix approximation using the lanczos bidiagonalization process with applications. SIAM J. Sci. Comput. 21, 6, 2257–2274. Google ScholarDigital Library
38. Simoncini, V., and Szyld, D. B. 2003. Theory of inexact krylov subspace methods and applications to scientific computing. SIAM J. Sci. Comput. 25, 2, 454–477. Google ScholarDigital Library
39. Trefethen, L. N., and Bau, I. 1997. Numerical linear algebra. SIAM, xii+361.Google Scholar
40. Wang, J., Dong, Y., Tong, X., Lin, Z., and Guo, B. 2009. Kernel nyström method for light transport. Proc. SIGGRAPH. Google ScholarDigital Library
41. Wang, O., Fuchs, M., Fuchs, C., Davis, J., Seidel, H.-P., and Lensch, H. P. A. 2010. A context-aware light source. Proc. ICCP.Google Scholar
42. Wetzstein, G., and Bimber, O. 2007. Radiometric compensation through inverse light transport. Pacific Graphics, 391–399. Google ScholarDigital Library
43. Zhang, L., and Nayar, S. 2006. Projection defocus analysis for scene capture and image display. Proc. SIGGRAPH. Google ScholarDigital Library
