“Nonlinear optimization framework for image-based modeling on programmable graphics hardware” by Hillesland, Molinov and Grzeszczuk

Graphics hardware is undergoing a change from fixed-function pipelines to more programmable organizations that resemble general purpose stream processors. In this paper, we show that certain general algorithms, not normally associated with computer graphics, can be mapped to such designs. Specifically, we cast nonlinear optimization as a data streaming process that is well matched to modern graphics processors. Our framework is particularly well suited for solving image-based modeling problems since it can be used to represent a large and diverse class of these problems using a common formulation. We successfully apply this approach to two distinct image-based modeling problems: light field mapping approximation and fitting the Lafortune model to spatial bidirectional reflectance distribution functions. Comparing the performance of the graphics hardware implementation to a CPU implementation, we show more than 5-fold improvement.

1. BISHOP, C. M. 1995. Neural Networks for Pattern Recognition. Clarendon Press. Google Scholar
2. BOLZ, J., FARMER, I., GRINSPUN, E., AND SCHRÖDER, P. 2003. Sparse Matrix Solvers on the GPU: Conjugate Gradients and Multigrid. ACM Transactions on Graphics 22, 3 (July). (Proceedings of ACM SIGGRAPH 2003) Google ScholarDigital Library
3. CARR, N. A., HALL, J. D., AND HART, J. C. 2002. Ray Engine. 2000 SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1–10.Google Scholar
4. CHEN, W.-C., BOUGUET, J.-Y., CHU, M. H., AND GRZESZCZUK, R. 2002. Light Field Mapping: Efficient Representation and Hardware Rendering of Surface Light Fields. ACM Transactions on Graphics 21, 3 (July), 447–456. ISSN 0730-0301 (Proceedings of ACM SIGGRAPH 2002). Google ScholarDigital Library
5. DENNIS, J. E. J., AND SCHNABEL, R. B. 1996. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Classics in Applied Mathematics, 16. SIAM. Google Scholar
6. FURUKAWA, R., KAWASAKI, H., IKEUCHI, K., AND SAKAUCHI, M. 2002. Appearance Based Object Modeling Using Texture Database: Acquisition, Compression and Rendering. Eurographics Rendering Workshop 2002. Google ScholarDigital Library
7. HARRIS, M. J., COOMBE, G., SCHEUERMANN, T., AND LASTRA, A. 2002. Physically-Based Visual Simulation on Graphics Hardware. 2002 SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1–10. Google Scholar
8. HOFF, K., CULVER, T., KEYSER, J., LIN, M., AND MANOCHA, D. 1999. Fast Computation of Generalized Voronoi Diagrams Using Graphics Hardware. In Proceedings of SIGGRAPH 99, Computer Graphics Proceedings, Annual Conference Series, 277–286. Google ScholarDigital Library
9. KAUTZ, J., AND MCCOOL, M. D. 1999. Interactive Rendering with Arbitrary BRDFs using Separable Approximations. Eurographics Rendering Workshop 1999 (June). Google Scholar
10. KHAILANY, B., DALLY, W. J., RIXNER, S., KAPASI, U. J., MATTSON, P., NAMKOONG, J., OWENS, J. D., TOWLES, B., AND CHANG, A. 2001. Imagine: Media Processing with Streams. IEEE Micro (March/April), 35–46. Google Scholar
11. KRÜGER, J., AND WESTERMANN, R. 2003. Linear Algebra Operators for GPU Implementation of Numerical Algorithms. ACM Transactions on Graphics 22, 3 (July). (Proceedings of ACM SIGGRAPH 2003). Google ScholarDigital Library
12. LAFORTUNE, E. P. F., FOO, S.-C., TORRANCE, K. E., AND GREENBERG, D. P. 1997. Non-Linear Approximation of Reflectance Functions. Proceedings of SIGGRAPH 97 (August), 117–126. Google ScholarDigital Library
13. LINDHOLM, E., KILGARD, M. J., AND MORETON, H. 2001. A User-Programmable Vertex Engine. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, 149–158. Google Scholar
14. MCALLISTER, D. K., LASTRA, A., AND HEIDRICH, W. 2002. Efficient Rendering of Spatial Bidirectional Reflectance Distribution Functions. Eurographics Rendering Workshop 2002 (June). Google Scholar
15. MCCOOL, M. D., ANG, J., AND AHMAD, A. 2001. Homomorphic Factorization of BRDFs for High-Performance Rendering. Proceedings of SIGGRAPH 2001 (August), 171–178. Google Scholar
16. NISHINO, K., SATO, Y., AND IKEUCHI, K. 1999. Eigen-Texture Method: Appearance Compression Based on 3D Model. In Proceedings of the IEEE Computer Science Conference on Computer Vision and Pattern Recognition (CVPR-99), 618–624.Google Scholar
17. PURCELL, T. J., BUCK, I., MARK, W. R., AND HANRAHAN, P. 2002. Ray Tracing on Programmable Graphics Hardware. ACM Transactions on Graphics 21, 3 (July), 703–712. ISSN 0730-0301 (Proceedings of ACM SIGGRAPH 2002). Google ScholarDigital Library
18. SATO, Y., WHEELER, M. D., AND IKEUCHI, K. 1997. Object Shape and Reflectance Modeling from Observation. Proceedings of SIGGRAPH 97 (August), 379–388. Google ScholarDigital Library
19. STRZODKA, R., AND RUMPF, M. 2001. Nonlinear Diffusion in Graphics Hardware. Proceedings EG/IEEE TCVG Symposium on Visualization, 75–84. Google Scholar
20. THOMPSON, C. J., HAHN, S., AND OSKIN, M. 2002. Using Modern Graphics Architectures for General-Purpose Computing: A Framework and Analysis. Proceedings of 35th International Symposium on Microarchitecture (MICRO-35). Google Scholar
21. YANG, R., WELCH, G., AND BISHOP, G. 2002. Real-Time Consensus-Based Scene Reconstruction using Commodity Graphics Hardware. Proceedings of Pacific Graphics. Google Scholar
22. YU, Y., DEBEVEC, P. E., MALIK, J., AND HAWKINS, T. 1999. Inverse Global Illumination: Recovering Reflectance Models of Real Scenes From Photographs. Proceedings of SIGGRAPH 99 (August), 215–224. Google Scholar