“Fast High-Resolution Appearance Editing Using Superimposed Projections” by Aliaga, Yeung, Law, Sajadi and Majumder

We present a system that superimposes multiple projections onto an object of arbitrary shape and color to produce high-resolution appearance changes. Our system produces appearances at an improved resolution compared to prior works and can change appearances at near interactive rates. Three main components are central to our system. First, the problem of computing compensation images is formulated as a constrained optimization which yields high-resolution appearances. Second, decomposition of the target appearance into base and scale images enables fast swapping of appearances on the object by requiring the constrained optimization to be computed only once per object. Finally, to make high-quality appearance edits practical, an elliptical Gaussian is used to model projector pixels and their interaction between projectors. To the best of our knowledge, we build the first system that achieves high-resolution and high-quality appearance edits using multiple superimposed projectors on complex nonplanar colored objects. We demonstrate several appearance edits including specular lighting, subsurface scattering, inter-reflections, and color, texture, and geometry changes on objects with different shapes and colors.

Aliaga, D., Law, A., and Yeung, Y. 2008. A virtual restoration stage for real-world objects. ACM Trans. Graph. 27, 5. Google ScholarDigital Library
Aliaga, D. and Xu, Y. 2010. A self-calibrating method for photogeometric acquisition of 3D objects. IEEE Trans. Pattern Anal. Mach. Intell. 32, 4, 747–754. Google ScholarDigital Library
Bandyopadhyay, D., Raskar, R., and Fuchs, H. 2001. Dynamic shader lamps painting on moveable objects. In Proceedings of the IEEE/ACM International Symposium on Augmented Reality (ISAR’01). Google ScholarDigital Library
Bimber, O., Coriand, F., Kleppe, A., Bruns, E., Zollmann, S., and Langlotz, T. 2005a. Superimposing pictorial artwork with projected imagery. IEEE Multimedia 12, 1, 16–26. Google ScholarDigital Library
Bimber, O., and Emmerling, A. 2006. Multifocal projection: A multiprojector technique for increasing focal depth. IEEE Trans. Vis. Comput. Graph. 12, 4, 658–667. Google ScholarDigital Library
Bimber, O., Emmerling, A., and Klemmer, T. 2005b. Embedded entertainment with smart projectors. IEEE Comput. 38, 1, 48–55. Google ScholarDigital Library
Bimber, O. and Iwai, D. 2008. Superimposing dynamic range. ACM Trans. Graph. 27, 5. Google ScholarDigital Library
Coleman, T. F. and Li, Y. 1996. A reflective Newton method for minimizing a quadratic function subject to bounds on some of the variables. SIAM J. Opt. 6, 4, 1040–1058. Google ScholarDigital Library
Chuang, Y., Zongker, D., Hindorff, J., Curless, B., Salesin, D., and Szeliski, R. 2000. Environment matting extensions: Towards higher accuracy and real-time capture. In Proceedings of the ACM SIGGRAPH Conference. 121–130. Google ScholarDigital Library
Damera-Venkata, N. and Chang, N. L. 2009. Display supersampling. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
Debevec, P. and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of the ACM SIGGRAPH Conference. 369–378. Google ScholarDigital Library
Durand, F. and Dorsey, J. 2002. Fast bilateral filtering for the display of high dynamic range images. In Proceedings of the ACM SIGGRAPH Conference. 257–266. Google ScholarDigital Library
Garg, G., Talvala, E., and Levoy, M. 2006. Symmetric photography: Exploiting data-sparseness in reflectance fields. In Proceedings of the Eurographics Workshop on Rendering. 251–262. Google ScholarDigital Library
Grossberg, M. D., Peri, H., Nayar, S. K., and Belhumeur, P. N. 2004. Making one object look like another: Controlling appearance using a projector-camera system. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogniation. Vol. 1. 452–459.Google Scholar
Grundhöfer, A. and Bimber, O. 2006. Real-Time adaptive radiometric compensation. IEEE Trans. Vis. Comput. Graph. 14, 1, 97–108. Google ScholarDigital Library
Law, A., Aliaga, D., and Majumder, A. 2010. Projector placement planning for high quality visualizations on real-world colored objects. IEEE Trans. Vis. Comput. Graph. 16, 6. Google ScholarDigital Library
Law, A., Aliaga, D., Sajadi, B., Majumder, A., and Pizlo, Z. 2011. Perceptually-Based appearance modification for compliant appearance editing. Comput. Graph. Forum 0, 0.Google Scholar
Majumder, A. 2005. Is spatial super-resolution possible with multiple overlapping projectors? In Proceedings of the IEEE International Conference on Audio, Speech and Signal Processing (ICASSP’05). Vol. 4, 209–212.Google Scholar
Mallick, S. P., Zickler, T., Kriegman, D. J., and Belhumeur, P. N. 2005. Beyond lambert: Reconstructing specular surfaces using color. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 619–626. Google ScholarDigital Library
Nayar, S. K., Peri, H., Grossberg, M. D., and Belhumeur, P. N. 2003. A projection system with radiometric compensation for screen imperfections. In Proceedings of the ICCV Workshop on Projector-Camera Systems.Google Scholar
Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3D geometry. ACM Trans. Graph. 24, 3, 536–543. Google ScholarDigital Library
Ng, R., Ramamoorthi, R., and Hanrahan, P. 2003. All-Frequency shadows using non-linear wavelet lighting approximation. ACM Trans. Graph. 22, 3, 376–381. Google ScholarDigital Library
Okazaki, T., Okatani, T., and Deguchi, K. 2009. Shape reconstruction by combination of structured-light projection and photometric stereo using projector-camera system. In Proceedings of the Pacific Rim Symposium on Advances in Image and Video Technology. 410–422. Google ScholarDigital Library
Park, S. C., Park, M. K., and Kang, M. G. 2003. Super-Resolution image reconstruction: A technical overview. IEEE Signal Process. Mag. 20, 3, 21–36.Google ScholarCross Ref
Rau, A., Gill, G., Majumder, A., Towles, H., and Fuchs, H. 2003. PixelFlex2: A comprehensive automatic casually aligned multi-projector display. In Proceedings of the IEEE International Workshop on Projector-Camera Systems.Google Scholar
Raskar, R., van Baar, J., Beardsley, P., Willwacher, T., Rao, S., and Forlines, C. 2003. iLamps: Geometrically aware and self-configuring projectors. ACM Trans. Graph. 22, 3, 809–818. Google ScholarDigital Library
Raskar, R., Welch, G., Low, K. L., and Bandyopadhyay, D. 2001. Shader lamps: Animating real objects with image-based illumination. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques. 89–102. Google ScholarDigital Library
Ruzon, M. and Tomasi, C. 2000. Alpha estimation in natural images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 18–25.Google Scholar
Seitz, S. M., Matsushita, Y., and Kutulakos, K. N. 2005. A theory of inverse light transport. In Proceedings of the IEEE International Conference on Computer Vision. 1440–1447. Google ScholarDigital Library
Sen, P., Chen, B., Garg, G., Marschner, S. R., Horowitz, M., Levoy, M., and Lensch, H. P. A. 2005. Dual photography. ACM Trans. Graph. 24, 3, 745–755. Google ScholarDigital Library
Sen, P. and Darabi, S. 2009. Comprehensive dual photography. Comput. Graph. Forum 28, 2, 609–618.Google ScholarCross Ref
Summet, J., Flagg, M., Cham, T., Rehg, J., and Sukthankar, R. 2006. Shadow elimination and blinding light suppression for interactive projected displays. IEEE Trans. Vis. Comput. Graph. 13, 3, 508– 517. Google ScholarDigital Library
Wang, J., Dong, Y., Tong, X., Lin, Z., and Guo, B. 2009. Kernel Nystrom method for light transport. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
Wetzstein, G. and Bimber, O. 2007. Radiometric compensation through inverse light transport. In Proceedings of the 15th Pacific Conference on Computer Graphics and Applications. 391–399. Google ScholarDigital Library
Xu, Y. and Aliaga, D. 2009. An adaptive correspondence algorithm for modeling scenes with strong inter-reflections. IEEE Trans. Vis. Comput. Graph. 15, 3, 465–480. Google ScholarDigital Library
Yang, R., Majumder, A., and Brown, M. 2005. Camera based calibration techniques for seamless multi-projector displays. IEEE Trans. Vis. Comput. Graph. 11, 2, 193–206. Google ScholarDigital Library
Zhang, L. and Nayar, S. 2006. Projection defocus analysis for scene capture and image display. ACM Trans. Graph. 25, 3, 907– 915. Google ScholarDigital Library