“DisCo: Display-Camera Communication Using Rolling Shutter Sensors” by Jo, Gupta and Nayar

  • ©Kensei Jo, Mohit Gupta, and Shree K. Nayar

Conference:


Type:


Title:

    DisCo: Display-Camera Communication Using Rolling Shutter Sensors

Session/Category Title: USER INTERFACES

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We present DisCo, a novel display-camera communication system. DisCo enables displays and cameras to communicate with each other while also displaying and capturing images for human consumption. Messages are transmitted by temporally modulating the display brightness at high frequencies so that they are imperceptible to humans. Messages are received by a rolling shutter camera that converts the temporally modulated incident light into a spatial flicker pattern. In the captured image, the flicker pattern is superimposed on the pattern shown on the display. The flicker and the display pattern are separated by capturing two images with different exposures. The proposed system performs robustly in challenging real-world situations such as occlusion, variable display size, defocus blur, perspective distortion, and camera rotation. Unlike several existing visible light communication methods, DisCo works with off-the-shelf image sensors. It is compatible with a variety of sources (including displays, single LEDs), as well as reflective surfaces illuminated with light sources. We have built hardware prototypes that demonstrate DisCo’s performance in several scenarios. Because of its robustness, speed, ease of use, and generality, DisCo can be widely deployed in several applications, such as advertising, pairing of displays with cell phones, tagging objects in stores and museums, and indoor navigation.

References:


    1. Li-Wei Chan, Hsiang-Tao Wu, Hui-Shan Kao, Ju-Chun Ko, Home-Ru Lin, Mike Y. Chen, Jane Hsu, and Yi-Ping Hung. 2010. Enabling beyond-surface interactions for interactive surface with an invisible projection. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (UIST’10). ACM, New York, NY, 263–272. DOI:http://dx.doi.org/10.1145/1866029.1866072 
    2. Abbas Cheddad, Joan Condell, Kevin Curran, and Paul McKevitt. 2010. Digital image steganography: Survey and analysis of current methods. Signal Processing 90, 3, 727–752. DOI:http://dx.doi.org/10.1016/j.sigpro.2009.08.010 
    3. Christos Danakis, Mostafa Afgani, Gordon Povey, Ian Underwood, and Harald Haas. 2012. Using a CMOS camera sensor for visible light communication. In Proceedings of the IEEE Workshop on Optical Wireless Communications.
    4. H. Elgala, R. Mesleh, and H. Haas. 2009. Predistortion in optical wireless transmission using OFDM. In Proceedings of the International Conference on Hybrid Intelligent Systems. 
    5. Fujifilm. 2016. EXR CMOS Technology. Retrieved June 4, 2016, from http://finepix.com/exr_cmos/en/.
    6. A. Grundhofer, M. Seeger, F. Hantsch, and O. Bimber. 2007. Dynamic adaptation of projected imperceptible codes. In Proceedings of the International Symposium on Mixed and Augmented Reality. 
    7. ISO. 2006. QR Code 2005 Bar Code Symbology Specification. ISO/IEC 18004:2006,.
    8. K. Kagawa, J. Ohta, and J. Tanida. 2009. Dynamic reconfiguration of differential pixel output for CMOS imager dedicated to WDM-SDM indoor optical wireless LAN. IEEE Photonics Technology Letters 21, 18, 1308–1310.
    9. K. Kamijo, N. Kamijo, and Zhang Gang. 2008. Invisible barcode with optimized error correction. In Proceedings of the IEEE International Conference on Image Processing. DOI:http://dx.doi.org/10.1109/ICIP.2008.4712185
    10. Yoshifumi Kitamura, Takashige Konishi, Sumihiko Yamamoto, and Fumio Kishino. 2001. Interactive stereoscopic display for three or more users. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’01). ACM, New York, NY, 231–240. DOI:http://dx.doi.org/10.1145/383259.383285 
    11. Mitsunori Matsushita, Makoto Iida, Takeshi Ohguro, Yoshinari Shirai, Yasuaki Kakehi, and Takeshi Naemura. 2004. Lumisight table: A face-to-face collaboration support system that optimizes direction of projected information to each stakeholder. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (CSCW’04). ACM, New York, NY, 274–283. DOI:http://dx.doi.org/10.1145/1031607.1031653 
    12. N. Matsushita, D. Hihara, T. Ushiro, S. Yoshimura, J. Rekimoto, and Y. Yamamoto. 2003. ID CAM: A smart camera for scene capturing and ID recognition. In Proceedings of the International Symposium on Mixed and Augmented Reality. 
    13. Ankit Mohan, Grace Woo, Shinsaku Hiura, Quinn Smithwick, and Ramesh Raskar. 2009. Bokode: Imperceptible visual tags for camera based interaction from a distance. ACM Transactions on Graphics 28, 3, Article No. 98. DOI:http://dx.doi.org/10.1145/1531326.1531404 
    14. S. K. Nayar and T. Mitsunaga. 2000. High dynamic range imaging: Spatially varying pixel exposures. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
    15. Richard van Nee and Ramjee Prasad. 2000. OFDM for Wireless Multimedia Communications. Artech House Publishers, Norwood, MA. 
    16. OmniVision. 2011. OV4689. Retrieved June 4, 2016, from http://www.ovt.com/products/sensor.php?id=136.
    17. Samuel David Perli, Nabeel Ahmed, and Dina Katabi. 2010. PixNet: Interference-free wireless links using LCD-camera pairs. In Proceedings of the 16th Annual International Conference on Mobile Computing and Networking (MobiCom’10). 137–148. 
    18. Ramesh Raskar, Amit Agrawal, and Jack Tumblin. 2006. Coded exposure photography: Motion deblurring using fluttered shutter. ACM Transactions on Graphics 25, 3, 795–804. 
    19. M. S. Z. Sarkera, I. Takai, M. Andoh, K. Yasutomi, S. Itoh, and S. Kawahito. 2009. A CMOS imager and 2-D light pulse receiver array for spatial optical communication. In Proceedings of the IEEE Asian Solid-State Circuits Conference.
    20. Jan-Jaap van De Beek, Magnus Sandell, and Per Ola Börjesson. 1997. ML estimation of time and frequency offset in OFDM systems. IEEE Transactions on Signal Processing 45, 7, 1800–1805. 
    21. J. Vucic, C. Kottke, S. Nerreter, K.-D. Langer, and J. W. Walewski. 2010. 513 Mbit/s visible light communications link based on DMT-modulation of a white LED. Journal of Lightwave Technology 28, 24, 3512–3518.
    22. Grace Woo, Andy Lippman, and Ramesh Raskar. 2012. VRCodes: Unobtrusive and active visual codes for interaction by exploiting rolling shutter. In Proceedings of the International Symposium on Mixed and Augmented Reality. http://dx.doi.org/10.1109/ISMAR.2012.6402539 
    23. S. Yoshimura, T. Sugiyama, K. Yonemoto, and K. Ueda. 2001. A 48 kframe/s CMOS image sensor for real-time 3-D sensing and motion detection. In Proceedings of the IEEE International Solid-State Circuits Conference.
    24. Wenjia Yuan, Kristin Dana, Ashwin Ashok, Marco Gruteser, and Narayan Mandayam. 2013. Spatially varying radiometric calibration for camera-display messaging. In Proceedings of the IEEE Conference on Signal and Information Processing.

ACM Digital Library Publication:


Overview Page: