“CoLux: multi-object 3D micro-motion analysis using speckle imaging” by Smith, Desai, Agarwal and Gupta
Conference:
Type(s):
Title:
- CoLux: multi-object 3D micro-motion analysis using speckle imaging
Session/Category Title: Imaginative Imaging
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present CoLux, a novel system for measuring micro 3D motion of multiple independently moving objects at macroscopic standoff distances. CoLux is based on speckle imaging, where the scene is illuminated with a coherent light source and imaged with a camera. Coherent light, on interacting with optically rough surfaces, creates a high-frequency speckle pattern in the captured images. The motion of objects results in movement of speckle, which can be measured to estimate the object motion. Speckle imaging is widely used for micro-motion estimation in several applications, including industrial inspection, scientific imaging, and user interfaces (e.g., optical mice). However, current speckle imaging methods are largely limited to measuring 2D motion (parallel to the sensor image plane) of a single rigid object. We develop a novel theoretical model for speckle movement due to multi-object motion, and present a simple technique based on global scale-space speckle motion analysis for measuring small (5–50 microns) compound motion of multiple objects, along all three axes. Using these tools, we develop a method for measuring 3D micro-motion histograms of multiple independently moving objects, without tracking the individual motion trajectories. In order to demonstrate the capabilities of CoLux, we develop a hardware prototype and a proof-of-concept subtle hand gesture recognition system with a broad range of potential applications in user interfaces and interactive computer graphics.
References:
1. E. Archbold and A. E. Ennos. 1972. Displacement measurement from double exposure laser photographs. Optical Acta (1972).Google Scholar
2. J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk. 2012. Non-invasive imaging through opaque scattering layers. Nature 491 (2012). Google ScholarCross Ref
3. O. Cossairt, N. Matsuda, and M. Gupta. 2014. Digital refocusing with incoherent holography. In IEEE ICCP. Google ScholarCross Ref
4. J. C. Dainty. 1975. Laser Speckle and Related Phenomena. Springer. Google ScholarCross Ref
5. A. Davis, M. Rubinstein, N. Wadhwa, G. Mysore, F. Durand, and W. T. Freeman. 2014. The Visual Microphone: Passive Recovery of Sound from Video. ACM Trans. Graph. (2014).Google Scholar
6. L. Ek and N. E. Molin. 1971. Detection of the nodal lines and the amplitude of vibration by speckle interferometry. Opt. Commun. 2 (1971). Google ScholarCross Ref
7. J. Engel, T. Schöps, and D. Cremers. 2014. LSD-SLAM: Large-Scale Direct Monocular SLAM. In Proc. ECCV. Google ScholarCross Ref
8. M. Françon. 1979. Laser Speckle and Applications in Optics. Academic Press.Google Scholar
9. I. Freund. 1990. Looking Through Walls and Around Corners. Physica A: Statistical Mechanics and its Applications (1990).Google Scholar
10. J. García, Z. Zalevsky, P. García-Martínez, C. Ferreira, M. Teicher, and Y. Beiderman. 2008. Three-dimensional mapping and range measurement by means of projected speckle patterns. Applied Aptics (June 2008).Google Scholar
11. W J Godinez and K. Rohr. 2015. Tracking multiple particles in fluorescence time-lapse microscopy images via probabilistic data association. IEEE Trans Med Imaging (2015).Google Scholar
12. J. W. Goodman. 2000. Statistical Optics. Wiley-Interscience.Google Scholar
13. J. W. Goodman. 2007. Speckle Phenomena in Optics: Theory and Applications. Roberts and Company Publishers.Google Scholar
14. D. A. Gregory. 1976. Basic physical principles of defocused speckle photography: A tilt topology inspection technique. Optics and Laser Technology (1976).Google Scholar
15. D. A. Gregory. 1978. Topological Speckle and Surface Inspection. In Speckle Metrology. Chapter 8.Google Scholar
16. S. Gupta, D. Morris, S. Patel, and D. Tan. 2012. SoundWave: Using the Doppler Effect to Sense Gestures. In Proc. ACM CHI. Google ScholarDigital Library
17. P. Jacquot and P. K. Rastogi. 1979. Speckle motions induced by rigid-body movements in freespace geometry: an explicit investigation and extension to new cases. Applied Optics (1979).Google Scholar
18. M. L. Jakobsen, H. T. Yura, and S. G. Hanson. 2012. Spatial filtering velocimetry of objective speckles for measuring out-of-plane motion. Applied Optics 51, 9 (2012). Google ScholarCross Ref
19. K. Jo, M. Gupta, and S. Nayar. 2015. SpeDo: 6 DOF Ego-Motion Sensor Using Speckle Imaging. In Proc. ICCV. Google ScholarDigital Library
20. B. Judkewitz, R. Horstmeyer, I. M. Vellekoop, I. N. Papadopoulos, and C. Yang. 2015. Translation correlations in anisotropically scattering media. Nature Physics (2015).Google Scholar
21. J. C. Silveira Jacques Junior, S. R. Musse, and C. R. Jung. 2010. Crowd Analysis Using Computer Vision Techniques. IEEE Signal Processing Magazine 27, 5 (2010). Google ScholarCross Ref
22. K. Kapinchev, A. Bradu, F. Barnes, and A. Podoleanu. 2015. GPU Implementation of Cross-Correlation for Image Generation in Real Time. In Proc. ICSPCS. Google ScholarCross Ref
23. O. Katz, P. Heidmann, M. Fink, and S. Gigan. 2014. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nature Photonics 8 (2014). Google ScholarCross Ref
24. O. Katz, E. Small, and Y. Silberberg. 2012. Looking around corners and through thin turbid layers in real time with scattered incoherent light. Nature Photonics 6 (2012). Google ScholarCross Ref
25. J. Lien, N. Gillian, M. E. Karagozler, P. Amihood, C. Schwesig, E. Olson, H. Raja, and I. Poupyrev. 2016. Soli: Ubiquitous Gesture Sensing with Millimeter Wave Radar. ACM Trans. Graph. (2016).Google ScholarDigital Library
26. S. Rollie and K. Sundmacher. 2010. Tracking the clustering dynamics in ternary particle mixtures by flow cytometry. Powder Technology 202 (2010). Google ScholarCross Ref
27. J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finocchio, A. Blake, M. Cook, and R. Moore. 2013. Real-time human pose recognition in parts from single depth images. Commun. ACM (2013).Google Scholar
28. S. K. Sinha. 1988. Improving the accuracy and resolution of particle image or laser speckle velocimetry. Experiments in Fluids (1988).Google Scholar
29. S.W. Smith. 2002. Digital Signal Processing: A Practical Guide for Engineers and Scientists. California Technical Publishing.Google Scholar
30. P. Synnergren. 1997. Measurement of three-dimensional displacement fields and shape using electronic speckle photography. Optical Engineering (1997).Google Scholar
31. H. J. Tiziani. 1972. A study of the use of laser speckle to measure small tilts. Opt. Commun. 5 (1972).Google Scholar
32. H. J. Tiziani. 1978. Vibration Analysis and Deformation Measurement. In Speckle Metrology. Chapter 5. Google ScholarCross Ref
33. N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman. 2013. Phase-Based Video Motion Processing. ACM Trans. Graph. (2013).Google Scholar
34. S. Wang, J. Song, J. Lien, I. Poupyrev, and O. Hilliges. 2016. Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum. In Proc. ACM UIST. Google ScholarDigital Library
35. F. Weichert, D. Bachmann, B. Rudak, and D. Fisseler. 2013. Analysis of the Accuracy and Robustness of the Leap Motion Controller. Sensors (2013).Google Scholar
36. H.-Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand, and W. T. Freeman. 2012. Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Trans. Graph. (2012).Google Scholar
37. C. Xu, N. Ashwin, X. Zhang, and L. Cheng. 2015. Estimate Hand Poses Efficiently from Single Depth Images. IJCV (2015).Google Scholar
38. Z. Zalevsky, Y. Beiderman, I. Margalit, S. Gingold, M. Teicher, V. Mico, and J. Garcia. 2009. Simultaneous remote extraction of multiple speech sources and heart beats from secondary speckles pattern. Optics Express (2009).Google Scholar
39. C. Zhao, K.-Y. Chen, M. T. I. Aumi, S. Patel, and M. S. Reynolds. 2014. SideSwipe: Detecting In-air Gestures Around Mobile Devices Using GSM Signal. In Proc. ACM UIST. Google ScholarDigital Library
40. J. Zizka, A. Olwal, and R. Raskar. 2011. SpeckleSense: Fast, Precise, Low-cost and Compact Motion Sensing using Laser Speckle. In Proc. ACM UIST. Google ScholarDigital Library
