“Snapshot difference imaging using correlation time-of-flight sensors”
Conference:
Type(s):
Title:
- Snapshot difference imaging using correlation time-of-flight sensors
Session/Category Title: High Performance Imaging
Presenter(s)/Author(s):
Abstract:
Computational photography encompasses a diversity of imaging techniques, but one of the core operations performed by many of them is to compute image differences. An intuitive approach to computing such differences is to capture several images sequentially and then process them jointly. In this paper, we introduce a snapshot difference imaging approach that is directly implemented in the sensor hardware of emerging time-of-flight cameras. With a variety of examples, we demonstrate that the proposed snapshot difference imaging technique is useful for direct-global illumination separation, for direct imaging of spatial and temporal image gradients, for direct depth edge imaging, and more.
References:
1. Bamji, C., P. O’Connor, T. Elkhatib, S. Mehta, B. Thompson, L. Prather, D. Snow, O. Akkaya, A. Daniel, A. Payne, T. Perry, M. Fenton, and V.-H. Chan. 2015. A 0.13 um CMOS System-on-Chip for a 512X424 Time-of-Flight Image Sensor With Multi-Frequency Photo-Demodulation up to 130 MHz and 2 GS/s ADC. IEEE Journal of Solid-State Circuits 50, 1 (2015), 303–319. Cross Ref
2. Chen, T., H. P. Lensch, C. Fuchs, and H.-P. Seidel. 2007. Polarization and phase-shifting for 3D scanning of translucent objects. In Proc. IEEE CVPR. IEEE, 1–8. Cross Ref
3. Darmont, A. 2012. High Dynamic Range Imaging: Sensors and Architectures. SPIE.
4. Gottardi, M., N. Massari, and S. A. Jawed. 2009. A 100 μW 128 X 64 Pixels Contrast-Based Asynchronous Binary Vision Sensor for Sensor Networks Applications. IEEE Journal of Solid-State Circuits 44, 5 (2009), 1582–1592. Cross Ref
5. Hansard, M., S. Lee, O. Choi, and R. Horaud. 2012. Time of Flight Cameras: Principles, Methods, and Applications. Springer.
6. Heide, F., M. B. Hullin, J. Gregson, and W. Heidrich. 2013. Low-Budget Transient Imaging using Photonic Mixer Devices. ACM Trans. Graph. (Proc. SIGGRAPH) 32, 4 (2013), 45:1–45:10.
7. Heide, F., L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich. 2014. Imaging in scattering media using correlation image sensors and sparse convolutional coding. Opt. Express 22, 21 (Oct 2014), 26338–26350. Cross Ref
8. Hontani, H., G. Oishi, and T. Kitagawa. 2014. Local estimation of high velocity optical flow with correlation image sensor. In European Conference on Computer Vision. Springer, 235–249.
9. Hwang, Y., J.-S. Kim, and I. S. Kweon. 2012. Difference-based image noise modeling using Skellam distribution. IEEE Trans. PAMI 34, 7 (2012), 1329–1341.
10. Kadambi, A., R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar. 2013. Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles. ACM Trans. Graph. 32, 6 (2013), 167.
11. Kim, H., S. Leutenegger, and A. J. Davison. 2016. Real-time 3D reconstruction and 6-DoF tracking with an event camera. In European Conference on Computer Vision. Springer, 349–364.
12. Koppal, S. J., I. Gkioulekas, T. Young, H. Park, K. B. Crozier, G. L. Barrows, and T. E. Zickler. 2013. Toward Wide-Angle Microvision Sensors. IEEE Trans. Pattern Anal. Mach. Intell. 35 (2013), 2982–2996.
13. Lange, R., P. Seitz, A. Biber, and R. Schwarte. 1999. Time-of-flight range imaging with a custom solid state image sensor. In Industrial Lasers and Inspection (EUROPTO Series). International Society for Optics and Photonics, 180–191.
14. Lichtsteiner, P., C. Posch, and T. Delbruck. 2008. A 128X128 120 dB 15 s latency asynchronous temporal contrast vision sensor. Solid-State Circuits, IEEE Journal of 43, 2 (2008), 566–576.
15. Liu, C. and J. Gu. 2014. Discriminative illumination: Per-pixel classification of raw materials based on optimal projections of spectral BRDF. IEEE Trans. PAMI 36, 1 (2014), 86–98.
16. Ma, W.-C., T. Hawkins, P. Peers, C.-F. Chabert, M. Weiss, and P. Debevec. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Proc. EGSR. Eurographics Association, 183–194.
17. Matsuda, N., O. Cossairt, and M. Gupta. 2015. MC3D: Motion Contrast 3D Scanning. In Proc. ICCP.
18. Nayar, S. K. and V. Branzoi. 2003. Adaptive Dynamic Range Imaging: Optical Control of Pixel Exposures Over Space and Time. In ICCV. 1168–1175.
19. Nayar, S. K., G. Krishnan, M. D. Grossberg, and R. Raskar. 2006. Fast separation of direct and global components of a scene using high frequency illumination. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3 (2006), 935–944.
20. O’Toole, M., S. Achar, S. G. Narasimhan, and K. N. Kutulakos. 2015. Homogeneous codes for energy-efficient illumination and imaging. ACM Trans. Graph. (Proc. SIGGRAPH) 34, 4 (2015), 35.
21. O’Toole, M., R. Raskar, and K. N. Kutulakos. 2012. Primal-dual coding to probe light transport. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4 (2012), 39.
22. Raskar, R., K.-H. Tan, R. Feris, J. Yu, and M. Turk. 2004. Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering Using Multi-flash Imaging. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3 (2004), 679–688.
23. Schmidt, M. 2011. Analysis, modeling and dynamic optimization of 3D time-of-flight imaging systems. Ph.D. Dissertation. University of Heidelberg.
24. Shrestha, S., F. Heide, W. Heidrich, and G. Wetzstein. 2016. Computational Imaging with Multi-Camera Time-of-Flight Systems. ACM Trans. Graph. (Proc. SIGGRAPH) (2016).
25. Skellam, J. G. 1946. The Frequency Distribution of the Difference Between Two Poisson Variates Belonging to Different Populations. Journal of the Royal Statistical Society 109, 3 (1946), 296–296. http://www.jstor.org/stable/2981372 Cross Ref
26. Solhusvik, J., J. Kuang, Z. Lin, S. Manabe, J. Lyu, H. Rhodes, et al. 2013. A comparison of high dynamic range CIS technologies for automotive applications. In Proc. 2013 Int. Image Sensor Workshop (IISW).
27. Tadano, R., A. K. Pediredla, and A. Veeraraghavan. 2015. Depth Selective Camera: A Direct, On-Chip, Programmable Technique for Depth Selectivity in Photography. In 2015 IEEE International Conference on Computer Vision (ICCV). IEEE.
28. Tomasi, C. and R. Manduchi. 1998. Bilateral filtering for gray and color images. In Proc. IEEE ICCV. IEEE, 839–846.
29. Tumblin, J., A. Agrawal, and R. Raskar. 2005. Why I want a gradient camera. In Proc. IEEE CVPR, Vol. 1. 103–110.
30. Wan, G., X. Li, G. Agranov, M. Levoy, and M. Horowitz. 2012. CMOS image sensors with multi-bucket pixels for computational photography. Solid-State Circuits, IEEE Journal of 47, 4 (2012), 1031–1042.
31. Wang, A. and A. Molnar. 2012. A Light-Field Image Sensor in 180 nm CMOS. IEEE Journal of Solid-State Circuits 47, 1 (jan 2012), 257–271. Cross Ref
32. Wang, A., S. Sivaramakrishnan, and A. Molnar. 2012. A 180nm CMOS image sensor with on-chip optoelectronic image compression. In Custom Integrated Circuits Conference (CICC), 2012 IEEE. IEEE, 1–4.
33. Wei, D., P. Masurel, T. Kurihara, and S. Ando. 2006. Optical flow determination with complex-sinusoidally modulated imaging. In Signal Processing, 2006 8th International Conference on, Vol. 2. IEEE.
34. Weikersdorfer, D., D. B. Adrian, D. Cremers, and J. Conradt. 2014. Event-based 3D SLAM with a depth-augmented dynamic vision sensor. In Robotics and Automation (ICRA), 2014 IEEE International Conference on. IEEE, 359–364.
35. Willassen, T., J. Solhusvik, R. Johansson, S. Yaghmai, H. Rhodes, S. Manabe, D. Mao, Z. Lin, D. Yang, O. Cellek, E. Webster, S. Ma, and B. Zhang. 2015. A 1280X 1080 4.2 μm split-diode pixel HDR sensor in 110nm BSI CMOS process. In Int. Image Sensor Workshop.
36. Wolff, L. B. 1990. Polarization-based material classification from specular reflection. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 11 (1990), 1059–1071.
37. Woodham, R. J. 1980. Photometric method for determining surface orientation from multiple images. Optical engineering 19, 1 (1980), 191139–191139.
38. Zomet, A. and S. K. Nayar. 2006. Lensless imaging with a controllable aperture. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, Vol. 1. IEEE, 339–346.
