“Capturing the human figure through a wall” by Adib, Hsu, Mao, Katabi and Durand
Conference:
Type(s):
Title:
- Capturing the human figure through a wall
Session/Category Title: Tracking and Transients
Presenter(s)/Author(s):
Abstract:
We present RF-Capture, a system that captures the human figure — i.e., a coarse skeleton — through a wall. RF-Capture tracks the 3D positions of a person’s limbs and body parts even when the person is fully occluded from its sensor, and does so without placing any markers on the subject’s body. In designing RF-Capture, we built on recent advances in wireless research, which have shown that certain radio frequency (RF) signals can traverse walls and reflect off the human body, allowing for the detection of human motion through walls. In contrast to these past systems which abstract the entire human body as a single point and find the overall location of that point through walls, we show how we can reconstruct various human body parts and stitch them together to capture the human figure. We built a prototype of RF-Capture and tested it on 15 subjects. Our results show that the system can capture a representative human figure through walls and use it to distinguish between various users.
References:
1. Abdelnasser, H., Youssef, M., and Harras, K. A. 2015. Wigest: A ubiquitous wifi-based gesture recognition system. In IEEE INFOCOM.
2. Adib, F., and Katabi, D. 2013. See through walls with Wi-Fi! In ACM SIGCOMM.
3. Adib, F., Kabelac, Z., Katabi, D., and Miller, R. C. 2014. 3D Tracking via Body Radio Reflections. In Usenix NSDI.
4. Adib, F., Kabelac, Z., and Katabi, D. 2015. Multi-Person Localization via RF Body Reflections. In Usenix NSDI.
5. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: reconstruction and parameterization from range scans. In ACM Transactions on Graphics (TOG).
6. Appleby, R., and Anderton, R. N. 2007. Millimeter-wave and submillimeter-wave imaging for security and surveillance. Proceedings of the IEEE.
7. Beckmann, P., and Spizzichino, A. 1987. The scattering of electromagnetic waves from rough surfaces. Artech House, Inc.
8. Bocca, M., Kaltiokallio, O., Patwari, N., and Venkatasubramanian, S. 2013. Multiple target tracking with rf sensor networks. Mobile Computing, IEEE Transactions on.
9. Chai, J., and Hodgins, J. K. 2005. Performance animation from low-dimensional control signals. In ACM Transactions on Graphics (TOG), vol. 24, ACM, 686–696.
10. Charvat, G., Kempel, L., Rothwell, E., Coleman, C., and Mokole, E. 2010. An ultrawideband (UWB) switched-antenna-array radar imaging system. In IEEE ARRAY.
11. Chetty, K., Smith, G., and Woodbridge, K. 2012. Through-the-wall sensing of personnel using passive bistatic wifi radar at standoff distances. IEEE Trans. Geoscience and Remote Sensing.
12. Cooper, K. B., Dengler, R. J., Llombart, N., Bryllert, T., Chattopadhyay, G., Schlecht, E., Gill, J., Lee, C., Skalare, A., Mehdi, I., et al. 2008. Penetrating 3-d imaging at 4-and 25-m range using a submillimeter-wave radar. Microwave Theory and Techniques, IEEE Transactions on.
13. Dengler, R., Cooper, K., Chattopadhyay, G., Mehdi, I., Schlecht, E., Skalare, A., Chen, C., and Siegel, P. 2007. 600 ghz imaging radar with 2 cm range resolution. In Microwave Symposium, 2007. IEEE/MTT-S International.
14. Depatla, S., Buckland, L., and Mostofi, Y. 2015. X-ray vision with only wifi power measurements using rytov wave models. IEEE Transactions on Vehicular Technology, special issue on Indoor Localization, Tracking, and Mapping.
15. Dogaru, T., and Le, C. 2008. Validation of xpatch computer models for human body radar signature. Tech. rep., Army Research Laboratory.
16. Dogaru, T., Nguyen, L., and Le, C. 2007. Computer models of the human body signature for sensing through the wall radar applications. Tech. rep., Army Research Laboratory.
17. 1993. Understanding the Fcc Regulations for Low-power, Nonlicensed Transmitters. Office of Engineering and Technology Federal Communications Commission.
18. Forbes, 2013. How Does A Fighter Jet Lock Onto And Keep Track Of An Enemy Aircraft? http://www.forbes.com/sites/quora/2013/07/17/how-does-a-fighter-jet-lock-onto-and-keep-track-of-an-enemy-aircraft/.
19. Gall, J., Stoll, C., De Aguiar, E., Theobalt, C., Rosenhahn, B., and Seidel, H.-P. 2009. Motion capture using joint skeleton tracking and surface estimation. In IEEE CVPR.
20. Ganapathi, V., Plagemann, C., Koller, D., and Thrun, S. 2010. Real time motion capture using a single time-of-flight camera. In IEEE CVPR.
21. Gonzalez-Ruiz, A., Ghaffarkhah, A., and Mostofi, Y. 2014. An integrated framework for obstacle mapping with see-through capabilities using laser and wireless channel measurement. IEEE Sensors Journal.
22. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. In EUROGRAPHICS.
23. Hasler, N., Rosenhahn, B., Thormahlen, T., Wand, M., Gall, J., and Seidel, H.-P. 2009. Markerless motion capture with unsynchronized moving cameras. In IEEE CVPR.
24. Heide, F., Xiao, L., Heidrich, W., and Hullin, M. 2014. Diffuse mirrors: 3d reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors. In IEEE CVPR.
25. Herda, L., Fua, P., Plankers, R., Boulic, R., and Thalmann, D. 2000. Skeleton-based motion capture for robust reconstruction of human motion. In Computer Animation, IEEE.
26. Huffman, C., Hayes, J., and Ericson, L. 2014. Through-the-wall sensors (ttws) for law enforcement: Test & evaluation. In ManTech Advanced Systems International.
27. Ihrke, I., Kutulakos, K. N., Lensch, H., Magnor, M., and Heidrich, W. 2010. Transparent and specular object reconstruction. In Computer Graphics Forum.
28. Jia, Y., Kong, L., Yang, X., and Wang, K. 2013. Through-wall-radar localization for stationary human based on life-sign detection. In IEEE RADAR.
29. Joshi, K., Bharadia, D., Kotaru, M., and Katti, S. 2015. Wideo: Fine-grained device-free motion tracing using rf backscatter. In Usenix NSDI.
30. Kirmani, A., Hutchison, T., Davis, J., and Raskar, R. 2009. Looking around the corner using transient imaging. In IEEE CVPR.
31. Le, C., Dogaru, T., Lam, N., and Ressler, M. 2009. Ultrawideband (uwb) radar imaging of building interior: Measurements and prediction. In IEEE Transactions on Geoscience and Remote Sensing.
32. Li, L., McCann, J., Pollard, N., and Faloutsos, C. 2010. Bolero: a principled technique for including bone length constraints in motion capture occlusion filling. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation.
33. Liu, G., and McMillan, L. 2006. Estimation of missing markers in human motion capture. The Visual Computer 22, 9–11, 721–728.
34. Liu, D., Chen, X., and Yang, Y.-H. 2014. Frequency-based 3d reconstruction of transparent and specular objects. In IEEE CVPR.
35. Lucy, L. B. 1974. An iterative technique for the rectification of observed distributions. The astronomical journal 79, 745.
36. Mahafza, B. R. 2013. Radar systems analysis and design using MATLAB. Chapman & Hall.
37. Mori, G., Ren, X., Efros, A. A., and Malik, J. 2004. Recovering human body configurations: Combining segmentation and recognition. In IEEE CVPR.
38. Mostofi, Y. 2012. Cooperative wireless-based obstacle/object mapping and see-through capabilities in robotic networks. Mobile Computing, IEEE Transactions on.
39. Nannuru, S., Li, Y., Zeng, Y., Coates, M., and Yang, B. 2013. Radio-frequency tomography for passive indoor multitarget tracking. Mobile Computing, IEEE Transactions on.
40. Orfanidis, S. J. 2002. Electromagnetic waves and antennas. Rutgers University New Brunswick, NJ.
41. Park, S. I., and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. In ACM Transactions on Graphics (TOG), vol. 25, ACM, 881–889.
42. Poppe, R. 2007. Vision-based human motion analysis: An overview. Computer vision and image understanding 108, 1, 4–18.
43. Pu, Q., Jiang, S., Gollakota, S., and Patel, S. 2013. Whole-home gesture recognition using wireless signals. In ACM MobiCom.
44. Ralston, T., Charvat, G., and Peabody, J. 2010. Real-time through-wall imaging using an ultrawideband multiple-input multiple-output (MIMO) phased array radar system. In IEEE ARRAY.
45. Raskar, R., Nii, H., deDecker, B., Hashimoto, Y., Summet, J., Moore, D., Zhao, Y., Westhues, J., Dietz, P., Barnwell, J., et al. 2007. Prakash: lighting aware motion capture using photosensing markers and multiplexed illuminators. In ACM Transactions on Graphics (TOG), vol. 26, ACM, 36.
46. Richards, M. A. 2005. Fundamentals of radar signal processing. Tata McGraw-Hill Education.
47. Roetenberg, D., Luinge, H., and Slycke, P. 2009. Xsens mvn: full 6dof human motion tracking using miniature inertial sensors. Xsens Motion Technologies BV, Tech. Rep.
48. Schwartz, J. L., and Steinberg, B. D. 1998. Ultrasparse, ultrawideband arrays. Ultrasonics, Ferroelectrics, and Frequency Control, IEEE Transactions on.
49. Seifeldin, M., Saeed, A., Kosba, A., El-Keyi, A., and Youssef, M. 2013. Nuzzer: A large-scale device-free passive localization system for wireless environments. Mobile Computing, IEEE Transactions on.
50. Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., and Moore, R. 2013. Real-time human pose recognition in parts from single depth images. Communications of the ACM 56, 1, 116–124.
51. Szeliski, R. 2010. Computer vision: algorithms and applications. Springer Science & Business Media.
52. Tse, D., and Vishwanath, P. 2005. Fundamentals of Wireless Communications. Cambridge University Press.
53. Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M. G., and Raskar, R. 2012. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature Communications 3, 745.
54. VICON T-Series. http://www.vicon.com. VICON.
55. Vlasic, D., Adelsberger, R., Vannucci, G., Barnwell, J., Gross, M., Matusik, W., and Popović, J. 2007. Practical motion capture in everyday surroundings. In ACM Transactions on Graphics (TOG), vol. 26, ACM, 35.
56. Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. In ACM Transactions on Graphics (TOG), vol. 27, ACM, 97.
57. Wang, J. M., Fleet, D. J., and Hertzmann, A. 2008. Gaussian process dynamical models for human motion. Pattern Analysis and Machine Intelligence, IEEE Transactions on 30, 2, 283–298.
58. Wang, J., Vasisht, D., and Katabi, D. 2014. Rf-idraw: virtual touch screen in the air using rf signals. In Proceedings of the 2014 ACM conference on SIGCOMM, ACM, 235–246.
59. Wang, Y., Liu, J., Chen, Y., Gruteser, M., Yang, J., and Liu, H. 2014. E-eyes: device-free location-oriented activity identification using fine-grained wifi signatures. In ACM MobiCom.
60. Wilson, J., and Patwari, N. 2011. See-through walls: Motion tracking using variance-based radio tomography networks. In IEEE Transactions on Mobile Computing.
61. Woodward, R. M., Cole, B. E., Wallace, V. P., Pye, R. J., Arnone, D. D., Linfield, E. H., and Pepper, M. 2002. Terahertz pulse imaging in reflection geometry of human skin cancer and skin tissue. Physics in Medicine and Biology.
62. Xu, Y., Wu, S., Chen, C., Chen, J., and Fang, G. 2012. A novel method for automatic detection of trapped victims by ultrawideband radar. Geoscience and Remote Sensing, IEEE Transactions on.
63. Ye, M., Wang, H., Deng, N., Yang, X., and Yang, R. 2014. Real-time human pose and shape estimation for virtual try-on using a single commodity depth camera. IEEE transactions on visualization and computer graphics 20, 4, 550–559.
64. Youssef, M., Mah, M., and Agrawala, A. 2007. Challenges: device-free passive localization for wireless environments. In ACM MobiCom.
65. Zebra MotionWorks. https://www.zebra.com/us/en/solutions/location-solutions/sports-tracking.html. Zebra.
66. Zhuge, X., Savelyev, T., Yarovoy, A., Ligthart, L., Matuzas, J., and Levitas, B. 2008. Human body imaging by microwave uwb radar. In IEEE EuRAD.
