Through-Wall Human Pose Estimation Using Radio Signals

Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba, Dina Katabi

Research output: A Conference proceeding or a Chapter in BookConference contribution

54 Citations (Scopus)

Abstract

This paper demonstrates accurate human pose estimation through walls and occlusions. We leverage the fact that wireless signals in the WiFi frequencies traverse walls and reflect off the human body. We introduce a deep neural network approach that parses such radio signals to estimate 2D poses. Since humans cannot annotate radio signals, we use state-of-the-art vision model to provide cross-modal supervision. Specifically, during training the system uses synchronized wireless and visual inputs, extracts pose information from the visual stream, and uses it to guide the training process. Once trained, the network uses only the wireless signal for pose estimation. We show that, when tested on visible scenes, the radio-based system is almost as accurate as the vision-based system used to train it. Yet, unlike vision-based pose estimation, the radio-based system can estimate 2D poses through walls despite never trained on such scenarios. Demo videos are available at our website.

Original languageEnglish
Title of host publicationProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages7356-7365
Number of pages10
ISBN (Electronic)9781538664209
ISBN (Print)9781538664209
DOIs
Publication statusPublished - Dec 2018
Externally publishedYes
EventIEEE Conference on Computer Vision and Pattern Recognition -
Duration: 1 Jan 2011 → …

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Conference

ConferenceIEEE Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR
Period1/01/11 → …

    Fingerprint

Cite this

Zhao, M., Li, T., Alsheikh, M. A., Tian, Y., Zhao, H., Torralba, A., & Katabi, D. (2018). Through-Wall Human Pose Estimation Using Radio Signals. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 7356-7365). [8578866] (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/CVPR.2018.00768