Human Sensing with mmWave Systems: from RADAR to Integrated Sensing and Communication

Contactless perception of human activity holds the potential to revolutionize the ways we interact with technology and our surroundings, enabling completely new remote, unobtrusive monitoring systems. In this context, the use of Millimeter-Wave (mmWave) reflected radio signals to detect, track, and analyze the movement of people leveraging the Radio Detection and Ranging (RADAR) principle has sparked great interest from academia and the industry alike. This is motivated, on the one hand, by the high sensitivity and robustness of such frequencies in perceiving and identifying small-scale motion of the human body parts. On the other hand, mmWave sensing is less invasive than widely adopted camera systems and raises significantly fewer privacy concerns by only sensing the activity-related information, without capturing any visual representation of the scene. However, despite its promising features, mmWave human sensing poses several challenges both at the signal processing and system design levels. The high sensitivity of mmWaves, to which they owe their appeal for accurate sensing, makes the mathematical modeling of the reflections on the human body extremely complex. Moreover, the high attenuation suffered by mmWave signals, paired with their low penetration through obstacles, poses the question of what kind of mmWave transceivers should be used, how to deploy them in order to provide good coverage, and how to combine the obtained information with other sensors. Regarding this aspect, a fundamental point is the feasibility of Integrated Sensing And Communication (ISAC), i.e., repurposing the existing (and future) communication infrastructure to jointly perform sensing tasks and communication. Indeed, future wireless networks are expected to be extremely dense, with billions of connected devices continuously exchanging signals, which could be exploited to gain pervasive sensing capabilities at almost zero cost. This thesis makes substantial contributions in the field of mmWave human sensing by advancing the state-of-the-art along two research lines. First, we focus on pure sensing, exploring the potential of dedicated mmWave RADAR devices for indoor people tracking and identification. In this sense, we develop algorithms that can exploit the reflected signal properties to obtain the position in space of multiple subjects, and extract Doppler-related features of their gait (i.e., their individual way of walking) to recognize their identities. Then, we utilize such algorithms to solve the important and timely problem of unobtrusive crowd monitoring in indoor environments. We demonstrate how it is possible to fuse the gait information from the mmWave reflections with infrared imaging to jointly perform body temperature screening, interpersonal distance estimation, and gait-based contact tracing. Second, we leverage mmWave RADAR signal processing methods to address ISAC, proposing the first approach to retrofit next-generation mmWave Wi-Fi Access Points (APs) into multipurpose devices that, in addition to providing connectivity, can also detect, track, and recognize the movements of people in their surroundings. To this end, we leverage the properties of the mmWave channel to reconstruct human movement features from irregular and sparse communication packets, thus fully reusing them for sensing purposes. In this way, we achieve a substantial reduction in the overhead and channel occupation of the sensing process. The methodology underpinning the work presented in this thesis is to integrate and jointly develop standard signal processing techniques and data-driven machine learning algorithms. Our claims are backed by extensive on-field experimentation supported by cutting-edge mmWave RADAR and ISAC research testbeds. We strongly believe that this approach represents the most promising way to develop future mmWave sensing systems and to achieve the envisioned goal of pervasive, human-oriented remote perception technology.Contactless perception of human activity holds the potential to revolutionize the ways we interact with technology and our surroundings, enabling completely new remote, unobtrusive monitoring systems. In this context, the use of Millimeter-Wave (mmWave) reflected radio signals to detect, track, and analyze the movement of people leveraging the Radio Detection and Ranging (RADAR) principle has sparked great interest from academia and the industry alike. This is motivated, on the one hand, by the high sensitivity and robustness of such frequencies in perceiving and identifying small-scale motion of the human body parts. On the other hand, mmWave sensing is less invasive than widely adopted camera systems and raises significantly fewer privacy concerns by only sensing the activity-related information, without capturing any visual representation of the scene. However, despite its promising features, mmWave human sensing poses several challenges both at the signal processing and system design levels. The high sensitivity of mmWaves, to which they owe their appeal for accurate sensing, makes the mathematical modeling of the reflections on the human body extremely complex. Moreover, the high attenuation suffered by mmWave signals, paired with their low penetration through obstacles, poses the question of what kind of mmWave transceivers should be used, how to deploy them in order to provide good coverage, and how to combine the obtained information with other sensors. Regarding this aspect, a fundamental point is the feasibility of Integrated Sensing And Communication (ISAC), i.e., repurposing the existing (and future) communication infrastructure to jointly perform sensing tasks and communication. Indeed, future wireless networks are expected to be extremely dense, with billions of connected devices continuously exchanging signals, which could be exploited to gain pervasive sensing capabilities at almost zero cost. This thesis makes substantial contributions in the field of mmWave human sensing by advancing the state-of-the-art along two research lines. First, we focus on pure sensing, exploring the potential of dedicated mmWave RADAR devices for indoor people tracking and identification. In this sense, we develop algorithms that can exploit the reflected signal properties to obtain the position in space of multiple subjects, and extract Doppler-related features of their gait (i.e., their individual way of walking) to recognize their identities. Then, we utilize such algorithms to solve the important and timely problem of unobtrusive crowd monitoring in indoor environments. We demonstrate how it is possible to fuse the gait information from the mmWave reflections with infrared imaging to jointly perform body temperature screening, interpersonal distance estimation, and gait-based contact tracing. Second, we leverage mmWave RADAR signal processing methods to address ISAC, proposing the first approach to retrofit next-generation mmWave Wi-Fi Access Points (APs) into multipurpose devices that, in addition to providing connectivity, can also detect, track, and recognize the movements of people in their surroundings. To this end, we leverage the properties of the mmWave channel to reconstruct human movement features from irregular and sparse communication packets, thus fully reusing them for sensing purposes. In this way, we achieve a substantial reduction in the overhead and channel occupation of the sensing process. The methodology underpinning the work presented in this thesis is to integrate and jointly develop standard signal processing techniques and data-driven machine learning algorithms. Our claims are backed by extensive on-field experimentation supported by cutting-edge mmWave RADAR and ISAC research testbeds. We strongly believe that this approach represents the most promising way to develop future mmWave sensing systems and to achieve the envisioned goal of pervasive, human-oriented remote perception technology