Toward Complex 3D Movement Detection to Analyze Human Behavior via Radio-Frequency Signals

A doctoral dissertation completed for the degree of Doctor of Science (Technology) to be defended, with the permission of the Aalto University School of Electrical Engineering, at a public examination held with remote technology on 15 September 2020 at 6:00pm. Zoom link: https://aalto.zoom.us/j/6549167529A driver's attention, parallel actions, and emotions directly influence driving behavior. Any secondary task, be it cognitive, visual, or manual, that diverts driver focus from the primary task of driving is a source of distraction. Longer response time, inability to scan the road, and missing visual cues can all lead to car crashes with serious consequences. Current research focuses on detecting distraction by means of vehicle-mounted video cameras or wearable sensors for tracking eye movements and head rotation. Facial expressions, speech, and physiological signals are also among the widely used indicators for detecting distraction. These approaches are accurate, fast, and reliable but come with a high installation cost, requirements related to lighting conditions, privacy intrusions, and energy consumption. Over the past decade, the use of radio signals has been investigated as a possible solution for the aforementioned limitations of today's technologies. Changes in radio-signal patterns caused by movements of the human body can be analyzed and thereby used in detecting humans' gestures and activities. Human behavior and emotions, in particular, are less explored in this regard and are addressed mostly with reference to physiological signals. The thesis exploited multiple wireless technologies (1.8~GHz, WiFi, and millimeter wave) and combinations thereof to detect complex 3D movements of a driver in a car. Upper-body movements are vital indicators of a driver's behavior in a car, and the information from these movements could be used to generate appropriate feedback, such as warnings or provision of directives for actions that would avoid jeopardizing safety. Existing wireless-system-based solutions focus primarily on either large or small movements, or they address well-defined activities. They do not consider discriminating large movements from small ones, let alone their directions, within a single system. These limitations underscore the requirement to address complex natural-behavior situations precisely such as that in a car, which demands not only isolating particular movements but also classifying and predicting them. The research to reach the attendant goals exploited physical properties of RF signals, several hardware-software combinations, and building of algorithms to process and detect body movements -- from the simple to the complex. Additionally, distinctive feature sets were addressed for machine-learning techniques to find patterns in data and predict states accordingly. The systems were evaluated by performing extensive real-world studies