4D smart environment sensing with advanced millimeter wave radar sensors and deep learning

Target detection systems are widely used in various scenarios such as autonomous vehicles, smart home or security systems etc. In the application of autonomous vehicles, human body detection technology can help avoid collisions around vehicles and warn drivers about pedestrians behind the vehicle. For smart home applications, temperature, humidity, noise, light can be controlled by detecting and tracking the people in the room. Moreover, multi-media providers can recommend different content according to the person who is using their service, which helps to provide a comfortable living environment and improve the quality of life. Millimeter wave (mmWave) radar, lidar, event camera and conventional cameras are wildly used in environment sensing and object detection. Frequency Modulated Continuous Wave (FMCW) Radar whose transmitting frequency is modulated by a specific signal is a typical mmWave radar. FMCW radar has a simpler structure, a lower peak transmit power, and a less complicated modulation. In this work, we propose a method to classify human or other objects based on PointNet++ and point cloud in an FMCW radar uniform planar array system. We propose a 3D point cloud frame sequence by concatenating 3D point cloud of the same object at different time point. Via this method, deep neural network can learn from both inter-frame information and intra-frame information, which could enhance the classification performance of dynamic shape changing objects. The result shows that the proposed method has an outstanding performance in walking human classification.Master of Science (Signal Processing