Geometric Computer Vision meets Deep Learning for Autonomous Driving Applications

Tesis doctoral presentada para lograr el título de Doctor por la Universidad Politécnica de Cataluña.--2021-05-17This dissertation intends to provide theoretical and practical contributions on the development of deep-learning algorithms for autonomous driving applications. The research is motivated by the need of deep neural networks (DNNs) to get a full understanding of the surrounding area and to be executed on real driving scenarios with real vehicles equipped with specific hardware, such as memory constrained (DSP or GPU platforms) or multiple optical sensors, which constraints the algorithm’s development forcing the designed deep networks to be accurate, with minimum number of operations and low-memory consumption. The main objective of this thesis is, on one hand, the research in the actual limitations of Deep Learning based algorithms that prevent them of being integrated in nowadays’ ADAS (Advanced Driver- Assistance Systems) functionalities, and on the other hand, the design and implementation of Deep Learning algorithms able to overcome such constraints to be applied on real autonomous driving scenarios, enabling their integration in low-memory hardware platforms and avoiding sensor redundancy. Deep Learning applications have been widely exploited over the last years but have some weak points that need to be faced and overcame in order to fully integrate Deep Learning into the development process of big manufacturers or automotive companies, like the time needed to design, train and validate and optimal network for a specific application or the vast knowledge from the required experts to tune hyperparameters of predefined networks in order to make them executable in the target platform and to obtain the biggest advantage of the hardware resources. During this thesis, we have addressed these topics and focused on the implementations of breakthroughs that would help in the industrial integration of Deep Learning based applications in the automobile industry. To this end, in the first part of this Thesis we focus on new approaches for vehicle pose estimation using only RGB planar images, avoiding the usage of other sensors that would lead to more costly and hard-to-manage systems. Furthermore, we investigate on the actual limitation of other hardware platforms different that the widely used GPUs (Graphical Processing Unit), such as FPGAs (Field-programmable Gate Array), to host a self-implemented algorithm that understands the surrounding scene of the ego-vehicle specifically designed to run fast and accurate, optimizing the FPGA resources. Finally, once we have done a deep research about the current constraints in the integration of Deep Learning based algorithms in the industry, we propose a new method to overcome them. We go deeper in the design of light DNN (Deep Neural Network) architectures and propose a framework for designing automatically DNNs to run on hardware constrained platforms. We test our development on commercial System on Chips (SoCs) such as Texas Instruments TDA2x SoC family, which are very typical in the automobile industry, obtaining remarkable results in image classification tasks in terms of accuracy, inference time and search time