Analyzing the latency overheads of acceleration in heterogeneous systems

Importance of low-latency heterogeneous systems in today’s world is immeasurable, which was proven in the last few years especially with the appearance of pandemic Covid-19 by forcing us to use online streaming technologies. By tackling more complex tasks and problems, algorithms require computational power that often exceeds the capabilities of pieces of technology used for computational tasks. In order to solve this problem, different methods are used. Hardware acceleration as a type of process acceleration is one of the most used methods in the field of AI mostly because it easily out-performs systems that are based on the general-purpose central processing units (CPUs). Their usage increases efficiency and reduces the time of execution needed to complete the process that was being set. This thesis investigated and provided an example of the implementation of hardware acceleration in heterogeneous systems and an analysis of latency overheads within that implementation. Results showed that hardware acceleration improves the time efficiency of complex systems and models, while simple systems can be solved with CPUs without including any hardware accelerator. Also, results showed that there is a connection between the total number of I/O and the execution time of the system which can be used later for more complex implementations