Ultrasonic testing is a non-destructive evaluation (NDE) technique that is used to inspect the integrity of the material and check if there are any defects in its internal structure. The acquisition of ultrasonic data is already done in an automated fashion using robotic manipulators, but the analysis of the data is still done manually by specially trained experts. Manual analysis is subject to human errors especially when a large amount of data needs to be inspected. The goal of this doctoral dissertation is to develop deep learning-based methods that can be used to efficiently and reliably detect defects from ultrasonic images. Deep learning methods have been achieving great results in many image analysis tasks during the past few years. ...