Electrical overhead line towers are painted to protect their metal surfaces from direct interaction with the environment. Subsequently, paint is applied to refurbish exposed towers. On a vast network, it is difficult to identify which line segments or towers require refurbishment. Industry practice involves taking aerial images of towers and classifying the level of paint defects, albeit manually. This process is labour-intensive and subjective. In this paper, we propose a prototype system based on deep learning to automatically identify towers at risk due to paint deterioration. We use a representative tower inspection data set from the industry with 343k images of 6,333 towers for development and evaluation. Each tower is classified as be...