Effects of approximation in computation on the accuracy and performance of deep neural network inference

Recently, deep learning is at the forefront of the state-of-the-art machine learning algorithms and has shown excellent results in a variety of applications such as medical field, consumer as well as autonomous vehicles. Convolutional Neural Network (CNN) - is the leading deep learning architecture that is mostly applied. However, huge dataset is needed to train with complex architecture to achieve precise learning. Inference can be performed when given a ready CNN model and its weight file to another user. Inference takes time with precise weights and huge dataset. To overcome this problem, and enhance the inference system, approximation computation will be applying in terms of weight for changed of decimal place. The smaller size of the dataset is used in the inference process to reduce the inference time. MobileNetV2 architecture is used with the new weight for inference. Also, open source libraries such as TensorFlow, Keras and python is used. GPU (NVIDIA GeForce GTX 1060 6GB 64 Bit) is used as training and inference platform. Inference time is shortened, and the accuracy of performance for new weights compare with the precise weight only has a small gap which still has a great performance for classification. This work has proved that with 4 decimal places is able to obtain the same accuracy for inference when compared to benchmark with 9 decimal places. Inference time for 4 decimal places is also less than benchmark time