Efficient Optimisation Framework for Convolutional Neural Networks with Secure Multiparty Computation

In recent years, deep learning has become an increasingly popular approach to modelling data, due to its ability to detect abstract underlying patterns in data. Its practical applications have been limited, however, by data privacy concerns, restricting its use in major sectors such as healthcare and banking. Secure multiparty computation (MPC) is a scheme which allows multiple parties to perform joint computations over private data, while keeping the content of their data secret. MPC can enable privacy-preserving machine learning, however current implementations are rarely applied in practice due to the prohibitively high cost of performing thousands of computations and transmitting data between parties. In this paper we propose a framework incorporating various optimisation approaches from the wider field of privacy-preserving deep learning, including privacy-preserving batch normalisation and polynomial approximation of activation functions, and evaluate their performance when applied to a privacy-preserving convolutional neural network (CNN), discussing the trade-off each offers in terms of their accuracy and efficiency. We experiment with parametric polynomial (PPoly) activations by deriving polynomial approximations to activation functions and allowing the network to tune the coefficients as learning weights. We will show that, in shallow CNNs, the application of batch normalisation in combination with a PPoly activation layer can result in faster convergence, with testing accuracy exceeding that achieved with an unencrypted network, at the cost of longer running times