Polynomial Approximations of ReLU for Secure CNN Inference in Homomorphic Encryption Environments

Abstract

In this paper, we introduce a novel approach to improve secure neural network inference by addressing the challenges posed by homomorphic encryption, specifically within the context of the CKKSscheme. A major limitation in homomorphic encryption is the inability to efficiently handle non-linear activation functions, such as ReLU, due to their nonpolynomial nature. We propose an innovative 7th-degree polynomial approximation of the ReLU function, generated using the Remez algorithm, which closely mimics ReLU's behavior while being fully compatible with encrypted operations. To further optimize performance, we introduce dynamic domain extension techniques, which allow for efficient scaling of inputs during polynomial evaluation, significantly reducing computational overhead. Our method is validated using the MNIST dataset, demonstrating secure inference on encrypted data with 97.93% accuracy, while achieving near-plaintext performance. This work represents a significant step forward in the practical application of homomorphic encryption for neural network inference, providing a more efficient and accurate approach to approximating non-linear functions under encryption.