Adaptive Quantization and Differential Privacy Federated Learning Framework

Federated Learning (FL) enables devices to collaboratively train machine learning models without sharing raw data, promoting privacy-preserving AI. However, practical deployment faces challenges in balancing the data privacy, communication overhead, and the training convergence rate. For instance, adding noise to local models to preserve privacy can increase the size of updates, exacerbating communication overhead and reducing the convergence rate, while coarse quantization reduces communication costs but can degrade model accuracy. This paper introduces a novel integration of diverse quantization schemes, including both uniform and adaptive quantization, synergistically paired with additive noise mechanisms, to optimally trade off the model/training precision/rate, communication overhead, and privacy protection. By adapting quantization levels based on training dynamics, including gradient variance and model convergence, our approach minimizes the learning error upper bound while ensuring theoretically quantified differential privacy and achieves significant savings in the number of communicated bits. To the best of our knowledge, this is the first work to integrate adaptive quantization with additive noise in FL. More importantly, we provide theoretical guarantees for differential privacy and convergence of the proposed framework and empirically evaluate its communication privacy tradeoffs. Experimental results on popular datasets like MNIST, CIFAR demonstrate that our method enables the training of convolutional neural networks with less than 4-bit quantization, achieving privacy budgets as low as 1.0, while maintaining accuracy that approaches the standard, non-differentially private FedAvg algorithm.