On Social Consensus Mechanisms for Federated Learning Aggregation

Abstract

The possibility of training Machine Learning models in a decentralized way has always been a challenge when maintaining data privacy and accuracy. Federated Learning stands out for allowing accurate inference by combining multiple models trained on their data locally. It uses the strategy of local learning, where each node learns from its individual data, and then groups the learned models into a single, unified one, thus preserving data privacy. However, this raises some points of attention, such as ensuring security, the accuracy of the aggregate model, and communication optimization between federated nodes. This article analyzes aggregation techniques based on Game Theory in the aggregation stage in federated learning networks, aiming to validate the exploration of new concepts and contribute to the evolution of future research. We implemented three mechanisms of the decision by consensus in the aggregation of the models, including the well-known majority voting, as well as two other mechanisms never previously used in the context of FL, namely weighted majority voting and Borda count. To properly validate, we proposed a reference pipeline based on the CIFAR-10 dataset. The proposed benchmark partitions and allocates the dataset into a number of clients, and sets up a common pipeline for them. Such a pipeline allows one to train multiple clients and then test different aggregations in a fairly, reproducible way. Moreover, the proposal increased the precision of individual inference by more than 50%, showing efficiency in using non-trivial consensus mechanisms, such as weighted majority voting.