Federated ensemble learning on long-tailed data with prototypes

Federated learning enables multiple participants to train models without sharing their raw data. However, long-tailed data with imbalanced sample sizes among clients deteriorates the model’s performance in federated learning. Additionally, existing studies on class prototypes are less effective for federated long-tailed issues, as the difference in class prototype representation between the head classes and tail classes exacerbates global model updates and instability among clients. Therefore, we propose a Federated Ensemble Prototypes Learning (FedEP) approach that employs ensemble class prototypes instead of local class prototypes to alleviate class representation bias. Specifically, each client partitions its local dataset into multiple subsets to derive subset class prototypes and filters biased subset class prototypes using a threshold to obtain ensemble class prototypes. The server then aggregates these ensemble prototypes to develop novel global ones, which guide local training without extra data. Concurrently, we track category probability differences to assess the degree of deviation among class prototypes during the iterative process. Furthermore, our method has proven effective and outperforms baseline approaches on long-tailed data across various experimental settings.