Leveraging Learning Metrics for Improved Federated Learning
This work addresses the challenge of enhancing model performance in federated learning settings, which is incremental as it applies an existing metric to a new domain.
The paper tackles the problem of improving federated learning by introducing the first federated learning metric aggregation method using Effective Rank (ER), a novel learning metric from explainable AI. The result shows that this approach outperforms baseline Federated Averaging.
Currently in the federated setting, no learning schemes leverage the emerging research of explainable artificial intelligence (XAI) in particular the novel learning metrics that help determine how well a model is learning. One of these novel learning metrics is termed `Effective Rank' (ER) which measures the Shannon Entropy of the singular values of a matrix, thus enabling a metric determining how well a layer is mapping. By joining federated learning and the learning metric, effective rank, this work will \textbf{(1)} give the first federated learning metric aggregation method \textbf{(2)} show that effective rank is well-suited to federated problems by out-performing baseline Federated Averaging \cite{konevcny2016federated} and \textbf{(3)} develop a novel weight-aggregation scheme relying on effective rank.