FEATHERS: Federated Architecture and Hyperparameter Search
This addresses the challenge of automated model design in privacy-sensitive distributed environments, such as under GDPR and CCPA, though it appears incremental by combining existing techniques like federated learning and differential privacy.
The paper tackles the problem of neural architecture search and hyperparameter optimization in distributed data settings with privacy constraints, introducing FEATHERS, a method that jointly optimizes both while adhering to differential privacy, and shows convergence on classification tasks without performance detriment.
Deep neural architectures have profound impact on achieved performance in many of today's AI tasks, yet, their design still heavily relies on human prior knowledge and experience. Neural architecture search (NAS) together with hyperparameter optimization (HO) helps to reduce this dependence. However, state of the art NAS and HO rapidly become infeasible with increasing amount of data being stored in a distributed fashion, typically violating data privacy regulations such as GDPR and CCPA. As a remedy, we introduce FEATHERS - $\textbf{FE}$derated $\textbf{A}$rchi$\textbf{T}$ecture and $\textbf{H}$yp$\textbf{ER}$parameter $\textbf{S}$earch, a method that not only optimizes both neural architectures and optimization-related hyperparameters jointly in distributed data settings, but further adheres to data privacy through the use of differential privacy (DP). We show that FEATHERS efficiently optimizes architectural and optimization-related hyperparameters alike, while demonstrating convergence on classification tasks at no detriment to model performance when complying with privacy constraints.