Privacy in Metalearning and Multitask Learning: Modeling and Separations
It addresses privacy risks in model personalization for individuals using shared data, but is incremental in formalizing and separating existing frameworks.
The paper systematically studies differentially private personalized learning, constructing a taxonomy of formal frameworks and threat models, and proves a novel separation between private multitask learning and private metalearning.
Model personalization allows a set of individuals, each facing a different learning task, to train models that are more accurate for each person than those they could develop individually. The goals of personalization are captured in a variety of formal frameworks, such as multitask learning and metalearning. Combining data for model personalization poses risks for privacy because the output of an individual's model can depend on the data of other individuals. In this work we undertake a systematic study of differentially private personalized learning. Our first main contribution is to construct a taxonomy of formal frameworks for private personalized learning. This taxonomy captures different formal frameworks for learning as well as different threat models for the attacker. Our second main contribution is to prove separations between the personalized learning problems corresponding to different choices. In particular, we prove a novel separation between private multitask learning and private metalearning.