A necessary and sufficient stability notion for adaptive generalization
This addresses the problem of ensuring reliable generalization in adaptive settings for machine learning and data analysis, though it appears incremental as it builds on existing stability and privacy concepts.
The paper introduces a new stability notion for computations that is necessary and sufficient to ensure generalization under adaptivity, specifically for bounded-sensitivity linear queries, and shows that all differentially private algorithms satisfy this notion.
We introduce a new notion of the stability of computations, which holds under post-processing and adaptive composition. We show that the notion is both necessary and sufficient to ensure generalization in the face of adaptivity, for any computations that respond to bounded-sensitivity linear queries while providing accuracy with respect to the data sample set. The stability notion is based on quantifying the effect of observing a computation's outputs on the posterior over the data sample elements. We show a separation between this stability notion and previously studied notion and observe that all differentially private algorithms also satisfy this notion.