Dissimilar Batch Decompositions of Random Datasets
This work addresses dataset batching for machine learning, but it appears incremental as it focuses on theoretical bounds without new empirical results.
The paper tackles the problem of decomposing random datasets into batches with restricted similarity between data points, deriving high probability bounds for the minimum batch size and analyzing tradeoffs with similarity constraints.
For better learning, large datasets are often split into small batches and fed sequentially to the predictive model. In this paper, we study such batch decompositions from a probabilistic perspective. We assume that data points (possibly corrupted) are drawn independently from a given space and define a concept of similarity between two data points. We then consider decompositions that restrict the amount of similarity within each batch and obtain high probability bounds for the minimum size. We demonstrate an inherent tradeoff between relaxing the similarity constraint and the overall size and also use martingale methods to obtain bounds for the maximum size of data subsets with a given similarity.