Generator Assisted Mixture of Experts For Feature Acquisition in Batch
This work addresses the challenge of efficient feature acquisition in batch for machine learning applications where time constraints make sequential acquisition impractical, though it appears incremental as it builds on prior sequential methods.
The paper tackles the problem of batch feature acquisition, where a subset of unobserved features is selected and acquired all at once to improve prediction accuracy, addressing the inefficiency of sequential acquisition in time-sensitive settings. The proposed method, which uses a feature generator and a mixture of experts model, outperforms existing methods on four datasets by achieving a better trade-off between accuracy and feature acquisition cost.
Given a set of observations, feature acquisition is about finding the subset of unobserved features which would enhance accuracy. Such problems have been explored in a sequential setting in prior work. Here, the model receives feedback from every new feature acquired and chooses to explore more features or to predict. However, sequential acquisition is not feasible in some settings where time is of the essence. We consider the problem of feature acquisition in batch, where the subset of features to be queried in batch is chosen based on the currently observed features, and then acquired as a batch, followed by prediction. We solve this problem using several technical innovations. First, we use a feature generator to draw a subset of the synthetic features for some examples, which reduces the cost of oracle queries. Second, to make the feature acquisition problem tractable for the large heterogeneous observed features, we partition the data into buckets, by borrowing tools from locality sensitive hashing and then train a mixture of experts model. Third, we design a tractable lower bound of the original objective. We use a greedy algorithm combined with model training to solve the underlying problem. Experiments with four datasets show that our approach outperforms these methods in terms of trade-off between accuracy and feature acquisition cost.