Epicure: Distilling Sequence Model Predictions into Patterns
This addresses the challenge of making high-precision predictions for rare sequences in tasks like function naming from source code, though it appears incremental as it builds on existing sequence models.
The paper tackles the problem of predicting high-entropy sequences by distilling model predictions into abstract patterns, showing that Epicure matches 61% more ground-truth names at a 10% false alarm rate compared to the best model prediction.
Most machine learning models predict a probability distribution over concrete outputs and struggle to accurately predict names over high entropy sequence distributions. Here, we explore finding abstract, high-precision patterns intrinsic to these predictions in order to make abstract predictions that usefully capture rare sequences. In this short paper, we present Epicure, a method that distils the predictions of a sequence model, such as the output of beam search, into simple patterns. Epicure maps a model's predictions into a lattice that represents increasingly more general patterns that subsume the concrete model predictions. On the tasks of predicting a descriptive name of a function given the source code of its body and detecting anomalous names given a function, we show that Epicure yields accurate naming patterns that match the ground truth more often compared to just the highest probability model prediction. For a false alarm rate of 10%, Epicure predicts patterns that match 61% more ground-truth names compared to the best model prediction, making Epicure well-suited for scenarios that require high precision.