Discovering outstanding subgroup lists for numeric targets using MDL
This addresses the issue of interpretable subgroup discovery for numeric targets in data mining, offering a method to reduce redundancy and reliance on heuristics, though it is incremental as it builds on existing SSD approaches.
The paper tackled the problem of redundant subgroups in subgroup set discovery by proposing a dispersion-aware formulation based on the minimum description length principle, resulting in SSD++, an algorithm that returns non-redundant, compact subgroups with strongly deviating means and small spread.
The task of subgroup discovery (SD) is to find interpretable descriptions of subsets of a dataset that stand out with respect to a target attribute. To address the problem of mining large numbers of redundant subgroups, subgroup set discovery (SSD) has been proposed. State-of-the-art SSD methods have their limitations though, as they typically heavily rely on heuristics and/or user-chosen hyperparameters. We propose a dispersion-aware problem formulation for subgroup set discovery that is based on the minimum description length (MDL) principle and subgroup lists. We argue that the best subgroup list is the one that best summarizes the data given the overall distribution of the target. We restrict our focus to a single numeric target variable and show that our formalization coincides with an existing quality measure when finding a single subgroup, but that-in addition-it allows to trade off subgroup quality with the complexity of the subgroup. We next propose SSD++, a heuristic algorithm for which we empirically demonstrate that it returns outstanding subgroup lists: non-redundant sets of compact subgroups that stand out by having strongly deviating means and small spread.