Subgroup Discovery in Unstructured Data
This addresses a bottleneck in knowledge discovery for researchers and practitioners working with unstructured data, enabling subgroup analysis in domains like image analysis, though it is an incremental advancement building on existing variational autoencoder frameworks.
The paper tackles the problem of subgroup discovery in unstructured, high-dimensional data like images, where traditional methods fail due to lack of well-defined attributes, by introducing a subgroup-aware variational autoencoder that learns representations leading to higher-quality subgroups, with experimental results demonstrating effectiveness in learning high-quality and interpretable subgroups.
Subgroup discovery is a descriptive and exploratory data mining technique to identify subgroups in a population that exhibit interesting behavior with respect to a variable of interest. Subgroup discovery has numerous applications in knowledge discovery and hypothesis generation, yet it remains inapplicable for unstructured, high-dimensional data such as images. This is because subgroup discovery algorithms rely on defining descriptive rules based on (attribute, value) pairs, however, in unstructured data, an attribute is not well defined. Even in cases where the notion of attribute intuitively exists in the data, such as a pixel in an image, due to the high dimensionality of the data, these attributes are not informative enough to be used in a rule. In this paper, we introduce the subgroup-aware variational autoencoder, a novel variational autoencoder that learns a representation of unstructured data which leads to subgroups with higher quality. Our experimental results demonstrate the effectiveness of the method at learning subgroups with high quality while supporting the interpretability of the concepts.