MCRapper: Monte-Carlo Rademacher Averages for Poset Families and Approximate Pattern Mining
This work addresses the need for statistically-sound pattern mining with guarantees on false positives and higher recall, which is incremental as it builds on existing techniques by combining previously separate capabilities.
The paper tackles the problem of efficiently computing Monte-Carlo Empirical Rademacher Averages for poset-structured function families, such as in pattern mining, and shows that MCRapper outperforms state-of-the-art methods in tasks like True Frequent Pattern mining, offering improved precision and recall guarantees.
We present MCRapper, an algorithm for efficient computation of Monte-Carlo Empirical Rademacher Averages (MCERA) for families of functions exhibiting poset (e.g., lattice) structure, such as those that arise in many pattern mining tasks. The MCERA allows us to compute upper bounds to the maximum deviation of sample means from their expectations, thus it can be used to find both statistically-significant functions (i.e., patterns) when the available data is seen as a sample from an unknown distribution, and approximations of collections of high-expectation functions (e.g., frequent patterns) when the available data is a small sample from a large dataset. This feature is a strong improvement over previously proposed solutions that could only achieve one of the two. MCRapper uses upper bounds to the discrepancy of the functions to efficiently explore and prune the search space, a technique borrowed from pattern mining itself. To show the practical use of MCRapper, we employ it to develop an algorithm TFP-R for the task of True Frequent Pattern (TFP) mining. TFP-R gives guarantees on the probability of including any false positives (precision) and exhibits higher statistical power (recall) than existing methods offering the same guarantees. We evaluate MCRapper and TFP-R and show that they outperform the state-of-the-art for their respective tasks.