Balancing Interpretability and Predictive Accuracy for Unsupervised Tensor Mining
This work addresses a domain-specific problem in unsupervised tensor mining for exploratory data analysis, representing an incremental improvement.
The paper tackles the challenge of estimating the number of latent factors (rank) in PARAFAC tensor decomposition to improve interpretability and predictive accuracy, with preliminary results showing that balancing this trade-off enhances rank estimation quality.
The PARAFAC tensor decomposition has enjoyed an increasing success in exploratory multi-aspect data mining scenarios. A major challenge remains the estimation of the number of latent factors (i.e., the rank) of the decomposition, which yields high-quality, interpretable results. Previously, we have proposed an automated tensor mining method which leverages a well-known quality heuristic from the field of Chemometrics, the Core Consistency Diagnostic (CORCONDIA), in order to automatically determine the rank for the PARAFAC decomposition. In this work we set out to explore the trade-off between 1) the interpretability/quality of the results (as expressed by CORCONDIA), and 2) the predictive accuracy of the results, in order to further improve the rank estimation quality. Our preliminary results indicate that striking a good balance in that trade-off benefits rank estimation.