CardOOD: Robust Query-driven Cardinality Estimation under Out-of-Distribution
This addresses performance degradation in database query optimization for OOD workloads, representing an incremental improvement by adapting existing techniques to a specific domain.
The paper tackles the problem of query-driven cardinality estimators degrading under out-of-distribution (OOD) workloads, presenting CardOOD, a framework that extends transfer/robust learning techniques to improve robustness, with experimental validation showing efficacy in mitigating OOD issues and integration into PostgreSQL.
Query-driven learned estimators are accurate, flexible, and lightweight alternatives to traditional estimators in query optimization. However, existing query-driven approaches struggle with the Out-of-distribution (OOD) problem, where the test workload distribution differs from the training workload, leading to performancedegradation. In this paper, we present CardOOD, a general learning framework designed to construct robust query-driven cardinality estimators that are resilient against the OOD problem. Our framework focuses on offline training algorithms that develop one-off models from a static workload, suitable for model initialization and periodic retraining. In CardOOD, we extend classical transfer/robust learning techniques to train query-driven cardinalityestimators, and the algorithms fall into three categories: representation learning, data manipulation, and new learning strategies. As these learning techniques are originally evaluated in computervision tasks, we also propose a new learning algorithm that exploits the property of cardinality estimation. This algorithm, lying in the category of new learning strategy, models the partial order constraint of cardinalities by a self-supervised learning task. Comprehensive experimental studies demonstrate the efficacy of the algorithms of CardOOD in mitigating the OOD problem to varying extents. We further integrate CardOOD into PostgreSQL, showcasing its practical utility in query optimization.