TabDeco: A Comprehensive Contrastive Framework for Decoupled Representations in Tabular Data
This work addresses the problem of representation learning for tabular data, which is crucial for applications in fields like finance and healthcare, but it appears incremental as it builds on existing contrastive learning frameworks with novel adaptations.
The paper tackles the challenge of adapting self-supervised contrastive learning to tabular data by introducing TabDeco, a method that uses attention-based encoding and contrastive learning to disentangle feature representations, resulting in consistent performance improvements over existing deep learning and gradient boosting methods like XGBoost, CatBoost, and LightGBM across benchmark tasks.
Representation learning is a fundamental aspect of modern artificial intelligence, driving substantial improvements across diverse applications. While selfsupervised contrastive learning has led to significant advancements in fields like computer vision and natural language processing, its adaptation to tabular data presents unique challenges. Traditional approaches often prioritize optimizing model architecture and loss functions but may overlook the crucial task of constructing meaningful positive and negative sample pairs from various perspectives like feature interactions, instance-level patterns and batch-specific contexts. To address these challenges, we introduce TabDeco, a novel method that leverages attention-based encoding strategies across both rows and columns and employs contrastive learning framework to effectively disentangle feature representations at multiple levels, including features, instances and data batches. With the innovative feature decoupling hierarchies, TabDeco consistently surpasses existing deep learning methods and leading gradient boosting algorithms, including XG-Boost, CatBoost, and LightGBM, across various benchmark tasks, underscoring its effectiveness in advancing tabular data representation learning.