Task-Agnostic Contrastive Pretraining for Relational Deep Learning
This work addresses scalability and reuse issues in RDL for database applications, though it appears incremental as it builds on existing RDL and contrastive learning principles.
The paper tackles the problem of task-specific supervised learning in Relational Deep Learning (RDL) by proposing a task-agnostic contrastive pretraining approach that uses row-level, link-level, and context-level objectives to capture structural and semantic heterogeneity. Preliminary results show that fine-tuning these pretrained models outperforms training from scratch on standard benchmarks, validating the method's effectiveness.
Relational Deep Learning (RDL) is an emerging paradigm that leverages Graph Neural Network principles to learn directly from relational databases by representing them as heterogeneous graphs. However, existing RDL models typically rely on task-specific supervised learning, requiring training separate models for each predictive task, which may hamper scalability and reuse. In this work, we propose a novel task-agnostic contrastive pretraining approach for RDL that enables database-wide representation learning. For that aim, we introduce three levels of contrastive objectives$-$row-level, link-level, and context-level$-$designed to capture the structural and semantic heterogeneity inherent to relational data. We implement the respective pretraining approach through a modular RDL architecture and an efficient sampling strategy tailored to the heterogeneous database setting. Our preliminary results on standard RDL benchmarks demonstrate that fine-tuning the pretrained models measurably outperforms training from scratch, validating the promise of the proposed methodology in learning transferable representations for relational data.