Universal Encoders for Modular Relational Deep Learning
For practitioners using relational databases in deep learning, this work addresses the generalization bottleneck of schema-specific models, enabling pretrained components for faster adaptation to new databases.
Relational Deep Learning (RDL) models multi-tabular databases as graphs but suffers from schema-specific retraining and monolithic architectures. The authors propose a modular Universal Row Encoder that decouples row encoding from graph message-passing, achieving cross-database knowledge transfer and improved convergence on RelBench benchmarks.
Relational Deep Learning (RDL) models multi-tabular databases as temporal heterogeneous graphs for end-to-end representation learning. While RDL is evolving rapidly, existing approaches face significant generalization obstacles. They are either schema-specific, requiring training from scratch for every new database, or they rely on monolithic architectures that entangle feature encoding with graph message-passing. Analyzing these limitations, we establish four core pillars for building foundational relational models: semantic granularity, structural topology, temporal causality, and unified optimization. Addressing these pillars, we propose a modular approach that decouples row encoding from graph message-passing. We introduce the Universal Row Encoder, a transformer-based module that integrates raw cell data with schema metadata$-$including column semantics, table names, and global distribution statistics$-$to produce table-width invariant row embeddings. By explicitly feeding global statistics to an intra-row self-attention mechanism, the encoder natively contextualizes unseen features and handles sparse data. Serving as a flexible "backend" for any downstream graph architecture, our pretrained encoder enhances cross-database knowledge transfer on the established RelBench benchmarks while improving learning convergence and memory footprint.