Transformers Meet Relational Databases
This work addresses the challenge of enabling end-to-end learning from database storage systems for AI practitioners, though it appears incremental as it builds on existing Transformer and relational model concepts.
The paper tackles the problem of applying Transformer models directly to relational databases by introducing a modular neural message-passing scheme that adheres to the relational model, resulting in superior performance compared to representative models across a wide range of datasets.
Transformer models have continuously expanded into all machine learning domains convertible to the underlying sequence-to-sequence representation, including tabular data. However, while ubiquitous, this representation restricts their extension to the more general case of relational databases. In this paper, we introduce a modular neural message-passing scheme that closely adheres to the formal relational model, enabling direct end-to-end learning of tabular Transformers from database storage systems. We address the challenges of appropriate learning data representation and loading, which are critical in the database setting, and compare our approach against a number of representative models from various related fields across a significantly wide range of datasets. Our results demonstrate a superior performance of this newly proposed class of neural architectures.