Towards Foundation Models for Relational Databases [Vision Paper]
This vision paper proposes a foundational approach for relational databases, which could impact data analysis and AI applications, but it is incremental as it builds on existing tabular representation learning.
The paper tackles the problem of learning representations from relational databases, which current methods fail to do by ignoring multi-table structures and scaling limitations, and presents a vision for scalable relational representation learning with initial promising results.
Tabular representation learning has recently gained a lot of attention. However, existing approaches only learn a representation from a single table, and thus ignore the potential to learn from the full structure of relational databases, including neighboring tables that can contain important information for a contextualized representation. Moreover, current models are significantly limited in scale, which prevents that they learn from large databases. In this paper, we thus introduce our vision of relational representation learning, that can not only learn from the full relational structure, but also can scale to larger database sizes that are commonly found in real-world. Moreover, we also discuss opportunities and challenges we see along the way to enable this vision and present initial very promising results. Overall, we argue that this direction can lead to foundation models for relational databases that are today only available for text and images.