PyTorch Frame: A Modular Framework for Multi-Modal Tabular Learning
This provides a practical tool for researchers and practitioners working with complex tabular data, though it is an incremental framework development.
The authors tackled the challenge of deep learning with multi-modal tabular data by developing PyTorch Frame, a modular PyTorch-based framework that simplifies model implementation and integrates external foundation models like LLMs, successfully applying it to complex datasets and combining it with PyTorch Geometric for relational database learning.
We present PyTorch Frame, a PyTorch-based framework for deep learning over multi-modal tabular data. PyTorch Frame makes tabular deep learning easy by providing a PyTorch-based data structure to handle complex tabular data, introducing a model abstraction to enable modular implementation of tabular models, and allowing external foundation models to be incorporated to handle complex columns (e.g., LLMs for text columns). We demonstrate the usefulness of PyTorch Frame by implementing diverse tabular models in a modular way, successfully applying these models to complex multi-modal tabular data, and integrating our framework with PyTorch Geometric, a PyTorch library for Graph Neural Networks (GNNs), to perform end-to-end learning over relational databases.