Making Table Understanding Work in Practice
This work addresses practical deployment issues for table understanding models, which is crucial for data integration and search tasks in enterprises, but it is incremental as it builds on existing methods.
The paper tackles the gap between high benchmark performance and practical applicability of deep learning models for table understanding, proposing a framework and SigmaTyper tool to address challenges like domain customization and lack of training data, with results including a hybrid model trained on GitTables and a human-in-the-loop approach.
Understanding the semantics of tables at scale is crucial for tasks like data integration, preparation, and search. Table understanding methods aim at detecting a table's topic, semantic column types, column relations, or entities. With the rise of deep learning, powerful models have been developed for these tasks with excellent accuracy on benchmarks. However, we observe that there exists a gap between the performance of these models on these benchmarks and their applicability in practice. In this paper, we address the question: what do we need for these models to work in practice? We discuss three challenges of deploying table understanding models and propose a framework to address them. These challenges include 1) difficulty in customizing models to specific domains, 2) lack of training data for typical database tables often found in enterprises, and 3) lack of confidence in the inferences made by models. We present SigmaTyper which implements this framework for the semantic column type detection task. SigmaTyper encapsulates a hybrid model trained on GitTables and integrates a lightweight human-in-the-loop approach to customize the model. Lastly, we highlight avenues for future research that further close the gap towards making table understanding effective in practice.