Sato: Contextual Semantic Type Detection in Tables
This work addresses the challenge of semantic type detection for data professionals in scenarios like data cleaning and search, offering a robust solution that improves over existing methods, though it is incremental in nature.
The paper tackles the problem of detecting semantic types of columns in relational tables, which is crucial for data preparation and retrieval tasks, and introduces Sato, a hybrid machine learning model that achieves support-weighted and macro average F1 scores of 0.925 and 0.735, significantly exceeding state-of-the-art performance.
Detecting the semantic types of data columns in relational tables is important for various data preparation and information retrieval tasks such as data cleaning, schema matching, data discovery, and semantic search. However, existing detection approaches either perform poorly with dirty data, support only a limited number of semantic types, fail to incorporate the table context of columns or rely on large sample sizes for training data. We introduce Sato, a hybrid machine learning model to automatically detect the semantic types of columns in tables, exploiting the signals from the context as well as the column values. Sato combines a deep learning model trained on a large-scale table corpus with topic modeling and structured prediction to achieve support-weighted and macro average F1 scores of 0.925 and 0.735, respectively, exceeding the state-of-the-art performance by a significant margin. We extensively analyze the overall and per-type performance of Sato, discussing how individual modeling components, as well as feature categories, contribute to its performance.