Tabular Learning: Encoding for Entity and Context Embeddings
This work addresses a preprocessing bottleneck in tabular data analysis for machine learning practitioners, offering an incremental improvement over existing encoding methods.
The study challenged the common use of Ordinal encoding for categorical data in tabular learning, finding it suboptimal for preprocessing and classification, and demonstrated that encoding based on string similarities improved performance for both entity and context embeddings, particularly with transformer architectures in multi-label tasks.
Examining the effect of different encoding techniques on entity and context embeddings, the goal of this work is to challenge commonly used Ordinal encoding for tabular learning. Applying different preprocessing methods and network architectures over several datasets resulted in a benchmark on how the encoders influence the learning outcome of the networks. By keeping the test, validation and training data consistent, results have shown that ordinal encoding is not the most suited encoder for categorical data in terms of preprocessing the data and thereafter, classifying the target variable correctly. A better outcome was achieved, encoding the features based on string similarities by computing a similarity matrix as input for the network. This is the case for both, entity and context embeddings, where the transformer architecture showed improved performance for Ordinal and Similarity encoding with regard to multi-label classification tasks.