RoTaR: Efficient Row-Based Table Representation Learning via Teacher-Student Training
This work addresses efficiency problems for users of table representation learning, but it appears incremental as it builds on existing methods with specific architectural and training enhancements.
The paper tackles efficiency and scalability issues in table representation learning by proposing RoTaR, a row-based method that generates reusable row representations with query-specific aggregation, achieving improved performance through techniques like cell-aware position embedding and teacher-student training.
We propose RoTaR, a row-based table representation learning method, to address the efficiency and scalability issues faced by existing table representation learning methods. The key idea of RoTaR is to generate query-agnostic row representations that could be re-used via query-specific aggregation. In addition to the row-based architecture, we introduce several techniques: cell-aware position embedding, teacher-student training paradigm, and selective backward to improve the performance of RoTaR model.