Intent Classification on Low-Resource Languages with Query Similarity Search
This addresses the challenge of expensive data annotation and language extension for intent classification, though it is incremental as it adapts existing similarity methods.
The paper tackled intent classification for low-resource languages by reframing it as a query similarity search problem, achieving reasonable performance in a zero-shot setting.
Intent classification is an important component of a functional Information Retrieval ecosystem. Many current approaches to intent classification, typically framed as a classification problem, can be problematic as intents are often hard to define and thus data can be difficult and expensive to annotate. The problem is exacerbated when we need to extend the intent classification system to support multiple and in particular low-resource languages. To address this, we propose casting intent classification as a query similarity search problem - we use previous example queries to define an intent, and a query similarity method to classify an incoming query based on the labels of its most similar queries in latent space. With the proposed approach, we are able to achieve reasonable intent classification performance for queries in low-resource languages in a zero-shot setting.