Human interaction classifier for LLM based chatbot
This work addresses improving service efficiency for a specific company's chatbot, but it is incremental as it compares existing methods on a new dataset.
This study tackled the problem of classifying human interactions for a chatbot to direct requests appropriately, finding that SVM and ANN models with Cohere embeddings achieved the best performance with superior F1 scores and faster execution times compared to LLM-based approaches.
This study investigates different approaches to classify human interactions in an artificial intelligence-based environment, specifically for Applus+ IDIADA's intelligent agent AIDA. The main objective is to develop a classifier that accurately identifies the type of interaction received (Conversation, Services, or Document Translation) to direct requests to the appropriate channel and provide a more specialized and efficient service. Various models are compared, including LLM-based classifiers, KNN using Titan and Cohere embeddings, SVM, and artificial neural networks. Results show that SVM and ANN models with Cohere embeddings achieve the best overall performance, with superior F1 scores and faster execution times compared to LLM-based approaches. The study concludes that the SVM model with Cohere embeddings is the most suitable option for classifying human interactions in the AIDA environment, offering an optimal balance between accuracy and computational efficiency.