Development of REGAI: Rubric Enabled Generative Artificial Intelligence
This addresses the need for more effective AI evaluation tools, though it appears incremental as it builds on existing RAG and LLM techniques.
The paper tackles the problem of improving large language model performance for evaluation tasks by introducing REGAI, a rubric-enabled generative AI technique that enhances both classical LLMs and RAG-based methods, with data showing performance improvements.
This paper presents and evaluates a new retrieval augmented generation (RAG) and large language model (LLM)-based artificial intelligence (AI) technique: rubric enabled generative artificial intelligence (REGAI). REGAI uses rubrics, which can be created manually or automatically by the system, to enhance the performance of LLMs for evaluation purposes. REGAI improves on the performance of both classical LLMs and RAG-based LLM techniques. This paper describes REGAI, presents data regarding its performance and discusses several possible application areas for the technology.