The Two Sides of the Coin: Hallucination Generation and Detection with LLMs as Evaluators for LLMs
This work addresses hallucination detection for LLM reliability, but it is incremental as it applies existing methods to a new shared task without introducing novel techniques.
The paper tackled the problem of hallucination detection in Large Language Models by evaluating four LLMs (Llama 3, Gemma, GPT-3.5 Turbo, GPT-4) for generating and detecting hallucinated content, using ensemble majority voting for detection, and provided insights into their strengths and weaknesses without reporting specific performance numbers.
Hallucination detection in Large Language Models (LLMs) is crucial for ensuring their reliability. This work presents our participation in the CLEF ELOQUENT HalluciGen shared task, where the goal is to develop evaluators for both generating and detecting hallucinated content. We explored the capabilities of four LLMs: Llama 3, Gemma, GPT-3.5 Turbo, and GPT-4, for this purpose. We also employed ensemble majority voting to incorporate all four models for the detection task. The results provide valuable insights into the strengths and weaknesses of these LLMs in handling hallucination generation and detection tasks.