Investigating Metamorphic Fuzz Oracle Enhancement via Large Language Models
For software security testing, this work automates the enhancement of fuzz oracles using LLMs, significantly improving bug detection capability beyond traditional crash-based oracles.
This paper introduces MetaFOE, an LLM-based framework that automatically generates and integrates metamorphic-based oracles into existing fuzz drivers. Evaluation on OSS-Fuzz drivers shows that MetaFOE generates 3,475 metamorphic relations (77.3% applicable), implements 12,351 meta drivers (6,228 valid), and after three hours of fuzzing, improves edge coverage by 18.7% and triggers 1,528 unique crashes.
Fuzz drivers are essential components of greybox fuzzing, as they encapsulate target interfaces, define test spaces, and largely determine fuzzing effectiveness. Existing fuzz drivers typically rely on crash-based oracles for security testing, overlooking library functionality and limiting bug detection capability. In this paper, we present the first study on metamorphic-based fuzz oracle enhancement (MFOE), which augments existing fuzz drivers with metamorphic-based oracles derived from metamorphic relations (MRs). Since constructing and integrating such oracles requires substantial domain knowledge, automating MFOE is challenging. To address this challenge, we propose MetaFOE, an LLM-based framework that automatically generates and integrates metamorphic-based oracles. We evaluate MetaFOE on OSS-Fuzz drivers using three modern LLMs and five prompt strategies. MetaFOE generates 3,475 MRs, of which 77.3% are applicable, and implements 12,351 meta drivers, with 6,228 being valid. After three hours of fuzzing, the valid meta drivers improve edge coverage by an average of 18.7% and trigger 1,528 unique crashes. Our results demonstrate both the effectiveness of metamorphic-based oracle enhancement and the feasibility of using LLMs to automate MFOE, providing valuable insights for advancing greybox fuzzing.