Deriving Semantics-Aware Fuzzers from Web API Schemas
This addresses the problem of security vulnerabilities in web APIs for developers and security testers, offering a novel method for an existing bottleneck.
The paper tackled the challenge of fuzzing stateful and structured web APIs by developing Schemathesis, a tool that derives semantics-aware fuzzers from API schemas, and found it handled over two-thirds of services without errors and detected 1.4 to 4.5 times more unique defects than the second-best fuzzer.
Fuzzing -- whether generating or mutating inputs -- has found many bugs and security vulnerabilities in a wide range of domains. Stateful and highly structured web APIs present significant challenges to traditional fuzzing techniques, as execution feedback is usually limited to a response code instead of code coverage and vulnerabilities of interest include silent information-disclosure in addition to explicit errors. Our tool, Schemathesis, derives structure- and semantics-aware fuzzers from web API schemas in the OpenAPI or GraphQL formats, using property-based testing tools. Derived fuzzers can be incorporated into unit-test suites or run directly, with or without end-user customisation of data generation and semantic checks. We construct the most comprehensive evaluation of web API fuzzers to date, running eight fuzzers against sixteen real-world open source web services. OpenAPI schemas found in the wild have a long tail of rare features and complex structures. Of the tools we evaluated, Schemathesis was the only one to handle more than two-thirds of our target services without a fatal internal error. Schemathesis finds 1.4 times to 4.5 times more unique defects than the respectively second-best fuzzer for each target, and is the only fuzzer to find defects in four targets.