Automatic Property-based Testing of GraphQL APIs
This work addresses the lack of automated test generation methods for GraphQL APIs, which is a problem for developers needing to ensure the quality of their GraphQL services.
This paper introduces a property-based method for automatically generating GraphQL queries to test GraphQL APIs. The method was evaluated on a real-world system, demonstrating its effectiveness in finding real bugs and its efficiency, covering a complete schema in seconds. It also found 73% of seeded faults.
In recent years, GraphQL has become a popular way to expose web APIs. With its raise of adoption in industry, the quality of GraphQL APIs must be also assessed, as with any part of a software system, and preferably in an automated manner. However, there is currently a lack of methods to automatically generate tests to exercise GraphQL APIs. In this paper, we propose a method for automatically producing GraphQL queries to test GraphQL APIs. This is achieved using a property-based approach to create a generator for queries based on the GraphQL schema of the system under test. Our evaluation on a real world software system shows that this approach is both effective, in terms of finding real bugs, and efficient, as a complete schema can be covered in seconds. In addition, we evaluate the fault finding capability of the method when seeding known faults. 73% of the seeded faults where found, with room for improvements with regards to domain specific behavior, a common oracle challenge in automatic test generation.