Alperen Keles

Alperen Keles, Jessica Shi, Nikhil Kamath et al.

Property-based testing is a mainstay of functional programming, boasting a rich literature, an enthusiastic user community, and an abundance of tools~ -- so many, indeed, that new users may have difficulty choosing. Moreover, any given framework may support a variety of strategies for generating test inputs; even experienced users may wonder which are better in any given situation. Sadly, the PBT literature, though long on creativity, is short on rigorous comparisons to help answer such questions. We present ETNA, a platform for empirical evaluation and comparison of PBT techniques. ETNA incorporates a number of popular PBT frameworks and testing workloads from the literature, and its extensible architecture makes adding new ones easy, while handling the technical drudgery of performance measurement. To illustrate its benefits, we use ETNA to carry out several experiments with popular PBT approaches in Rocq, Haskell, OCaml, Racket, and Rust, allowing users to more clearly understand best practices and tradeoffs.

Alperen Keles, Ethan Chou, Harrison Goldstein et al.

Database management systems (DBMSs) are notoriously complex, making them difficult to test effectively, especially during early development when many features are incomplete. Traditional testing tools like SQLancer and SQLSmith are highly effective for mature databases, but they struggle with high false positive rates and low actionability when applied to evolving systems. We present DIRT, a paradigm designed specifically for testing databases during development, which integrates a testing framework directly into the DBMS, enabling the random testing process to evolve in tandem with the system and reducing false positives by construction. We introduce generation actions, an abstraction for allowing database developers rather than testing experts to specify correctness properties. We evaluate DIRT on Turso, an actively developed SQLite-compatible OLTP engine, and show that it finds 23 unique, confirmed bugs--significantly outperforming off-the-shelf SQLancer variants in terms of true positive rate and usefulness of bug reports. Our results demonstrate that embedding testing infrastructure within the DBMS can dramatically improve its effectiveness and usability during development.

Alperen Keles

Hand-crafted mutants are increasingly used to evaluate fuzzing and property-based testing tools, but current tooling is fragmented and often forces trade-offs between readability, mutation preservation, and execution cost. We present a declarative framework for hand-crafted mutation analysis and management. First, we characterize five mutation representations: comment-based, preprocessor-based, patch-based, match-and-replace, and in-AST mutations. Second, we define a mutation algebra that supports selective execution, tag-based expansion, and higher-order combinations of mutants. Third, we describe a lossless conversion pipeline that maps mutation representations through a common intermediate form, including a strategy for extracting and normalizing in-AST mutations. We implement these ideas in Marauder, a prototype system for injecting, activating, resetting, and composing hand-crafted mutants across representations. This framework clarifies the design space of hand-crafted mutation systems and provides a practical foundation for more efficient and expressive mutation experiments.