Designing Robust API Monitoring Solutions
This work addresses the problem of robust API monitoring for software security researchers, offering incremental improvements through novel design insights and implementations.
The paper tackles the challenge of accurately and reliably tracing API calls in complex software stacks by identifying six key challenges and proposing design solutions. It presents two implementation variants, including the first general-purpose user-space tracer using hardware-assisted virtualization, and releases the SNIPER system as open source.
Tracing the sequence of library and system calls that a program makes is very helpful in the characterization of its interactions with the surrounding environment and ultimately of its semantics. Due to entanglements of real-world software stacks, accomplishing this task can be surprisingly challenging as we take accuracy, reliability, and transparency into the equation. To manage these dimensions effectively, we identify six challenges that API monitoring solutions should overcome and outline actionable design points for them, reporting insights from our experience in building API tracers for software security research. We detail two implementation variants, based on hardware-assisted virtualization (realizing the first general-purpose user-space tracer of this kind) and on dynamic binary translation, that achieve API monitoring robustly. We share our SNIPER system as open source.