ANIRA: An Architecture for Neural Network Inference in Real-Time Audio Applications
This addresses the need for efficient, real-time neural network inference in audio applications, but it is incremental as it builds on existing backends with optimizations.
The authors tackled the problem of real-time audio applications lacking suitable neural network inference tools by introducing anira, a cross-platform library that supports multiple backends and decouples inference from audio callbacks, resulting in benchmarks showing ONNX Runtime with the lowest runtimes for stateless models and LibTorch fastest for stateful models.
Numerous tools for neural network inference are currently available, yet many do not meet the requirements of real-time audio applications. In response, we introduce anira, an efficient cross-platform library. To ensure compatibility with a broad range of neural network architectures and frameworks, anira supports ONNX Runtime, LibTorch, and TensorFlow Lite as backends. Each inference engine exhibits real-time violations, which anira mitigates by decoupling the inference from the audio callback to a static thread pool. The library incorporates built-in latency management and extensive benchmarking capabilities, both crucial to ensure a continuous signal flow. Three different neural network architectures for audio effect emulation are then subjected to benchmarking across various configurations. Statistical modeling is employed to identify the influence of various factors on performance. The findings indicate that for stateless models, ONNX Runtime exhibits the lowest runtimes. For stateful models, LibTorch demonstrates the fastest performance. Our results also indicate that for certain model-engine combinations, the initial inferences take longer, particularly when these inferences exhibit a higher incidence of real-time violations.