Orpheus: A New Deep Learning Framework for Easy Deployment and Evaluation of Edge Inference
This addresses the problem of compatibility and complexity in existing frameworks for machine learning engineers and systems researchers working on edge inference, though it appears incremental as it builds on existing concepts.
The paper tackles the challenge of optimizing deep learning inference on edge devices by introducing Orpheus, a new framework designed for easy prototyping, deployment, and evaluation, with features like a small codebase and minimal dependencies.
Optimising deep learning inference across edge devices and optimisation targets such as inference time, memory footprint and power consumption is a key challenge due to the ubiquity of neural networks. Today, production deep learning frameworks provide useful abstractions to aid machine learning engineers and systems researchers. However, in exchange they can suffer from compatibility challenges (especially on constrained platforms), inaccessible code complexity, or design choices that otherwise limit research from a systems perspective. This paper presents Orpheus, a new deep learning framework for easy prototyping, deployment and evaluation of inference optimisations. Orpheus features a small codebase, minimal dependencies, and a simple process for integrating other third party systems. We present some preliminary evaluation results.