More Than A Shortcut: A Hyperbolic Approach To Early-Exit Networks
This addresses the trade-off between accuracy and computational cost for deploying event detection on devices like smartphones or IoT sensors, offering an incremental improvement over existing Early-Exit methods.
The paper tackles the problem of unreliable early predictions in Early-Exit networks for event detection on resource-constrained devices by proposing Hyperbolic Early-Exit networks (HypEE), which uses hyperbolic space and a novel entailment loss to enforce hierarchical structure, resulting in significant performance gains over baselines, especially at early exits.
Deploying accurate event detection on resource-constrained devices is challenged by the trade-off between performance and computational cost. While Early-Exit (EE) networks offer a solution through adaptive computation, they often fail to enforce a coherent hierarchical structure, limiting the reliability of their early predictions. To address this, we propose Hyperbolic Early-Exit networks (HypEE), a novel framework that learns EE representations in the hyperbolic space. Our core contribution is a hierarchical training objective with a novel entailment loss, which enforces a partial-ordering constraint to ensure that deeper network layers geometrically refine the representations of shallower ones. Experiments on multiple audio event detection tasks and backbone architectures show that HypEE significantly outperforms standard Euclidean EE baselines, especially at the earliest, most computationally-critical exits. The learned geometry also provides a principled measure of uncertainty, enabling a novel triggering mechanism that makes the overall system both more efficient and more accurate than a conventional EE and standard backbone models without early-exits.