Accelerating Deep Learning Classification with Error-controlled Approximate-key Caching
This work addresses the problem of real-time traffic measurement inefficiencies in networking for practitioners by offering an incremental improvement over existing caching methods.
The paper tackles the high computational cost of deep learning inference for real-time networking classification by proposing approximate-key caching with an auto-refresh error-correction algorithm, which reduces inference workload and increases throughput while controlling approximation errors, as validated through trace-driven evaluation against state-of-the-art similarity caching.
While Deep Learning (DL) technologies are a promising tool to solve networking problems that map to classification tasks, their computational complexity is still too high with respect to real-time traffic measurements requirements. To reduce the DL inference cost, we propose a novel caching paradigm, that we named approximate-key caching, which returns approximate results for lookups of selected input based on cached DL inference results. While approximate cache hits alleviate DL inference workload and increase the system throughput, they however introduce an approximation error. As such, we couple approximate-key caching with an error-correction principled algorithm, that we named auto-refresh. We analytically model our caching system performance for classic LRU and ideal caches, we perform a trace-driven evaluation of the expected performance, and we compare the benefits of our proposed approach with the state-of-the-art similarity caching -- testifying the practical interest of our proposal.