Behrooz Farkiani

Behrooz Farkiani, Fan Liu, Ke Yang et al.

We introduce Hermes, a general-purpose networking architecture that aims to improve service delivery over the Internet. Hermes delegates networking responsibilities from applications and services to proxies and is designed as a portable, adaptable solution to four fundamental challenges of efficient service delivery over the Internet: end-to-end traffic management, backward compatibility, data-plane security and privacy models, and adaptable communication layers. The design centers on an overlay of reconfigurable proxies and HTTP tunneling and proxying techniques, utilizing assisting components to extend proxy functionality when needed. Through prototyping and emulation, we demonstrate that Hermes improves key performance metrics across multiple use cases: it provides backward compatibility through protocol translation and tunneling, improves reliability by delegating retry logic to proxies, enables unified policy-based Layer 3 routing across network segments, and serves as an efficient substrate for future architectures like NDN, facilitating their operation over the Internet. Beyond evaluating Hermes across various use cases, we measured the overhead of Hermes' HTTP tunneling and proxying mechanisms and found it to be modest, typically under 2 ms per proxy pair traversal in an isolated collocated setup. Although the HTTP proxying and tunneling techniques used by Hermes increase single-connection processing overhead, we also show that, with up to 1,000 concurrent requests, proxies can amortize connection setup time and reduce end-to-end latency by utilizing connection pooling and multiplexing.

Fan Liu, Guanqi Li, Behrooz Farkiani et al.

Cloud services must continuously adapt replica counts to fluctuating demand while respecting fixed-period reliability budgets. Many horizontal autoscalers either react to instantaneous utilization or provision against a fixed predictive risk target. These policies do not explicitly account for how much of the period-level violation budget has already been consumed, so they can be overly conservative when the budget is healthy and insufficiently conservative when the budget is being depleted. We present BACC, a model-agnostic framework for budget-aware horizontal autoscaling. BACC separates three concerns that are often entangled in prior systems: workload prediction, online uncertainty calibration, and budget-paced capacity control. It wraps an arbitrary forecaster with Adaptive Conformal Inference (ACI) to calibrate workload uncertainty online, then uses a proportional--integral controller to adjust provisioning aggressiveness based on the observed pace of budget consumption. We instantiate BACC for CPU-threshold-based horizontal autoscaling in Kubernetes and evaluate it through trace-driven simulation and cluster replay experiments. Across five Azure Functions traces, three compliance levels, and two forecasting backends, BACC tracks the requested violation target closely, achieving mean absolute compliance gaps of 0.44 and 0.42 percentage points with ARIMA and Chronos, respectively. The Kubernetes experiments further show that the same controller improves CPU-threshold compliance over native HPA under deployment effects such as measurement delay and replica readiness.