Sharad Agarwal

Yuxin, Liu, Yoshimichi Nakatsuka et al.

Demonstrating the veracity of videos is a longstanding problem that has recently become more urgent and acute. It is extremely hard to accurately detect manipulated videos using content analysis, especially in the face of subtle, yet effective, manipulations, such as frame rate changes or skin tone adjustments. One prominent alternative to content analysis is to securely embed provenance information into videos. However, prior approaches have poor performance and/or granularity that is too coarse. To this end, we construct Vronicle -- a video provenance system that offers fine-grained provenance information and substantially better performance. It allows a video consumer to authenticate the camera that originated the video and the exact sequence of video filters that were subsequently applied to it. Vronicle exploits the increasing popularity and availability of Trusted Execution Environments (TEEs) on many types of computing platforms. One contribution of Vronicle is the design of provenance information that allows the consumer to verify various aspects of the video, thereby defeating numerous fake-video creation methods. Vronicle's adversarial model allows for a powerful adversary that can manipulate the video (e.g., in transit) and the software state outside the TEE. Another contribution is the use of fixed-function Intel SGX enclaves to post-process videos. This design facilitates verification of provenance information. We present a prototype implementation of Vronicle (to be open sourced), which relies on current technologies, making it readily deployable. Our evaluation demonstrates that Vronicle's performance is well-suited for offline use-cases.

Sharad Agarwal, Pooria Namyar, Alec Wolman et al.

Agentic AI systems that invoke external tools are powerful but costly, leading developers to default to large models and overspend inference budgets. Model routing can mitigate this, but existing routers are designed for chat completion rather than tool use. We present Switchcraft, the first (to the best of our knowledge) model router optimized for agentic tool calling. Switchcraft operates inline, selecting the lowest-cost model subject to correctness. We construct an evaluation framework on five function-calling benchmarks and train a DistilBERT-based classifier, deployed under a latency budget. Switchcraft achieves 82.9% accuracy -- matching or exceeding the best individual model -- while reducing inference cost by 84%, saving over $3,600 per million queries. We find that larger models do not consistently outperform smaller ones on tool-use tasks, and that nominally cheaper models can incur higher total cost due to token-intensive reasoning. Our work enables cost-aware agentic AI deployment without sacrificing correctness.

Madhav Kanda, Pedro Las-Casas, Alok Gautam Kumbhare et al.

LLM-based systems increasingly generate structured workflows for complex tasks. In practice, automatic evaluation of these workflows is difficult, because metric scores are often not calibrated, and score changes do not directly communicate the severity of workflow degradation. We introduce WorkflowPerturb, a controlled benchmark for studying workflow evaluation metrics. It works by applying realistic, controlled perturbations to golden workflows. WorkflowPerturb contains 4,973 golden workflows and 44,757 perturbed variants across three perturbation types (Missing Steps, Compressed Steps, and Description Changes), each applied at severity levels of 10%, 30%, and 50%. We benchmark multiple metric families and analyze their sensitivity and calibration using expected score trajectories and residuals. Our results characterize systematic differences across metric families and support severity-aware interpretation of workflow evaluation scores. Our dataset will be released upon acceptance.