Time Will Tell: Timing Side Channels via Output Token Count in Large Language Models
This work addresses a security vulnerability for users of LLMs, revealing a novel side-channel attack that can leak sensitive inference inputs, though it is incremental in the context of side-channel research.
This paper tackles the problem of extracting sensitive information from large language models (LLMs) by exploiting a side-channel based on the number of output tokens, demonstrating attacks that recover target languages in translation tasks with over 75% precision and output classes in classification tasks with over 70% precision across various models.
This paper demonstrates a new side-channel that enables an adversary to extract sensitive information about inference inputs in large language models (LLMs) based on the number of output tokens in the LLM response. We construct attacks using this side-channel in two common LLM tasks: recovering the target language in machine translation tasks and recovering the output class in classification tasks. In addition, due to the auto-regressive generation mechanism in LLMs, an adversary can recover the output token count reliably using a timing channel, even over the network against a popular closed-source commercial LLM. Our experiments show that an adversary can learn the output language in translation tasks with more than 75% precision across three different models (Tower, M2M100, MBart50). Using this side-channel, we also show the input class in text classification tasks can be leaked out with more than 70% precision from open-source LLMs like Llama-3.1, Llama-3.2, Gemma2, and production models like GPT-4o. Finally, we propose tokenizer-, system-, and prompt-based mitigations against the output token count side-channel.