Roy Weiss, Daniel Ayzenshteyn, Guy Amit, and Yisroel Mirsky, Ben Gurion University of the Negev
AI assistants are becoming an integral part of society, used for asking advice or help in personal and confidential issues. In this paper, we unveil a novel side-channel that can be used to read encrypted responses from AI Assistants over the web: the token-length side-channel. The side-channel reveals the character-lengths of a response's tokens (akin to word lengths). We found that many vendors, including OpenAI and Microsoft, had this side-channel prior to our disclosure.
However, inferring a response's content with this side-channel is challenging. This is because, even with knowledge of token-lengths, a response can have hundreds of words resulting in millions of grammatically correct sentences. In this paper, we show how this can be overcome by (1) utilizing the power of a large language model (LLM) to translate these token-length sequences, (2) providing the LLM with inter-sentence context to narrow the search space and (3) performing a known-plaintext attack by fine-tuning the model on the target model's writing style.
Using these methods, we were able to accurately reconstruct 27% of an AI assistant's responses and successfully infer the topic from 53% of them. To demonstrate the threat, we performed the attack on OpenAI's ChatGPT-4 and Microsoft's Copilot on both browser and API traffic.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
![](https://www.usenix.org/sites/all/modules/usenix/usenix_files/images/usenix-locked.png)
author = {Roy Weiss and Daniel Ayzenshteyn and Yisroel Mirsky},
title = {What Was Your Prompt? A Remote Keylogging Attack on {AI} Assistants},
booktitle = {33rd USENIX Security Symposium (USENIX Security 24)},
year = {2024},
isbn = {978-1-939133-44-1},
address = {Philadelphia, PA},
pages = {3367--3384},
url = {https://www.usenix.org/conference/usenixsecurity24/presentation/weiss},
publisher = {USENIX Association},
month = aug
}