Yazhou Tu, Liqun Shan, and Md Imran Hossen, University of Louisiana at Lafayette; Sara Rampazzi and Kevin Butler, University of Florida; Xiali Hei, University of Louisiana at Lafayette
In various scenarios from system login to writing emails, documents, and forms, keyboard inputs carry alluring data such as passwords, addresses, and IDs. Due to commonly existing non-alphabetic inputs, punctuation, and typos, users' natural inputs rarely contain only constrained, purely alphabetic keys/words. This work studies how to reveal unconstrained keyboard inputs using auditory interfaces.
Audio interfaces are not intended to have the capability of light sensors such as cameras to identify compactly located keys. Our analysis shows that effectively distinguishing the keys can require a fine localization precision level of keystroke sounds close to the range of microseconds. This work (1) explores the limits of audio interfaces to distinguish keystrokes, (2) proposes a μs-level customized signal processing and analysis-based keystroke tracking approach that takes into account the mechanical physics and imperfect measuring of keystroke sounds, (3) develops the first acoustic side-channel attack study on unconstrained keyboard inputs that are not purely alphabetic keys/words and do not necessarily follow known sequences in a given dictionary or training dataset, and (4) reveals the threats of non-line-of-sight keystroke sound tracking. Our results indicate that, without relying on vision sensors, attacks using limited-resolution audio interfaces can reveal unconstrained inputs from the keyboard with a fairly sharp and bendable "auditory eyesight."
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
![](https://www.usenix.org/sites/all/modules/usenix/usenix_files/images/usenix-locked.png)
author = {Yazhou Tu and Liqun Shan and Md Imran Hossen and Sara Rampazzi and Kevin Butler and Xiali Hei},
title = {Auditory Eyesight: Demystifying {μs-Precision} Keystroke Tracking Attacks on Unconstrained Keyboard Inputs},
booktitle = {32nd USENIX Security Symposium (USENIX Security 23)},
year = {2023},
isbn = {978-1-939133-37-3},
address = {Anaheim, CA},
pages = {175--192},
url = {https://www.usenix.org/conference/usenixsecurity23/presentation/tu},
publisher = {USENIX Association},
month = aug
}
![](https://www.usenix.org/sites/default/files/usenix_artifact_evaluation_available_125_update.png)
![](https://www.usenix.org/sites/default/files/usenix_artifact_evaluation_functional_125.png)
![](https://www.usenix.org/sites/default/files/usenix_artifact_evaluation_reproduced_125.png)