Privacy Leakage via Unrestricted Motion-Position Sensors in the Age of Virtual Reality: A Study of Snooping Typed Input on Virtual Keyboards
Document Type
Conference Proceeding
Publication Date
1-1-2023
Abstract
Virtual Reality (VR) has gained popularity in numerous fields, including gaming, social interactions, shopping, and education. In this paper, we conduct a comprehensive study to assess the trustworthiness of the embedded sensors on VR, which embed various forms of sensitive data that may put users' privacy at risk. We find that accessing most on-board sensors (e.g., motion, position, and button sensors) on VR SDKs/APIs, such as OpenVR, Oculus Platform, and WebXR, requires no security permission, exposing a huge attack surface for an adversary to steal the user's privacy. We validate this vulnerability through developing malware programs and malicious websites and specifically explore to what extent it exposes the user's information in the context of keystroke snooping. To examine its actual threat in practice, the adversary in the considered attack model doesn't possess any labeled data from the user nor knowledge about the user's VR settings. Extensive experiments, involving two mainstream VR systems and four keyboards with different typing mechanisms, demonstrate that our proof-of-concept attack can recognize the user's virtual typing with over 89.7% accuracy. The attack can recover the user's passwords with up to 84.9% recognition accuracy if three attempts are allowed and achieve an average of 87.1% word recognition rate for paragraph inference. We hope this study will help the community gain awareness of the vulnerability in the sensor management of current VR systems and provide insights to facilitate the future design of more comprehensive and restricted sensor access control mechanisms.
Identifier
85166483496 (Scopus)
ISBN
[9781665493369]
Publication Title
Proceedings IEEE Symposium on Security and Privacy
External Full Text Location
https://doi.org/10.1109/SP46215.2023.10179301
ISSN
10816011
First Page
3382
Last Page
3398
Volume
2023-May
Grant
CCF2211163
Fund Ref
National Science Foundation
Recommended Citation
Wu, Yi; Shi, Cong; Zhang, Tianfang; Walker, Payton; Liu, Jian; Saxena, Nitesh; and Chen, Yingying, "Privacy Leakage via Unrestricted Motion-Position Sensors in the Age of Virtual Reality: A Study of Snooping Typed Input on Virtual Keyboards" (2023). Faculty Publications. 2257.
https://digitalcommons.njit.edu/fac_pubs/2257
