KEYSTROKE INFERENCE ATTACKS AND DEFENSES IN EXTENDED REALITY PLATFORMS

Abstract

Virtual Reality (VR), Augmented Reality (AR), and the broader spectrum of Extended Real-ity (XR) technologies have experienced rapid consumer adoption, driven by advances in head- mounted displays, hand tracking, eye tracking, and spatial computing. However, this growth introduces an unprecedented attack surface encompassing novel sensor data streams, immersive interaction paradigms, and deeply personal biometric and behavioral telemetry. This thesis makes two principal contributions. First, it systematically reviews and synthe- sizes over 160 publications spanning 2012–2025, covering the full landscape of XR security and privacy. The literature is organized into seven thematic pillars: (1) threat modeling and risk assess- ment frameworks, (2) biometric and behavioral identification from XR sensor data, (3) authenti- cation mechanisms designed for immersive environments, (4) side-channel and inference attacks, (5) social and psychological threats including harassment and manipulation, (6) privacy-preserving countermeasures and defensive architectures, and (7) regulatory and ethical considerations. Second, this thesis presents original research demonstrating a novel infrared (IR) side-channel attack that enables non-invasive recovery of Personal Identification Numbers (PINs) entered in VR environments. By passively observing the IR emissions produced by standard VR controller hard- ware using a low-cost camera sensor lacking an IR-cut filter, the proposed approach tracks the spa- tiotemporal movement of IR emitter spots to reconstruct controller trajectories, identifies keypress events through motion dynamics analysis, and decodes the entered PIN by geometrically aligning the observed trajectory against candidate PIN templates. The system operates effectively even in complete darkness and requires no prior user-specific training. Extensive real-world experiments across multiple commercial VR platforms and diverse environmental conditions demonstrate the practicality and severity of this vulnerability. Together, the survey and the IR side-channel investigation reveal that XR devices collect richer personal data than any prior consumer technology, that motion and gaze data alone are sufficient to uniquely identify and profile users at scale, that novel optical side channels expose authentication credentials through previously unexplored vectors, and that existing defense mechanisms remain fragmented and largely inadequate. A unified research agenda is proposed aimed at achieving security and privacy by design in next-generation immersive systems.