Apple has released a patch for its Vision Pro mixed reality headset after researchers showed how an attacker could obtain data typed by a user by tracking their eyes.
One of the ways Vision Pro users can type is by using a virtual keyboard and looking at each of the keys they want to press.
Researchers from the University of Florida and Texas Tech University have demonstrated an attack method, dubbed GAZEploit, that can be used to infer what a Vision Pro user is typing by tracking the eye movement of their avatar.
An avatar, called by Apple a Persona, is a natural representation of the user’s face and hand movements within the Vision Pro environment. This is how others see the user during video calls, meetings and live streams.
The researchers found that an analysis of the avatar’s eye movements while the user is typing with their gaze can be used to reconstruct the keys they press on the Vision Pro virtual keyboard.
The GAZEploit attack was tested on data collected from 30 individuals and the researchers achieved significant accuracy for when users typed messages, passwords, URLs, emails, and passcodes (PINs).
“During gaze typing, users’ gazes shift between keys and fixate on the key to be clicked, resulting in saccades followed by fixations. Saccades refers to the period when users move their gaze rapidly from one object to another. Fixations refers to the period when users stare at an object,” the researchers explained.
“We developed an algorithm that calculates the stability of the gaze trace and sets a threshold to classify fixations from saccades. We use the gaze estimation points in these high stability regions as click candidates. Evaluation on our dataset shows precision and recall rate of 85.9% and 96.8% on identifying keystrokes within typing sessions,” they added.
Apple said the vulnerability, which it tracks as CVE-2024-40865, has been patched with the release of visionOS 1.3. The security advisory for visionOS 1.3 was published in late July, but it was updated by Apple on September 5 to include CVE-2024-40865.
Apple has addressed the issue by suspending Persona when the virtual keyboard is active.
This is not the first Vision Pro hack. A researcher showed recently how an attacker could have generated arbitrary objects in a room — specifically bats and spiders — simply by getting the user to visit a website.
Related: Apple Patches Vision Pro Vulnerability Used in Possibly ‘First Ever Spatial Computing Hack’
Related: Apple Patches Vision Pro Vulnerability as CISA Warns of iOS Flaw Exploitation
Related: Meta’s Virtual Reality Headset Vulnerable to Ransomware Attacks
