Compile | Vendii
Edit | Desert Shadow
Zhidong reported on September 13 that according to a report by Wired magazine yesterday, 6 computer scientists found a new attack method against Apple's MR headset Vision Pro and named it GAZEploit. The attack method leverages the device's eye-tracking technology to hack what the user enters on the virtual keyboard, including passwords, PINs, and other sensitive information.
Eye tracking is one of the ways the Vision Pro interacts with users. When using Vision Pro, your eyes are your mouse. As you enter text, you'll see a removable, resizable virtual keyboard. When you look at a letter, tap with two fingers to complete the typing.
However, it is this advanced eye tracking technology that can also be a source of security breaches. By analyzing the user's eye tracking data, the attacker can crack the content entered by the user on the virtual keyboard and successfully reproduce the sensitive information such as passwords and PIN codes entered by people through eye movement.
Researchers at GAZEploit notified Apple of the vulnerability in April, and Apple released a patch at the end of July that fixed the potential risk of a data breach.
GAZEploit paper link: https://sites.google.com/view/Gazeploit/
1. Just one video call is all it takes to steal your "eye secret"
When using Apple Vision Pro, users are likely to use Persona for live streaming or video conferences.
Persona is a digital avatar created by Apple that enables users to appear as avatars during video calls. This feature leverages multiple cameras and sensors in the headset to create a digital twin that looks and acts like a user by capturing data from scans of the user's face and 3D measurements.
In a video call, the user's Persona, including head, shoulders, and hands, are displayed in floating boxes for a more natural communication experience.
▲MacRumors, an information platform focused on Apple, reviews Persona (source: YouTube)
"These technologies ...... A user's facial biometric data, including eye tracking data, may be inadvertently exposed during a video call. The digital avatar will reflect the user's eye movements. The researchers wrote in a preprint paper detailing their findings.
According to the researchers, during the course of the GAZEploit attack, they did not gain access to the Vision Pro, so they were unable to see the user's view.
GAZEploit攻击只需依赖于2种可以从Persona的录像中提取的生物特征:眼睛纵横比(Eye Aspect Ratio)和视线估计(Gauze Estimation)。
"Knowing where users are looking is a very powerful ability." Alexandra Papoutsaki, associate professor of computer science at Pommona College, said. She has been researching eye tracking technology for a long time and has reviewed GAZEploit's research for Wired.
According to Papoutsaki, what stands out about this study is that it relies solely on the user's Persona video stream. In contrast, it is much more difficult for hackers to gain access to the Vision Pro and try to exploit the eye tracking data.
"Now, users have the potential to expose their actions just by streaming their Persona." She said.
Second, the success rate of the attack is 73%-92%, and it only takes 5 chances to guess the user's password and PIN code correctly
According to researcher Zihao Zhan, the GAZEploit attack was split into two parts.
First, the researchers created a way to identify whether a user is entering text while wearing the Vision Pro by analyzing the 3D avatar shared by the user.
They recorded the performance of 30 avatars as they completed various text input tasks and used the data to train a recurrent neural network, a deep learning model.
According to the researchers, when someone uses the Vision Pro for text input, their eyes focus on the button they are about to press and then quickly move on to the next key.
"When we make text input, the eye movements show some regular patterns." Zhan said.
Another researcher, Hanqiu Wang, added that these patterns are more common when typing text than when browsing the web or watching videos. "In tasks such as text input, blinking less frequently is due to greater concentration." He explained.
The second part of the study is to use geometric calculations to infer where the user will place the keyboard in the virtual space and its dimensions.
"The only requirement is that as long as we get enough eye tracking information to accurately reproduce the keyboard, then subsequent key inputs can be detected." Zhan explained.
By combining these two elements, they were able to predict which keystrokes the user might type. In a series of lab tests, the researchers were able to guess up to 5 times, although they didn't know the victim's text input habits, speed, or the exact location of the keyboard:
- Predict the letters in text messages with 92.1% accuracy;
- Guess the letters in the password with 77% accuracy;
- Guess the PIN code with 73% accuracy;
- Guess letters in emails, URLs, and web pages with 86.1% accuracy.
3. The results of the study have been reported to Apple, which announced that the vulnerability has been fixed
The GAZEploit attack was studied in a lab environment and has not yet been implemented in the real world against users using Persona. According to the researchers, in the actual use process, hackers may take advantage of data leakage to carry out attacks.
Theoretically, the hacker could share a file with the victim during a Zoom call, and the victim might then log in to a Google or Microsoft account. At this point, the attacker can record the victim's persona image and recover the victim's password through the GAZEploit attack, thereby accessing their account.
Researchers at GAZEploit reported their findings to Apple in April, and subsequently sent the proof-of-concept code to Apple.
Apple fixed the vulnerability in a software update for the Vision Pro at the end of July, which automatically paused the Persona feature when the user was using the virtual keyboard.
Apple says the issue has been resolved in visionOS 1.3. While Apple doesn't mention a fix for this vulnerability in the software update notes, it does go into detail in the company's security notes.
Apple assigned the vulnerability number CVE-2024-40865 to the vulnerability and advised Vision Pro users to download the latest software update.
▲Apple's description of the CVE-2024-40865 vulnerability in Apple's security notes (source: Apple's official website)
Conclusion: With the popularization of wearable devices, it is necessary to strengthen the awareness of privacy and security
As wearable devices become lighter, cheaper, and able to capture more and more personal biometric data, the issue of user privacy protection is becoming more and more prominent. The data collected by these devices involves not only personal health information, but also sensitive information such as location and activity habits, which will pose a serious threat to personal privacy if leaked or misused.
"As wearables such as smart glasses, XR and smartwatches become part of everyday life, people are often under-aware of the vast amounts of data they can collect about their behaviour and preferences, and the privacy security implications they can pose." Cheng Zhang, an assistant professor at Cornell University, said. His research work involves the development of wearable devices to help interpret human behavior.
"This paper clearly shows a specific risk of gaze typing, but that's just the tip of the iceberg." "While these technologies are being developed for positive purposes, we also need to be aware of the privacy security implications that may be involved and begin to take steps to reduce the potential risks that wearables may pose in the future." ”
Source: Wired Magazine