Connect with us

Hi, what are you looking for?


Data Protection

Researchers Extract Sounds From Still Images on Smartphone Cameras

A group of academic researchers devised a technique to extract sounds from still images captured using smartphone cameras with rolling shutter and movable lens structures.

A group of academic researchers has devised a technique to extract sounds from still images captured using smartphone cameras with rolling shutter and movable lens structures.

The movement of camera hardware, such as the Complementary Metal-oxide–Semiconductor (CMOS) rolling shutters and the moving lenses used for Optical Image Stabilization (OIS) and Auto Focus (AF), create sounds that are modulated into images as imperceptible distortions.

These types of smartphone cameras, the researchers explain in a research paper (PDF), create a “point-of-view (POV) optical-acoustic side channel for acoustic eavesdropping” that requires no line of sight, nor the presence of an object within the camera’s field of view.

Focusing on the limitations of this side channel – which relies on a “suitable mechanical path from the sound source to the smartphone” to support sound propagation, the researchers extract and analyze the leaked acoustic information identifying with high accuracy different speakers, genders, and spoken digits.

The academics relied on machine learning to recover information from human speech broadcast by speakers, in the context of an attacker that has a malicious application running on the smartphone but does not have access to the device’s microphone.

However, the threat model assumes that the attacker can captures a video with the victim’s camera and that they can acquire speech samples of the target individuals beforehand, to use them as part of the learning process.

Using a dataset of 10,000 samples of signal-digit utterances, the researchers performed three classification tasks (gender, identity, and digit recognition) and trained their model for each task. They used Google Pixel, Samsung Galaxy, and Apple iPhone devices for the experiments.

“Our evaluation with 10 smartphones on a spoken digit dataset reports 80.66%, 91.28%, and 99.67% accuracies on recognizing 10 spoken digits, 20 speakers, and 2 genders respectively,” the academics say.

Advertisement. Scroll to continue reading.

Lower quality cameras, the researchers say, would limit the potential information leakage associated with this type of attack. Keeping smartphones away from speakers and adding vibration-isolation dampening materials between the phone and the transmitting surface should also help.

Smartphone makers can mitigate the attack through higher rolling shutter frequencies, random-code rolling shutters, tougher lens suspension springs, and lens locking mechanisms.

“We believe the high classification accuracies obtained in our evaluation and the related work using motion sensors suggest this optical-acoustic side channel can support more diverse malicious applications by incorporating speech reconstruction functionality in the signal processing pipeline,” the researchers added.

Related: Researchers Demo Electromagnetic Fault Injection Attacks on Drones

Related: Open Source Tool For Hunting Node.js Security Flaws

Related: New Speculative Execution Attack Against Apple M1 Chips

Written By

Ionut Arghire is an international correspondent for SecurityWeek.

Click to comment

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing to stay informed on the latest threats, trends, and technology, along with insightful columns from industry experts.

Join us as we delve into the transformative potential of AI, predictive ChatGPT-like tools and automation to detect and defend against cyberattacks.


As cybersecurity breaches and incidents escalate, the cyber insurance ecosystem is undergoing rapid and transformational change.


Expert Insights

Related Content

Application Security

Cycode, a startup that provides solutions for protecting software source code, emerged from stealth mode on Tuesday with $4.6 million in seed funding.

Data Protection

The cryptopocalypse is the point at which quantum computing becomes powerful enough to use Shor’s algorithm to crack PKI encryption.

Artificial Intelligence

The CRYSTALS-Kyber public-key encryption and key encapsulation mechanism recommended by NIST for post-quantum cryptography has been broken using AI combined with side channel attacks.


The three primary drivers for cyber regulations are voter privacy, the economy, and national security – with the complication that the first is often...

Mobile & Wireless

Infonetics Research has shared excerpts from its Mobile Device Security Client Software market size and forecasts report, which tracks enterprise and consumer security client...

Artificial Intelligence

Two of humanity’s greatest drivers, greed and curiosity, will push AI development forward. Our only hope is that we can control it.

Application Security

Fortinet on Monday issued an emergency patch to cover a severe vulnerability in its FortiOS SSL-VPN product, warning that hackers have already exploited the...

Application Security

Virtualization technology giant VMware on Tuesday shipped urgent updates to fix a trio of security problems in multiple software products, including a virtual machine...