Researchers Using the Doppler Effect to Sense Gestures for Input Commands

Researchers Using the Doppler Effect to Sense Gestures

Building on its success with the XBox Kinect, which uses motion to influence the actions of a software program, Microsoft is looking at sound as its next frontier for data entry.

Researchers Sidhant Gupta, Dan Morris, Shwetak N Patel, and Desney Tan from Microsoft Research and from the University of Washington have published a paper that demonstrates how the devices can detect subtle changes in sound to intuit a user’s intent. For example, pointing at a particular location on a display screen.

Called SoundWave, the discovery started by accident when the researchers were using ultrasonic sensors in the lab and later realized the results they could not directly correlate were not entirely anomalous.

According to Microsoft, Gupta recalls sitting in his lab, measuring signals and “kind of jiggling my leg,” he said. “I saw the signal change when it should not have moved, so I thought there was a loose connection or something. But as soon as I got up from my chair to check, the error went away. I sat down to work, started moving my leg, and the signal changed again. After a couple of minutes of this, I realized it was detecting motion.”

The Doppler effect recognizes changes in light and sound frequencies associated with motion. When objects approach, they have a higher sound and different light frequency than objects that are moving away. So, if a user’s hand is approaching or moving away from the screen, the SoundWave could interpret that command. The research is granular enough that twists of the hand can also be differentiated.

With laptops, the microphone and speakers are mounted on the keyboard which faces up, not out toward the end user. Researchers say this could create a number of false events. A better choice, they say is the mobile device.

SoundWave uses the speakers on a device to emit a continuous, mostly non-audible sound that bounces off the body. As the body moves, the sound is measured for changes in frequency. The team found they could discern velocity, direction, proximity, and even the size of a moving object.

The researchers could then use these changes in frequency to simulate single and double taps and even a two-handed motion used in game playing. The researchers admit there are limitations. One is the tone emitted by Soundwave, which could be annoying to small children and animals. Another is piggy-backing the tone over the user’s choice of music; it could distort the music selection. And since Doppler only measures motion, the researchers also admit they would need a complimentary technology for static poses.

Still, it’s a pretty neat advancement, however, security consequences of its use remain unclear.