No-Touch Smartwatch Possible Using Sonar

(CN) - Consumers desire sleek and small electronic devices, which prompts manufacturers to increase efficiency and minimize physical clutter for modern smartphones and smartwatches. But a smaller screen size can also make typing accurately difficult and frustrating.

So researchers from the University of Washington have developed a new process that lets users interact with smart devices by performing gestures in mid-air or on a nearby surface - without additional hardware.

The process, called FingerIO, allows users to create search commands, adjust volume or scroll through menus without directly touching the device.

"FingerIO is the first system that achieves sub-centimeter level accuracy on unmodified off-the-shelf smartphones," lead author Rajalakshmi Nandakumar, a doctoral student in computer science and engineering at the University of Washington, said. "Unlike, other near-device interaction systems, FingerIO does not require the user to instrument their fingers with any additional sensors."

FingerIO creates a sonar system using a device's speakers by emitting inaudible sound waves. These sound waves bounce off of a user's finger, sending an "echo" that is recorded by the device's microphones and translates gestures onto the screen of a smartphone or smartwatch.

"Our system does not require line-of-sight and can work even if the phone is inside a pocket or hidden by a jacket," Nandakumar said.

Past technological approaches have relied on cameras or additional sonar-sensing hardware, which limited the accuracy of tracking finger movements and usage range.

"The key challenge was to reduce the error to sub-centimeter level on an off-the-shelf device," Nandakumar said. "The existing methods gave us an accuracy of only 3 to 4 centimeters, which was not sufficient for accurate finger tracking."

Sonar echoes are generally weak and not accurate enough to track a user's finger motion at a high resolution. To improve tracking resolution, the team used a type of signal that is used for a variety of wireless communications - including Wi-Fi and LTE - called Orthogonal Frequency Division Multiplexing. This signal type permits high-resolution tracking of phase changes in the echoes.

The team drew on a previous project they completed in 2015, which used sonar to identify sleep apnea. That project prompted them to consider alternative uses for their sonar technology, and the existing framework allowed them to complete FingerIO in roughly four months.

"After our previous work using active sonar on smartphones to detect sleep apnea, we started thinking about other applications of sonar and realized that similar techniques could be applied to track finger motion," Nandakumar said.

They tested their theory using an off-the-shelf Samsung Galaxy S4 smartphone and a smartwatch customized with two microphones. Smartwatches generally have one speaker, which allows finger tracking in one dimension.

The researchers will pursue improvements, such as incorporating additional microphones for tracking in three dimensions.

"Beyond that we're exploring options to commercialize this technology and make it available for use as a real product. In the meantime we're working on improving the system to make it more robust, or possibly track multiple fingers," Nandakumar said.

Nandakumar and her co-researchers will present their technology at the Association for Computing Machinery's CHI 2016 conference in San Jose, California.

Their research was funded by the National Science Foundation and Google.

Follow @SeanDuffyCNS

Subscribe to Closing Arguments

Sign up for new weekly newsletter Closing Arguments to get the latest about ongoing trials, major litigation and hot cases and rulings in courthouses around the U.S. and the world.

Email *

Submit