The chorus of Hotel California sings out from the mobile phone in Bryce Kellogg’s pocket. He waves his hand to turn down the volume. He flicks his fingers a couple of times and Stairway to Heaven plays. The phone never leaves his pocket.
It looks like a parlor trick, but the underlying technology, developed by a team of University of Washington electrical engineers and computer scientists, has potential to enable natural gesture control of the broadest range of electronic devices, even those with no batteries.
The technology—a prototype of which was attached to Kellogg’s off-the-shelf Android phone—is called AllSee. Kellogg and fellow doctoral student Vamsi Talla refined it over the course of the last year with Shyam Gollakota, a UW assistant professor of computer science and engineering. They’re demonstrating it this week at the Symposium on Networked Systems Design and Implementation, an international conference being held in Seattle. They’ve filed for a provisional patent with help from the UW Center for Commercialization, and they’ve already fielded multiple enquiries from phone manufacturers.
In the UW Networks and Wireless Lab, Kellogg, 24, and Talla, 26, show me how AllSee works. They’ve set up a prototype receiver; an RFID reader, which provides a radio signal for demonstration purposes; an oscilloscope that shows how the signal appears to the receiver; and a computer that displays the receiver’s interpretation of the gestures being performed.
Kellogg holds his hand near the receiver. The scope shows small peaks and troughs in the signal’s amplitude, indicating the power of the radio signal being reflected off his hand. He moves his hand closer and the peaks and troughs on the scope grow bigger. An image of a “push” gesture pops up on the screen.
“That’s how we tell the difference, for example, between a push gesture and a pull gesture,” he explains. “We go small to big, big to small. Just looking at the power of the reflected signal, we can detect which gesture you’re doing.”
Though we’re still getting used to touchscreen-based gestures, lots of devices today can detect hand or body motions. Prominent examples on the market today include the Kinect from Microsoft, the Leap Motion controller, and Samsung Air Gesture. The Kinect, a sophisticated controller sold as an accessory for Xbox game consoles and for Windows PCs, includes a camera, microphone, and infrared sensors. The Leap Motion connects to a Mac or PC and interprets hand and finger gestures using infrared sensors. Air Gesture, too, relies on infrared proximity and gesture sensors.
Since they rely on visible or invisible light, all require the gestures be made within sight of the devices—for the Samsung technology, gestures can be no more than 3 inches away, according to the company’s how-to guide. But since AllSee uses reflected ambient radio waves, it can detect gestures even if there’s an object or material in the way.
But it’s AllSee’s power effiency that really makes it stand out. Kinect, which does skeletal tracking of multiple players simultaneously, and facial and voice recognition, hooks into a larger, non-mobile device such as a PC or game console and therefore has much more power, and greater computational resources, to work with. The same is true of the Leap Motion device, which connects via USB to a computer and is designed to sit on a desktop. Samsung Air Gesture, a feature of the company’s Galaxy S4 phone, is perhaps the more apt comparison to AllSee.
But, Kellogg notes, “If you leave it on all the time, it just sucks your battery. What we’re trying to do is the same type of gesture recognition, but with passive components.”
AllSee consumes only 28 microwatts of power to perform an average of 15 gestures per minute—orders of magnitude less than similar systems. That includes the low-power controller that runs a simple algorithm to convert eight distinct gestures into instructions for the device, Talla says. (The Samsung technology only recognizes five gestures.) AllSee has no radio, just an antenna to receive ambient television or RFID signals. It even makes use of another UW innovation to gather small amounts of power from the signals themselves.
The technology, which costs less than $1, presents the possibility of outfitting virtually any
