If they weren’t outdated already, 1980s innovations like the Clapper and the medical alert pendants of “I’ve fallen and I can’t get up” fame may have met their 21st century match in a new gesture sensing technology developed by University of Washington researchers.
The WiSee parses and interprets small disruptions in a WiFi signal caused by human movements, translating them into commands for music systems, lights, thermostats, and other connected devices.
No longer is it “Clap On!, Clap Off!” to operate the lamp, but a simple, quiet raising or lowering of the arm.
“It’s a new way of doing it, and it’s more natural,” says Sidhant Gupta, a fourth-year PhD student at the UW, working on the project.
This whole-home motion sensing and gesture recognition technology—which is now merely a research prototype—has countless potential applications, from home security to home automation to entertainment. Indeed, Qifan Pu, a visiting student from China and former Microsoft Research Asia intern, was inspired to explore the technology after playing with the Xbox Kinect, which uses a camera and microphones to turn the body into a game controller.
“We kept thinking… it would be much cooler if we can control the Kinect from the kitchen where I’m going to grab a drink,” Pu says.
The camera-based Kinect must be able to see the person controlling it. WiSee works anywhere a WiFi signal reaches, meaning you could turn off the TV from the shower. And since it’s not actually seeing its subject, WiSee is “agnostic to lighting,” as Gupta puts it.
WiSee works by taking advantage of something that has previously been a nuisance for wireless networks.
WiFi routers are designed to correct for the Doppler shift that occurs when their signal bounces off people and objects in motion. WiSee captures the patterns in that shift resulting from specific gestures, which then become controls for other connected devices.
This is no mean feat. To carry lots of data, a WiFi signal can be as broad as 20 megahertz per channel. The variations in the signal caused by human movement amount to tens of hertz—orders of magnitude smaller, and hard to detect within the broad signal.
The key technical breakthrough that makes WiSee possible is an algorithm that splits the signal into 2-hertz chunks, making the variations stand out.
Pu, demonstrating WiSee in a room at the UW Allen Center for Computer Science & Engineering, points to a display of the local WiFi signal. “We are going to see these small Doppler shifts on this transformed narrow band signal,” he says.
The team has shown WiSee can detect nine distinct gestures with an average of 94 percent accuracy.
Parsing the signal in this way does not degrade it, Gupta says. It is merely processing it in a different way.
This video shows WiSee in action.
WiSee could ultimately become part of the software on board a WiFi router—though the router would need more processing power than today’s consumer products—easily adding this sort of gesture-recognition capability without any additional hardware.
“Really low-hanging fruit is simply a whole-home motion detector,” Gupta says. “You could imagine a security system where you don’t have to install a motion detector in each individual room.”
A gesture could also be used to summon emergency assistance. “Someone falls down and they cannot get up to dial 911,” he says. “And you simply do this gesture and it engages a 911 call or alerts your loved ones that there’s a trouble.”
There are also myriad applications for controlling home entertainment systems—potentially as a complement to other interfaces, such as Kinect—as well as lights, heat, and other appliances.
Since word of the invention spread earlier this month, the team—which also includes UW computer science professors Shyam Gollakota and Shwetak Patel—has been inundated with emails. Fellow academics, as well as device manufacturers, security companies, large display makers, and home automation and control systems firms have sent enquiries and ideas.
A paper describing the technology [PDF] is to be presented in the fall at the International Conference on Mobile Computing and Networking, in Miami, FL.
There is more work to be done to refine WiSee, such as making it more robust when there are multiple people in a space. Tests so far have shown that using a router with multiple antennas, it can distinguish among up to four people.
Security is another concern. “You don’t want someone walking by your apartment to wave and turn on your kettle or something like that,” Gupta says.
Asked whether they are planning a company to commercialize or license the technology, Gupta says, “It’s all in flux.”
Pu has just moved to U.C. Berkeley to pursue his PhD. Patel is co-founder of SNUPI Technologies, which is working on low-cost home sensor networks.
“This is a very new freshly baked research project,” Gupta says. “But there’s been a lot of interest and we’ll pay attention to it and see where it goes.”
