“to get computers to see and hear and reason like humans, or even better.”
Lee says he first encountered the term back in the late 1980s. But there were lots of early problems and the technology didn’t work as hoped. About six or seven years ago, the field picked up again, and it has exploded in more recent times, he says. Beginning in 2009, one of the original pioneers of the field, Geoffrey Hinton of the University of Toronto (he now works part-time for Google), collaborated with MSR to apply deep neural nets, first and foremost, to speech recognition. Lee says MSR researchers advanced this work and in 2011 showed it dramatically improved the potential of real-time, speaker-independent, automatic speech recognition: rather than having one word in 4 or 5 incorrect, the error rate is now one word in 7 or 8. “Within two years, with changes in big data and cloud computing, plus steady algorithmic improvements, this stuff actually works and works way better than anything we have seen before,” Lee says. Microsoft has since advanced it as the core of its speech processing, in products from Xbox to Bing to Windows Phone.
What’s more, the change goes far beyond Microsoft, Lee says. Nuance and Google have altered their speech product direction to be based on deep neural nets, he says. “The entire industry just changed on the dime through this.”
Telepresence
One of the biggest MSR efforts, Lee says, is a broad investigation into the technologies and algorithms around telepresence. There are lots of problems to address, such as how to convey things like body language over long distances. But it is important for researchers to step back and ask why they should be doing this, Lee says. If they are too mission-focused, the answer they come up with will be things like enabling better business meetings and saving travel costs. Those may well be commercially viable but not that imaginative—and maybe not that interesting over the long haul, he says. So Lee wants researchers to ask questions such as, “Would it be possible for musicians to collaborate through time and space?’ and “What would we learn by enabling children to really engage at a distance?”
“That kind of playfulness in what we’re trying to accomplish” is more likely to lead to surprising things that might pay off in much broader or bigger ways, he says.
Computational Social Sciences
Lee believes that the social sciences can be utterly transformed by computing in much the same way that the physical sciences and the life sciences already have been—enabling game-changing advances in everything from analyzing and predicting human behavior to forecasting the behavior of companies and even society at large. Lee sees some big commercial applications of all this. “Because unlike understanding the cosmos and the big bang, which is important,” he says, “really deeply understanding how society works and from that inferring lots of things about people can have really profound consequences for economics, public policy, psychology, and so on in the social sciences, but also for how Microsoft and other companies do their business.”
One specific way this might bear fruit is what he calls a new twist on machine learning. Typically machine learning is about finding statistically relevant correlations between data. “It gives you correlation, not causation,” he says. But with advanced techniques and big data sets, you can begin to find causes.
Ubiquitous Computing
“The technologies for sensing and embedding sensors not just on our bodies but everything around us, I think, will be pretty significant,” Lee says. This has long been a kind of holy prediction in computer sciences, but he says, “It actually looks like it finally will become real.”
A New Openness?
When I met a few years ago with Craig Mundie (at the time, Microsoft’s chief research and strategy officer and now senior advisor to the CEO), he had clearly asked MSR to be less open about its research, worried that others were developing business intelligence or advantage from what MSR sometimes shared about its work.
Lee seems to have a different view. “The importance of openness is really underrated,” he says. “If there’s one thing I worry about, not just for MSR but for the IT industry, it’s that secrecy seems to have become cool. I think that this is something that is a dangerous trend really for the whole industry.”
He says secrecy has even “crept into academic research.” In the field of deep neural nets, for instance, “I’m aware of eight or nine academic research groups that are just very hush-hush about what they are planning to show there.” Such secrecy, he says, is at the level where be believes it can be “a drag on the flow of ideas.”
“For MSR, one of the things that we’ll be working very hard to do and try to show some leadership in is not to just maintain our openness but grow it.”
An example of this is some recent work on Kinect, which employs gesture and speech interfaces, among other technologies being studied at MSR. Researchers don’t reveal specifically how they are applying their work to future products or what future applications of their ideas are envisioned. However, rather than clamming up on it all, “We have been publishing all the foundational work,” Lee says. “It’s a partnership. We depend a lot on the advances in the academic research community, and we need to participate as openly as possible.”
Closing Perspective: Watch for Some Big Payoffs from MSR
Lee has studied the history of industrial research and he spoke somewhat passionately about the long time it sometimes takes researchers to deliver seminal discoveries or inventions.
“I think MSR has the people now to do something really remarkable,” Lee says. “Can it do something as monumental as IBM Research did in its heyday, or Bell Labs? I do think that all the opportunities are there.”
