Microsoft, with a new CEO and a new corporate structure, is pushing hard to burnish its image as a source of innovation in business and consumer computing. Satya Nadella, Steve Ballmer’s replacement (and only the third CEO in the company’s 39-year history), continues to talk about Microsoft’s future as both a “mobile first” and a “cloud first” company, which means building on the now-completed Nokia acquisition and moving farther beyond its traditional emphasis on desktop operating systems and productivity software.
Meanwhile, Peter Lee, who took over last fall as managing director of Microsoft Research, says Microsoft is now more dependent than ever on new ideas from his 1,100-employee R&D operation. He says MSR must deliver both mission-focused innovations that address immediate business needs, and more disruptive or blue-sky innovations that could redefine Microsoft’s product lines years or decades down the road. Tellingly, Lee also thinks MSR researchers should be less secretive about what they’re working on—or at least, that’s what he told Xconomy CEO and editor-in-chief Bob Buderi in an interview last year.
In that spirit, the company invited a few journalists to MSR’s Silicon Valley TechFair at its Mountain View, CA, campus a couple of weeks ago. I was there to check out the 18 projects on offer, which ranged from new types of 3D displays to improved mobile interfaces and data visualization methods.
True to Lee’s four-quadrant map for representing computer research at Microsoft, some of the technologies had obvious and immediate uses in products like Office or Windows Phone 8.1 (which came out this month), while others didn’t seem relate to Microsoft’s existing business areas in any discernible way. Which is fine, in Lee’s world, since theoretical or curiosity-driven projects can turn out to have foundational implications.
I was able to spend time investigating five of the projects in particular. Here are few details, in order of most mission-focused to least (in my eyes, anyway).
1. Texting Without Lifting a Finger
Years ago, Seattle startup Swype pioneered the concept of a virtual QWERTY keyboard for touchscreen phones that would let you spell words by sliding your finger from key to key, rather than tapping; its software recognized the resulting squiggles. Nuance bought Swype in 2011 and continues to offer the Swype app for Android phones, but the idea has also been percolating elsewhere in the mobile world. At MSR, a team led by Jason Abacible, Time Paek, Bongshin Lee, and Asela Gunawardana used machine learning to develop an extremely error-tolerant shape writing system called Word Flow.
The researchers say it’s all about letting users trace out words using muscle memory, and then applying machine learning to disambiguate the resulting letter groupings. That means the system works even if the user is blindfolded or looking away from the screen. I wasn’t blindfolded when I tried Word Flow (which was included as a feature of Windows Phone 8.1), but I was able to write text messages like “The quick brown fox jumped over the lazy dog’s back” with nearly 100 percent accuracy on my first try. In January, a Seattle high school student using Word Flow on a Windows Phone achieved a Guinness World Record for the fastest text message on a touchscreen phone.
1. Making Cortana Smarter
Halo players know Cortana as the holographic AI character who supplies tactical tips to Master Chief. That made her (it?) a good model for the virtual personal assistant baked into Windows Phone 8.1; in this form, Cortana is a Siri-style, Bing-powered interface that lets owners query their phones about things like weather and news reports, flight and traffic information, and reminders and appointments. Microsoft engineers have strived to make Cortana a little more personalized than Siri or Google Now; for example, the system learns over time who’s in the user’s “inner circle” of friends and should therefore be allowed to interrupt designated “quiet hours” with incoming calls or texts.
But what Cortana knows about the outside world is largely limited to the domains where Bing’s search index is strong. If you ask a question like “Are there are any cheap Japanese restaurants open right now within five miles?” it can assemble a pretty accurate list, just based on Web data such as Yelp restaurant listings. If you ask what constellations will be visible from Tasmania tonight, it might have a harder time.
At Tech Fair, I talked about that problem with Larry Heck, a distinguished engineer at MSR and one of the contributors to the Cortana product. He says the number-one job for the Cortana team right now is to gather user data. The more people who use Windows Phone 8.1, the better developers will see which tasks and subjects Cortana users are asking about, and which answers they’re clicking on; this clickstream feedback will help Microsoft tune future responses and prioritize which new domains to bring into Cortana’s knowledge base.
3. Manipulating Data in 3-D
Last year I wrote about an MSR project called GeoFlow, designed to help Microsoft Excel users generate and explore interactive maps showing complex geospatial and temporal data. The technology was eventually incorporated into Office 365 under the name Power Map, and it’s all based on ideas about information display and storytelling that principal researcher Curtis Wong first developed for WorldWide Telescope, Microsoft’s virtual planetarium program.
Power Map is great for visualizing on a flat, 2-D display, but often such data would be even easier to explore in 3-D. At Tech Fair, Wong and his colleague Dave Brown, a senior research development engineer at MSR, were showing off a graphics platform code-named Holograph that makes special use of Microsoft’s giant Perceptive Pixel or “PPI” displays, touchscreen devices that come in 55-inch and 82-inch sizes. Brown developed virtual dials and buttons that
