Showing snapshots from vacations is nice, but what if you could use a phone or tablet to walk your friends through a three-dimensional reconstruction of Notre Dame or Angkor Wat, based on your photos? That’s just one application of “immersive” 3D imaging technology, which is coming to smartphones and tablets.
The biggest names in computing, including Google, Intel, and Microsoft, are investing in three-dimensional scanners and cameras. The technology can be used to not only produce 3D images, but also recognize faces and control video games through gestures, as Microsoft’s Kinect controller does.
One of the startups in this field is Mountain View, CA-based Matterport, which is betting that today’s emerging 3D imaging technology will lead to a new class of visual applications. In the coming months, the company plans to release software that will allow third-party developers to write custom applications using Matterport’s system for capturing images and generating high-quality 3D renderings, CEO Bill Brown says.
Matterport now makes a $4,500 camera aimed at professionals, specifically in the real-estate market. Using an iPad as a controller, a photographer or real-estate agent places a large camera on a tripod and scans a home’s interior spaces from different locations. Matterport’s cloud-based software stitches the images together in an hour or two, allowing the user to navigate through a 3D rendering of an interior space.
Opening up the Matterport software to app developers will allow someone without expertise in computer vision—a field of computer science for recognizing and processing images—to build 3D real estate listings with additional features, such as the ability to take measurements or tag items within a virtual building, Brown says.
Brown sees potential for consumer applications, too. A home improvement store, for instance could allow a person to plan home improvements from a 3D rendering of the home by trying different colors, floorings and furniture, all provided by partner companies.
These types of applications are becoming possible in part because 3D sensors, such as the Microsoft Kinect gesture recognition controller, have become so inexpensive. These sensors can recognize gestures as well as scan a space to recreate it in a 3D map. “The founders of Matterport looked at the Kinect sensor and said, ‘That’s a camera. We’re going to use that to create a new immersive media,’” says Brown.
Matterport plans release less-expensive versions of its camera. But ultimately, Matterport intends to work with 3D scanners from different companies and provide the means for building custom applications based on its 3D photography software.
And in the future, some of those 3D cameras will be on smartphones and tablets. Google’s Advanced Technology and Projects group (ATAP) is leading Project Tango, an initiative to build 3D motion and depth sensing into mobile gadgets. Although there aren’t many applications that take advantage of Google Tango’s hardware and software, the addition of 3D cameras to phones and tablets could give consumers a reason to upgrade sooner.
The technology behind this is challenging: Google has convened people from research labs, universities, and other tech companies with experience in robotics and computer vision to develop add-on hardware and cloud-based software.
Google ATAP has built a prototype and is recruiting developers to write applications. For example, aisle411, a company that specializes in retail store mapping, developed a tablet app that allows a consumer to walk around Walgreens stores with a tablet on the shopping cart.
As the person walks around the store, the app provides a picture of shopping aisles overlaid with images on the screen, such as offers on products or directions around the store. “We can overlay navigation, rewards, and personalized specialized offers that turns the store into an immersive, game-like experience,” aisle411 CEO Nathan Pettyjohn says in a promotional video.
Another use for these sensors is indoor navigation for robots, which can use cheap sensors to build maps that allow them to move autonomously. Cambridge, MA-based Paracosm is building 3D imaging software and targeting a number of uses, including gaming, augmented reality, mobile devices, virtual reality headsets, and robotics.
For business reasons, Matterport is currently focusing on real estate and expects to expand to construction or other industries. But Brown says the application of this technology is broad. For example, a person could take a 3D picture of a device and then do a Web search based on the picture.
“The professional market (for its camera) is just the first step in a progression that leads to new and unique visual media,” he says. “We’re pretty positive future mobile cameras will include 3D data.
