[Corrected 5/15/14, 3:41 pm. See below.] The fact that Norrie Daroga considers Walt Disney the forefather of his company’s interactive virtual characters is telling, and not just because Geppetto Avatars was partly named after the woodcarver who created Pinocchio, the puppet who dreamed of being a real boy, in Disney’s classic 1940 animated film, “Pinocchio.”
Disney’s team understood that the secret of a successful animated movie was crafting characters that can touch audiences emotionally. Geppetto Avatars, a Mequon, WI-based startup with offices in San Francisco, Los Angeles, and Boston, is seeking that emotional bond, too. But it is also trying to do something harder, something that Disney could only imagine. It is building virtual characters, or avatars, that actually interact with people, characters that could become complementary caregivers, teachers—maybe even friends. As co-founder and chief science officer Mark Meadows says in a video on the company’s website, Geppetto is creating “smart puppets that can listen to you, read your emotions, and understand you and reply in real time.”
There’s a place for such empathetic characters in today’s health care system, Geppetto believes. Take the thorny and expensive problem of asthma patients making frequent hospital visits because they experience repeated asthma attacks. One way to help prevent those attacks is to check in with asthma patients every day—yet understaffed nursing departments often don’t have the resources to contact patients that frequently. Geppetto’s characters can. Patients could chat daily with an avatar on their tablets, smartphones, or laptops, says Daroga, the company’s CEO and other co-founder. The software would log how the patients are feeling and how often they’re using their rescue inhalers. That info would then be sent wirelessly back to the caregivers, who can intervene with patients at high risk for another attack. The avatar can also look online at the forecasts for weather, pollen, and pollution, and warn patients not to go outside when conditions are bad, while also sending email or text messages or making a phone call reminding them to take their meds.
During the interview with Xconomy, Daroga played a video of an exchange between David, a middle-aged man with asthma, and Sophie, an avatar dressed in a lab coat and holding a patient chart. She asks why he had five asthma attacks the previous week (he says he’s catching a cold), and what his condition is hindering him from doing (“walk up stairs, play with the kids, go see a movie,” he responds).
Green and red dots are superimposed over David’s image on the screen, indicating the observations by the facial recognition software. Next to it is an octagonal-shaped heat map with emotions listed on the edges, like fear, shame, indignation, and confidence. As David speaks, blue and white-colored octagonal clouds shift around the grid and hover in the sectors corresponding to his perceived emotions. For example, Sophie explains that the local weather forecast indicates a 65 percent increased risk of an asthma attack. “You got some popcorn in the house?” she adds in a lower-pitched voice. “Ah sure, I get it, not going out today,” David replies, as the heat map indicates he’s feeling fear and shame at that moment. Sophie advises David to keep his inhaler nearby, and he tells her to send him a text message reminding him when to use it.
In addition to improving the care of patients like David, Geppetto’s virtual assistants could speed diagnoses of diseases by conversing with patients, cross-referencing their responses with online scientific databases, and creating a list of possible disorders the patients might have. That would save doctors time, and maybe even improve healthcare.
The avatars also could help pharmaceutical companies that are developing drugs for cognitive diseases like Alzheimer’s or depression, Daroga says. Being observed or interviewed by Geppetto’s avatars could be a more reliable way to measure patients’ improvements or declines while taking the drug than observations by doctors or nurses. Humans just aren’t as good at objectively and scientifically charting small changes, he explains.
The avatar, on the other hand, could track a patient’s daily speech improvement over the course of a month, or which emotions the patient often shows, measured by things like the tone of voice, word choices, and facial expressions, Daroga says.
“Now after three weeks, the conversations [display] 90 percent more confidence or 90 percent more happiness, you’ve got a touch point,” Daroga says.
The potential applications for Geppetto’s avatars go beyond healthcare. The company also plans to market its platform to education companies that could use it to help children with learning disabilities improve their reading, spelling, and other skills. The two-way conversation is more engaging than a video, and the avatar can customize the instruction to each child, Daroga says, ending the lesson early if it senses the child is getting frustrated, for example.
If a soothing virtual teacher or caregiver who’s attuned to your every emotion sounds a bit like science fiction, well, that’s not surprising. Hollywood and pop culture have
