app. The University of Michigan has hosted summer camps for deaf and hard-of-hearing children, and he’s let the kids try out early prototypes.
“The aim of this is to increase their access to the hearing world, to ease the communication barrier,” he says.
The app features a three-dimensional avatar on the screen that does the signing. Users can rotate the character around to different angles, speed it up, slow it down, even play the sign backward, if they want to.
Gilbert says it is important that the avatar not be just a cartoon character, but a full motion-capture computer representation of a real human. That is the only way the full syntax of sign language can be achieved, with all its subtleties.
“The grammar of sign language is not entirely in the hands,” Gilbert says. “It’s very much in the body and facial expressions. Facial grammar can actually change the meaning of a handshape. And so we wanted to be able to capture it as realistically as possible.”
All this development requires time in a motion-capture studio. He and Yu did their prototype at the University of Michigan’s 3-D Lab and plan on doing more at a studio at Henry Ford Community College in Dearborn.
And, since Gilbert and Yu are entirely self-funded—and doing this while they also hold down day jobs—progress is sometimes a bit slow. Gilbert has a PhD in atmospheric and space sciences, and he works in the space department at U-M. Gilbert says the company will seek venture funding, but not until they have a fully functioning prototype to shop around.
Because of the time and expense involved in getting into a studio, the full-function app will probably take another year of development. What Mobile Sign Language Systems will release around the holidays is a kind of baby step along the way—in some sense. It’s an app of baby sign language.
It’s been trendy for a few years now for parents to communicate with their prelingual children using sign language. The theory is that babies can make their needs known to their parents using sign language a lot sooner than they can learn words. Educational courses and products have developed around this theory.
Gilbert, a father of three, communicated with his kids that way when they were infants, and he hopes that it is more than just a trend. So, he and Yu put together a short dictionary of baby signs—basic
