Creating Web-based marketplaces for “crowdsourcing”—farming out digital piece-work to freelancers around the world—has been a hot business idea for the last half-decade. IStockphoto does it for stock images, TopCoder does it for enterprise software, Innocentive does it as a supplement to corporate R&D, and uTest does it for software quality assurance testing. But when push comes to shove, how well do these companies know and trust their networks of contributors, most of whom they’ve never met? A couple of weeks ago Boston-based uTest found itself at the mercy of its own crowd—and emerged with an encouraging story to tell.
At uTest, 25,000 software testers in more than 160 countries log on over the Web to do on-demand testing of software applications, helping the applications’ makers catch bugs and glitches, troubleshoot usability problems, and simulate performance under realistic loads. Twice a month, uTest pays its active testers via Paypal or Payoneer (a New York-based network that delivers payments using prepaid Mastercard cards).
According to Matt Johnston, uTest’s vice president of marketing and community, uTest ran into a glitch of its own the evening of Saturday, May 15, as it disbursed payments for the first half of May. It transferred funds to Paypal and Payoneer twice—meaning that all testers who had done any work in early May received double their usual fee.
“I won’t go into specific numbers, but it was a non-trivial amount of money,” Johnston tells Xconomy. (He also wrote about the episode on uTest’s blog last week.) “We’re talking well into the five figures”—way more than the venture-funded startup could afford to lose on a bookkeeping error.
But right away, uTest started to benefit from an unexpected side effect of being in the software-testing business. It wasn’t PayPal or Payoneer who detected the problem, but the testers themselves. And being testers, they sent in bug reports.
“If you saw an extra $200 sitting in your bank statement, you might go, ‘Huh,'” says Johnston. “But software testers, they say ‘Oh, this is an interesting defect.’ They are wired to point out flaws. The first reports we got were not just ‘Hey, you paid me twice’—they were, ‘It looks like I’ve got two separate transaction IDs for the exact same amount, seven seconds apart.’ They were diagnosing it for us, like it was just another software defect.”
This was late Saturday night, Johnston says. The company had to decide quickly what to do about the problem. PayPal and Payoneer politely informed uTest that there was nothing they could do—there’s no such thing as an “undo” button for electronic payment (for good reason—the idea of PayPal debiting your bank account without your consent is more than a little scary). “We ultimately came to the conclusion that we had to be really transparent about it, and tell our community
