Hack/Reduce Gets Down to Big-Data Business with First Hackathon

In a very busy week for tech events, you should know that this one is BYOD. No, not “bring your own device.” Bring your own dataset. But don’t fret if you don’t have one; there’s also plenty of data available to work on.

The inaugural big-data hackathon at Cambridge, MA-based hack/reduce is happening tomorrow, Nov. 17, at the nonprofit’s workspace on Third and Binney near Kendall Square. The goal is to bring lots of techies together (most of whom don’t know each other) to work on crunching huge amounts of data across a wide range of fields, such as music and entertainment, stock markets, shopping, Web search queries, cancer genomics, transportation, travel discovery, and the social graph.

Overall it’s a nice counterpoint to the MassTLC unConference, going on today, which is mostly about talking and networking, and less about doing actual work. (Hack/reduce officially opened with a big party of its own last week.)

Hack/reduce co-founder Fred Lalonde, whose day job is CEO at big-data travel startup Hopper, is expecting more than 100 people on Saturday (mostly coders, but not all). The hackathon is free, and people can stop by anytime. It starts around 9 am, when folks will pitch their project ideas and form teams; then the coding will go on all afternoon, and the groups will present their demos and results from around 6-8 pm. The audience favorite will receive one year of computing resources at hack/reduce.

“It’s usually great fun,” says Lalonde, who has helped organize similar events in Montreal and other cities. As goals of the hackathon, he cites making “connections between geeks” and “intersecting datasets.” (The latter I took to mean combining disparate data in a way that provides insights into things like customer behavior or health trends.)

Lalonde is encouraging a couple of things from participants. One is big-data BYOD. “Even if they’re not a coder, if they work for a company or organization that has a public dataset, the best thing they can do is come in and make that available to the community,” he says. The other is participation from local life sciences researchers. “I want to make a special call to the biotech community,” he says.

That’s partly because in past big-data hackathons, some of the most useful results have come from health-tech researchers—doing things like discovering and analyzing genome-based patterns of diseases based on anonymous datasets, Lalonde says.

Not that it’s all life-and-death stuff. Lalonde recalls one of the funniest hackathon demos being a comprehensive list of all the ambiguous URLs out there—innocent domain names that can be read in a lewd way. Like www.expertsexchange.com, www.mp3shits.com, and my personal favorite, www.speedofart.com (that project must have required serious infrastructure). “There’s some ridiculous stuff,” he admits.

Hack/reduce is still looking for a full-time executive director to succeed Abby Fichtner, who left recently. As for official ongoing projects, Lalonde says he has a stack of more than 100 applications to go through. Of those, a few will be selected and announced, starting next month.

“What I’m hoping is that people are actually going to realize they want to work on big data more than just at an event like this,” he says. “There’s close to $1 million of compute [resources] available for anyone who gets into the space.”

Author: Gregory T. Huang

Greg is a veteran journalist who has covered a wide range of science, technology, and business. As former editor in chief, he overaw daily news, features, and events across Xconomy's national network. Before joining Xconomy, he was a features editor at New Scientist magazine, where he edited and wrote articles on physics, technology, and neuroscience. Previously he was senior writer at Technology Review, where he reported on emerging technologies, R&D, and advances in computing, robotics, and applied physics. His writing has also appeared in Wired, Nature, and The Atlantic Monthly’s website. He was named a New York Times professional fellow in 2003. Greg is the co-author of Guanxi (Simon & Schuster, 2006), about Microsoft in China and the global competition for talent and technology. Before becoming a journalist, he did research at MIT’s Artificial Intelligence Lab. He has published 20 papers in scientific journals and conferences and spoken on innovation at Adobe, Amazon, eBay, Google, HP, Microsoft, Yahoo, and other organizations. He has a Master’s and Ph.D. in electrical engineering and computer science from MIT, and a B.S. in electrical engineering from the University of Illinois, Urbana-Champaign.