references. In addition, the venture partners give weight to a potential portfolio company that is gaining traction as reflected in its revenues, “with the implicit assumption that if customers are buying a vendor’s product, they are evaluating it and it performs its function and meets their needs,” he says.
Aside from their own in-house testing, security customers may have varying degrees of access to information that could help them identify low-performing cybersecurity products.
Banks have internal groups that assess security services and share that intelligence within their own circles, Tseitlin says. “If you’re not part of those inner cliques, you’re not going to get access to that information,” he says. For the average company, there’s no reliable, comprehensive public data bank with ratings of cybersecurity company effectiveness, he says.
“There really isn’t any kind of independent Consumer Reports kind of thing,” Tseitlin says.
Cybersecurity companies themselves gain insights and earn money by sharing with other security vendors what they’ve learned about breaches they’ve observed, or handled, or caught slipping past their shields. Such data exchange agreements tend to require that nothing is disclosed about the clients who fell victim to the breach, Tseitlin says. Information about the breaches helps security companies protect their other clients, he says.
Security firm customers may glean intel from various watchdogs that publicly disclose their discoveries about weaknesses in cybersecurity software. For example, Google’s Project Zero team conducts research on “any popular software which is security critical,” according to a recent job announcement. Early this year, that Google team called out Irving, TX-based cybersecurity company Trend Micro for an alleged defect in its malware shield that allowed hackers to peek inside users’ password manager files, Ars Technica reported. Trend Micro responded quickly with a fix.
Another birddog is former Washington Post security writer Brian Krebs, whose blog KrebsonSecurity dissected a shake-up at Foster City, CA-based cybersecurity company Norse in January as its CEO was fired. Krebs quoted Norse employees saying that the company’s dazzling displays of Internet attack traffic might not be backed up by a robust security product. Norse countered some of Krebs’ reporting in a later response.
Tseitlin says security customers are making choices among cybersecurity companies while they’re grappling with larger uncertainties.
Assessing the level of data security for any business—-whether an e-commerce site or a government contractor or cybersecurity companies themselves—-is a complex task in a connected era. For one thing, most businesses are running their operations on Web-based applications sourced from dozens of other companies, Tseitlin says. The security of the business depends on the security of each of those cloud service vendors, he says.
Businesses often contract with multiple security companies that shield their clients’ data in different ways. It may be tough to quantify how much more secure a business customer may be because of the services of one particular cybersecurity company.
“There really is no good set of best practices, or known ways to quantify security risk,” Tseitlin says. Without an agreed-upon set of metrics, it’s hard for a company’s chief information security officer to argue that spending $5 million more on a new security service will yield a known return on investment by taking data protection to a higher level, he says.
Such a set of generally accepted standards, like the GAAP accounting practices companies use to report their financial returns, is sorely lacking in the industry, Tseitlin says.
“Finance has GAAP. There’s nothing like that in security,” Tseitlin says.
