Earlier this year, Twitter admitted they lost personal information on 250,000 or so users to hackers. Other organizations, including the New York Times and the Federal Reserve, reported hackers had been inside their systems. The list of high profile hackings is so long that perhaps the day has come when companies should expect they will get hacked.
Already, businesses must comply with policies about document access and document retention under Sarbanes-Oxley and other regulations. They may soon be required to do more to prevent hacking—or face liability when they don’t. Fortunately, businesses may be able to eliminate one angle of attack using big data.
What exactly is the link between big data and hacking? We live in a world where, thanks to the combination of technologies such as Hadoop and NoSQL with big data service providers, companies can now store and analyze most, if not all, of their data. (Previously companies could store and analyze only a fraction of their information with huge price tags if using legacy storage systems.)
If this new data is not being properly managed, viruses and malware hidden inside company data can’t be located. Delayed action malware is designed to “sleep” in the data until it’s no longer being managed or monitored by the company. When no one is “watching”, the malware wakes up and contacts hackers to ask them, “What can I do for you, Mr. Hacker?” When companies can’t monitor their total data store, including their massive pool of historical data, Mr. Hacker will.
Thankfully, data scientists can now build analytic applications to detect problems before they occur. We’ve seen this first hand in the last several months.
Two major enterprises we work with carry an enormous amount of Internet traffic through their networks. In the past, that data flew by and they couldn’t search for abnormalities. Now they use big data analytics to store complete network traffic data sets, and can catch abnormal traffic for closer examination. For example, they can find compromised machines while they lie dormant, detect hackers trying known approaches to breaking in and testing new ones, map the spread of infections to limit damage, and identify opportunities to cut off attacks in progress.
The bottom line is this: If large companies are not utilizing big data technologies in their approach to hacker defense, then they are exposing themselves to needless risk and costs. Ultimately, this impacts the consumer, whose personal information has been turned over to the hands of nameless, faceless hackers—the very hackers who invest in cutting edge technology every day.
If hacking can be prevented, will corporations be forced to take steps to do so? From our perspective, we are on the cusp of an era where the answer will be yes.
