If you would like to be notified of when Deb Shinder releases the next part in this article series please sign up to our WindowSecurity.com Real-Time Article Update newsletter.
If you would like to read the first part in this article series please go to Big Data: The Security Perspective (Part 1).
Introduction
There are at least two distinct aspects involved when we talk about the big data phenomenon and security. In Part 1 of this two-part series, we discussed the challenges of securing the data itself, when it exists in exceptionally large volumes that are often distributed across multiple physical locations. In this, Part 2, we will turn to a discussion of expert opinion regarding whether and how big data analytics can be used to detect and predict attacks, thus making it a tool for enhancing the overall security of your network.
The first question: Is it all a bunch of hype?
As with most popular IT buzzwords, much of the “buzz” surrounding big data analytics is put out there by vendors in an attempt to sell you products and/or by fanboyz (or fangirlz) who are so infatuated with a particular technology solution that they’re blind to its flaws or limitations. Because security is a major concern for most organizations (and mandated for those who fall under the purview of various governmental and industry regulations), this is particularly true in regard to big data security analytics.
As Randy Franklin Smith explains in his webinar on Cutting through the Hype: What is Big Data Security Analytics, once you sort through the exaggerated claims, you’ll find that there really are some important ways in which security efforts can benefit from big data analytics.
Gartner VP/fellow Neil McDonald said last summer that big data analysis to identify potentially dangerous network activity is becoming a necessity rather than a luxury. He predicted that it will be in use by 40% of enterprises by 2016. And some believe it’s already here. A study done by ESG in late 2012 showed that 44% of enterprises would already classify their security analytics as “big data” and the same percentage believed they would be in that position within two years.
But what does that mean, exactly? McDonald sees it as a way of combining security and operations data sets in something he refers to as “BI for IT.” In other words, we’re talking about massive amounts of security data that, as with other types of big data, is difficult or impossible to process using traditional methods and applications.
The evolution of security information management
To understand how big data security analytics has come about, we first need to look at the concept of SIM (security information management), which is itself a part of SIEM (security information and event management). The SIM component is concerned with storing, analyzing and reporting on security log data, while the SEM (security event management) component handles the monitoring and collection of the data that goes into the logs and real time notification and alerts based that information.
Big data security analytics is an outgrowth of SIM/SEM/SIEM – it goes further by collecting a more comprehensive set of data throughout the enterprise, and then uses advanced pattern recognition, advanced statistical analysis, heuristics and other behavioral analysis to detect deviations from the norm. This is aimed at identifying and preventing types of attacks that more traditional tools miss, such as APTs (Advanced Persistent Threats).
Whereas in the past, companies have taken a reactive approach to security – both in reaction to the threats themselves and in reaction to government and industry compliance mandates – the big data security analytics approach is more proactive. It involves collecting the data necessary to assess risks before or during an attack, rather than waiting to react after an attack has already occurred and damage has been done. Ultimately, the goal is to integrate security data into the overall business intelligence and make it part of the decision-making process.
What companies have discovered, in the effort to improve their security models and make security a more integral part of their business processes, is that the sheer volume of data and the complexity of new threat types makes it difficult to use the tools they were able to use in the past. There are many such tools out there, but they often don’t work together so that the processing of analyzing information is fragmented. Big data security analytics will involve collecting and processing terabytes, petabytes or even more of security-relevant information from various sources such as log files and applications that monitor and record system, network and user behaviors.
New technologies such as mobile and cloud computing have created new security challenges. At the same time attackers are growing more sophisticated, IT departments are losing much of the iron-clad control they had over their networks in the past. The hybrid IT trend results in combining on-premises resources with those hosted by cloud providers, and the BYOD trend means users are choosing and bringing in smart phones, tablets and laptops that are not owned and issued by the company. All of this creates an ever-increasing amount of information that must be analyzed in order to keep a handle on the security situation at all times. And that’s where big data security analytics come in.
Seeing the big picture
Current security analysis tools tend to be specialized; each is designed to detect particular types of threats, such as malware or network intrusion attempts. The problem is the same one that occurs in medicine, for example, if a patient is seen only by a handful of specialists and there is no “gateway” physician such as a family practitioner or internist who is trained to look at the “big picture” of the patient’s overall health.
Specialists (in any field) tend to focus almost exclusively on their own specialty area. If you come into a financial windfall, a tax attorney would tend to see only the tax implications, a family law attorney would focus on protecting the money in a change in your marital status, and so forth. While specialized security analysis tools have their place, large organizations need (and most are lacking) the capability to get an overall view of security across the enterprise.
Something else that keeps us from seeing the big picture is limiting the types of data we analyze. Restricting it to traditional security log files can cause you to miss early indications of security problems or attacks. Even unstructured data sources such as blog and social networking posts or emails can provide clues. Big data is all about handling both structured and unstructured data – but the more types of data you collect for analysis, the greater the total volume of data that must be analyzed, and that leads right back around to the need for new and better tools that can handle these data types and scale to handle the increased amounts of data.
Incorporating game theory
Security is serious business, so it might seem frivolous to use the word “game” in relation to developing your security strategy – but in actuality, game theory is about strategic decision-making, and that applies to much more than just play time. It’s a mathematical model for predicting behavior that has been used in business and politics, biology, philosophy and computer science.
The use of game theory in computer security strategizing is too complex to delve into in any detail here, but if you’re mathematically inclined, you might find this paper interesting: Game Strategies in Network Security by Kong-wei Lye and Jeannette Wing, Carnegie Mellon University.
For those who prefer a more simplistic approach, there’s another way that you can look to games for examples of the direction we should be headed in adapting enterprise security strategies to increasingly complex networks and increasingly sophisticated threats. In most games of strategy, from football to chess, it’s important to plan for both defensive and offensive tactics. This can be seen as an extension of the need for both reactive and proactive measures.
Security has traditionally been all about the defense. We even call our security plan “defense in depth.” Defensive security measures include firewalls, anti-malware and anti-virus software, security updates to address vulnerabilities in software, server “hardening” and so forth. These are not going away anytime soon – but big data security analytics is about adding a more “offensive” strategy that includes continuous monitoring and automation of the response processes.
Detection is key, with a high percentage of security breaches being undetected by the organizations at the time of occurrence and a low percentage of security professionals expressing confidence that they would know when systems were breached. But detection is only half the battle – the difference between catastrophic impact on the organization’s productivity, reputation and bottom line vs. minimal impact depends on what happens when the breach is detected – and how quickly it happens.
Even with modern SIEM solutions, manual (human) examination of the data is often necessary at some point, in order to eliminate the false positives. Better analytics tools would be able to reduce that administrative overhead and automate more of the process, which in turn would speed up response time. Of course, when time is of the essence and machines are doing the work, the performance of the hardware and software performing the analytics takes on even more importance. The distributed nature of big data lends itself to massively parallel processing, whereby the processing is spread out over multiple systems/processors to achieve processing speeds that wouldn’t otherwise be possible.
Summary
In Parts 1 and 2 of this series, we’ve provided an overview of how massive amount of data can be secured and what big data security analytics is about and how it works. In Part 3, we’ll wrap up this three-part article by taking a look at some of the commercial big data security analysis solutions that are currently out there, as well as what may be on the horizon in this new category of security product.
If you would like to be notified of when Deb Shinder releases the next part in this article series please sign up to our WindowSecurity.com Real-Time Article Update newsletter.
If you would like to read the first part in this article series please go to Big Data: The Security Perspective (Part 1).
