What is the point of analysis anyway? Perhaps this sounds like a bit of a shocking or radical question, but I’d argue that it is one that sorely needs to be asked — and answered. What do I mean by that? Allow me to elaborate.
Analysis, sometimes referred to as investigation or forensics, is an integral part of the incident response process and the incident handling life cycle. Many security professionals perform analysis in one form or another or oversee analysis functions during the course of their daily work schedules. One thing I’ve noticed over the course of my career is that analysis isn’t always well understood for the critical function it is within a larger strategic process. Furthermore, the overall purpose of analysis isn’t always well understood either. Because of this, analysis is something many organizations struggle with, or at the very least find challenging.
Before we can understand the purpose of analysis within the greater incident response process, we need to better understand the decision-making process. I’m hoping that the reasons for why this is the case will become clearer as we get further along in this piece.
I’ve always been fascinated by the manner in which different people make decisions. There are some people who can quickly understand the decision that needs to be made, gather the facts relevant to that decision, make a decision about what action to take, and subsequently act decisively. Other people seem to meander about, unable to grasp what they’re actually trying to decide, collect hordes of information irrelevant to the decision that needs to be made, cannot make a decision, and are subsequently unable to act decisively.
I’m sure we all know people who fall into one of these two categories, or perhaps somewhere in between. But why is it that some people are so much better at making decisions than others? The decision-making process is one that has been studied quite a bit in psychology, as well as elsewhere. For those of us that are not psychologists, Wikipedia offers an executive summary of the decision-making field, as well as a few different models for the decision-making process:
For discussion purposes, let’s work with Kristina Guo’s six-part DECIDE model of decision-making, published in 2008:
1. Define the problem
2. Establish or Enumerate all the criteria (constraints)
3. Consider or Collect all the alternatives
4. Identify the best alternative
5. Develop and implement a plan of action
6. Evaluate and monitor the solution and examine feedback when necessary
If the DECIDE model reminds you a bit of analysis, investigation, or forensics, I’m not surprised. If it doesn’t, I’d argue that it should. Why? It all comes back to my original question. What is the point of analysis anyway?
The point of analysis is to converge to a decision and subsequently take action on that decision. Within the framework of the incident response process or the incident handling life cycle, that decision generally boils down to two questions: Is response necessary? And if so, what type or level of response is necessary? Of course, to answer those questions intelligently, we need to be able to do analysis properly.
When we understand analysis within this context, we can begin to understand why so many organizations find analysis to be so challenging. It’s all too easy to get bogged down in the weeds, details, and minutiae and lose sight of the larger goal of analysis. How can an organization avoid this? Let’s adapt Guo’s DECIDE model to the security profession:
1. Define the problem: What exactly are we trying to accomplish? For example, if we’re trying to vet and quality an alert, when would this be considered complete? Or, as another example, if we’re trying to piece together the puzzle of what exactly happened before, during, and after an intrusion, when would we consider this puzzle assembled? Or, as yet another example, if we’re trying to identify gaps in our telemetry, at what point would we consider our work thorough enough? This is arguably the most important phase of the decision-making process, as it can prevent us from going down one or more rabbit holes.
2. Establish or Enumerate all the criteria (constraints): How do we evaluate whether or not we are progressing towards the goal we set in the first phase? How will we understand when we have arrived at a conclusion? In this phase, we essentially create a decision-making matrix for ourselves. Later on in the process, we will use these criteria and the matrix we build from them to make a timely and informed decision.
3. Consider or Collect all the alternatives: In this phase we begin to dig into the data to understand what the data tell us regarding the criteria we established in the previous phase. Unfortunately, this phase is where many organizations jump into analysis, which is precisely why they often struggle with it. Why? As I’ve written in previous pieces, the questions are more important than the answers. The answers will flow naturally if the right questions are asked. How do we know which are the right questions to ask? We generate them based upon the criteria we established in the second phase toward the goal of accomplishing the goal we set in the first phase. If we jump right into this phase, we miss out on all the benefits the first two phases bring us.
4. Identify the best alternative: If done properly, analysis, investigation, or forensics will provide factual information upon which we can evaluate our criteria. Perhaps there is more than one course of action that can be taken. We need to choose the best course of action based upon the data from the third phase, the criteria from the second phase, and our ultimate goal as defined in the first phase.
5. Develop and implement a plan of action: This is where we set out to address the two fundamental questions I mentioned earlier. Is response necessary? And if so, what type or level of response is necessary? If we’ve worked through the first four phases correctly, we should be able to answer these questions intelligently. A timely and informed decision should flow naturally with the evidence to back it up.
6. Evaluate and monitor the solution and examine feedback when necessary: It goes without saying that our work is never done. Our goals can always be adjusted as business needs and the threat landscape change. Our criteria can always be tweaked to ensure they continue to fit the reality we face on a daily basis. The people, process, and technology we use to collect our data points can and should change over time to meet our changing n
eeds. And so on.
Analysis is something that organizations sometimes struggle with. It’s often the case that this struggle results from an improper or incomplete understanding of what analysis really is and what its essential purpose is. Understanding where analysis fits within the strategic framework of the incident response process and the incident handling life cycle can help an organization improve its analysis capability, its incident response process, and its security posture as a whole.
