There Are Often Too Many Disjointed Systems Involved in Producing Particular Metrics
In my experience, there aren’t too many security organizations out there who think they’ve achieved the pinnacle of efficiency. Rather, most security organizations realize that, regardless of how much progress they’ve made towards improving maturity and efficiency, there is still more work to be done. This constant desire to improve is healthy and helps ensure that a security team doesn’t begin to fall behind to the detriment of its organization’s security posture.
There are many angles to be taken around improving efficiency. In the past, my pieces on the topic of metrics seem to have piqued the attention of readers. I see this as evidence of the interest that SecurityWeek’s audience has regarding the subject. As such, it seems fitting to offer seven ways to improve efficiency in your security metrics program:
1. Document: While it may not seem like it, documentation can greatly aid in improving metrics efficiency. Aside from the obvious benefits, documenting the source for each metric, the data required to calculate it, the steps involved in calculating it, the frequency of it, and the metric’s intended audience has another advantage. Taking the time to gather, understand, and write down the above parameters allows the security organization to study them. In some cases, inefficiencies around each metric may present themselves during the documentation process. In other cases, it may become clear that there is too much manual labor involved in reporting specific metrics. In still other cases, it may become apparent that there are too many disjointed systems involved in producing particular metrics. These are just some of the benefits that come from documenting the metrics process.
2. Consolidate data: One of the most time consuming parts of producing metrics on a regular basis is gathering, manipulating, and massaging the data. This is particularly the case when the data come from a large number of sources. This challenge is not unique to metrics, of course. Nonetheless, there are steps that can be taken here to improve efficiency. Can data be pulled from its source, normalized, and loaded into a centralized repository on a regular basis? Or, better yet, can it be pushed to a centralized repository continually? How can the centralized repository be configured to provide the required flexibility and power needed to facilitate the automated calculation of metrics? How can the number of stovepiped data sources required to produce metrics be greatly reduced?
3. Get away from email: Believe it or not, email is still the de facto source for too many types of data. Needless to say, if your security team sends and receives emails on a regular basis as a way of requesting and receiving data required for metrics, it’s a sign that there are large inefficiencies in the process. Identify the data being exchanged via email and understand if there is a more efficient way that data can be leveraged for metrics. This effort will pay large efficiency dividends.
4. Automate: It goes without saying that automation is a great way to increase efficiency in many areas of security, metrics included. Unfortunately, there is a key point about this topic that many security professionals miss. Automation shouldn’t be done for automation’s sake. The best way to leverage automation is to identify activities involved in the metrics process that are extremely time consuming and labor intensive. Those should be the first candidates for automation. Once those have been automated, the next most time consuming activities should be identified and so on. Approaching automation this way ensures that it will be used to improve processes and increase efficiency, rather than the opposite.
5. Streamline reporting: Do you report metrics in multiple different formats, on multiple different schedules, and/or to multiple different audiences? If the answer to one or more of these questions is yes, it’s likely that streamlining reporting could be a great way to bring increased efficiency to your metrics program. While it may never be possible to reduce or eliminate certain formats, schedules, or audiences, that doesn’t mean that efficiencies can’t be introduced. Examine your metrics closely to see if some are similar or overlapping and can be reused for different audiences. See if different formats can be consolidated or combined into one or a few standard formats. Investigate whether or not schedules can be aligned to each other, or, failing that, whether or not metrics can be calculated automatically to reduce the need to repeatedly calculate them manually for different reporting schedules.
6. Zero-in on audiences: You know that report you’re producing monthly for a particular audience? Yeah, that’s the one. Do they read it? Does it bring them value? Is it important to the work they do? Do they provide feedback on the material they receive or ask you any clarifying questions about it? If you were to change the content or format of the report, would anyone notice? If you don’t know the answer to one or more of these questions, it might be time to find out. Why bother investing precious resources in something that may not be necessary? It may very well be that you can provide a more valuable report to a particular audience with less effort. That is, if you even need to provide them a report at all.
7. Don’t overcomplicate: I’m a firm believer that the most elegant solutions are simple. If producing a set of metrics or a given report is tediously complicated, it may be a sign that something has gone terribly awry. Examine the use case for the metrics or report closely. Chances are, there is a simpler way to do it. Find that way and run with it.
Related Reading: Leveraging Gap Analysis to Drive Security Metrics
Related Reading: Using Gap Analysis to Fix a Leaky Enterprise
Related Reading: CISOs and the Quest for Cybersecurity Metrics Fit for Business