This is the beginning of a series of postings I’ll be doing on security metrics. It’s a topic that I don’t think we, as a community, have a particularly good grasp of – probably because security, as a field, is only just beginning to professionalize to the point where (in some markets) it’s getting more than a nod as a necessary evil. I can’t even imagine the number of times in my career that I have heard a security practitioner say something like, “We have to speak to executives in the language of business!” which often gets mistaken for “use lots of PowerPoint and buzzwords” but which really means: Be able to quantify what you’re talking about. And that’s where metrics come in.
 |
| Lord Kelvin – If you really understand metrics, maybe you’ll have a unit of measurement named after you, like he did. What is the unit of measurement for computer security, anyway? |
During the course of this series I’m going to hit on a range of topics from why metrics are important and what they are, to bottom-up analysis of your business process, and top-down analysis of your mission, then the problems of normalization and data-sharing, as well as suggestions on how to present data. I’m not going to pretend to you that metrics are insanely exciting as a field, because they aren’t. On the other hand, metrics are how you learn where you’re going and, as the great quip goes, “If you don’t know where you’re going, how will you know when you get there?” William Thompson, Lord Kelvin, once observed that “If you can not measure it, you can not improve it.” That, in short, is almost all we’d have to say about metrics in computer security – our goal is to improve, and right now many of us are following popular fads or traditions, instead of seriously studying what we’re doing.
Thompson also said, “There is nothing new to be discovered in physics now, All that remains is more and more precise measurement.” which goes to show you that it’s a bad idea to think that your field of study is immune to change. Computer security is hardly immune to change – in fact it’s more characterized by constant flux than anything else – which is what makes it so hard: we are chasing a mixture of configurations and practices that surround a variety of applications and protocols all of which are mutating at a very high rate. When I started in this field the first firewall I built needed to effectively handle 5 protocols (DNS, NNTP, SMTP, FTP, and TELNET) and it didn’t even need to support a full command-set for those protocols. Today, the complexity of security has grown out of pace with the number of applications and protocols.
In other words, it’s a good time to be alive. Enjoy the ride.
Next up: Why should you care about metrics? We’ll look at where metrics fit into the organizational structure and why they are an important part of executive knowledge.