I recently wrote in this space about the rise of the phrase “continuous monitoring” and the confusion it causes. In a nutshell, federal organizations, facing FISMA mandates, have very clear guidance on the meaning of the phrase. What they mean by it makes a ton of sense, but doesn’t match what many people outside the Beltway think of first when they hear the term.
This is a pity, because the NIST Risk Management Framework behind all this represents some excellent thought leadership on the most important problem in security today: complexity. The idea boils down to the application of automation to situational awareness, to help tame the burgeoning complexity of our infrastructure, to help us understand where our defenses are weak, and ultimately, to help us better withstand the inevitable breaches. Evidence shows they are on the right track – attackers are getting in easily, because our environments are too complex, our defensive counter-measures aren’t covering all the gaps, and the attackers can persistently seek out the gaps we leave open. So the FISMA effort is trying to drive federal agencies to up their game – don’t just fill in audit paperwork once every three years, get to a point where you can continuously see and understand your risk posture. All good, right?
The problem seems to be in the label. If you take the phrase continuous monitoring out of context of the descriptions of the RMF, and interview randomly selected security professionals who don’t work for government agencies, they seem to break it down as “hmm – continuous means real time, and monitoring means snooping, so we’re talking about live wire packet capture on a grand scale”. Now, as we know, the NSA has been busted for fantasizing about doing exactly that, but that’s called PRISM – that’s something else. As I discussed in a previous article, the monitoring in NIST RMF is awareness of yourself – of your controls, of your readiness and to the extent possible, of your intruders.
What about continuous? That choice of term begs a rather obvious question – how fast? Here, I find everyone tends to resort to hand waving. Enthusiasts say real time, whereas pragmatists say near real time, and NIST documents are careful to stay in the second camp. Nobody really ventures to say what these terms mean. In my travels, I’ve found the smartest folks for this discussion are military – that community is quite sensitive to the differences between too slow, fast enough or immediate. In conversations with them, I’ve found it’s useful to define four speeds of reaction, differentiated by how decisions are made. Think of activity as a loop – get some input, think about it, act, and then get some more input.
The fastest loop in this context is pure machine decision making – the machine senses, the machine reacts. This is the only category that I think we should call real time (in the sense of an RTOS). And as a result, if we don’t trust the machine to operate totally on its own, we can’t really talk about real time. Consider how confident you would need to be to deploy automated machine guns to guard your secrets. Or think about why we still have pilots on aircraft – we clearly can build planes that fly themselves, but interestingly, nobody will buy tickets to ride on them.
Speaking of air travel, one step slower in the hierarchy of awareness loops is the kind of decision making used by well-trained, agile fighter pilots. The Air Force has studied this in great depth, and has a name for it – the OODA loop, named for the way pilots process information during a dogfight. The better pilots Observe, Orient, Decide and Act faster than the worse ones – truly “the quick and the dead”, in either sense of the word quick. The OODA loop is certainly fast, but it’s not machine speed. It doesn’t need to be – there’s a human at the joystick.
There’s a slower cycle – the speed of organizational decisions. These don’t happen as fast as pilot reflexes, and generally involve at least a little reflection. A cycle time of daily would be an idealistic goal for a lot of business decision processes – a few can move that fast.
The fourth possible speed is the current speed – the way we do things before we try to automate and up our game. In this case, we’re talking about the assessment of security controls, and the original, pre-reform FISMA requirements amounted to generating results once every three years. Everyone agrees that this is far too slow.
So how fast should continuous monitoring be? To be practical about it, we’d be doing spectacularly well if we could get from current speed to business speed or daily. The good news is the technology of automation is ready to take us there and to even faster speeds – to our equivalent of the OODA loop. Some fools, and vendors who prey on them, still insist on talking about real time, but I have to be frank: they are talking nonsense. Automation of situational awareness shouldn’t target real time, for the simple reason that machines aren’t aware of anything. They compute really well, but where’s the HAL 9000 from “2001: A Space Odyssey”? It turns out that decades of AI work on machine awareness have amounted to “that problem is harder than we thought – a lot harder”. (We can’t even get The Frame Problem right yet, and if we can’t do that, we’re not making much headway.) If we won’t deploy automatic machine guns to guard banks, why would we want truly real-time, machine-based response to ongoing attacks?
As an aside, there are apparently some automated machine guns deployed along the Korean DMZ, but it’s pretty clear this is due to a rather unsettling lack of concern for collateral damage. In our commercial networks, we’re not quite so willing to have robots shooting at Bambi.
Bringing it back to continuous monitoring, when understood as it’s meant by NIST, it really is the right idea, at the right time. For federal agencies it’s mandatory, and if you’re operating critical infrastructure or working with the government, you can expect these standards to extend to you soon. Wise organizations get a step ahead, not least because the NIST guidelines are good guidelines – they point to the right uses of automation to solve the real problems we have today that allow far too many breaches to succeed. The only stumbling block is the unfortunate misreading of the label. Continuous doesn’t mean real time, and monitoring doesn’t simply translate to “buy more sensors”. Continuous monitoring is a call for maturity in our risk management and assessment processes – check that we’ve locked the doors and that the barn door is closed before the inevitable attack comes. Automated, closed-loop, machine-based, real-time defensive responses, without a human in the loop are just a trope in the movie business, and should remain as such. Skynet is not the way forward!
Related Reading: Continuous Monitoring and the Confusion It Causes
Related Reading: Continuous Monitoring for Year-Round Coal Avoidance
