By now, many security and fraud professionals understand the risks that bots introduce to our online applications and to our businesses in general. In a previous piece, I discussed and summarized some of these risks to help security and fraud teams understand the need to articulate the threat of bots to executives and the board in their own language. Indeed, this type of communication has been increasingly common, resulting in higher awareness around the bot problem.
Not surprisingly, as awareness of the bot problem has grown, so has the stream of marketing material aimed at enterprise buyers. Regardless of which risks security and fraud teams are concerned about, they need a way to cut through the marketing rhetoric in order to properly evaluate bot solutions. How can enterprise buyers objectively evaluate bot solutions? How can they evaluate who can truly deliver what they promise, what approaches will be effective in their environments, and which vendors will be able to stay one step ahead of the evolving threat landscape?
While there are likely many different approaches here, I’ve highlighted a few points that I believe are important ones for enterprises to consider when evaluating bot solutions:
- R&D: Lots of bot management vendors collect telemetry data. But what different vendors do with that data has a huge impact on the efficacy of their solutions. Continually analyzing, dissecting, and investigating the telemetry data is a must for a bot management solution to be effective. Questions that need to be asked consistently and continuously include: What do the data tell us? What is the right data to collect? How can we reliably and accurately differentiate between human and machine traffic? Successful R&D also includes identifying gaps in telemetry data and understanding what additional telemetry data needs to be collected in order for the solution to perform at maximal efficacy.
- Machine learning: Machine learning is an important part of detecting and understanding which traffic is originating from a human and which traffic is originating from a bot. Many vendors tout their machine learning capabilities and the power of their models. Of course, good models are important, and many of the top vendors do have good models. So what differentiates the most effective bot management solutions from the rest? The secret lies in the data – the better the data that goes into a model, the more accurate and reliable the predictions that come out of the model. Even the greatest machine learning model will not accurately differentiate between human and automated traffic if it does not receive the appropriate data as input.
- Verification: During my years on the operational side, there were more than a few occasions where vendors insisted that we turn on their latest and greatest detection rules and/or signatures. Not surprisingly, in many cases, this resulted in a deluge of false positives and noise that clogged up the work queue. In one instance, the avalanche of false positives even crashed the SIEM. The best bot management vendors test and verify their rules thoroughly before releasing them. To those vendors, bombarding a customer with a large number of false positives after an update would be considered a colossal failure.
- Obfuscation: Obfuscating the bot management solution’s Javascript to hide it from attackers is essential. I am often amazed by how many vendors do not do this, thus making it significantly easier for attackers to know they are hitting a page with a bot management solution. The attackers can then easily circumvent the solution – as one example, the attackers may simply modify the page, remove the bot management solution’s Javascript, and proceed with their attacks as if there were no solution in place at all. Obfuscation is not a once and done process – it is an iterative one. Proper obfuscation that can withstand attacker workarounds requires studying attackers, reverse engineering their tactics, techniques, and procedures, and constantly releasing new and modified obfuscation.
- Advanced analysis: Last, but certainly not least, incorporating learnings into the bot management solution vastly improves efficacy. Many vendors, unfortunately, develop and market solutions that address a certain level of sophistication. Yet, they don’t continuously study attacker retooling, incorporate that learning into their solutions, and improve their offering. This leads to the result that bot management solutions will sometimes be somewhat effective for a few weeks until the attackers realize that their target has implemented a bot management solution. At that point, the attackers often retool, and if the solution is not able to handle the added level of sophistication, the bot management solution becomes completely ineffective. The best bot management vendors continuously perform off-line or second stage analysis to ensure that their solutions remain consistently effective.
When it comes to bot management solutions, iterative solutions reign supreme. Those vendors that study attackers and continually feed that knowledge back into the solution have much higher efficacy rates than those that do not. Similarly, vendors that are diligent in collecting the best and the right data, vetting rules, and ensuring their solutions are secured from attacker tampering do far better than those that don’t. These points, along with others, are important for enterprises to keep in mind as they evaluate a bot management solution.
Related: Bringing Bots and Fraud to the Boardroom
Related: All About the Bots: What Botnet Trends Portend for Security Pros
