Security Experts:

Evaluating an Intelligence Vendor: Key Questions to Consider

Choosing an Intelligence Offering is a Decision That Shouldn’t be Taken Lightly

Choosing the right intelligence vendor is in many ways like finding a needle in a haystack. There are virtually countless vendors and seemingly endless intelligence offerings. Given these conditions, it can be easy for even the most tenured professionals to lose sight of their organization’s needs and, more importantly, how to assess which vendor or offering is best suited for those needs.

The following questions are designed to help security and risk professionals accurately evaluate intelligence vendors and offerings:

1. What type(s) of intelligence do you offer? 

With so many intelligence offerings, it’s important for prospective consumers to have a sense of what intelligence—if any—a vendor actually provides. Common answers one might hear from vendors include:

● Threat Intelligence: this is often used as an umbrella phrase to encompass many other types of intelligence. And because not all of these types are created equally, it’s crucial for prospective consumers to dig deeper.

● Cyber Threat Intelligence (CTI): CTI has long been integral to the success of any network defense or perimeter security initiative. But since it focuses primarily on cyber threat detection and indicators of compromise (IoCs), CTI is generally only suitable for supporting tactical cybersecurity use cases.

● Business Risk Intelligence (BRI): Unlike CTI, BRI provides a strategic decision advantage that supports not just cybersecurity teams but all business functions. As such, BRI is suitable for organizations seeking to support a diverse array of use cases and address enterprise-wide risk.

2. What data sources are used to produce the intelligence?

The best intelligence is derived from relevant, high-value data sources. Indeed, the primary facets of the cyber (and often physical) threat landscape tend to originate and develop within the confines of various underground communities. Often, it’s only after a potential threat becomes a tangible security incident or breach that any indicators of the threat’s existence reach the open web. 

Although Deep & Dark Web (DDW) and open web data are useful in certain instances, the most successful intelligence vendors recognize that what is critical is to have the technology and subject matter expertise to be able to gather data at scale from the most critical environments on the internet, many of which are extremely difficult to access and reside within the DDW. This nuance is why it’s important for prospective consumers to understand the data sources from which an intelligence offering is produced. 

3. How many intelligence analysts do you have? What are their qualifications and areas of expertise?

The size and capability of a vendor’s analyst team can help shed light on the quality and relevance of its intelligence offerings. Typically, a vendor with few human analysts needs to rely heavily on automation to produce data, but that approach leaves contextualizing the data into intelligence to its clients. While vendors should strive to automate mundane tasks such as routine data collection so their analysts can focus on complex problem-solving and analysis, vendors that rely solely on automation aren’t really producing intelligence at all. Interpreting, contextualizing, and processing raw data into intelligence requires ample human expertise that cannot be effectively replaced with pure automation.

For example, the DDW is home to countless communities where adversaries congregate and develop malicious schemes. Since many of these adversaries don’t operate in English, a vendor’s intelligence analysts need to possess the necessary linguistic skills. And in many cases, simply being fluent in Russian, Arabic, Mandarin, Turkish, Farsi, Spanish, French, and other languages, isn’t enough—analysts also need to have a keen understanding of the cultural nuances, social norms, idioms, and slang that exist within different DDW communities. Despite promising advances in artificial intelligence and other automated technologies, such tools aren’t yet capable of mimicking this level of human expertise.

4. How do your customers consume your intelligence?

Intelligence is most easily and effectively consumed when it is “finished.” Indeed, finished intelligence is derived from relevant data that has been contextualized, deeply analyzed, and packaged along with all the details needed to support decision making and spur action. In other words, finished intelligence is actionable in and of itself, and doesn’t require users to seek additional context or analysis before making a decision.

However, not all vendors provide finished intelligence. Many focus solely on delivering indicators of compromise (IoCs) and keyword alerts. While valuable, these offerings tend to require customers to conduct additional research and analysis in order to determine the extent to which an IoC or alert is even relevant to their organization. 

5. What types of use cases does your intelligence support?

Different types of intelligence are typically suitable for different business functions and use cases. As I mentioned, CTI, for example, can support cybersecurity and IT teams, while BRI is far more strategic and diverse. When evaluating any vendor or offering, consider your organization’s intelligence needs not just in the present, but also how they might change as the business grows, scales, and evolves in the long-term. Learning what types of use cases are common among a vendor’s customers can provide additional insight into just how suitable an intelligence offering might be for your organization. 

In addition to supporting traditional cybersecurity use cases, the right intelligence might also help reveal, for example, malicious actors seeking to compromise your executive team’s physical safety, threats posed by malicious insiders, unknown security vulnerabilities that exist within your company’s supply chain, or emerging fraud schemes targeting your company’s customers. 

Ultimately, the intelligence vendor landscape will always be complex and ripe with seemingly indistinguishable offerings. The five aforementioned questions can help security and risk professionals better assess how well an offering aligns with their organization’s needs, but I must emphasize that choosing an intelligence offering is a decision that shouldn’t be taken lightly. Regardless of what type of intelligence your organization opts to consume, remember that the value lies not in how it is marketed but rather in the extent to which that intelligence supports timely and effective decision making.

view counter
Josh Lefkowitz is the CEO of Flashpoint, which delivers Business Risk Intelligence (BRI) to empower organizations worldwide with meaningful intelligence and information that combats threats and adversaries. Lefkowitz has worked extensively with authorities to track and analyze terrorist groups. He has also served as a consultant to the FBI's senior management team and worked for a top tier, global investment bank. Lefkowitz holds an MBA from Harvard University and a BA from Williams College.