Security Experts:

Connect with us

Hi, what are you looking for?

SecurityWeekSecurityWeek

Malware & Threats

Facebook Discusses its ‘ThreatData’ Security Framework

Facebook shared details recently about an internal security framework the company uses to collect and leverage information about malicious activity on the Web.

Facebook shared details recently about an internal security framework the company uses to collect and leverage information about malicious activity on the Web.

Known as ThreatData, the framework is composed of three primary parts: feeds, data storage and real-time response. Feeds collect information from a specific source, and are implemented through a light-weight interface. The data can be in nearly any format and is transformed by the feed into a simple schema Facebook calls ThreatDatum.

Once the feed has transformed the raw data, it is fed into two of the social network’s existing data repositories: Hive and Scuba. Hive is used to answer questions based on long-term data, such as whether or not the threat has been seen before, while Scuba focuses on answering questions about the present day.

“Given the pace of criminals today, one of the hard parts is actually keeping track of all the data related to malware, phishing, and other risks,” blogged Mark Hammell, Internet threat researcher at Facebook. “We wanted an easier way to organize our work and incorporate new threat information we receive so that we can do more to protect people.”

“When we began sketching out a system to solve this problem, we encountered issues others have faced: every company or vendor uses their own data formats, a consistent vocabulary is rare, and each threat type can look very different from the next,” he continued. “With that in mind, we set about building what we now call ThreatData, a framework for importing information about badness on the Internet in arbitrary formats, storing it efficiently, and making it accessible for both real-time defensive systems and long-term analysis.”

To help, Facebook built a processor to examine ThreatDatum at the time of logging and act on new threats. For example, all malicious URLs collected from any feed are sent to the same blacklist used to protect people on Facebook.com. Additionally, “interesting” malware file hashing are automatically downloaded from known malware repositories, stored and sent for automated analysis, Hammell explained.

As part of the ThreatData framework, the company is expanding its capabilities to “decorate the data with additional context at logging time,” the researcher blogged. “For example, we add Autonomous System, ISP, and country-level geocoding on every malicious or victimized IP address logged to the repository. As a result, we can understand where threats are coming from, arranged by type of attack, time, and frequency.”

“We’re constantly finding new ways to improve and extend the ThreatData framework to encompass new threats and make smarter decisions with the ones we’ve already identified,” he blogged.

Written By

Click to comment

Expert Insights

Related Content

Malware & Threats

Microsoft plans to improve the protection of Office users by blocking XLL add-ins from the internet.

Cybercrime

CISA, NSA, and MS-ISAC issued an alert on the malicious use of RMM software to steal money from bank accounts.

Cybercrime

A recently disclosed vBulletin vulnerability, which had a zero-day status for roughly two days last week, was exploited in a hacker attack targeting the...

Cybercrime

Chinese threat actor DragonSpark has been using the SparkRAT open source backdoor in attacks targeting East Asian organizations.

Cyberwarfare

Russia-linked cyberespionage group APT29 has been observed using embassy-themed lures and the GraphicalNeutrino malware in recent attacks.

Malware & Threats

Security researchers are warning of a new wave of malicious NPM and PyPI packages designed to steal user information and download additional payloads.

Cybercrime

No one combatting cybercrime knows everything, but everyone in the battle has some intelligence to contribute to the larger knowledge base.

Malware & Threats

Cybercrime in 2017 was a tumultuous year "full of twists and turns", with new (but old) infection methods, a major return to social engineering,...