The initial results of a survey conducted by a Google researcher show that a large part of public vulnerability research involves fuzzing tools.
Google researcher Michał Zalewski, also known as “lcamtuf,” has carried out a study to determine how most security flaws are discovered, and by whom.
“Our industry tends to glamorize vulnerability research, with a growing number of bug reports accompanied by flashy conference presentations, media kits, and exclusive interviews,” the expert wrote in a blog post. “But for all that grandeur, the public understands relatively little about the effort that goes into identifying and troubleshooting the hundreds of serious vulnerabilities that crop up every year in the software we all depend on.”
lcamtuf has analyzed public flaws that have been assigned Common Vulnerabilities and Exposures (CVE) identifiers. The researcher compiled a list of high-impact vulnerabilities affecting commonly used software, and sent out anonymous surveys to the individuals who reported them.
The survey asked respondents about how they found the issues (manual analysis, automated bug discovery, or luck), the tools and methods that had been used to discover the flaws, the method used to demonstrate the impact of the issues, the channel used for disclosure, communications with the vendor, motivation, and time spent on the project.
Zalewski says the current response rate for the survey is roughly 80 percent and he expects to come up with more detailed conclusions over the next two months. Based on the responses received so far, the researcher has determined that fuzzers are omnipresent, the most widely used being American Fuzzy Lop (AFL), an open source, coverage-assisted fuzz testing tool developed by Zalewski himself.
While static analysis frameworks, concolic testing and other automated tools are often used in academic research, the experts who took part in lcamtuf’s survey haven’t relied on such resources.
On the other hand, memory diagnostic tools such as ASAN and Valgrind are used by many bug hunters to analyze vulnerabilities, and the Google researcher calls them “an untold success story of vulnerability research.”
Interestingly, the survey reveals that most serious vulnerabilities are uncovered by people who work on this full time, employed by vendors. Hobbyists and bug bounty hunters follow closely, lcamtuf said.
Finally, the study shows that only a small number of serious flaws are disclosed outside a vendor advisory. According to the researcher, this makes it “extremely dangerous to rely on press coverage (or any other casual source) for evaluating personal risk.”
Zalewski has pointed out that not all vulnerabilities make it to the public domain; some flaws are found by internal security teams and they don’t reach customers.
“Since it is difficult to collect comprehensive and unbiased data about such programs, there is always some speculation involved when discussing the similarities and differences between this work and public security research,” the expert said.