A team of researchers has introduced the concept of smart greybox fuzzing, which they claim is much more efficient in finding vulnerabilities in libraries that parse complex files compared to existing fuzzers.
Fuzzing is used to find software vulnerabilities by sending malformed input to the targeted application. If the software crashes or behaves unexpectedly, it could indicate the presence of a security flaw. There are three main types of fuzzing: blackbox fuzzing, where the tester has zero knowledge of the target; whitebox fuzzing, where there is full knowledge of the target and the testing is done through the source code; and greybox fuzzing, where the tester has some knowledge of the targeted application.
Five researchers from the National University of Singapore, the Monash University in Australia, and the University Politehnica of Bucharest in Romania have been looking at ways to improve greybox fuzzing and they claim to have achieved impressive results.
Building upon the American Fuzzy Lop (AFL) fuzzer developed by security expert Michał Zalewski (aka lcamtuf), the researchers created a tool called AFLsmart, which implements what they call smart greybox fuzzing (SGF).
According to the experts, AFLsmart is highly efficient in analyzing libraries that parse structurally complex files, such as audio, video, image, document and database files.
In coverage-based greybox fuzzing, the fuzzer is provided a seed file and it randomly flips, deletes, copies or adds bits in order to generate new files that can be parsed by the tested library in order to find potential vulnerabilities. The problem, however, is that in the case of complex file formats, bit flips do not generate valid files.
The researchers have overcome this challenge by defining what they call “innovative mutation operators” that work on the virtual file structure rather than the bit level, which helps ensure that files remain valid.
“We introduce a novel validity-based power schedule that enables SGF to spend more time generating files that are more likely to pass the parsing stage of the program, which can expose vulnerabilities much deeper in the processing logic,” the researchers explained in a paper.
In their experiments, they tested 11 popular open source libraries that process executable binary (ELF), image, audio and video files. The list includes Binutils, LibPNG, ImageMagick, LibJPEG-turbo, LibJasper, FFmpeg, LibAV, WavPack, and OpenJPEG.
These libraries were tested with AFLsmart and, for comparison, with the AFL, AFLfast and Peach fuzzers. AFLsmart uncovered 33 vulnerabilities, double compared to AFL and AFLfast – Peach did not find any flaws.
The new tool discovered a total of 42 vulnerabilities in the tests conducted by the researchers, and 17 of these issues have so far been assigned CVE identifiers. Of the 33 aforementioned bugs, 8 have been assigned CVEs. The types of security holes include assertion failures, heap and stack buffer overflows, null pointer dereferences, and divided-by-zero bugs.
The researchers said they plan on making the AFLsmart fuzzer available as open source.
Related: Microsoft Uses Neural Networks to Improve Fuzzing
Related: Google Wants More Projects Integrated With OSS-Fuzz