Smaller open source projects tend to be more secure than proprietary applications, but the opposite is the case for software with more than a million lines of code, according to a new report from Coverity.
There is an accepted industry standard for defects in software: one in every thousand lines of code, or a “defect density” of 1.0. Coverity ran its vulnerability-scanning technology against over 68 million lines of code from open source projects and 381 million lines from proprietary products to calculate the number of defects found per thousand lines of code.
The average defect density for open source applications was 0.69, compared to 0.68 for proprietary software, Coverity found in its 2012 Coverity Scan Open Source report released Tuesday.
“The increasing number of open source and commercial projects that have embraced static analysis have raised the bar for the entire industry,” said Jennifer Johnson, Chief Marketing Officer for Coverity, a provider of software development testing solutions.
The report suggests that organizations implementing formal testing processes in their development lifecycle have some effects on software quality.
“Development testing is no longer a nice-to-have, it’s a must-have,” said Johnson. Not testing is a competitive disadvantage, she said.
The overall size of the program appeared to impact code quality. Small proprietary applications with 500,000 to a million lines of code had a defect density of 0.98. The density dropped to 0.66 for proprietary applications with more than a million lines of code. Open source projects between 500,000 to a million lines of code had a defect density of just 0.44. However, that figure rose to 0.75 for projects with more than a million lines of code.
At less than 100,000 lines, the difference was narrower, with the defect density for open source and proprietary code at 0.40 and 0.51 respectively. The density increased to 0.60 and 0.66, respectively, for applications between 100,000 and 499,999 lines of code.
Coverity’s report is interesting in light of last week’s report from WhiteHat Security, which found that building in secure coding practices as part of the software development lifecycle does not necessarily translate to more secure code.
Open source projects are generally feature-specific and maintained by a core group of committed developers, Coverity said. As the project grow in size and scope and more developers come on board, developers become reluctant to make changes to one part of the code as it may have unexpected effects elsewhere. On the other hand, smaller projects may not garner much attention in enterprise environments. Once they hit a certain size and visibility, the organization is more likely to implement formal testing processes to ensure software quality, Coverity said.
For open source projects, the larger size doesn’t mean the quality of the codebase suffers, the report pointed out. Most of the million-lines-of-code projects are “heavily adopted in the industry, have the backing and support of a commercial company and still have above average software quality,” the report said.
The Linux 3.8 kernel has 7.6 million lines of code, and according to Coverity’s scan results, a defect density of 0.59. “Linux remains a benchmark for quality,” the company said, noting that its defect density has been consistently below 1.0 since 2008, and below 0.7 since 2011.
Of the 374 total projects scanned, over 229,000 defects were found and fixed, according to an infographic from Coverity. Of the 20,000 or so defects found in open source projects, 36 percent were considered “high risk”. Resource leaks, memory corruption and illegal memory access, were the most common high-risk defects identified in the report. All of these issues are considered to be difficult to detect without automated code analysis, the company said.
Applications written in C, C++, and Java can be scanned using Coverity’s engine. LibreOffice, MariaDB, NetBSD, NGINIX, Git, zsh, Thunderbird, and Firefox are among the well-known open source projects that have been scanned by Coverity.
The full report from Coverity is available here in PDF format.
Related: Following Best Development Practices Does Not Always Mean Better Security
Related: 71% of Apps Use Components With Severe or Critical Security Flaws
Related: Experts Debate – Is Software Security a Waste of Time?