Software engineers tracking the quality of software bill of materials have stumbled on a startling discovery: Barely 1% of all SBOMs being generated today meets the “minimum elements” defined by the U.S. government.
According to new data from software supply chain security startup Chainguard, SBOMs being generated by existing tools fail to meet the minimum data fields needed inside an SBOM to enable the management of software vulnerabilities, licenses, and inventory tracking.
“Only one percent of SBOMs were entirely conformant with the minimum elements. The minimum elements appear to be a high bar for SBOMs. Further research will need to address whether the standard is too high, whether SBOM generation tools must evolve, or whether the underlying software artifacts lack necessary package metadata,” Chainguard security data scientist John Speed Meyers explained.
Chainguard’s researchers collected about 3,000 SBOMs for analysis using four SBOM creation tools from a list of popular Docker Hub containers and used an NTIA conformance checker tool to measure SBOM conformance with minimum elements.
The team said the minimum element data fields include information about each software component (supplier, name, version, unique ID, relationships) and also metadata about the SBOM itself, including the author and the time of creation.
After parsing the data, the Chainguard team found the majority of SBOMs lacked specified suppliers for their components while about 1,000 SBOMs failed to specify a name or version for all components.
The latest Chainguard discovery is sure to add fuel to an ongoing debate over the value and quality of SBOMs to help mitigate supply chain attacks.
A high-powered lobbying outfit representing some of the biggest names in technology has already signaled strong objection to the government’s SBOM mandate, arguing that “it is premature and of limited utility” because SBOMs are not currently scalable or consumable.
The ITI lobbying outfit, which counts Amazon, Microsoft, Apple, Intel, AMD, Lenovo, IBM, Cisco, Samsung, TSMC, Qualcomm, Zoom and Palo Alto Networks among its prominent members, described the current SBOM process as immature.
“At this time, it is premature and of limited utility for software producers to provide an SBOM. We ask that OMB discourage agencies from requiring artifacts until there is a greater understanding of how they ought to be provided and until agencies are ready to consume the artifacts that they request,” the group said.
In its research, Chainguard called attention to the ITI objections, cautioning that its findings are not meant to be viewed as evidence for what it called a cynical argument that SBOMs are “immature” and not yet “consumable.”
“This analysis suggests that standard SBOMs already provide a great deal of information but not enough to satisfy the minimum elements. Additionally, this research implies that the push to make SBOMs “everywhere” should be accompanied by an effort to measure and improve the quality of SBOMs,” the company said.
A tool-by-tool analysis suggests that none of the tools appear to consistently create minimum elements-compliant SBOMs,” Chainguard added.
Still, the company is advising caution against dismissing the usefulness of SBOMs. “The results suggest lots of variability: some SBOMs are high-quality, some are low-quality,” it said.
The SBOM mandate was included in a cybersecurity executive order issued last May, sending security leaders scrambling to understand the ramifications and prepare for downstream side-effects.
Related: Big Tech Vendors Object to US Gov SBOM Mandate
Related: Microsoft Releases Open Source Toolkit for Generating SBOMs
Related: Cybersecurity Leaders Scramble to Decipher SBOM Mandate
