Today we find ourselves using cloud native technologies to increase flexibility, scaling and cost savings in many respects. The modern cloud stack using IaaS, abstracts the hardware maintenance component away and you are left with everything above such as the operating system and software.
Golden images have been a simple concept used in practice for a long time. It reminds me a bit of the AOL marketing campaign that everyone saw and knew. The concept was that each year a new CD appeared in the mail with a version number, and people installed the software from the CD onto their PC. (Amazing to imagine how far we have come) The idea was that we know there is a known good version that has been approved and tested, pre-bundled, so no downloads were needed from external sources, especially over slow dial-up internet connections.
There is a lot of debate around the best way to create golden images and how to maintain them, as well as the software involved such as AWS image builder, Terraform or Packer. One idea is to keep them as simple as possible such that they have broad compatibility and can be configured by downstream systems. The other side is to configure them as much as possible beforehand to speed up builds and remove downloaded dependencies. Here is a typical workflow you might have in your image building and deployment process.
For simplicity, I’m going to focus on the first build step as defined in the image above. The 1st build is foundational and sets the stage for all of your other applications. Pick a reputable and official image that will be supported for a long time. You want to ensure your runtime is installed correctly and the image is hardened to your needs. In addition, you will download, sign and version your image to reduce reliance on third party hosted servers and minimize source hijacking threads. Also, consider including these three areas to help empower your images from the start and to minimize configuration drift.
Logging agent configuration – Logs are extremely important to monitor processes, crashes and anything else that you want turned on by default.
Telemetry agent – Organizations need visibility into the health of clusters, so it’s imperative that you are collecting reliable telemetry and sending the data somewhere for processing. This is paramount for both security and troubleshooting down the road.
Security agent – Depending on how your environment is set up, you may want to consider a security agent at this level to ensure every endpoint is monitored. Consideration here is key.
Images can run rampant within an organization, often having various flavors based on the needs of each team. When it comes to security, change management is a key part of NIST 800-53 as well as other system frameworks. In order to be successful with your golden builds, you will need to ensure you have coverage of the following:
- Approved baseline configurations accepted by the organization
- Default rule to use built-in software as a first option
- Expectation of regular of security and feature upgrades
- Process to upgrade pipelines to the latest build
- Retire old images and promote new images on a standard timeline
Part of securing the supply chain means that companies need to know how to fix issues when they discover them. Having a SBOM or other inventory list is a great place to start, but being able to take action when a vulnerability is found is the second part. By having a golden image, or set of known golden images, you will put a process in place that allows you to quickly fix and deploy within your organization.
Related: The SBOM Bombshell
Related: SBOMs – Software Supply Chain Security’s Future or Fantasy?
