Do you want your Development teams to use a Linux AMI (Amazon Machine Image) offered in the Amazon Market Place published by Joe Smith that could have malware embedded in it?
Most of the “big” firms do not use AMIs (Amazon Machine Images) available in the Amazon Market Place. The “big” firms do not allow use of publicly available AMIs and build their own AMIs, with their own security stack embedded in the AMIs.
But, how do you know the AMIs are being built with the right security stack and the OS at the right patch level? You need to test the AMIs as best as you can before you make them available for your Development teams to use.
Couple of simple options could be to:
- Write server spec test cases to test the AMIs meet your specifications
- Implement some sort of static analysis on the AMIs. But, currently I am not aware of any products that can do this.
But, these are not effective and there is no way to know if the test cases are written and the AMIs are safe to use.
How about you deploy the AMIs in a lower SysLevel and run scanning tools against a EC2 instance that uses the AMI? So, before you promote the AMIs for general use, have a private test VPC where you deploy an EC2 instance with the AMI and run your scanning tools such as Amazon Inspector, Qualys etc., to make sure that the OS is at the right patch level. This gives your better results and the confidence that the AMI is safe to use.
Once you are confident that AMIs are safe to use, promote the AMIs for use by Development teams. You can rely upon tagging to indicate what AMIs are available for production use.
Now that you have made a safe AMI available for Development teams to use, how do you know they continue to be safe, given that the threat environment changes and new attacks emerge constantly.
You need to create a farm of EC2 instance with AMIs that are currently in use in production and constantly scan them on a schedule basis. This allows you to be on top of new threats. If a vulnerability is detected, you need to have line of sight in to all the EC2 instance that are using that vulnerable image. After you identify all the EC2 instances, you have couple of options:
- Patch the EC2 instance in place
- Re-hydrate with EC2 instances that have the patched AMIs
The options you pick depends on your environment.
Now that we considered new AMI development and ongoing scanning, the EC2 server farm that you setup for scheduled scans also allows you to verify zero day vulnerabilities.
Building this automation for scanning your AMIs and promoting safe AMIs to use, will allow your vulnerability scanning to be effective for cloud native infrastructure. Without this option, you scan all available instances in the cloud and the instances that you scan might be part of auto scale and you will never have an accurate way to map the vulnerability back to a specific instance that need to be patched.
Having a server farm with all the AMIs that are currently promoted to be used by the development teams, you make the scanning pretty effective and you can take the results and map it back to all the EC2 instances currently using the vulnerable AMIs and limits the number of servers you need to scan and use your scanning licences in a cost effective manner.
Please share your approach to making AMIs safe for your development teams to use.