Creating an Image Pipeline with EC2 Image Builder

Credits Educative.io

Let’s create our own image that we can use to create EC2 instances with pre-installed Python and boto3. We will do this via EC2 Image Builder.

We create a SNS topic and a create an email subscriber to it (arn:aws:sns:us-east-1:332498689295:CL_SNS_Topic)
Then we create a IAM role for EC2 (under “AWS service”) and assign “EC2InstanceProfileForImageBuilder” and “EC2InstanceProfileForImageBuilder” policies.
We create build component that’s used to specify the image operating system, configure settings, and perform any tasks that are necessary while creating an image. We do this via “EC2 Image Builder” with the following definition document:

name: LinuxWithBoto3
schemaVersion: 1.0
phases:
  - name: build
    steps:
      - name: Boto3
        action: ExecuteBash
        inputs:
          commands:
            - 'sudo yum install python3 -y'
            - 'sudo yum install gcc openssl-devel bzip2-devel libffi-devel zlib-devel -y'
            - 'cd /opt'
            - 'sudo wget https://www.python.org/ftp/python/3.8.12/Python-3.8.12.tgz'
            - 'sudo tar xzf Python-3.8.12.tgz'
            - 'cd Python-3.8.12'
            - 'sudo ./configure --enable-optimizations'
            - 'sudo make altinstall'
            - 'pip3.8 install pip --upgrade'
            - 'pip3.8 install boto3'

Now we create image pipeline, also under “EC2 Image Builder”, with Amazon Machine Image (AMI) (and not Docker Image). This process has 6 steps:
- select build component (the one we created above),
- select recipe (that will also select SNS topic we create above to receive notifications about Image Builder process)
- infrastructure configurations (optional)
- distribution settings (optional) We then build and “Run pipeline” which might take 15-20 minutes to build. When it’s done we now have Image that has installed Python and boto3 (and of course can have anything we need as well).

We can now go to EC2, create instance, and select image that we created in a previous step. If we now connect to the instance we can see that Python, boto3 are pre-installed, which was the goal.

To add more packages, we can add them to the build component definition document (save it as a new version), and then rebuild the image.