The CI/CD and DevOps Blog

Learn about various tried-and-tested strategies that will help you ship code faster

Provision an AWS VPC using Ansible

This tutorial explains how to manually provision a AWS Virtual Private Cloud(VPC) using Ansible. Before you start, you should be familiar with the following concepts:

The best way to get started is to install ansible and run playbooks manually on your local machine to provision a VM. The first section of this tutorial explains how to do that. However, manual execution isn't the best and most efficient way to run ansible playbooks, so we will take a look at the challenges and learn how to automate this workflow with a step-by-step tutorial.

Provision AWS EC2 virtual machine with Ansible

This tutorial explains how to manually provision an AWS EC2 virtual machine using Ansible. Before you start, you should be familiar with the following concepts:

 The best way to get started is to install ansible and run playbooks manually on your local machine to provision a VM. The first section of this tutorial explains how to do that. However, manual execution isn't the best and most efficient way to run ansible playbooks, so we will take a look at the challenges and learn how to automate this workflow.

Spinning up Custom Nodes Programmatically

Most customers use nodes provided by Shippable to run their CI/CD workflows. However, you also have another option called Custom Nodes, which allows you to attach your own build nodes, which can be inside your VPC and/or behind your firewall, to run your CI/CD workflows

Custom nodes provide many key advantages:

  • Security: Your build machines can be inside your VPC and/or behind your firewall, which gives you the ability to configure access, IAM, etc. We even have a way of configuring these machines so that you do not have to grant Shippable SSH access! This means your code never leaves your firewall and no external entity can access your machines.
  • Faster build times: You can leave your build machines running all the time, which eliminates the occasional 2-3 mins per build that is added when new machines are spun up on Shippable's hosted infrastructure.
  • Docker caching: If you use Docker for your build workflows like pulling Docker images from a registry or building Docker images, your build machines will already have these images and this will speed up your builds.

Provisioning AWS infrastructure with Terraform

Provisioning and updating infrastructure is a the first step in setting up your development, beta, or production environments. Hasicorp's Terraform format is fast becoming very popular for this use case.  We love Terraform at Shippable due to its easy declarative syntax, similar to our pipelines syntax. Other advantages are: 

  • Infrastructure as Code: Infrastructure is described using a high-level configuration syntax. This allows a blueprint of your datacenter to be versioned and treated as you would any other code. Additionally, infrastructure can be shared and re-used.

  • Execution Plans: Terraform has a "planning" step where it generates anexecution plan. The execution plan shows what Terraform will do when you call apply. This lets you avoid any surprises when Terraform manipulates infrastructure.

  • Resource Graph: Terraform builds a graph of all your resources, and parallelizes the creation and modification of any non-dependent resources. Because of this, Terraform builds infrastructure as efficiently as possible, and operators get insight into dependencies in their infrastructure.

  • Change Automation: Complex changesets can be applied to your infrastructure with minimal human interaction. With the previously mentioned execution plan and resource graph, you know exactly what Terraform will change and in what order, avoiding many possible human errors.

At Shippable, we use Terraform to provision all our environments and automate the provisioning using our Pipelines feature. If you're interested in taking a look at our terraform scripts and pipelines config, we have made our repositories public so you can check them out:  

Interested in trying it yourself? The following example walks you through a sample project that provisions two t2.micro instances on AWS. We've kept it simple for easy understanding, but you can also automate provisioning of complex environments as seen in our beta infra scripts above.