Setup a Container Cluster on AWS with Terraform Part 2-Provision a CLUSTER

- By Ambarish Chitnis on January 03, 2018

This blog is the Part 2 in the series of blogs to provision an ECS cluster using Terraform. In Part 1 of the blog, we had completed the first step of setting up a VPC. In this blog, we will cover the remaining steps that will complete the provisioning of an ECS cluster and get a Wordpress instance running on it. 

 

Setting up the variables

The Terraform files that we will create in steps below use input variables You can create all these files yourself or simply clone the sample here on GitHub

 

1. Create a variables.tf file to define these variables

variables.tf
# main creds for AWS connection
variable "aws_access_key_id" {
  description = "AWS access key"
}

variable "aws_secret_access_key" {
  description = "AWS secret access key"
}

variable "ecs_cluster" {
  description = "ECS cluster name"
}

variable "ecs_key_pair_name" {
  description = "EC2 instance key pair name"
}

variable "region" {
  description = "AWS region"
}

variable "availability_zone" {
  description = "availability zone used for the demo, based on region"
  default = {
    us-east-1 = "us-east-1"
  }
}

########################### Test VPC Config ################################

variable "test_vpc" {
  description = "VPC name for Test environment"
}

variable "test_network_cidr" {
  description = "IP addressing for Test Network"
}

variable "test_public_01_cidr" {
  description = "Public 0.0 CIDR for externally accessible subnet"
}

variable "test_public_02_cidr" {
  description = "Public 0.0 CIDR for externally accessible subnet"
}

########################### Autoscale Config ################################

variable "max_instance_size" {
  description = "Maximum number of instances in the cluster"
}

variable "min_instance_size" {
  description = "Minimum number of instances in the cluster"
}

variable "desired_capacity" {
  description = "Desired number of instances in the cluster"
}

 

2. Specify the values for the variables in a terraform.tfvars file. 

Replace the placeholders (environment variables) with specific values. These placeholders get dynamically replaced when the Shippable workflow is setup and executed, whose configuration can also be seen here on GitHub.

terraform.tfvars

ecs_cluster="$ECS_CLUSTER"
ecs_key_pair_name="$ECS_KEY_PAIR_NAME"
aws_access_key_id = "$PECT_AWS_KEYS_INTEGRATION_ACCESSKEY"
aws_secret_access_key = "$PECT_AWS_KEYS_INTEGRATION_SECRETKEY"
region = "$REGION"
test_vpc = "$TEST_VPC"
test_network_cidr = "$TEST_NETWORK_CIDR"
test_public_01_cidr = "$TEST_PUBLIC_01_CIDR"
test_public_02_cidr = "$TEST_PUBLIC_02_CIDR"
max_instance_size = "$MAX_INSTANCE_SIZE"
min_instance_size = "$MIN_INSTANCE_SIZE"
desired_capacity = "$DESIRED_CAPACITY"

 

Step 2: Setup IAM Roles

IAM roles are required for the ECS container agent and ECS service scheduler. We also create an instance profile to pass the role information to the EC2 instances when they are launched.

 

ecs-service-role.tf
resource "aws_iam_role" "ecs-service-role" {
    name                = "ecs-service-role"
    path                = "/"
    assume_role_policy  = "${data.aws_iam_policy_document.ecs-service-policy.json}"
}

resource "aws_iam_role_policy_attachment" "ecs-service-role-attachment" {
    role       = "${aws_iam_role.ecs-service-role.name}"
    policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceRole"
}

data "aws_iam_policy_document" "ecs-service-policy" {
    statement {
        actions = ["sts:AssumeRole"]

        principals {
            type        = "Service"
            identifiers = ["ecs.amazonaws.com"]
        }
    }
}
 
ecs-instance-role.tf
resource "aws_iam_role" "ecs-instance-role" {
    name                = "ecs-instance-role"
    path                = "/"
    assume_role_policy  = "${data.aws_iam_policy_document.ecs-instance-policy.json}"
}

data "aws_iam_policy_document" "ecs-instance-policy" {
    statement {
        actions = ["sts:AssumeRole"]

        principals {
            type        = "Service"
            identifiers = ["ec2.amazonaws.com"]
        }
    }
}

resource "aws_iam_role_policy_attachment" "ecs-instance-role-attachment" {
    role       = "${aws_iam_role.ecs-instance-role.name}"
    policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role"
}

resource "aws_iam_instance_profile" "ecs-instance-profile" {
    name = "ecs-instance-profile"
    path = "/"
    roles = ["${aws_iam_role.ecs-instance-role.id}"]
    provisioner "local-exec" {
      command = "sleep 10"
    }
}

 

Step 3: Setup ALB

Now let's setup an ALB to load balance traffic across all our instances.

 

application-load-balancer.tf

resource "aws_alb" "ecs-load-balancer" {
    name                = "ecs-load-balancer"
    security_groups     = ["${aws_security_group.test_public_sg.id}"]
    subnets             = ["${aws_subnet.test_public_sn_01.id}", "${aws_subnet.test_public_sn_02.id}"]

    tags {
      Name = "ecs-load-balancer"
    }
}

resource "aws_alb_target_group" "ecs-target-group" {
    name                = "ecs-target-group"
    port                = "80"
    protocol            = "HTTP"
    vpc_id              = "${aws_vpc.test_vpc.id}"

    health_check {
        healthy_threshold   = "5"
        unhealthy_threshold = "2"
        interval            = "30"
        matcher             = "200"
        path                = "/"
        port                = "traffic-port"
        protocol            = "HTTP"
        timeout             = "5"
    }

    tags {
      Name = "ecs-target-group"
    }
}

resource "aws_alb_listener" "alb-listener" {
    load_balancer_arn = "${aws_alb.ecs-load-balancer.arn}"
    port              = "80"
    protocol          = "HTTP"

    default_action {
        target_group_arn = "${aws_alb_target_group.ecs-target-group.arn}"
        type             = "forward"
    }
}

 

 

Step 4: Set up Autoscaling group and launch configuration 

A launch configuration specifies the instance template that is used by an Autoscaling group to launch EC2 instances. This should be customized based on the requirements of your application.

 

launch-configuration.tf

resource "aws_launch_configuration" "ecs-launch-configuration" {
    name                        = "ecs-launch-configuration"
    image_id                    = "ami-fad25980"
    instance_type               = "t2.xlarge"
    iam_instance_profile        = "${aws_iam_instance_profile.ecs-instance-profile.id}"

    root_block_device {
      volume_type = "standard"
      volume_size = 100
      delete_on_termination = true
    }

    lifecycle {
      create_before_destroy = true
    }

    security_groups             = ["${aws_security_group.test_public_sg.id}"]
    associate_public_ip_address = "true"
    key_name                    = "${var.ecs_key_pair_name}"
    user_data                   = <<EOF
                                  #!/bin/bash
                                  echo ECS_CLUSTER=${var.ecs_cluster} >> /etc/ecs/ecs.config
                                  EOF
}

 

autoscaling-group.tf

resource "aws_autoscaling_group" "ecs-autoscaling-group" {
    name                        = "ecs-autoscaling-group"
    max_size                    = "${var.max_instance_size}"
    min_size                    = "${var.min_instance_size}"
    desired_capacity            = "${var.desired_capacity}"
    vpc_zone_identifier         = ["${aws_subnet.test_public_sn_01.id}", "${aws_subnet.test_public_sn_02.id}"]
    launch_configuration        = "${aws_launch_configuration.ecs-launch-configuration.name}"
    health_check_type           = "ELB"
  }

 

Step 5: Create the ECS Cluster

We now create the ECS cluster.

 

cluster.tf
resource "aws_ecs_cluster" "test-ecs-cluster" {
    name = "${var.ecs_cluster}"
}

 

Step 6: Create the Task definition

A task definition defines the containers that run on the ECS cluster. Here we have define two containers required to run WordPress, which are linked together along with their CPU and memory requirements.

 

task-definition.tf
data "aws_ecs_task_definition" "wordpress" {
  task_definition = "${aws_ecs_task_definition.wordpress.family}"
}

resource "aws_ecs_task_definition" "wordpress" {
    family                = "hello_world"
    container_definitions = <<DEFINITION
[
  {
    "name": "wordpress",
    "links": [
      "mysql"
    ],
    "image": "wordpress",
    "essential": true,
    "portMappings": [
      {
        "containerPort": 80,
        "hostPort": 80
      }
    ],
    "memory": 500,
    "cpu": 10
  },
  {
    "environment": [
      {
        "name": "MYSQL_ROOT_PASSWORD",
        "value": "password"
      }
    ],
    "name": "mysql",
    "image": "mysql",
    "cpu": 10,
    "memory": 500,
    "essential": true
  }
]
DEFINITION
}

 

Step 7: Create the Service

Finally, we define the service. A service specified the desired number of instances (replicas) of the task definition to run on the cluster. The load balancer distrubutes traffic across all the instances of the task. The ECS Service scheduler automatically creates new task instances should any instance terminate unexpectedly.

 

resource "aws_ecs_service" "test-ecs-service" {
  	name            = "test-ecs-service"
  	iam_role        = "${aws_iam_role.ecs-service-role.name}"
  	cluster         = "${aws_ecs_cluster.test-ecs-cluster.id}"
  	task_definition = "${aws_ecs_task_definition.wordpress.family}:${max("${aws_ecs_task_definition.wordpress.revision}", "${data.aws_ecs_task_definition.wordpress.revision}")}"
  	desired_count   = 2

  	load_balancer {
    	target_group_arn  = "${aws_alb_target_group.ecs-target-group.arn}"
    	container_port    = 80
    	container_name    = "wordpress"
	}
}

 

Run Your Script

Once you have created these files, we need to define the variables used in the script. You can create these files yourself or simply clone the sample here on GitHub. Note that the order that you have these in your file doesn't matter, as Terraform will evaluate each action and determine the appropriate order to execute the steps.

If you're not already there, change to the directory that holds the script:
$ cd provision-cluster

 

Check that you have everything set up correctly:
$ terraform plan

 

If everything looks good, run the scripts for real (note -- you will be provisioning resources on AWS and may incur charges):
$ terraform apply

 

Shippable sample

I have also created a Shippable sample that creates the underlying infrastructure for a container cluster (executes steps 1-7) and also deprovisions the infrastructure in a workflowYou can find all of the code for the sample here on GitHub

To run the sample, you will need to do the following:

If you're not familiar with Shippable, here are some basic concepts you should know before you start:

  • Configuration: The Assembly Lines configuration for Shippable resides in a shippable.yml file. The repository that contains this config in your source control is called a Sync Repository, aka syncRepo. A syncRepo is added through your Shippable UI to add your Assembly Line. 
  • Jobs are executable units of your pipeline and can perform any activity such as CI, provisioning an environment, deploying your application, or running pretty much any custom script. A simple way to think of it is, if something can execute in the shell of your laptop, it can execute as a Job.
  • Resources typically contain information needed for Jobs to execute, such as credentials, pointer to a cluster on a Container Engine or an image on a Hub, or any key-value pairs.  Resources are also used to store information produced by a job which can be then accesses by downstream jobs.
  • Integrations are used to configure connections to third-party services, such as AWS, Docker Hub, GKE, Artifactory, etc.
  • The Single Pane of Glass view shows a real-time, interactive view of your Assembly Line(s).

 

Try Shippable 


Add comments below...