Spinning up Custom Nodes Programmatically

- By Ambarish Chitnis on September 21, 2017

Most customers use nodes provided by Shippable to run their CI/CD workflows. However, you also have another option called Custom Nodes, which allows you to attach your own build nodes, which can be inside your VPC and/or behind your firewall, to run your CI/CD workflows

Custom nodes provide many key advantages:

  • Security: Your build machines can be inside your VPC and/or behind your firewall, which gives you the ability to configure access, IAM, etc. We even have a way of configuring these machines so that you do not have to grant Shippable SSH access! This means your code never leaves your firewall and no external entity can access your machines.
  • Faster build times: You can leave your build machines running all the time, which eliminates the occasional 2-3 mins per build that is added when new machines are spun up on Shippable's hosted infrastructure.
  • Docker caching: If you use Docker for your build workflows like pulling Docker images from a registry or building Docker images, your build machines will already have these images and this will speed up your builds.

The most common way to add custom nodes is through the Shippable UI.

 

It all starts with a customer..

Recently, a customer came to us with an interesting requirement. They wanted to provision custom nodes using an API, rather than our dashboard since they use EC2 spot instances. As and when nodes become available, they wanted to register them dynamically using an API.

Shippable's UI itself uses a clusterNodes API endpoint to provision custom nodes and runs a script thereafter to initialize the node and register it with our SaaS service. So, we knew this was possible but we had to make it easy for our customer to integrate it into their EC2 spot instance workflow.

So I went to our favorite editor, wrote some scripts, and gave an elegant solution to our customer. I thought it would be a great idea to share it broadly with our Shippable community. 

 

Prerequisites

1) The minimum hardware and software requirements that need to be met by a custom node can be found here

2) We will be using Shippable API to provision Custom Nodes. Shippable API is exposed to only paid customers. Generate an API token using these instructions.
3) We will be provisioning Custom Nodes for a specific Subscription in your Shippable dashboard. The scripts we will use later on need the subscription ID. Locate the Subscription ID of your subscription by navigating to the Subscription dashoard and scrolling to the bottom of the page.

 

Step-by-Step Instructions 

There are three ways to create Custom Nodes and we have created scripts that you can use as-is to jump start this process. All the scripts described below can be found in our samples repository here - https://github.com/devops-recipes/create-custom-nodesClone the repo and use the script that best meets your requirement.

 

#1: Using an Ansible Playbook

For Ansible fans, we have also created an Ansible playbook that works specifically with AWS hosts. The playbook provisions an AWS node, initializes it as a custom node and registers it with your Shippable subscription.

Playbook location:  https://github.com/devops-recipes/aws-provision-shippable-nodes.

1. Customize the AWS node you want to provision by modifying this file.

2. Set the API_TOKEN and SUBSCRIPTION_ID environment variables.

3. Run the following command:

ansible-playbook ansible-ec2-shippableNode-provision.yml

 

#2: From within your provisioning script/ playbook/laptop

Use this script if you want to register a custom node from within one of your own scripts that provisions your custom node on the cloud, Chef/Puppet/Ansible playbook, or even your laptop. This script by default uses a pem file called test.pem to authenticate with a custom node. Modify the script to use your specific pem file that has access to the custom node that you provisioned.

Script: register-custom-nodes-remote.sh

Usage: register-custom-nodes-remote.sh <API TOKEN> <SUBSCRIPTION ID> <UNIQUE NAME FOR CUSTOM NODE>

 

#3: From A Shell On The Custom Node Machine

Since this script runs from the custom node itself, it does not any credentials for establishing SSH access to the node.

Script: register-custom-node.sh

Usage: register-custom-node.sh <API TOKEN> <SUBSCRIPTION ID> <UNIQUE NAME FOR CUSTOM NODE> 

Topics: continuous integration (CI), infrastructure