Docker image creation, tagging and traceability in Shippable

- By Tom Trahan on August 25, 2015

By now, you've almost surely heard about Docker. Here at Shippable, we've been using Docker and containers for almost two years now and the benefits to how we manage our software delivery and deployment process are impressive. So impressive, in fact, that we're convinced it will revolutionize how all companies do it, making the current DevOps practices obsolete faster than many predict. 

Ultimately, the goal is to deliver value that's derived from software into customers' hands as soon as possible. With Docker images, many of the challenges faced within traditional DevOps approaches are overcome as the creation and deployment of immutable images becomes the new way to deliver software.

The latest features from Shippable make this within reach for any software delivery team as we make it easy for you to create Docker images, push them to your registry of choice, then deploy them without the use of separate DevOps tools or code.  

However, in order to effectively take advantage of immutable Docker images and avoid image chaos, you must have an effective strategy for managing your images. With Shippable, you do this within your CI/CD process flow by automatically creating and tagging your images, with full traceability from final image back to all of the elements used to create it.

Creating Docker images

Shippable makes it easy to create images, whether you're experienced with Docker or not. We provide three primary methods for doing so:

  • Create image from standard Shippable CI/CD flow (no Docker knowledge required)
  • Create image based on CI/CD build container that starts with a base image created from a Dockerfile
  • Create image at conclusion of CI/CD flow created from a Dockerfile

 

The following diagram should hopefully make it easier to understand these options:

docker-image-creation-shippable

 

Create image from standard Shippable CI/CD flow

Behind the scenes, everyone using Shippable is actually already taking advantage of Docker. With the recent release, we've now made it possible for you to take the Docker container we create as part of each CI build process in the background and push it to an image registry of your choice (we support Docker Hub, Google Container Registry, Quay.io, and any private registry).

To create a Docker image using this method, you'll just require a few settings to be made on your Shippable project. Instructions for this can be found here

That's it! All you need is a registry account and without learning any Docker syntax, you're creating and pushing Docker images that can be deployed anywhere.   

Create image using Pre-CI Docker Build

Many of you are already experienced with using Dockerfile syntax and Docker Build to create images. With Shippable, you can use a Dockerfile stored in your repo to build the base image that we'll use for your CI build container. After that, Shippable will execute all of the CI instructions you've provided in your shippable.yml, including adding your source code.  Upon successful CI build, we'll create a Docker image from your CI build container and push it to your image registry of choice.

To create a Docker image using this method, you'll just require a few settings to be made on your Shippable project, as well as a Dockerfile in the root of your repository. Instructions for this can be found here.  

Create image using Post-CI Docker Build

Post-CI Docker Build combines the previous two methods to enable you to construct your build container to use for CI via the standard Shippable CI process, but specify Shippable to use a Dockerfile stored in your repo to build the final image that we push to your image registry upon successful completion of the CI process. This is used by customers who wish to strip out all test artifacts from the final image so that the immutable image promoted to further environments (e.g. production) is as lightweight as possible by excluding artifacts unnecessary to end-user use of the application.

To create a Docker image using this method, you'll just require a few settings to be made on your Shippable project, as well as a Dockerfile in the root of your repository. Instructions for this can be found here.  

Versioning Docker images

Once you begin working with Docker images, you'll quickly realize that it can be difficult to keep track of the images you create in an orderly manner (search for how to use the "latest" tag in Docker for an example of this complexity). With Shippable, you can eliminate this confusion and opportunity for error by automatically tagging your images on every successful CI run.

Shippable provides three options for automatically tagging the Docker images created within your Shippable CI/CD process:

  • Tag with source code repo branch name and Shippable build number, e.g. master.15
  • Tag with the full source code commit SHA hash
  • Tag with a custom tag that you specify, such as "latest"

By having Shippable automatically tag your images, you eliminate the chance of conflicting tags generated by different developers.

Tracing Docker images

Another common issue Docker users quickly face is how to trace the genealogy of their images. You'll find yourself frequently asking "what version of our code is in this image" or "what version of the dependencies is in this image".  Once an image has been created, it can be difficult to determine what's in it if the relevant details aren't tracked when the image is created and made easily searchable.  With Shippable, this traceability from image back to its origins is fully tracked and searchable.

Shippable provides multiple options for automatically tagging the Docker images created within your Shippable CI/CD process, two of which allow for full traceability back to the the specific commit that caused the image to be built:

  • Tag with source code repo branch name and Shippable build number, e.g. master.15
  • Tag with the full source code commit SHA hash

With either of these methods, you can easily find the build and source code commit information for any image in your repository that was built and pushed by Shippable.

For example, let's say I'd like to find the information on the master.30 image below that was pushed by Shippable to Docker Hub:

aye0aye-docker-hub-radar-api

To find everything I need to know about this image, I simply need to navigate to the CI build in Shippable that created this image to get access to the who, what and when related to this image:  

 

aye0aye-shippable-radar-api-master-30

And I can drill down on the commit SHA hash linked to the build (upper right - "5a9af8f") to see the exact state of all source code, shippable.yml build instructions and Dockerfile instructions that were used to generate the image:

 
aye0aye-github-radar-api-master-30
 

So, I'm always just a few clicks away from being able to identify the who, what and when related to my Docker images. Whether you use Shippable to automatically manage your Docker images or not, if you're getting started using Docker images, this is an area you must plan for ahead of time. Without an effective strategy for tagging your images and tracing their genealogy, you'll likely find yourself quickly overwhelmed by the images your developers and operations people create.

 

Topics: Docker, containers