Build, test and deploy applications independently from a monorepo

- By Chetan Tarale on March 25, 2017

In our previous blog posts, Our journey to microservices: mono repo vs multiple repositories, we shared our thoughts and experiences on our approach with monorepo. We received a few questions after that blog on how CI and deploys go with the monorepo.

In this article we will learn how to run CI, build and deploy applications independently from a monorepo. On each PR/commit we will run tests on the service which has changed build a docker image from it and push it to a registry. This image can then be deployed to to a cluster on any supported Container ServiceWe will use Shippable for this scenario.



Our monorepo has two projects. Both of them are simple node js apps, www is a front-end app and will run selenium tests, api is the back-end app which we will test using superagent and chai. The initial setup can be a bit tricky and depends on how projects in your repository are structured. Detecting what has changed in the repository is important.  We will call a shell script from shippable.yml which will detect what has changed and trigger tests. When all our test pass successfully we will build a docker image and then push it to the registry.


language: node_js

- selenium

- ./

This yml will run the below shell script which will detect changes folders(i.e services in our repository and run tests). Also note that we are specifying all the services required in shippable.yml.

#!/bin/bash -e
export IGNORE_FILES=$(ls -p | grep -v /)

detect_changed_folders() {
echo "detecting changes for this build"
folders=`git diff --name-only $SHIPPABLE_COMMIT_RANGE | sort -u | awk 'BEGIN {FS="/"} {print $1}' | uniq`
export changed_components=$folders

run_tests() {
for component in $changed_components
echo "----------------------------------------------"
echo "$component has changed"
if ! [[ " ${IGNORE_FILES[@]} " =~ "$component" ]]; then
echo "-------------------Running tests for $component---------------------"
cd "$component"
./ "$component"
cd ..
echo "Ignoring this change"

if [ "$IS_PULL_REQUEST" != true ]; then
echo "skipping because it's a PR"


Step 1. Enable the project on shippable.

Now on navigating to the pipelines tab you can view your project on SPOG.


On every build triggered the changed folders are detected and a image is built and pushed to the registry. In our example the file that we have in every folder is responsible for triggering tests, building an image and pushing it to the registry.

Step 2.  Now that our images are build we will use pipelines to deploy these images. We will begin our resources and jobs in shippable.resources.yml and respectively.

Here is how we will define resources and jobs for our pipeline.

- name: api-img
type: image
integration: dockerhub
sourceName: "shippableSamples/sampleapp"
versionName: master.3

- name: www-img
type: image
integration: dockerhub
sourceName: "shippableSamples/sampleapp"
versionName: master.3

- name: ddc-cluster
type: cluster
integration: ddc-cluster

- name: www-options #required
type: dockerOptions #required
portMappings: #optional
- "51000:51000"

- name: api-options #required
type: dockerOptions #required
portMappings: #optional
- "50000:50000"

- name: dockerhub
type: integration
integration: "dockerhub"

www-img and api-img are the two images that will be built when the ci job runs. These images are the OUT's from the ci job.

dockerhub is an resource of type  integration and this points to the regisrty where we will push our image. ddc-cluster is an integration of type cluster and this is where we will deploy our app. www-options and api-options are the options that can be appended to a docker image to be pushed.

We will define the following jobs in our jobs yml

  • A runCI job that builds our  images
  • Two manifest jobs to generate a manifests
  • Two deploy jobs that deploys our app
- name: monorepo_sample_runCI
type: runCI
- OUT: api-img
- OUT: www-img
- script: echo 'on success !!!!!'
- script: echo 'Failed job .... :('

- name: www-man
type: manifest
- IN: www-options
- IN: www-img
force: true

- name: www-deploy
type: deploy
- IN: www-man
- IN: ddc-cluster

- name: api-man
type: manifest
- IN: api-options
- IN: api-img
force: true

- name: api-deploy
type: deploy
- IN: api-man
- IN: ddc-cluster

 Now push these files and add this repo as syncRepo from Shippable UI.Shippable will read the jobs and resources ymls and create your pipeline.



We have successfully set up our pipeline. For any changes to a project within in the monorepo, CI for that corresponding project is triggered, a docker image is built and deployed.

Links to repositories used in this tutorial

Topics: Docker, devops, continuous delivery, microservices

Add comments below...