Configuring Multi-Stage CI

- By Ambarish Chitnis on October 09, 2017

In this blog, we demonstrate how to use the Shippable platform to perform Multi-Stage CI on your repositories. The key benefit of Multi-Stage CI is to split a time-consuming CI process into smaller stages to detect issues in code quality / tests as early as possible and shorten the feedback loop on every checkin. This often entails refactoring or designing your application into smaller components and testing each component in isolation first before running more expensive integration tests of your component with other components in the system.

What is multi-stage CI?

In our multi-stage CI scenario, we split the CI of a Node.js app into several stages.

  • Stage 1: Stage 1 runs on every PR and lints the source code in the repository to find style errors. To learn more about the benifits of linting your javascript code, look at this article. The idea behind Stage 1 is to perform a quick code quality check on every PR and shorten the feedback loop for any errors in coding style and bugs found during static analysis. This allows developers to quickly find and fix issues in their PRs.
  • Stage 2: Stage 2 runs on successful completion of Stage 1. In Stage 2, we run a small subset of tests to quickly validate the PR.
  • Stage 3: Stage 3 runs on the merged commit to the repository. Here we run a broader set of core unit tests that take longer to run than Stage 2.

This is a pictorial representation of the workflow we're going to configure. The green boxes are jobs and the grey boxes are the input resources for the jobs. The workflow is defined across 3 configuration files:, shippable.resources.yml, and shippable.yml.




Sample project

The code for this example is in GitHub: 

You can fork the repository to try out this sample yourself or just follow instructions to configure your own use case. 


1. Author Stage 1 of the workflow


A. Define app_gitRepo 

Add the following to your shippable.resources.yml file: 

  - name: app_gitRepo
    type: gitRepo
    integration: github
      sourceName: devops-recipes/multi-stage-ci-app     # pointer to your repository
      buildOnPullRequest: true                          # update this resource when a pull request is opened or updated
      buildOnCommit: false                              # do not update this resource for commits

  • Description: app_gitRepo is a gitRepo resource which is a pointer to the git repository that contains your source code. We have configured our gitRepo resource to only trigger on PRs.
  • Shippable Integrations needed: default SCM account integration or other source control providers.

By default, you will already have an account integration with whichever SCM provider you've used to log into Shippable. If your source code repository is on that SCM account, you should use it as is.

If your source code repository is on another SCM account, create an integration for it by using one of the following supported SCM docs:


B. Define lint_runSH_job 

Add the following to your file: 


  - name: lint_runSh_job
    type: runSh
      - IN: app_gitRepo 
showBuildStatus: true # job status is shown on your PR page
- TASK: - script: pushd $APP_GITREPO_PATH/gitRepo # env var pointing to directory containing files - script: npm install --only=dev - script: $APP_GITREPO_PATH/gitRepo/node_modules/.bin/eslint app.js # run lint on app.js
  • Description: lint_runSh_job is a runSh job that lets you run any shell script. This job implements the first stage of our multi-stage CI workflow. It takes app_gitRepo as an input and runs scripts to lint the application source code. 

  • Scripts: We have specified the package details for running eslint in the package.json in the devdependencies section. The scripts pushd into the source code directory using environment variables for the gitRepo resource, run npm install and execute esLint.


CCommit your shippable.resources.yml and files and add config to your Shippable account.

Once you have these configuration files as described above, commit them to your repository. The and shippable.resources.yml can be committed to the same app repository, or to a separate repository.

The repository containing your jobs and resources ymls is called a Sync repository and basically represents your workflow configuration.

Follow these instructions to import your configuration files into your Shippable account.


Stage 1 is complete at this point. We've configured the following:



2. Author Stages 2 and 3 of the workflow


A. Enable your repository

To set up the next stages, we will first setup CI for our application and author the logic to run the different unit test suites for commits and PRs. The prerequisites are to componentize your unit tests and organize them in two separate suites for each stage. We can then trigger them separately very easily.

To setup Shippable CI on your applictation repository, you need to do three things

  • Sign in with your GitHub or Bitbucket credentials.
  • Create the CI config file shippable.yml and commit to the root of your repository. The next section will go into the details of how to author shippable.yml file. If you're using a fork of our sample repository, you do not need to do this step since the repo already contains the config file. 
  • Enable your repository so that we can set up webhooks on your behalf. 

 Please note that enabling a project on shippable automatically creates a job and resource for you:

  • A runCI job that represents your CI workflow and is named with (your_repo_name)_runCI. In our case, this is multi-stage-ci-app_runCI. This is configured via shippable.yml in the next step.
  • A ciRepo resource pointing to your repository, named with (your_repository)_ciRepo. In our case, this is multi-stage-ci-app_ciRepo  


B. Author stages 2 and 3 in a shippable.yml file.

In this shippable.yml, we demonstrate the following key concepts for our Node.js application:

  • Stage 2: Detecting CI triggered by a PR 
  • Stage 3: Detecting CI triggered for a merge commit 
  • Running unit tests using mocha and code coverage using Istanbul
  • Configuring email notifications of your build
language: node_js

    - TEST_RESULTS_DIR=$SHIPPABLE_REPO_DIR/shippable/testresults
    - CODE_COVERAGE_DIR=$SHIPPABLE_REPO_DIR/shippable/codecoverage
    - MOD_LOC=$SHIPPABLE_REPO_DIR/node_modules/.bin/

    - shippable_retry npm install
    - mkdir -p $TEST_RESULTS_DIR && mkdir -p $CODE_COVERAGE_DIR
    - pushd $TESTS_LOC_DIR
    - |
      # Stage 2: For PRs, run basic tests
if [ "$IS_PULL_REQUEST" == true ] then echo "running the test suite for a PR build" $MOD_LOC/mocha name.spec.js -R mocha-junit-reporter --reporter-options mochaFile=$TEST_RESULTS_DIR/testresults.xml
# Stage 3: For commits and PR merges, run the core and important test suites for commits else echo "running the core test suite for a commit or manually triggered build" $MOD_LOC/mocha name.spec.js users.spec.js -R mocha-junit-reporter --reporter-options mochaFile=$TEST_RESULTS_DIR/testresults.xml fi - $MOD_LOC/istanbul --include-all-sources cover -root "$SHIPPABLE_REPO_DIR/routes" $SHIPPABLE_REPO_DIR/node_modules/mocha/bin/_mocha -- -R spec-xunit-file --recursive "$TESTS_LOC_DIR/**/*.spec.js" - $MOD_LOC/istanbul report cobertura --dir $CODE_COVERAGE_DIR - popd
# send notifications. reference at integrations: notifications: - integrationName: email type: email on_success: change on_failure: always on_pull_request: never

 For a complete reference on the configuration, please visit our shippable.yml documentation

After you have authored and committed your shippable.yml file and enabled your project, run the project manually from the project dashboard. This should run the core tests for your application, which is like running Stage 3. 


CCommit your shippable.yml file.


D. Turn off automatic triggers for Pull Requests

Next, we need to ensure that our stage 2 workflow doesn't trigger for Pull Requests automatically. This is because we want it to be triggered only after the Stage 1 finishes. To turn off triggers for Pull Requests, follow instructions for Configuring Job Triggers and turn off triggers for PRs.


Stage 2 is now complete. We have authored the configuration for stages 2 and 3 :


If you notice, some text is striken through. This is because we have yet to connect Stage 1 to trigger Stage 2.


3. Connecting the Stages

We have now authored the configuration for all stages, but we still need to connect stage 1 to stage 2. This will ensure that when the lint_runSH_job completes successfully, it will trigger stage 2.   

To connect the stages, we augment the file:


  - name: lint_runSh_job
    type: runSh
      - IN: app_gitRepo
      - TASK:
        - script: pushd $APP_GITREPO_PATH/gitRepo
        - script: npm install --only=dev
        - script: $APP_GITREPO_PATH/gitRepo/node_modules/.bin/eslint app.js
      - OUT: multi-stage-ci-app_ciRepo          # update the resource when this job completes
        replicate: app_gitRepo             # copy all information from multi_ci_gitRepo to multi-stage-ci-app_ciRepo


  • To connect Stage 1 to Stage 2, we first specify the OUT on the Stage 1 job (lint_runSh_job). This is the ciRepo resource which was created when you enabled the repository, and is an input to the runCI job. We also use the replicate directive to copy the metadata of the PR webhook from the gitRepo resource into the ciRepo resource so that the runCI job will be triggered with the right PR information. 
  • Commit your file after making this change.
  • Your pipeline should look like this in your Shippable dashboard SPOG view. Navigate to the SPOG UI from your subscription dashboard by clicking on the eye icon followed by Show SPOG View.


Screen Shot 2017-10-09 at 6.32.53 AM.png


5. Trigger your pipeline

To test the pipeline, raise a PR from a fork of your repository on the master branch. This will trigger the workflow, run lint and then if that passes,  then run your unit tests.

Click on the jobs in the SPOG to see the execution details of the latest run of your jobs.

Screen Shot 2017-10-09 at 7.36.24 AM.png


Screen Shot 2017-10-09 at 7.37.01 AM.png


Use the grid view to look at the entire history of runs for a specific job.

Screen Shot 2017-10-09 at 7.41.05 AM.png

The history view for the linting job shows you the status of all commits that passed and failed the linting tests.


Screen Shot 2017-10-09 at 7.40.52 AM-1.png


The history view for the CI job shows you the status of unit tests that were executed for all PR, commit and time trigger builds.

And that's it! This completes your multi-stage CI workflow!

Sign up to try this out:

Try Shippable

Topics: continuous integration (CI), multi-stage-ci