In this blog, we demonstrate how to use the Shippable platform to perform Multi-Stage CI on your repositories. The key benefit of Multi-Stage CI is to split a time-consuming CI process into smaller stages to detect issues in code quality / tests as early as possible and shorten the feedback loop on every checkin. This often entails refactoring or designing your application into smaller components and testing each component in isolation first before running more expensive integration tests of your component with other components in the system.
What is multi-stage CI?
In our multi-stage CI scenario, we split the CI of a Node.js app into several stages.
- Stage 2: Stage 2 runs on successful completion of Stage 1. In Stage 2, we run a small subset of tests to quickly validate the PR.
- Stage 3: Stage 3 runs on the merged commit to the repository. Here we run a broader set of core unit tests that take longer to run than Stage 2.
This is a pictorial representation of the workflow we're going to configure. The green boxes are jobs and the grey boxes are the input resources for the jobs. The workflow is defined across 3 configuration files: shippable.jobs.yml, shippable.resources.yml, and shippable.yml.
The code for this example is in GitHub:
- devops-recipes/multi-stage-ci-pipeline: This repository has the Shippable configuration files to setup the multi-stage CI pipeline.
devops-recipes/multi-stage-ci-app: This repository contains the source code and unit tests for a NodeJs app.
You can fork the repository to try out this sample yourself or just follow instructions to configure your own use case.
1. Author Stage 1 of the workflow
Add the following to your shippable.resources.yml file:
- name: app_gitRepo type: gitRepo integration: github pointer: sourceName: devops-recipes/multi-stage-ci-app # pointer to your repository buildOnPullRequest: true # update this resource when a pull request is opened or updated buildOnCommit: false # do not update this resource for commits
- Description: app_gitRepo is a gitRepo resource which is a pointer to the git repository that contains your source code. We have configured our gitRepo resource to only trigger on PRs.
- Shippable Integrations needed: default SCM account integration or other source control providers.
By default, you will already have an account integration with whichever SCM provider you've used to log into Shippable. If your source code repository is on that SCM account, you should use it as is.
If your source code repository is on another SCM account, create an integration for it by using one of the following supported SCM docs:
Add the following to your shippable.jobs.yml file:
jobs: - name: lint_runSh_job type: runSh steps: - IN: app_gitRepo
showBuildStatus: true # job status is shown on your PR page - TASK: - script: pushd $APP_GITREPO_PATH/gitRepo # env var pointing to directory containing files - script: npm install --only=dev - script: $APP_GITREPO_PATH/gitRepo/node_modules/.bin/eslint app.js # run lint on app.js
Description: lint_runSh_job is a runSh job that lets you run any shell script. This job implements the first stage of our multi-stage CI workflow. It takes
app_gitRepoas an input and runs scripts to lint the application source code.
Scripts: We have specified the package details for running
package.jsonin the devdependencies section. The scripts
pushdinto the source code directory using environment variables for the gitRepo resource, run
npm installand execute
C. Commit your shippable.resources.yml and shippable.jobs.yml files and add config to your Shippable account.
Once you have these configuration files as described above, commit them to your repository. The shippable.jobs.yml and shippable.resources.yml can be committed to the same app repository, or to a separate repository.
The repository containing your jobs and resources ymls is called a Sync repository and basically represents your workflow configuration.
Follow these instructions to import your configuration files into your Shippable account.
Stage 1 is complete at this point. We've configured the following:
2. Author Stages 2 and 3 of the workflow
A. Enable your repository
To set up the next stages, we will first setup CI for our application and author the logic to run the different unit test suites for commits and PRs. The prerequisites are to componentize your unit tests and organize them in two separate suites for each stage. We can then trigger them separately very easily.
To setup Shippable CI on your applictation repository, you need to do three things
- Sign in with your GitHub or Bitbucket credentials.
- Create the CI config file shippable.yml and commit to the root of your repository. The next section will go into the details of how to author shippable.yml file. If you're using a fork of our sample repository, you do not need to do this step since the repo already contains the config file.
- Enable your repository so that we can set up webhooks on your behalf.
Please note that enabling a project on shippable automatically creates a job and resource for you:
- A runCI job that represents your CI workflow and is named with (your_repo_name)_runCI. In our case, this is
multi-stage-ci-app_runCI.This is configured via shippable.yml in the next step.
- A ciRepo resource pointing to your repository, named with (your_repository)_ciRepo. In our case, this is
B. Author stages 2 and 3 in a shippable.yml file.
In this shippable.yml, we demonstrate the following key concepts for our Node.js application:
- Stage 2: Detecting CI triggered by a PR
- Stage 3: Detecting CI triggered for a merge commit
- Running unit tests using mocha and code coverage using Istanbul
- Configuring email notifications of your build
language: node_js env: global: - TEST_RESULTS_DIR=$SHIPPABLE_REPO_DIR/shippable/testresults - CODE_COVERAGE_DIR=$SHIPPABLE_REPO_DIR/shippable/codecoverage - TESTS_LOC_DIR=$SHIPPABLE_REPO_DIR/tests - MOD_LOC=$SHIPPABLE_REPO_DIR/node_modules/.bin/ build: ci: - shippable_retry npm install - mkdir -p $TEST_RESULTS_DIR && mkdir -p $CODE_COVERAGE_DIR - pushd $TESTS_LOC_DIR - | # Stage 2: For PRs, run basic tests
if [ "$IS_PULL_REQUEST" == true ] then echo "running the test suite for a PR build" $MOD_LOC/mocha name.spec.js -R mocha-junit-reporter --reporter-options mochaFile=$TEST_RESULTS_DIR/testresults.xml
# Stage 3: For commits and PR merges, run the core and important test suites for commits else echo "running the core test suite for a commit or manually triggered build" $MOD_LOC/mocha name.spec.js users.spec.js -R mocha-junit-reporter --reporter-options mochaFile=$TEST_RESULTS_DIR/testresults.xml fi - $MOD_LOC/istanbul --include-all-sources cover -root "$SHIPPABLE_REPO_DIR/routes" $SHIPPABLE_REPO_DIR/node_modules/mocha/bin/_mocha -- -R spec-xunit-file --recursive "$TESTS_LOC_DIR/**/*.spec.js" - $MOD_LOC/istanbul report cobertura --dir $CODE_COVERAGE_DIR - popd
# send notifications. reference at http://docs.shippable.com/ci/send-notifications/ integrations: notifications: - integrationName: email type: email on_success: change on_failure: always on_pull_request: never
For a complete reference on the configuration, please visit our shippable.yml documentation.
After you have authored and committed your shippable.yml file and enabled your project, run the project manually from the project dashboard. This should run the core tests for your application, which is like running Stage 3.
C. Commit your shippable.yml file.
D. Turn off automatic triggers for Pull Requests
Next, we need to ensure that our stage 2 workflow doesn't trigger for Pull Requests automatically. This is because we want it to be triggered only after the Stage 1 finishes. To turn off triggers for Pull Requests, follow instructions for Configuring Job Triggers and turn off triggers for PRs.
Stage 2 is now complete. We have authored the configuration for stages 2 and 3 :
If you notice, some text is striken through. This is because we have yet to connect Stage 1 to trigger Stage 2.
3. Connecting the Stages
We have now authored the configuration for all stages, but we still need to connect stage 1 to stage 2. This will ensure that when the
lint_runSH_job completes successfully, it will trigger stage 2.
To connect the stages, we augment the shippable.jobs.yml file:
jobs: - name: lint_runSh_job type: runSh steps: - IN: app_gitRepo - TASK: - script: pushd $APP_GITREPO_PATH/gitRepo - script: npm install --only=dev - script: $APP_GITREPO_PATH/gitRepo/node_modules/.bin/eslint app.js - OUT: multi-stage-ci-app_ciRepo # update the resource when this job completes replicate: app_gitRepo # copy all information from multi_ci_gitRepo to multi-stage-ci-app_ciRepo
- To connect Stage 1 to Stage 2, we first specify the
OUTon the Stage 1 job (
lint_runSh_job). This is the ciRepo resource which was created when you enabled the repository, and is an input to the runCI job. We also use the replicate directive to copy the metadata of the PR webhook from the gitRepo resource into the ciRepo resource so that the runCI job will be triggered with the right PR information.
- Commit your shippable.jobs.yml file after making this change.
- Your pipeline should look like this in your Shippable dashboard SPOG view. Navigate to the SPOG UI from your subscription dashboard by clicking on the eye icon followed by
Show SPOG View.
5. Trigger your pipeline
To test the pipeline, raise a PR from a fork of your repository on the master branch. This will trigger the workflow, run lint and then if that passes, then run your unit tests.
Click on the jobs in the SPOG to see the execution details of the latest run of your jobs.
Use the grid view to look at the entire history of runs for a specific job.
The history view for the linting job shows you the status of all commits that passed and failed the linting tests.
The history view for the CI job shows you the status of unit tests that were executed for all PR, commit and time trigger builds.
And that's it! This completes your multi-stage CI workflow!
Sign up to try this out: