Extend your CI workflows using Assembly Lines

- By Devashish Meena on September 20, 2018

Do more with less - No one disagrees with this famous quote, but how often do we really take a step back to think about pushing the envelope with what we have right now ? At Shippable, we've tried to constantly ask this question with every feature we've built. This ideology manifests itself in the capabilities the current Shippable workflows have, compared to what they used to a few years back. All this while keeping things simple and with zero additional costs to the customers. A lot of our customers started using Assembly Lines after the launch, a year go. Most of them didn't need much help but we do admit that some steps in extending traditional CI with the new Assembly Lines are a bit complicated. The objective of this post is to provide a detailed, step by step guide, to enable any CI job on Shippable to use the power of Assembly Lines 

Scenario

We'll move a sample repository that already has a CI job on Shippable to use Assembly lines. Once this is done, we'll enable a nightly build on the job

You can jump right into the Steps if you want to skip over the FAQ's. 

 

FAQ's

Q. Do I really have to do this ?

A. No, any CI jobs will keep running as-is and no one needs to do this.

 

Q. This sounds interesting, but what do I get by enabling Assembly Lines?

A. DevOps Assembly Lines give you the flexibility of combining multiple jobs according to your custom workflows, state management, triggers, rule engine etc. You can read more about it here.

 

Q. Why can't Shippable CI do all the Assembly Line stuff automatically?

A. It can, well, most of it. The primary reason why CI jobs cannot automatically be hooked up as Assembly Line jobs is because of backward compatibility. Enabling Assembly Lines was a big architectural change and we wanted to ensure we didn't break any existing builds. One compromise we had to make was requiring a few manual steps for anyone who wants to switch. That being said, we've enhanced the CI jobs too to work seamlessly with Assembly Lines.

 

Q. Ok, so what changes are required in CI yml ?

A. No changes to the CI definition. We'll just add some new entries in the shippable.yml file to enable Assembly Line components.

 

Q. What are these new yml entries you're talking about?

A. We're just doing two things - telling Shippable to parse  shippable.yml file to also read all the Assembly Line configuration along with CI steps (which

Shippable was already doing) and giving a name to the CI job, so it can be linked with other jobs. The new entries will be added to configure the Assembly Line.

 

Q. How long will this take?

A. About 15 mins. This is just a 3 step process which won't take long.

 

Q. How much do I have to pay for this?

A. Nothing, all the features of Assembly Lines are free. 

 

Steps

 

Step 1: Setup CI 

If this is not currently set up, use instructions here to set up a sample CI project. Enabling CI on any project will automatically add a job of type runCI. The CI build works and shows correct test and coverage reports, great ! Lets move on.

 

Step 2: Naming the CI job

As mentioned in the FAQ's, we'll now give this CI job a name so that it can be hooked up with any number of jobs in Assembly Lines. We'll also add a time resource which will trigger this job at 11:00 PM UTC everyday. Just add the following snippet at the top of the shippable.yml file, commit and push. 

resources:
- name: trigger_nightly
type: time
versionTemplate:
interval: "0 23 * * *"

jobs:
- name: basic-node_runCI
type: runCI
steps:
- IN: trigger_nightly

... rest of CI configuration as-is

 

The job again runs fine. Here's the proof ! But you'll notice that nothing changed in the job execution. The job steps are the same as well as reports and other console output. This is because we've only defined the job but not yet told Shippable to use this shippable.yml file for creating the Assembly Lines. Lets do that now.

 

Step 3: Enable Assembly Lines

Navigate to the Subscriptions page and click on the `+` icon.

 

Find the project using the search box

 

Click the `+` button in the column with header `Add Assembly Line`, fill in the details with default settings and click `Create`. This step will need a Subscription Integration to be created first. Integrations allow you to decouple the permissions from Assembly Line workflows. This, in turn, helps in cleanly managing permissions while keeping the workflows intact. 

 

This will create an rSync job since you've just added a new syncRepo. This is just a fancy way of telling Shippable: Hey, use this repository to define my Assembly Line jobs and resources and don't change anything you were doing with this earlier.

Well that's about it. Once you click the SPOG tab on the Subscription dashboard, you'll be able to see a configuration like this.

 

You can trigger any job directly from SPOG(by right-clicking on it) or from the Jobs list(in the Jobs tab).

 

Bonus Step: Add another job 

This isn't really needed, but I'd love to show how powerful Assembly Lines are while being really simple to configure at the same time. We'll just add one more step after the CI is complete and run some arbitrary commands. Remember we named the CI job earlier? This is where we'll use it. Update the shippable.yml with following snippet added to `jobs` section, commit and push.

 

jobs:
- name: basic-node_runCI
type: runCI
steps:
- IN: trigger_nightly

- name: post_ci_steps
type: runSh
steps:
- IN: basic-node_runCI
- TASK:
name: "Run these commands after CI job"
script:
- echo "OMG, the CI stuff worked"
- echo "What's the machine config here?"
- uname -a
- uptime
- echo "Ok, I'm done."

... rest of yml remains as-is

 

You can now see that the SPOG has been updated with the new runSh job which is linked with the previous job. The new job runs immediately after the CI job completes successfully. And this took less than a minute to add !!

 Conclusion 

I've tried to provide a basic howto migration guide from CI to Assembly lines with zero interruptions. What you get is:

- No need to upgrade the billing plans. The node parallelization limits still apply to all the jobs in Assembly Lines.

- Simple three step update process. We're here to help if you get stuck.

- Automation-as-code. You can create complex scenarios using the basic concepts provided by Shippable Assembly Lines while keeping everything in your SCM of choice.

- Leverage all the capabilities of Assembly Lines like triggers, managed jobs, runSh jobs, multi-platform support and much more. You can read this to get started.

- Full code for this blog is available here: https://github.com/ric03uec/basic-node 

We'd love to hear about your use case and would be happy to help out if you need any instructions or guidance on best practices to create the workflow that fits your needs. Feel free to reach out.

 

Topics: multi-stage-ci, Assembly lines, Workflows