Matrix builds for minor Java versions with Maven, Artifactory and Shippable

- By Tom Trahan

BACK TO BLOG HOME

I recently had someone ask whether Shippable supports testing their builds against multiple minor Java versions. The answer is "YES!" and in this blog I'll share two ways to do it and walk through an example setup.

java-maven-artifactory.png

Shippable provides standard build machine images for testing against Oracle JDK v8 and v7, as well as OpenJDK v7 and v6. These come pre-loaded (as do Maven, Ant, and Gradle) and by merely calling out which one(s) you'd like to build and test with in your shippable.yml file, your builds will run without any further installation on your part. You can read more about that in our docs

If you need to test against a minor release version, however, you'll need to choose between two approaches and set up your shippable.yml to handle it.  

Two approaches you can take include:

  • Install the required minor version as part of your build process
  • Add the minor version to the Shippable standard image and use that image for your builds

With the first option, you won't have to manage build images as a separate process, while with the second option you'll achieve shorter build duration by avoiding the time needed for downloading and installing the minor versions.

I'll walk you through the first approach. You can find the code for both examples here:

Sample 1: Install during build process (branch = matrix-minor)
Sample 2: Add minor versions to standard image (branch = matrix)

 

Steps required to install the minor version during your build process:

  1. Specify the minor versions to use in the env: section of the shippable.yml
  2. For each minor version desired, add a script for installing and registering the minor version
  3. Add logic to specify the appropriate version to use

First, you'll need to identify the minor versions that you wish to have available to your builds. For this sample, I have three available minor versions identified: oracle-java-1.8.0_51oracle-java-1.8.0_91, and oracle-java-1.8.0_92. You can name them whatever you'd like, but you'll need to use the names again in step 2.  You'll put these in the env: section of the shippable.yml as follows:

language: java

jdk:
  - oraclejdk8

env:
  matrix:
    - VERSION_1=oracle-java-1.8.0_51
    - VERSION_2=oracle-java-1.8.0_91
    - VERSION_3=oracle-java-1.8.0_92

Note that this will run three builds for every commit, one for each version in the matrix. I've also indicated to use the Shippable standard Java image by specifying the language: and jdk: sections.

Next, you'll need to add a script to install and register the minor version within the build. In my example, I've created a directory in my repo called java_versions to hold each of the install scripts and added three scripts using the name specified in the env: section. For instance, you'll see a file called oracle-java-1.8.0_51.sh (don't forget to make the files executable).

The scripts should follow this format:

#!/bin/bash -e

echo "========== Installing oracle-java8-minor-release-1.8.0_51 =========="
echo oracle-java8-minor-release shared/accepted-oracle-license-v1-1 select true | debconf-set-selections

# create dir for minor version
mkdir -p /usr/lib/jvm/oracle-java-1.8.0_51
cd /usr/lib/jvm

# download java minor version
wget --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u51-b16/jdk-8u51-linux-x64.tar.gz

# unzip
tar -zxf jdk-8u51-linux-x64.tar.gz -C /usr/lib/jvm/oracle-java-1.8.0_51

# move unzipped files up one level (note - must use cp, not mv due to file system bug)
cp -aR /usr/lib/jvm/oracle-java-1.8.0_51/jdk1.8.0_51/. /usr/lib/jvm/oracle-java-1.8.0_51/

# # remove original unzipped directory
rm -rf /usr/lib/jvm/oracle-java-1.8.0_51/jdk1.8.0_51/

# register version
update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/oracle-java-1.8.0_51/jre/bin/java" 51
update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/oracle-java-1.8.0_51/bin/javac" 51

Note that the URL for downloading a JDK version is not always consistent across versions, so take care to ensure you have it correct for each version. For instance, for version 1.8.0_51, the path includes '8u51-b16'. However, for version 1.8.0_91, it's '8u91-b14'. Also note that the priority I've set in the update-alternatives command just matches the version number.  You can set this to any appropriate value.

Once you have added the install scripts for each of the minor versions you require, the last step is to add logic to specify the appropriate version to use within a particular build. So, first, I've added logic to specify the appropriate install script to use based on which build in the matrix this is for. That occurs in this line in the shippable.yml where I use the Shippable-provided environment variable for SHIPPABLE_JOB_NUMBER to differentiate between jobs (e.g. job 1, job 2, and job 3) and identify the appropriate version to build.

build:
  ci:
    # identify and set appropriate java version for this build
    - tmp=VERSION_$SHIPPABLE_JOB_NUMBER && export BUILD_VERSION=${!tmp}

Next, I added and called a script to tell the build machine to use the appropriate version. You'll find that logic in the set_java_version.sh script.

#!/bin/bash -e

#
# Function to install and set up JAVA version depending on env variable
# called BUILD_VERSION in shippable.yml
#

STANDARD_VERSIONS="java-8-oracle java-7-oracle java-7-openjdk-amd64 java-6-openjdk-amd64"

if [[ ! $STANDARD_VERSIONS =~ $BUILD_VERSION ]]; then
  export INSTALL_VERSION=$BUILD_VERSION.sh
  . $SHIPPABLE_BUILD_DIR/java_versions/$INSTALL_VERSION
fi

setup_jdk() {
  export JAVA_HOME="/usr/lib/jvm/$BUILD_VERSION";
  export PATH="$PATH:/usr/lib/jvm/$BUILD_VERSION/bin";
  java_path="/usr/lib/jvm/$BUILD_VERSION/jre/bin/java";
  javac_path="/usr/lib/jvm/$BUILD_VERSION/bin/javac";

    if [ -f $java_path ]; then
      update-alternatives --set java $java_path
    fi

    if [ -f $javac_path ]; then
      update-alternatives --set javac $javac_path
    fi

    java -version
}

# Call setup JAVA version
setup_jdk

Note that I skip the installation if the value of $BUILD_VERSION references one of the included java versions in the Shippable standard image. If not, it will execute the appropriate install script. After completing installation, it will update the necessary environment variables and register the new version with the update-alternatives command.

That's it. You're now able to install any number of particular Java version on top of the Shippable standard Java image as part of a set of matrix builds. The only limit to how many you can test simultaneously is the size of your Shippable subscription.

Check out the other sample in the GitHub repo for an example on how to add minor versions to the Shippable standard Java image ahead of time and use that image for your build machines.

 

Bonus example:

Although not required to achieve the matrix builds for minor Java versions, I'll point out that in this example you'll also find the details on how to dynamically add Artifactory credentials to your Maven settings.xml file in order to securely encrypt your password within Shippable and avoid exposing this information in plain text in your repo.

First, note that I've included the standard .m2 Maven directory within my repo that holds my settings.xml file. In place of the passwords normally stored in plain text by Maven in the <servers> section, I've supplied placeholders in the form '{PASSWORD}'.

<servers>
  <server>
    <username>admin</username>
    <password>{PASSWORD}</password>
    <id>central</id>
  </server>
  <server>
    <username>admin</username>
    <password>{PASSWORD}</password>
    <id>snapshots</id>
  </server>
</servers>

Within the shippable.yml, I then add to the env: section an encrypted secure variable to store the password (see the docs for instructions on doing so). For this example, I named the secure password PASSWORD.

Ensure you're in the directory holding your source code using the Shippable-provided environment variable:

    - cd $SHIPPABLE_BUILD_DIR

Then execute a sed command to find and replace the placeholder with the secure password:

    - sed -i "s/{PASSWORD}/$PASSWORD/" .m2/settings.xml

Then, I execute the Maven command with an additional flag to specify the location of my updated settings.xml file:

    - mvn -B -s .m2/settings.xml -U clean deploy

With those settings, you can securely download from and deploy to your Artifactory repository using Maven without exposing your credentials.

Now, try out the sample on Shippable.

Try Shippable                                 

 

Topics: maven, artifactory, java, matrix builds