Build Matrices

Availability

Build matrices use build profiles and plans. Support for multiple configuration profiles per repo and chaining together profiles in a build plan is in private release. This is a pre-release feature and should be used at your own discretion.

Usage Scenarios

Build matricies provide a syntax for users to quickly define multiple profiles along various axis:

  • Running the same tests with multiple different language versions
  • Running tests using different values of the same environment variables

Concepts

Build Profiles & Plans
Solano CI uses Build Profiles and Plans to implement Matrices. In order to fully understand how Build Matrices work, please refer to the documentation on Build Profiles and Plans.
SPLIT
Solano CI uses the SPLIT key to denote an array of possible values. This array will be expanded into separate configurations with each value of the array being substituted in the location of the original SPLIT.

Example: Running tests on multiple versions of ruby

The following configuration file runs a very simple test on two different version of ruby. As you can see, using the SPLIT: key, we can define an array of ruby versions that we wish to have our test run against.

tests:
  - `ruby -v`
ruby_version:
  SPLIT:
    - "1.9.3"
    - "2.1.2"

Solano CI will expand this configuration into two separate build profiles, one for each ruby version. These profiles will be automatically scheduled and run sequentially.

  1. ruby_version: 1.9.3
  2. ruby_version: 2.1.2

Example: Running tests with multiple versions of ruby and multiple environment variables

This configuration runs a simple test on two different versions of ruby as well as three different values of the INTEGRATION environment variable.

tests:
  - `ruby -v`
  - `echo $INTEGRATION`
ruby_version:
  SPLIT:
    - "1.9.3"
    - "2.1.2"
environment:
  SPLIT:
    - INTEGRATION: true
    - INTEGRATION: false
    - INTEGRATION: maybe

The matrix will expand into the full intersection of both SPLIT arrays. Six separate sessions will be run in Solano CI to satisfy the different combinations:

  1. ruby_version: 1.9.3, INTEGRATION: false
  2. ruby_version: 1.9.3, INTEGRATION: maybe
  3. ruby_version: 1.9.3, INTEGRATION: true
  4. ruby_version: 2.1.2, INTEGRATION: false
  5. ruby_version: 2.1.2, INTEGRATION: maybe
  6. ruby_version: 2.1.2, INTEGRATION: true

Running A Specific Matrix

Matrices are expanded into unique profile names, which are easily visible on the session’s report page. The profile names are in the format of {build_profile_name}_{matrix_index}. In a build with no defined profiles and a matrix of one axis with two selections (as in the first example) the profiles will be named default_1 and default_2 respectively.

Limitations

  • Matrices can not currently be expanded for profiles scheduled with plan_vars