Setup Hooks

Solano CI automates the setup and provisioning of most applications. Some applications, however, require special setup actions or a configuration file to start properly. Typical examples include

  • Generating configuration files, or copying them from examples
  • Special database setup and seeding
  • Compiling/packaging assets (e.g., from CoffeeScript or Sass for Rails apps)

Before your tests run on a target VM, Solano CI will look for a collection of setup hooks that configure the environment for tests. Solano CI uses a similar mechanism to run customizations after a build completes.

Setup Hook Syntax

Hooks are defined by adding a key-value pair to the hooks section of your solano.yml. This section is a hash and not a list so each hook is defined by a single command. The key is the name of the hook and the value is a string containing the command to run. The command is passed to the shell for execution. In particular:

---
hooks:
  worker_setup: bundle exec rake db:migrate db:seed # OK!
  post_setup:
    - echo "Step 1"       # NOT OK! -- must be a string, not a list
    - echo "Step 2"
  post_build: echo "Deploy Step 1"        # NOT OK! - hooks is a hash
  post_build: echo "Deploy Step 2"        #  Only one of these will run

Available Setup Hooks

Setup hooks correspond to build stages:

Hook Name Before/After Tests Parallel? Failure Result [1] Description
boot Before No Build Error Runs immediately after the repository is checked out, before any other setup occurs
package_setup Before No Build Error Runs after repository is checked out; must implement package dependency installation
pre_setup Before No Build Error Runs after repository is checked out and dependencies are installed
worker_setup Before Yes [2] Build Error Runs once for each worker in the system
post_setup [3] Before No [4] Build Error Runs after the worker_setup hook but before the build starts
N/A Tests Run Here Yes N/A Tests Run in Parallel after post_setup and before post_worker
post_worker [5] After Yes [6] Build Failure Runs once in each worker after that worker is done
post_build After No Build Failure Runs once globally after the build is complete

Notes

[1]Hook success/failure is determined by its exit code. Failing hooks that run before tests start will cause the build to stop with an error. Failing hooks that run after tests complete will cause the build to be marked as a failure.
[2]worker_setup hooks run in parallel and therefore must be parallel safe. See Build And Setup Failures for further details if you are running into trouble with your hooks.
[3]post_setup hooks run before all tests in a worker have been started.
[4]The post_setup hook runs in parallel on Solano’s V2runtime infrastructure. See the Custom Queues and V2runtime section below.
[5]post_worker runs after all of the tests in a worker have completed.
[6]post_worker hooks run in parallel and therefore must be parallel safe.

The following hook configuration is a simple example:

---
hooks:
  # load the database, dump it to file, and precompile assets
  pre_setup: bundle exec rake solano:db_load_and_dump assets:precompile
  # load the db dump file created by the pre_setup hook
  worker_setup: bundle exec rake solano:worker_db_load
  # Attach artifacts to build
  post_worker: ./scripts/collect-artifacts.sh
  # Custom deployment and notifications
  post_build: ./scripts/deploy-and-notify.sh

Hooks have access to the full range of Setting Environment Variables.

Note

If you override any hook with a script that you supply, the Solano CI-supplied defaults for all other hooks will be skipped.

Typical Usage

The boot hook is rarely ever needed. Since it runs before dependencies are installed and major subsystems (such as databases) are initialized, typical uses would be moving configuration files from non-standard locations to where Solano CI expects them, and to change dependency installation specifications for package managers that Solano CI automatically runs (bundler, pip, git submodule installation).

The pre_setup hook is typically used to load databases, install dependencies that are not automatically installed by Solano CI, and download any other external requirements. Often after an initial database load is performed, a database dump file is created to allow the worker_setup hook to quickly load per-worker databases. If dependencies are installed in the pre_setup hook, faster build times can often be acheived by Caching Dependencies.

The worker_setup hooks run once for each test worker and is typically used to prepare per-worker functionality. This can include database loading, Elasticsearch setup, setting up services, etc. Often a database dump file created in the pre_setup hook is used to load each worker’s database. The TDDIUM_TID environment variable can be used to identify each worker.

Currently, the post_setup hook is typically used to precompile assets and generate fixtures, as these often depend on the database already being loaded. For tasks that write to the file system, we recommended these tasks be moved into the pre_setup hook and executed after their dependent tasks. For example:

hooks:
  pre_setup: RAILS_ENV=test bundle exec rake solano:db_setup assets:precompile

The post_worker hook runs once on each worker after tests have been completed. Typical uses include per-worker Attaching Files And Artifacts and/or uploading worker generated results to third-party services.

The behavior of the post_build hook is often determined by the results of the build. Upon a passing build, the code can be deployed, and on any result, custom notifications can be run. (Solano CI natively supports many popular notification services.)

Custom Queues and V2runtime

Customers using Solano CI’s beta Custom Queues UI or otherwise using V2runtime workers should keep in mind the following:

  • Immediately after the pre_setup hook runs, the filesystem is cloned for the individual workers. Preparatory tasks that modify the filesystem, such as assets:precompile, should usually be run in the pre_setup hook (after any other dependant tasks).
  • Any worker_setup tasks should not depend on the TDDIUM_TID environment variable to determine if databases should be loaded. Instead, check if the individual worker’s database is empty.
  • Due to increased worker isolation, the post_setup hook will be run on each worker. Tasks that were called by this hook likely should be moved to the worker_setup hook, or preferably to the pre_setup hook if possible.

Ruby Setup Hooks (Deprecated)

By default, when Solano CI detects a Ruby on Rails application it will provide a default implementation of the database setup hook, which you may override. The default implementation should work for most Ruby applications. Read it in full in this gist: https://gist.github.com/1519804. There are separate hooks for Mongo and Redis setup. Customized database hooks are appropriate when you have checked in a raw SQL schema dump or find that the default implementation does not work for your application.

The default mechanism for customizing a build in Solano CI is to define a set of setup hooks in your solano.yml. If you are using ruby, you can also define a set of rake tasks. Note, however, that if you define any hooks via solano.yml, all of your hooks must be defined in solano.yml and any rake tasks will be ignored.

  • pre_setup_hook: the equivalent rake task is tddium:pre_hook
  • worker_setup_hook: the equivalent rake task is tddium:db_hook
  • post_setup_hook: the equivalent rake task is tddium:post_hook
  • post_worker_hook: the equivalent rake task is tddium:post_worker_hook
  • post_build_hook: the equivalent rake task is tddium:post_build_hook

Setup Hook Rails Environment

For Rails applications, the hooks run by default in the development environment; that is, with RAILS_ENV=development. The setting for RAILS_ENV when hooks are evaluated is separate from the environment settings for your tests. Generally, tests will run in the test environment but migrations will be applied in the development environment. You can change the default to the test environment by adding the following to config/solano.yml in your repository:

---
rake:
  rails_env: 'test'

Once you have configured your hook tasks, you should verify that they can run successfully.

Examples

Example: Deployment

For deployment and other post-build tasks, see Post Build Tasks.

Example: Populating Config Files (pre_setup hook)

One common case of the pre_setup hook is to install configuration files that you don’t want to check in to git.

For example, say you use a billing provider, and your application reads keys from config/billing_provider.yml. You don’t want these keys in git, but you’ve saved sample keys in config/billing_provider.example.yml so your developers can get up and running quickly.

You can write a pre_setup hook task like the following to use your example:

namespace :solano do
  desc "Solano environment pre-run setup"
  task :pre_setup_hook do
    system("cd config && ln -s billing_provider.example.yml billing_provider.yml")
  end
end

Example: Loading Raw SQL Dump (worker_setup hook)

Complex applications sometimes require the use of database features such as triggers or stored procedures that are not readily expressible in schema.rb. In these cases, a raw SQL dump is checked into the repository and loaded into the database by the worker_setup hook. A sample solano:worker_setup_hook rake task that you can adapt for your own use:

namespace :solano do
  desc 'Load database for eeach Solano worker'
  task :worker_setup_hook do
    Rake::Task[ 'db:create'].invoke
    cmd = 'psql -U "$TDDIUM_DB_PG_USER" -f db/development_structure.sql -d "$TDDIUM_DB_PG_NAME"'
    system cmd
  end
end

If you are using mysql, don’t forget to specify the password with something like this:

namespace :solano do
  desc 'Load database for eeach Solano worker'
  task :worker_setup_hook do
    Rake::Task[ 'db:create'].invoke
    cmd = 'mysql --socket="$TDDIUM_DB_MYSQL_SOCKET" --user="$TDDIUM_DB_MYSQL_USER" --password="$TDDIUM_DB_MYSQL_PASSWORD" "$TDDIUM_DB_MYSQL_NAME" < db/development_structure.sql'
    system cmd
  end
end