Solano CI automates the setup and provisioning of most applications. Some applications, however, require special setup actions or a configuration file to start properly. Typical examples include
- Generating configuration files, or copying them from examples
- Special database setup and seeding
- Compiling/packaging assets (e.g., from CoffeeScript or Sass for Rails apps)
Before your tests run on a target VM, Solano CI will look for a collection of setup hooks that configure the environment for tests. Solano CI uses a similar mechanism to run customizations after a build completes.
Setup Hook Syntax¶
Hooks are defined by adding a key-value pair to the hooks section of your solano.yml. This section is a hash and not a list so each hook is defined by a single command. The key is the name of the hook and the value is a string containing the command to run. The command is passed to the shell for execution. In particular:
--- hooks: worker_setup: bundle exec rake db:migrate db:seed # OK! post_setup: - echo "Step 1" # NOT OK! -- must be a string, not a list - echo "Step 2" post_build: echo "Deploy Step 1" # NOT OK! - hooks is a hash post_build: echo "Deploy Step 2" # Only one of these will run
Available Setup Hooks¶
Setup hooks correspond to build stages:
|Hook Name||Before/After Tests||Parallel?||Failure Result ||Description|
|early/boot ||Before||No||Build Error||Runs immediately after the repository is checked out, before any other setup occurs|
|package_setup||Before||No||Build Error||Runs after repository is checked out; must implement package dependency installation|
|pre_setup||Before||No||Build Error||Runs after repository is checked out and dependencies are installed|
|worker_setup||Before||Yes ||Build Error||Runs once for each worker in the system|
|post_setup ||Before||No ||Build Error||Runs after the worker_setup hook but before the build starts|
|N/A||Tests Run Here||Yes||N/A||Tests Run in Parallel after post_setup and before post_worker|
|post_worker ||After||Yes ||Build Failure||Runs once in each worker after that worker is done|
|post_build||After||No||Build Failure||Runs once globally after the build is complete|
|||Hook success/failure is determined by its exit code. Failing hooks that run before tests start will cause the build to stop with an error. Failing hooks that run after tests complete will cause the build to be marked as a failure.|
|||While the |
The following hook configuration is a simple example:
--- hooks: # move subsystem configuration/dependencies into place before subsystem starts early: mv solr-solano solr # load the database, dump it to file, and precompile assets pre_setup: bundle exec rake solano:db_load_and_dump assets:precompile # load the db dump file created by the pre_setup hook worker_setup: bundle exec rake solano:worker_db_load # Attach artifacts to build post_worker: ./scripts/collect-artifacts.sh # Custom deployment and notifications post_build: ./scripts/deploy-and-notify.sh
Hooks have access to the full range of Setting Environment Variables.
If you override any hook with a script that you supply, the Solano CI-supplied defaults for all other hooks will be skipped.
The boot hook is rarely ever needed. Since it runs before dependencies are installed and major subsystems (such as databases) are initialized, typical uses would be moving configuration files from non-standard locations to where Solano CI expects them, and to change dependency installation specifications for package managers that Solano CI automatically runs (bundler, pip, git submodule installation).
The pre_setup hook is typically used to load databases, install dependencies that are not automatically installed by Solano CI, and download any other external requirements. Often after an initial database load is performed, a database dump file is created to allow the worker_setup hook to quickly load per-worker databases. If dependencies are installed in the pre_setup hook, faster build times can often be acheived by Caching Dependencies.
The worker_setup hooks run once for each test worker and is typically used to prepare per-worker functionality. This can include database loading, Elasticsearch setup, setting up services, etc. Often a database dump file created in the pre_setup hook is used to load each worker’s database. The TDDIUM_TID environment variable can be used to identify each worker.
Currently, the post_setup hook is typically used to precompile assets and generate fixtures, as these often depend on the database already being loaded. For tasks that write to the file system, we recommended these tasks be moved into the pre_setup hook and executed after their dependent tasks. For example:
hooks: pre_setup: RAILS_ENV=test bundle exec rake solano:db_setup assets:precompile
The post_worker hook runs once on each worker after tests have been completed. Typical uses include per-worker Attaching Files And Artifacts and/or uploading worker generated results to third-party services.
The behavior of the post_build hook is often determined by the results of the build. Upon a passing build, the code can be deployed, and on any result, custom notifications can be run. (Solano CI natively supports many popular notification services.)
Custom Queues and V2runtime¶
Customers using Solano CI’s beta Custom Queues UI or otherwise using V2runtime workers should keep in mind the following:
- Immediately after the pre_setup hook runs, the filesystem is cloned for the individual workers. Preparatory tasks that modify the filesystem, such as assets:precompile, should usually be run in the pre_setup hook (after any other dependant tasks).
- Any worker_setup tasks should not depend on the TDDIUM_TID environment variable to determine if databases should be loaded. Instead, check if the individual worker’s database is empty.
- Due to increased worker isolation, the post_setup hook will be run on each worker. Tasks that were called by this hook likely should be moved to the worker_setup hook, or preferably to the pre_setup hook if possible.
Ruby Setup Hooks (Deprecated)¶
By default, when Solano CI detects a Ruby on Rails application it will provide a default implementation of the database setup hook, which you may override. The default implementation should work for most Ruby applications. Read it in full in this gist: https://gist.github.com/1519804. There are separate hooks for Mongo and Redis setup. Customized database hooks are appropriate when you have checked in a raw SQL schema dump or find that the default implementation does not work for your application.
The default mechanism for customizing a build in Solano CI is to define a set of setup hooks in your solano.yml. If you are using ruby, you can also define a set of rake tasks. Note, however, that if you define any hooks via solano.yml, all of your hooks must be defined in solano.yml and any rake tasks will be ignored.
- pre_setup_hook: the equivalent rake task is tddium:pre_hook
- worker_setup_hook: the equivalent rake task is tddium:db_hook
- post_setup_hook: the equivalent rake task is tddium:post_hook
- post_worker_hook: the equivalent rake task is tddium:post_worker_hook
- post_build_hook: the equivalent rake task is tddium:post_build_hook
Setup Hook Rails Environment¶
For Rails applications, the hooks run by default in the development environment; that is, with RAILS_ENV=development. The setting for RAILS_ENV when hooks are evaluated is separate from the environment settings for your tests. Generally, tests will run in the test environment but migrations will be applied in the development environment. You can change the default to the test environment by adding the following to config/solano.yml in your repository:
--- rake: rails_env: 'test'
Once you have configured your hook tasks, you should verify that they can run successfully.
Example: Populating Config Files (pre_setup hook)¶
One common case of the pre_setup hook is to install configuration files that you don’t want to check in to git.
For example, say you use a billing provider, and your application reads keys from config/billing_provider.yml. You don’t want these keys in git, but you’ve saved sample keys in config/billing_provider.example.yml so your developers can get up and running quickly.
You can write a pre_setup hook task like the following to use your example:
namespace :solano do desc "Solano environment pre-run setup" task :pre_setup_hook do system("cd config && ln -s billing_provider.example.yml billing_provider.yml") end end
Example: Loading Raw SQL Dump (worker_setup hook)¶
Complex applications sometimes require the use of database features such as triggers or stored procedures that are not readily expressible in schema.rb. In these cases, a raw SQL dump is checked into the repository and loaded into the database by the worker_setup hook. A sample solano:worker_setup_hook rake task that you can adapt for your own use:
namespace :solano do desc 'Load database for eeach Solano worker' task :worker_setup_hook do Rake::Task[ 'db:create'].invoke cmd = 'psql -U "$TDDIUM_DB_PG_USER" -f db/development_structure.sql -d "$TDDIUM_DB_PG_NAME"' system cmd end end
If you are using mysql, don’t forget to specify the password with something like this:
namespace :solano do desc 'Load database for eeach Solano worker' task :worker_setup_hook do Rake::Task[ 'db:create'].invoke cmd = 'mysql --socket="$TDDIUM_DB_MYSQL_SOCKET" --user="$TDDIUM_DB_MYSQL_USER" --password="$TDDIUM_DB_MYSQL_PASSWORD" "$TDDIUM_DB_MYSQL_NAME" < db/development_structure.sql' system cmd end end