Demystifying GitLab CI/CD Variables

   This blog post is Unfiltered   

GitLab provides a lot of flexibility when it comes to defining and using variables. They are extremely useful for controlling your jobs and pipelines, and they help you avoid hard-coding values in your .gitlab-ci.yml configuration file. Through this post, I aim to weave a larger picture by bringing together all (or most) of the information around defining and handling variables so it is easy to understand the scope and capabilities. To avoid overloading readers with information, the documentation is surfaced contextually depending on the task in focus.

In GitLab CI/CD, variables can be used to customize your jobs by defining and storing values that they could make use of. When you use variables, you don't have to hard code values. In GitLab, these variables could be defined by going to Settings » CI/CD » Variables, or by simply defining them in the .gitlab-ci.yml file.

Variables are useful for configuring third party services for different environments, like testing, staging, production, etc. You can modify the services attached to those environments by simply modifying the variable that points to the API endpoint the services need to use. You can also use variables to configure your jobs, and then have them available as environment variables within the jobs when they start running.

The relationship between variables and environments

Software development as a process includes stages when you test your product before rolling it out to users. Environments are used to define what those stages look like and it may differ from team to team or organization to organization.

On the other hand, variables are data values that are likely to change as a result of user interaction with a product. For example, their age, preference, or any input you could possibly think of that might determine their next step in the product task-flow.

We often hear the term environment variable. These are variables that are defined in a given environment, but outside the application. They provide developers the ability to configure values in their code. How are they helpful? Using environment variables ensures that your code is flexible. They allow you to modify an application deployed to a certain environment without making any change to your code. You can run tests or even integrate third party services by merely changing a configuration environment variable outside the application.

Variables and Scopes in GitLab CI/CD

.gitlab-ci.yml defined variables

In GitLab, you can add variables that are available in the job environment. They are meant to store non-sensitive project configuration, like RAILS_ENV or DATABASE_URL, in the .gitlab-ci.yml file. You can reuse this variable wherever the value is needed, in multiple jobs or scripts. If the value changes, you only need to up date the variable once, and the change is reflected everywhere the variable is used.

Group-level environment variables

Project CI/CD variables

Moving a step above the repository specific requirements, you can define CI/CD variables in project settings, but have them made available to CI/CD pipelines. These are stored out of the repository (not in the .gitlab-ci.yml file), but are still available for use in the CI/CD configuration and scripts. By storing them outside the .gitlab-ci.yml file, you can keep the values of things like tokens and passwords secret.

Group and Instance CI/CD variables

Some variables are relevant at the group level, or even organization level, and could be useful to all projects in a group or instance. You can define these in the group or instance settings, and all projects within those scopes can use the variables without actually needing to know the values. For example, a common value that needs to be updated in multiple projects can easily be managed if can keep it up to date in a single place. Alternatively, multiple projects could use a specific password without actually needing to know the value of the password itself.

Jobs and pipelines as environments

The GitLab CI/CD variables, besides being used as environment variables, also work in the scope of the .gitlab-ci.yml configuration file, unrelated to any environment. They can be stored in the project/group/instance settings and be made available to jobs in pipelines.

For example:

job:
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
  - echo "This job ran on the $CI_COMMIT_BRANCH branch."

The variable in the script section runs in the scope of the job in which it was defined. This scope is the "job environment". That is, when the job starts, the runner stars up a docker container and runs the job in that environment. The runner will make that variable (and all other predefined or custom variables) available to the job, and it can display their value in the log output if needed.

But the variable is also used in the if: section, to determine when the job should run. That in itself is not an environment, which is why we call these CI/CD variables. They can be used to dynamically configure your CI/CD jobs, as well as be used as environment variables when the job is running.

Pre-defined variables

A number of variables are pre-defined when a GitLab CI/CD pipeline starts. You can immediately access values for things like commit, project, or pipeline details without needing to define the variables themselves.

Custom CI/CD variables

When you create a CI/CD variable in the settings, GitLab gives you more configuration options for the variable. Use these extra configuration options for stricter control over more sensitive variables:

1. Environment scope: If a variable only ever needs to be used in one specific environment, you can set it to only ever be available in that environment. For example, you can set a deploy token to be available in the production environment only.

2. Protected Variables: Similar to the environment scope, you can set a variable to be available only when the pipeline runs on a protected branch, like your default branch.

3. Masked: Variables that contain secrets should always be masked. This lets you use the variable in job scripts without the risk of the variable value being exposed. If someone tries to output it in a job log with a command like echo $VARIABLE, the job log will only show echo [masked]. There are, however, certain limitations to the types of values that can be masked. Additionally, through secret management offerings, GitLab also enables its users to leverage HashiCorp Vault to securely manage keys, tokens, and other secrets at the project level by installing it as a managed application within a Kubernetes Cluster, allowing them to separate those from other CI/CD variables for security reasons.

4. Variable type: A few applications require configuration to be passed to it in the form of a file. If you use an application that requires this, you can set a CI/CD variable to be a "File" type variable. This means that when the runner makes the variable available in the environment, it actually writes it out to a temporary file, and stores the path to the file as the value. You can then pass the path to the file to any applications that need it.

---

Alongwith the listed ways of using and defining variables, GitLab recently introduced a feature that generates pre-filled variables from .gitlab-ci.yml file when there's a need to override a variable or run a pipeline manually. This reduces the chances of running into an error and makes running pipeline easier.

Read more about GitLab CI/CD Variables in the documentation.

Special thanks to Marcel Amirault for contributing content to the blog post.

from GitLab https://ift.tt/3fWA4nr #GitLab #DevSecOps