4. Job Control and Execution Flow

Chapter 3 decided which pipelines exist and how they connect. This chapter goes one level down: which jobs appear in a given pipeline, when each one is allowed to start, what happens when it fails or hangs, and how many copies of it run at the same time. These controls make one .gitlab-ci.yml behave correctly for a feature branch, a merge request, a release tag and a nightly schedule. They also keep pipelines fast, cheap and safe to deploy from.

<a id="41-rules"></a>

4.1 Rules

rules is the keyword that decides, job by job, whether a job is added to a pipeline and how it behaves once added. This subchapter covers each type of rule clause, how a list of rules is evaluated, and the expression language used inside rules:if.

rules:if

Why it matters. A single configuration file has to serve many situations. Unit tests should run everywhere, a deployment only on the default branch, a release job only for version tags. Without conditions, every job would run in every pipeline. rules:if is the most common way to attach a condition to a job, and most other job-control features build on it.

What it is. rules is a list of rules. Each rule can contain one or more clauses (if, changes, exists) and attributes that apply when the rule matches (when, allow_failure, variables, needs, start_in, interruptible). rules:if holds a boolean expression over CI/CD variables. GitLab evaluates it when it creates the pipeline, before any job runs.

Think of rules as a bouncer with a guest list read from top to bottom. The first line that describes you decides whether you get in and which room you go to. If no line describes you, you do not get in.

deploy-production:
  stage: deploy
  script: [./deploy.sh production]
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH   # compare two variables
      when: manual                                  # match: add the job as a manual job
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/      # regex match on a semantic version tag
      when: on_success                              # match: add the job, run automatically
    # no match: the job is not added to the pipeline

The attributes a matching rule can set:

Attribute	Effect when the rule matches	Default
when	on_success, on_failure, always, manual, delayed or never	on_success
allow_failure	Whether a failure of this job blocks the pipeline	false (also for when: manual in rules)
start_in	Delay for when: delayed	none
variables	Variables added to or overriding the job's variables	none
needs	Replaces the job's needs	job-level needs
interruptible	Overrides the job's interruptible	job-level value

Which variables can rules:if see? Only those known when the pipeline is created:

Available in rules:if	Not available in rules:if
Predefined variables such as CI_COMMIT_BRANCH, CI_PIPELINE_SOURCE	Variables exported in script
Project, group and instance variables	Variables from dotenv report artifacts
Pipeline variables (Run pipeline, API, trigger, schedule)	Runner-only variables such as CI_JOB_ID, CI_RUNNER_*
Global and job-level variables from the YAML	Variables set by the same rule's rules:variables
Variables set by workflow:rules:variables

only/except cannot be used in the same job as rules. Put when and start_in inside the rules rather than at job level, so each condition carries its own behavior.

Key takeaways

• rules is an ordered list. A job is added only if at least one rule matches.
• rules:if evaluates a variable expression at pipeline creation, so only variables known at that time can be used.
• A matching rule can change when, allow_failure, variables, needs, start_in and interruptible for the job.
• rules and only/except are mutually exclusive in a job.

Practice

1. (Beginner) Write a lint job that runs on every branch except main, and a deploy job that runs only on main. Push to both branches and confirm which jobs appear.
2. (Intermediate) Add a release job that runs automatically for tags matching v1.2.3 and is a manual job for any other tag. Test it with v1.2.3 and v1.2.3-rc1.
3. Interview question: "A job has rules: - if: $BUILD_FLAVOR == "debug", and BUILD_FLAVOR is exported in the before_script of the same job. The job never appears. Why?" Hint: consider when rules are evaluated versus when scripts run.

rules:changes

Why it matters. In a repository with documentation, a frontend and a backend, a typo fix in README.md should not trigger a 40-minute backend test suite. rules:changes adds a job only when files matching given paths changed, which saves time and runner minutes, especially in monorepos.

What it is. rules:changes lists file paths or glob patterns. The clause matches when at least one changed file matches at least one pattern. "Changed" depends on the pipeline type:

Pipeline type	Files compared
Merge request pipeline	All changes in the MR, compared to the target branch
Branch pipeline (push to an existing branch)	Commits in the push, compared to the previous commit of the branch before the push
New branch, tag, schedule, Run pipeline, API, trigger	No push diff exists, so changes always evaluates to true
Any pipeline with changes:compare_to	Diff between the pipeline's commit and the given ref

The last two rows explain the most common surprise: on a new branch, every changes rule matches and every job runs. compare_to fixes this by giving GitLab an explicit base to compare against.

backend-tests:
  script: [make -C backend test]
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes:                                    # if AND changes must both match
        - backend/**/*                            # ** matches any depth of subdirectories
        - shared/{proto,schemas}/**/*             # {a,b} matches either alternative
        - .gitlab-ci.yml                          # CI config changes should re-run tests

frontend-tests:
  script: [npm --prefix frontend test]
  rules:
    - if: $CI_COMMIT_BRANCH
      changes:
        paths:
          - frontend/**/*
        compare_to: refs/heads/main               # compare against main, not against the previous push

Glob notes:

• Patterns are relative to the repository root and use Ruby File.fnmatch with path matching, dot files and {a,b} enabled.
• dir/* matches files directly in dir. dir/**/* matches files at any depth.
• A deleted file counts as a change to its path.
• paths accepts a limited number of entries per changes section (50 at the time of writing). Use broader globs instead of long file lists.

Push of commit C on feature/login (branch existed before):
  A ── B ── C            changes compares B..C  -> only files changed by C

First push of a new branch:
  (no previous commit of this branch)            -> changes = true for every rule

Key takeaways

• rules:changes matches when at least one changed file matches at least one pattern.
• MR pipelines compare against the target branch, branch pipelines against the commit before the push.
• New branches, tags, schedules and API-triggered pipelines have no diff, so changes is always true.
• changes:compare_to sets an explicit base ref and makes branch-pipeline behavior predictable.

Practice

1. (Beginner) Create docs-build and app-tests jobs that run only when files under docs/ or src/ change, respectively. Change only a doc file in an MR and confirm only docs-build runs.
2. (Intermediate) Push a brand-new branch containing only a doc change and observe that both jobs run. Fix it with compare_to and verify again.
3. Interview question: "A team relies on rules:changes in branch pipelines to skip expensive tests. After a force push, tests that should have run were skipped. What happened?" Hint: think about which commits the push diff contains after history is rewritten, and what MR pipelines compare instead.

rules:exists

Why it matters. Shared templates are used by many projects with different contents. A template's docker-build job only makes sense if the project has a Dockerfile, and a helm-lint job only if there is a chart. rules:exists lets a job decide based on what is in the repository, not on what changed.

What it is. rules:exists lists paths or glob patterns. The clause matches if at least one file matching any pattern exists in the repository at the pipeline's commit. Since GitLab 17.x it can also check another project and ref with exists:project and exists:ref.

docker-build:
  image: docker:27
  services: [docker:27-dind]
  script: [docker build -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA" .]
  rules:
    - exists:
        - Dockerfile                    # only if the project ships a Dockerfile

helm-lint:
  image: alpine/helm:3.15.4
  script: [helm lint charts/*]
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      exists:
        - charts/*/Chart.yaml           # any chart directory

check-shared-config:
  script: [echo "Shared config present"]
  rules:
    - exists:
        paths: [configs/defaults.yml]
        project: platform/shared-config # look in another project
        ref: main                       # at this ref

Clause	Question it answers	Typical use
changes	"Did these files change?"	Skipping work for untouched parts of a monorepo
exists	"Do these files exist?"	Generic templates that adapt to the project's content

For performance, GitLab caps the number of file checks performed for glob patterns. Past the cap, glob-based exists clauses are treated as matching, so prefer specific paths in very large repositories.

Key takeaways

• rules:exists matches if any file matching the patterns exists at the pipeline's commit.
• It is ideal for shared templates that should only activate in projects with the relevant files.
• exists:project and exists:ref check files in another project.
• Specific paths are cheaper and more predictable than broad globs in large repositories.

Practice

1. (Beginner) Write a python-tests job that runs only if pyproject.toml or requirements.txt exists. Include it in a project with neither file and confirm it is absent.
2. (Intermediate) Build a hidden template job .container-scan that real jobs extend, which activates only when a Dockerfile exists anywhere in the repository.
3. Interview question: "Why might a platform team prefer rules:exists over a project-level CI/CD variable like ENABLE_DOCKER=true to turn template jobs on?" Hint: think about configuration drift, discoverability and what the repository already tells you.

rules:variables

Why it matters. Often the same job runs in several situations with small differences: deploying to staging from main and to production from tags, or running a reduced test set on feature branches. Duplicating the job for each case invites drift. rules:variables lets each rule set its own variable values for one job definition.

What it is. rules:variables is a map of variables that is applied only when that rule matches. These values override job-level and global variables with the same name. They become available to the job's scripts and to other keywords that support variable expansion, such as environment:name.

variables:
  TEST_SCOPE: "smoke"                            # global default

deploy:
  stage: deploy
  script:
    - ./deploy.sh "$DEPLOY_ENV" --replicas "$REPLICAS"
  environment:
    name: $DEPLOY_ENV                            # expanded from the rule's variables
  rules:
    - if: $CI_COMMIT_TAG                         # releases
      variables:
        DEPLOY_ENV: production
        REPLICAS: "6"                            # quote numbers: variable values are strings
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      variables:
        DEPLOY_ENV: staging
        REPLICAS: "2"

tests:
  script: [./run-tests.sh --scope "$TEST_SCOPE"]
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      variables:
        TEST_SCOPE: "full"                       # overrides the global "smoke"
    - when: on_success                           # elsewhere keep the global value

Pipeline on main:   deploy(DEPLOY_ENV=staging, REPLICAS=2)     tests(TEST_SCOPE=full)
Pipeline on v2.0.0: deploy(DEPLOY_ENV=production, REPLICAS=6)  tests(TEST_SCOPE=smoke)
Pipeline on feat/x: deploy not added                           tests(TEST_SCOPE=smoke)

A rule's variables are set after the rule is evaluated, so they cannot be used in the same rule's if, or in an earlier rule of the same job.

Key takeaways

• rules:variables applies variables only when its rule matches.
• They override job-level and global YAML variables with the same name.
• They can drive other keywords that expand variables, such as environment:name.
• They cannot influence the if of the rule that defines them.

Practice

1. (Beginner) Make a build job that sets BUILD_MODE=release on tags and BUILD_MODE=debug everywhere else, and prints it.
2. (Intermediate) Merge two near-identical jobs, deploy-staging and deploy-production, into one job driven by rules:variables. Keep the production deployment manual.
3. Interview question: "When would you keep two separate deploy jobs instead of one job with rules:variables?" Hint: consider protected environments, different needs, readability in the pipeline graph and audit trails.

rules:needs

Why it matters. Chapter 3 showed that needs controls both ordering and artifact flow. Sometimes those dependencies differ by situation: a production deployment should wait for a security scan that only runs on the default branch, while a review deployment should not. rules:needs lets each rule choose the job's dependencies.

What it is. When a rule with needs matches, its list replaces the job-level needs entirely. The syntax is the same as job-level needs: job names, or the long form with job, artifacts and optional. A rule without needs keeps the job-level value.

stages: [build, test, deploy]

build:
  stage: build
  script: [make build]

security-scan:
  stage: test
  script: [make scan]
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

unit-tests:
  stage: test
  script: [make test]

deploy:
  stage: deploy
  script: [./deploy.sh]
  needs: [build, unit-tests]                     # default dependencies
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      needs: [build, unit-tests, security-scan]  # replaces the job-level list on main
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      when: manual                               # keeps the job-level needs

main:  build ──> unit-tests ──┐
              └─> security-scan ─┴─> deploy
MR:    build ──> unit-tests ──> deploy (manual)        security-scan not in the pipeline

Every job listed by a matching rule must exist in the pipeline, or must be marked optional: true, otherwise pipeline creation fails with a configuration error.

Key takeaways

• rules:needs replaces, not extends, the job-level needs when its rule matches.
• It supports the same short and long forms as job-level needs.
• It is useful when dependencies only exist in some pipeline types.
• Listed jobs must be present in that pipeline or be optional.

Practice

1. (Beginner) Reproduce the example above and compare the pipeline graphs of an MR pipeline and a main pipeline.
2. (Intermediate) Add an e2e-tests job that only runs on main, and make deploy depend on it there with rules:needs. Then remove e2e-tests' rule temporarily and explain the result.
3. Interview question: "Compare rules:needs with job-level needs using optional: true. When is each the better choice?" Hint: one encodes intent per situation, the other tolerates absence. Think about accidental dependency loss.

Rule Evaluation Order

Why it matters. Most rule bugs are ordering bugs: a broad rule placed first swallows cases that a later, more specific rule was meant to handle. Knowing exactly how GitLab walks the list makes rules predictable and easy to review.

What it is. GitLab evaluates rules with these steps:

1. Rules are checked top to bottom.
2. Within one rule, all clauses (if, changes, exists) must match. They are combined with AND.
3. The first matching rule wins. Its attributes are applied and later rules are ignored.
4. If a matching rule has when: never, the job is excluded.
5. If no rule matches, the job is excluded.
6. A rule with no clauses (only when, variables, ...) always matches. It acts as a catch-all and belongs at the end.

              ┌──────────────────────────────┐
  rule 1 ───> │ all clauses true?            │── yes ──> apply rule 1's when/attrs ──> done
              └──────────────────────────────┘
                     │ no
                     v
  rule 2 ───> same check ... ── yes ──> apply rule 2 ──> done
                     │ no
                     v
                    ...
                     │ no rule matched
                     v
              job NOT added (implicit when: never)

The classic exclusion pattern places when: never rules first, then inclusion rules:

integration-tests:
  script: [make integration]
  rules:
    - if: $CI_COMMIT_MESSAGE =~ /\[skip integration\]/   # 1. explicit opt-out
      when: never
    - if: $CI_PIPELINE_SOURCE == "schedule"               # 2. schedules use their own job
      when: never
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"    # 3. MRs: only if code changed
      changes: [src/**/*, tests/**/*]
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH         # 4. always on main

Reordering changes behavior. If rule 4 were first, a [skip integration] commit on main would still run the tests.

Workflow rules (section 4.4) run before job rules. If the workflow excludes the pipeline, no job rules are evaluated at all.

Pattern	Meaning
- when: never at the end	Redundant: the job is already excluded when nothing matches
- when: on_success at the end	Catch-all: add the job in every remaining case
A rule with only changes	Matches in every pipeline type, including both MR and branch pipelines
Several clauses in one rule	AND
Several rules	OR, but first match decides the attributes

Key takeaways

• Rules are evaluated in order, and the first match decides everything.
• Clauses inside one rule are ANDed. Separate rules are effectively ORed.
• No match means the job is excluded, so a final when: never is unnecessary.
• Put exclusions first, specific inclusions next and catch-alls last.

Practice

1. (Beginner) Given four rules on a job, write down which rule matches for a push to main, a tag v1.0.0 and an MR pipeline. Then verify with the pipeline editor's validation (simulate default branch) and real pipelines.
2. (Intermediate) A job uses - changes: [src/**/*] as its only rule. Explain why it appears twice when an MR is open, and rewrite the rules so it runs once.
3. Interview question: "How do you express 'run on main OR on tags, but never when the commit message contains [wip]' with rules?" Hint: there is no NOT operator for whole expressions, but rule order can give you one.

Expression Syntax and Operators

Why it matters. rules:if expressions look like shell, but they are a small, separate language. Misunderstanding it, for example by forgetting that regex patterns are unanchored or that an empty variable counts as "not set", leads to jobs that silently run or vanish.

What it is. The supported elements:

Element	Example	Meaning
Variable presence	$CI_COMMIT_TAG	True if defined and not empty
Equality	$CI_COMMIT_BRANCH == "main"	String equality
Inequality	$CI_COMMIT_BRANCH != "main"	String inequality
Variable vs variable	$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH	Compares two values
Null check	$DEPLOY_ENV == null	True if the variable is undefined
Empty check	$DEPLOY_ENV == ""	True if defined but empty
Regex match	$CI_COMMIT_REF_NAME =~ /^release\/.+$/	RE2 regular expression match
Regex non-match	$CI_COMMIT_REF_NAME !~ /^wip\//	True if the pattern does not match
Case-insensitive	$CI_COMMIT_MESSAGE =~ /deploy/i	i flag
Logical AND / OR	$A && $B, $A || $B	&& binds tighter than ||
Grouping	($A || $B) && $C	Parentheses override precedence

Important details:

• Regex is unanchored. /main/ matches maintenance and domain-fix. Use ^ and $ for whole-value matches.
• RE2 syntax is used, so lookaheads and backreferences are not supported.
• The right side of =~ can be a variable containing a pattern such as /^release-.*$/. Its content is treated as a regex.
• No arithmetic or ordering comparisons (<, >). Values are strings.
• No unary NOT for whole expressions. Use !=, !~ or a preceding when: never rule.
• Quoting. YAML needs quotes when an expression starts with a character such as (, ' or !. Quoting the whole expression with single quotes is always safe.

variables:
  RELEASE_BRANCH_PATTERN: '/^release\/\d+\.\d+$/'    # a regex stored in a variable

publish:
  script: [./publish.sh]
  rules:
    # Parentheses: tags or release branches, and only from push events
    - if: '($CI_COMMIT_TAG || $CI_COMMIT_BRANCH =~ $RELEASE_BRANCH_PATTERN) && $CI_PIPELINE_SOURCE == "push"'

nightly-report:
  script: [./report.sh]
  rules:
    # Case-insensitive opt-in via commit message, or any scheduled pipeline
    - if: $CI_COMMIT_MESSAGE =~ /\[report\]/i || $CI_PIPELINE_SOURCE == "schedule"

cleanup:
  script: [./cleanup.sh]
  rules:
    - if: $CLEANUP_TARGET == null                     # variable not defined at all
      when: never
    - if: $CLEANUP_TARGET == ""                       # defined but empty
      when: never
    - when: on_success

Precedence example:  $A || $B && $C   is evaluated as   $A || ($B && $C)

Key takeaways

• Expressions support presence checks, ==, !=, =~, !~, &&, || and parentheses.
• A bare $VAR is true only if the variable is defined and non-empty.
• Regex uses RE2 and is unanchored. Anchor with ^ and $ for exact matches.
• && binds tighter than ||. Use parentheses and single quotes for clarity.

Practice

1. (Beginner) Write rules that match branches named hotfix/<anything> but not hotfix-notes. Test both names.
2. (Intermediate) Store the allowed deploy branch pattern in a project variable and use it on the right side of =~. Change the pattern without editing .gitlab-ci.yml and confirm the effect.
3. Interview question: "Why does if: $CI_COMMIT_BRANCH =~ /main/ cause production deployments from a branch named fix-maintenance-page?" Hint: compare matching a substring with matching a whole value.

<a id="42-legacy-onlyexcept"></a>

4.2 Legacy only/except

Before rules, jobs were controlled with only and except. They still work and still appear in older projects, templates and blog posts, so you need to read them fluently. This subchapter explains how they work, why they were superseded, and how to translate them to rules.

only and except Keywords

Why it matters. Many existing pipelines, including some long-lived internal templates, still use only/except. Maintaining or migrating them means understanding exactly what they include and exclude, including defaults that are not visible in the YAML.

What it is. only defines when a job is added. except defines when it is not added. Each accepts either a simple list of refs, or a map with these keys:

Key	Matches on	Example values
refs	Branch or tag names, regexes, or special keywords	main, /^release-.*$/, tags, merge_requests
variables	Variable expressions, like rules:if	$CI_COMMIT_MESSAGE =~ /run-e2e/
changes	Changed file paths	src/**/*
kubernetes	An active Kubernetes integration	active

Special refs keywords:

Keyword	Pipelines matched
branches	Any branch pipeline
tags	Any tag pipeline
merge_requests	Merge request pipelines
api, web, schedules, triggers, pipelines, pushes, chat, external, external_pull_requests	Pipelines from that source

How conditions combine:

• A job with none of rules, only or except behaves as only: [branches, tags]. It runs in branch and tag pipelines, but not in MR pipelines.
• Inside only, multiple values of one key are ORed. Multiple keys are ANDed: the job is added only if every key has at least one match.
• Inside except, the job is excluded if any key matches.
• If a job matches both only and except, except wins and the job is excluded.

build:
  script: [make build]
  only:
    - main                       # a simple list is shorthand for only:refs
    - /^release-\d+\.\d+$/       # regex on the ref name
    - tags

e2e-tests:
  script: [make e2e]
  only:
    refs: [merge_requests]       # AND
    changes: [web/**/*]          # both keys must match
  except:
    variables:
      - $CI_MERGE_REQUEST_LABELS =~ /skip-e2e/   # any match excludes the job

nightly-cleanup:
  script: [./cleanup.sh]
  only: [schedules]

Key takeaways

• only includes and except excludes. except wins when both match.
• Without rules/only/except, a job implicitly has only: [branches, tags].
• Under only, keys are ANDed and values within a key are ORed. Under except, any matching key excludes.
• merge_requests must be listed explicitly to run a job in MR pipelines.

Practice

1. (Beginner) Write an only configuration that runs a job on main and on tags starting with v. Confirm it does not run in MR pipelines.
2. (Intermediate) Given only: {refs: [branches], variables: [$RUN_SLOW]} and except: {refs: [main]}, list which of these pipelines contain the job: push to main with RUN_SLOW=1, push to feat/a with RUN_SLOW=1, push to feat/a without it.
3. Interview question: "An engineer adds only: [merge_requests] to every job. Afterwards, nothing runs on main after merges. Why?" Hint: only replaces the implicit default rather than adding to it.

Limitations of only/except

Why it matters. Knowing why only/except were superseded helps you explain migrations to teammates and spot bugs in legacy configurations that rules would have avoided.

What it is. The main limitations:

Limitation	Consequence	rules equivalent
Inclusion/exclusion only	Cannot make a job manual, delayed or allowed to fail per condition	Each rule sets when, allow_failure, start_in
No per-condition variables or needs	Requires duplicating jobs for each situation	rules:variables, rules:needs
Fixed combination logic (AND across keys, OR within a key, except overrides)	Some conditions are awkward or impossible to express	Ordered rules with full &&/||/()
Implicit default only: [branches, tags]	Hidden behavior that surprises readers	Explicit catch-all rule
Easy duplicate pipelines with merge_requests + branches	Same commit tested twice	Combined with workflow:rules patterns (4.4)
No new features	Newer capabilities such as exists, compare_to and rule-level needs are rules-only	All new job-control features target rules

GitLab documents only/except as no longer actively developed. They are not removed, but rules is the recommended keyword, and a single job cannot mix the two.

A concrete example of the expressiveness gap: "run automatically on main, manually on other branches" needs two jobs with only/except, but one job with rules:

# Legacy: two jobs, two places to maintain the script
deploy-auto:
  script: [./deploy.sh]
  only: [main]

deploy-manual:
  script: [./deploy.sh]
  when: manual
  only: [branches]
  except: [main]

# rules: one job, behavior per condition
deploy:
  script: [./deploy.sh]
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    - if: $CI_COMMIT_BRANCH
      when: manual

Key takeaways

• only/except can only include or exclude a job. They cannot vary its behavior per condition.
• Their combination logic is fixed and their default is implicit.
• New features are added only to rules, and the two cannot be mixed in one job.
• Prefer rules for all new configuration.

Practice

1. (Beginner) Try adding both rules and only to one job in the pipeline editor and note the validation error.
2. (Intermediate) Find a requirement from your own project that needs two jobs with only/except but one job with rules. Write both versions.
3. Interview question: "Your organization has 300 projects using a shared template with only/except. Is migrating to rules worth it, and how would you roll it out?" Hint: weigh duplicate pipelines and missing features against risk, and think about template versioning and incremental adoption.

Migrating to rules

Why it matters. Migration is not a mechanical search-and-replace. Because the two systems have different defaults, a careless translation can add jobs to MR pipelines, drop them from tag pipelines, or create duplicate pipelines. A systematic mapping keeps behavior identical first, then lets you improve it.

What it is. A translation table for the common cases:

only / except	rules
(nothing)	- if: $CI_COMMIT_BRANCH || $CI_COMMIT_TAG (or no rules plus workflow:rules that exclude MR pipelines)
only: [main]	- if: $CI_COMMIT_BRANCH == "main"
only: [/^release-.*$/]	- if: $CI_COMMIT_REF_NAME =~ /^release-.*$/
only: [tags]	- if: $CI_COMMIT_TAG
only: [branches]	- if: $CI_COMMIT_BRANCH
only: [merge_requests]	- if: $CI_PIPELINE_SOURCE == "merge_request_event"
only: [schedules]	- if: $CI_PIPELINE_SOURCE == "schedule"
only: [web] / [api] / [triggers]	- if: $CI_PIPELINE_SOURCE == "web" / "api" / "trigger"
only: {variables: [$X == "1"]}	- if: $X == "1"
only: {changes: [src/**/*]}	- changes: [src/**/*] (combine with an if to avoid duplicates)
except: [main]	- if: $CI_COMMIT_BRANCH == "main" with when: never, then a catch-all
only: {refs: [...], variables: [...]}	One rule combining both conditions with &&

The general recipe:

1. Translate each except condition into a when: never rule at the top.
2. Translate only conditions into inclusion rules. Combine keys that were ANDed into one if with &&, or one rule with if plus changes.
3. Move any job-level when, allow_failure or start_in into the inclusion rules.
4. Watch for the implicit default: a legacy job with no conditions ran on branches and tags only.
5. Compare the resulting pipelines in each scenario (branch, MR, tag, schedule) before and after.

# Before
integration:
  script: [make integration]
  when: manual
  only:
    refs: [branches, merge_requests]
    changes: [api/**/*]
  except:
    refs: [main]
    variables: [$SKIP_INTEGRATION]

# After
integration:
  script: [make integration]
  rules:
    - if: $CI_COMMIT_BRANCH == "main"                  # except: refs main
      when: never
    - if: $SKIP_INTEGRATION                            # except: variables
      when: never
    - if: $CI_PIPELINE_SOURCE == "merge_request_event" || $CI_COMMIT_BRANCH   # only: refs
      changes: [api/**/*]                              # only: changes (ANDed with if)
      when: manual                                     # job-level when moved into the rule

The "after" version still creates the job in both a branch pipeline and an MR pipeline for the same commit, exactly like the original. Once behavior matches, add workflow:rules (4.4) to remove the duplication.

Key takeaways

• Translate except into leading when: never rules and only into inclusion rules.
• ANDed only keys become a single rule with combined clauses.
• Move job-level when, allow_failure and start_in into the rules.
• Reproduce existing behavior first, verify each pipeline type, then improve with workflow:rules.

Practice

1. (Beginner) Convert only: [main, tags] and except: [schedules] into rules.
2. (Intermediate) Migrate a three-job legacy pipeline (build, test, deploy) to rules. Record which jobs appear for a push to a feature branch, an MR, main and a tag, before and after, and confirm they are identical.
3. Interview question: "After migrating to rules, the team notices jobs now run in MR pipelines that never ran there before. What translation mistake is likely?" Hint: compare a catch-all rule with the legacy implicit default.

<a id="43-job-execution-behavior"></a>

4.3 Job Execution Behavior

Once a job is in a pipeline, a second set of keywords decides when it starts, whether a human must start it, how its failure affects the pipeline, whether it is retried, how long it may run and whether it can be canceled. This subchapter covers each of them.

when: on_success, on_failure, always

Why it matters. Not every job should run only when everything before it passed. Cleanup should run even after a failure, and a notification or rollback should run only after a failure. when encodes these relationships so they do not have to be scripted by hand.

What it is. when sets the condition, relative to earlier jobs, under which a job runs. It can be set at job level or, preferably, inside a rule.

Value	The job runs...
on_success (default)	When all jobs in earlier stages (or all jobs in needs) succeeded or are allowed to fail
on_failure	When at least one job in earlier stages (or in needs) failed
always	Regardless of the status of earlier jobs
manual	Only when started by a user (next topic)
delayed	After a delay (topic after next)
never	Never. Used in rules to exclude the job

"Earlier jobs" means jobs in earlier stages. If the job uses needs, it means only the needed jobs.

stages: [build, test, deploy, cleanup]

build:
  stage: build
  script: [make build]

test:
  stage: test
  script: [make test]

deploy:
  stage: deploy
  script: [./deploy.sh]                     # when: on_success (default)

notify-failure:
  stage: cleanup
  script: [./notify.sh "Pipeline $CI_PIPELINE_ID failed"]
  when: on_failure                          # only if something earlier failed

teardown-test-env:
  stage: cleanup
  script: [./teardown.sh]
  when: always                              # free resources whatever happened

All green:         build (ok) ─> test (ok) ─> deploy (ok) ─> teardown (ok)        notify-failure: skipped
test fails:        build (ok) ─> test (failed) ─> deploy skipped ─> notify (ok), teardown (ok)

A canceled pipeline is not a failed one: canceling stops pending jobs, including always and on_failure jobs that have not started. For per-job cleanup that must survive a failed script, use after_script (chapter 2) instead of a separate job.

Key takeaways

• on_success is the default. on_failure runs only after an earlier failure. always runs regardless.
• With needs, "earlier jobs" means only the needed jobs.
• when: never is used in rules to exclude a job.
• Canceling a pipeline does not trigger on_failure jobs.

Practice

1. (Beginner) Add an on_failure job and an always job to a pipeline. Make a test fail and observe which jobs run.
2. (Intermediate) Make a rollback job that runs only if deploy failed, not if a test failed. Use needs to narrow the scope of on_failure.
3. Interview question: "Why is a when: always cleanup job in the last stage not a reliable way to delete cloud resources created during tests?" Hint: think about cancellation, timeouts and where that cleanup logic belongs.

Manual Jobs

Why it matters. Some steps need a human decision: deploying to production, running an expensive benchmark or rotating a key. Manual jobs keep these steps inside the pipeline, with the same code, artifacts and audit trail, but wait for a person to click Run.

What it is. A job with when: manual is created in the pipeline but does not start automatically. It shows a play button in the pipeline graph. A key distinction is blocking versus non-blocking:

Setting	Behavior	Pipeline status while waiting
allow_failure: true (default for job-level when: manual)	Non-blocking. Later stages continue without it	passed once other jobs finish
allow_failure: false (default for when: manual inside rules)	Blocking. Later stages and dependents wait	blocked

stages: [build, deploy, verify]

build:
  stage: build
  script: [make build]

deploy-production:
  stage: deploy
  script: [./deploy.sh production]
  environment: production
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual                         # inside rules: blocking by default
      manual_confirmation: "Deploy to production now?"   # optional confirmation dialog

smoke-tests:
  stage: verify
  script: [./smoke.sh production]          # waits for the manual deploy to succeed

benchmark:
  stage: verify
  script: [make bench]
  when: manual                             # job-level: non-blocking by default

main pipeline:   build (ok) ──> deploy-production (manual, blocking) ──> smoke-tests (waiting)
                                     ^ pipeline status: blocked
                 benchmark (manual, optional) — pipeline can pass without it

Who can run a manual job:

• Users who can push to (or merge into) the job's ref. For protected branches, that means users allowed to merge.
• If the job deploys to a protected environment, only users and groups allowed to deploy there.
• When running a manual job from the UI, users can provide extra variables for that run.

A blocking manual job in an MR pipeline with "Pipelines must succeed" enabled blocks merging until someone runs it. That is usually unintended, so in MR pipelines manual jobs are often made non-blocking with allow_failure: true.

Key takeaways

• when: manual jobs wait for a user to start them.
• Job-level manual jobs are non-blocking by default. Manual jobs from rules are blocking by default.
• A blocking manual job puts the pipeline in blocked status and holds later stages and dependents.
• Protected environments restrict who can run manual deployment jobs. manual_confirmation adds a confirmation prompt.

Practice

1. (Beginner) Add a manual job at job level and another via rules. Compare the pipeline status and whether later stages wait.
2. (Intermediate) Configure a production deployment as a blocking manual job, protect the production environment so only one group can deploy, and verify that a Developer outside that group cannot run it.
3. Interview question: "An MR cannot be merged because its pipeline is 'blocked', but all tests passed. What happened and what are two ways to fix it?" Hint: look for manual jobs and their default allow_failure value in rules.

Delayed Jobs

Why it matters. Some actions should happen automatically, but not immediately: a canary that runs for 30 minutes before full rollout, or a scheduled switchover after a warm-up period. Delayed jobs add a timer while still allowing a person to start the job early or stop it.

What it is. when: delayed with start_in makes a job wait for the given duration after it would normally start (when its previous stage or needs complete). start_in accepts human-readable durations such as 30 seconds, 10 minutes, 1 hour or 1 day, up to one week.

stages: [canary, rollout]

deploy-canary:
  stage: canary
  script: [./deploy.sh --canary 10%]

full-rollout:
  stage: rollout
  script: [./deploy.sh --all]
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: delayed
      start_in: 30 minutes               # timer starts when deploy-canary succeeds

00:00  deploy-canary starts
00:04  deploy-canary succeeds  ──> full-rollout status: scheduled (countdown 30:00)
00:34  full-rollout starts automatically
       (anytime before: "Run now" starts it, "Unschedule" turns it into a manual job)

Action in the UI	Effect
Run now (play)	Starts the job immediately
Unschedule	Stops the timer. The job becomes a manual job
Cancel pipeline	Stops the timer with the rest of the pipeline

A delayed job is a simple timer, not a health check. For the canary example, the canary stage should still verify metrics, and fail, if the canary is unhealthy.

Key takeaways

• when: delayed plus start_in waits a fixed time before running, up to one week.
• The countdown starts when the job would otherwise start.
• Users can run a delayed job early or unschedule it into a manual job.
• Pair delays with real verification rather than relying on time alone.

Practice

1. (Beginner) Create a job with start_in: 2 minutes and watch its status change from scheduled to running. Repeat and use Run now.
2. (Intermediate) Build a canary flow where a verification job checks an endpoint after deployment, and the delayed full rollout only happens if verification passed.
3. Interview question: "When is a delayed job a better choice than a pipeline schedule, and vice versa?" Hint: compare "relative to this pipeline's progress" with "at a fixed time of day".

allow_failure and Exit Codes

Why it matters. Some checks are informative rather than mandatory: a new linter being rolled out, a flaky experimental test suite, a dependency audit with known findings. Failing the whole pipeline for them would block delivery, but hiding them would lose the signal. allow_failure keeps the result visible without blocking.

What it is. With allow_failure: true, a failed job shows a warning status (orange), later stages continue and the pipeline finishes as passed with a warning. allow_failure:exit_codes narrows this to specific exit codes, so only known, acceptable failures are tolerated.

new-linter:
  script: [./lint-strict.sh]
  allow_failure: true                    # any failure is tolerated, shown as a warning

dependency-audit:
  script:
    - ./audit.sh                         # exits 0 = clean, 3 = known low-severity findings, other = error
  allow_failure:
    exit_codes: [3]                      # tolerate only code 3; any other failure fails the pipeline

integration-tests:
  script: [make integration]
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      allow_failure: false               # mandatory on main
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      allow_failure: true                # advisory in MRs while the suite stabilizes

Job result	allow_failure: false	allow_failure: true	exit_codes: [3]
Exit 0	passed	passed	passed
Exit 3	failed, blocks	warning	warning
Exit 1	failed, blocks	warning	failed, blocks

Your script must return the right exit code for this to work. In a shell script, set -e makes the shell exit with the failing command's code, and wrappers such as cmd || exit 3 map outcomes explicitly.

allow_failure also interacts with other features covered in this chapter: it determines whether a manual job is blocking, and a job that needs an allowed-to-fail job still runs if that job fails.

Key takeaways

• allow_failure: true turns a failure into a warning and lets the pipeline continue.
• allow_failure:exit_codes tolerates only specific exit codes, keeping unexpected errors fatal.
• allow_failure can vary per rule, for example advisory in MRs and mandatory on main.
• Allowed failures should be temporary or deliberate. Review warnings regularly.

Practice

1. (Beginner) Make a job that exits with code 1 and set allow_failure: true. Note the job and pipeline statuses.
2. (Intermediate) Write a script that exits 0, 2 or 5 depending on an input variable, and configure the job so only exit code 2 is tolerated. Test all three cases.
3. Interview question: "Half of your pipeline jobs have allow_failure: true. What risks does that create and how would you reduce it?" Hint: think about ignored warnings, exit_codes and making checks blocking on the default branch.

retry Keyword

Why it matters. Not all failures are caused by the code under test. A runner can be lost, a network call can time out, a cloud VM can be preempted. Re-running these by hand wastes time. retry lets GitLab automatically retry jobs, ideally only for failure types that are worth retrying.

What it is. retry sets how many times a failed job is retried, from 0 (default) to 2. The long form adds filters:

Option	Purpose
retry:max	Number of retries (0 to 2)
retry:when	Failure reasons that trigger a retry. Default: always (any reason)
retry:exit_codes	Exit codes that trigger a retry

Common retry:when values:

Value	Failure reason
runner_system_failure	The runner failed to set up or run the job
stuck_or_timeout_failure	The job got stuck or timed out
api_failure	An API failure between runner and GitLab
scheduler_failure	The job could not be scheduled on a runner
unknown_failure	Unknown reason
script_failure	The script itself failed (non-zero exit)
job_execution_timeout	The job exceeded its timeout
data_integrity_failure	A structural integrity problem was detected

default:
  retry:
    max: 2
    when:                                 # retry infrastructure problems everywhere
      - runner_system_failure
      - stuck_or_timeout_failure
      - api_failure
      - scheduler_failure

e2e-tests:
  script: [npx playwright test]
  retry:
    max: 2
    exit_codes: [137]                     # 137 = killed (often out-of-memory); script_failure in general is not retried

attempt 1: runner_system_failure ─> retry
attempt 2: runner_system_failure ─> retry
attempt 3: passed                 ─> job passed (earlier attempts kept as "retried")

Retrying script_failure across the board hides flaky tests and lengthens pipelines when the code really is broken. Prefer retrying infrastructure reasons, and fix or quarantine flaky tests.

Some job phases have their own retry variables, independent of retry:

Variable	Retries
GET_SOURCES_ATTEMPTS	Fetching the repository
ARTIFACT_DOWNLOAD_ATTEMPTS	Downloading artifacts
RESTORE_CACHE_ATTEMPTS	Restoring the cache
EXECUTOR_JOB_SECTION_ATTEMPTS	Some executor sections (for example Docker setup)

Key takeaways

• retry retries a failed job up to 2 times. The default is no retries.
• retry:when and retry:exit_codes restrict retries to failures worth retrying.
• Retrying infrastructure failures is good practice. Retrying all script failures hides real problems.
• Phase-specific variables retry source fetching, artifact downloads and cache restores separately.

Practice

1. (Beginner) Write a job that fails randomly half of the time (for example with $RANDOM) and add retry: 2. Observe the retried attempts in the job list.
2. (Intermediate) Configure a default retry for infrastructure failures only, and a job that additionally retries exit code 137. Confirm that a normal script failure is not retried.
3. Interview question: "A team set retry: 2 on every job to cope with flaky tests. What are the costs, and what would you propose instead?" Hint: consider pipeline duration, runner cost, masking real regressions and test quarantine.

Job timeout

Why it matters. A job stuck on a network call or an interactive prompt can occupy a runner for hours, blocking other work and wasting money. Timeouts put an upper bound on every job, and choosing them deliberately catches hung jobs quickly without killing legitimately long ones.

What it is. Three settings interact:

Setting	Where	Default	Role
Project timeout	Settings > CI/CD > General pipelines	60 minutes	Default for every job in the project
Job timeout	.gitlab-ci.yml (job or default)	none	Overrides the project timeout for that job, up or down
Runner maximum job timeout	Runner settings	none	Hard cap for any job on that runner

The effective timeout is the job timeout if set, otherwise the project timeout, and in either case capped by the runner's maximum if the runner has one.

default:
  timeout: 20m                           # most jobs should be quick

unit-tests:
  script: [make test]                    # inherits 20m

nightly-full-suite:
  script: [make test-all]
  timeout: 3h 30m                        # human-readable durations
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"

Project timeout 60m, runner max 2h:
  unit-tests          -> 20m  (job timeout)
  nightly-full-suite  -> 2h   (job asked for 3h30m, capped by runner)
  job without timeout -> 60m  (project timeout; default overridden here by 20m)

When a job times out, it fails with the reason job_execution_timeout, which retry:when can target. Within a job, the runner can also limit parts of it with variables such as RUNNER_SCRIPT_TIMEOUT and RUNNER_AFTER_SCRIPT_TIMEOUT. This leaves time for after_script cleanup before the job timeout is reached.

Key takeaways

• A job's effective timeout is its timeout, else the project timeout, capped by the runner's maximum.
• Job timeout can be shorter or longer than the project timeout.
• Set short defaults and lengthen only the jobs that need it, so hung jobs are caught fast.
• Timeouts appear as job_execution_timeout, which can be retried selectively.

Practice

1. (Beginner) Write a job that runs sleep 600 with timeout: 2m. Note the failure reason shown.
2. (Intermediate) Set a default timeout of 15 minutes, override it for one long job, and run that job on a runner with a lower maximum timeout. Explain the effective timeout you observe.
3. Interview question: "Your runners are often saturated by jobs that hang until the 60-minute project timeout. How do you find these jobs and reduce the waste?" Hint: job duration data, default: timeout, runner maximum timeouts and fixing interactive prompts.

interruptible Keyword

Why it matters. Developers often push several commits in quick succession. Without intervention, each push starts a full pipeline and older pipelines keep running even though their results no longer matter. interruptible marks which jobs are safe to cancel when a newer pipeline supersedes them, freeing runners for the result that counts.

What it is. interruptible: true tells GitLab that a job may be canceled while running if a newer pipeline starts on the same ref. It is false by default. It works together with the auto-cancel features in section 4.4. Jobs that are safe to interrupt are those with no lasting side effects: builds, tests and linting. Deployments, database migrations and publishing are not safe.

default:
  interruptible: true                     # most jobs are safe to cancel

build:
  script: [make build]

test:
  script: [make test]

deploy-staging:
  script: [./deploy.sh staging]
  interruptible: false                    # never cut a deployment in half
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

With the default auto-cancel behavior (conservative, section 4.4):

Pipeline #1 (commit A): build (ok) ─> test (running, interruptible)
Push commit B ─> Pipeline #2 starts
  -> pipeline #1 is canceled: only interruptible jobs have started

Pipeline #1 (commit A): build (ok) ─> test (ok) ─> deploy-staging (running, NOT interruptible)
Push commit B ─> Pipeline #2 starts
  -> pipeline #1 is NOT canceled: an uninterruptible job has started

interruptible can also be set per rule with rules:interruptible, for example interruptible in MR pipelines but not on main.

Key takeaways

• interruptible: true marks a job as safe to cancel when a newer pipeline on the same ref starts.
• The default is false. Setting default: interruptible: true and opting out for risky jobs is a common pattern.
• Never mark deployments, migrations or publishing jobs as interruptible.
• Its effect depends on the auto-cancel mode configured in workflow:auto_cancel (4.4).

Practice

1. (Beginner) Mark a long-running test job as interruptible, push two commits quickly and observe the first pipeline being canceled.
2. (Intermediate) Add a non-interruptible deploy job that starts early, repeat the experiment, and explain why the first pipeline now continues.
3. Interview question: "What could go wrong if a Terraform apply job were marked as interruptible?" Hint: think about state locks and partially applied infrastructure.

<a id="44-workflow-control"></a>

4.4 Workflow Control

Job rules decide which jobs are in a pipeline. The workflow keyword works one level higher: it decides whether a pipeline is created at all, what it is called, and how it reacts to newer pipelines. This subchapter covers workflow:rules, workflow:name, auto-cancellation and the patterns that prevent duplicate branch and MR pipelines.

workflow:rules

Why it matters. Without workflow:rules, every push creates a pipeline, and every job must repeat conditions such as "not for schedules" or "not for drafts". That scatters policy across dozens of jobs and wastes resources on pipelines with nothing useful to do. workflow:rules centralizes the decision of which events deserve a pipeline.

What it is. workflow:rules uses the same syntax as job rules (if, changes, exists, variables), with two differences:

• when can only be always (default) or never.
• The result applies to the whole pipeline. If no rule matches, no pipeline is created.

Workflow rules are evaluated first. Job rules are only evaluated for pipelines that pass the workflow.

Event (push, MR, schedule, ...)
        │
        v
 workflow:rules ── no match / never ──> no pipeline created
        │ match
        v
 job rules, job by job ──> jobs added to the pipeline
        │
        v
 pipeline with at least one job? ── no ──> no pipeline created

workflow:
  rules:
    - if: $CI_COMMIT_TITLE =~ /^Draft:/i                   # skip work-in-progress commits
      when: never
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      variables:
        DEPLOY_TARGET: review                              # available to every job, and to job rules
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      variables:
        DEPLOY_TARGET: staging
    - if: $CI_COMMIT_TAG
      variables:
        DEPLOY_TARGET: production
    - if: $CI_PIPELINE_SOURCE == "schedule"
    # anything else (pushes to feature branches without an MR, API calls, ...): no pipeline

deploy:
  script: [./deploy.sh "$DEPLOY_TARGET"]
  rules:
    - if: $DEPLOY_TARGET                                   # workflow variables can be used in job rules

GitLab ships two templates with ready-made workflows:

Template	Behavior
Workflows/Branch-Pipelines.gitlab-ci.yml	Branch and tag pipelines only
Workflows/MergeRequest-Pipelines.gitlab-ci.yml	MR pipelines, plus pipelines for the default branch and tags

include:
  - template: Workflows/MergeRequest-Pipelines.gitlab-ci.yml

Key takeaways

• workflow:rules decides whether a pipeline is created at all. No match means no pipeline.
• Only when: always and when: never are allowed.
• workflow:rules:variables sets variables for the whole pipeline, visible to job rules too.
• Centralizing event policy in workflow keeps job rules short and consistent.

Practice

1. (Beginner) Add workflow:rules that create pipelines only for MRs, the default branch and tags. Push to a feature branch without an MR and confirm no pipeline appears.
2. (Intermediate) Use workflow:rules:variables to set LOG_LEVEL=debug for MR pipelines and LOG_LEVEL=info elsewhere, and make one job run only when LOG_LEVEL is debug.
3. Interview question: "What is the difference between excluding a scheduled pipeline with workflow:rules and giving every job a when: never rule for schedules?" Hint: consider whether a pipeline object exists, runner usage, and maintenance as jobs are added.

workflow:name

Why it matters. The pipeline list usually shows commit titles, which say nothing about why a pipeline ran: was it a nightly security scan, a release, a manual hotfix deploy? workflow:name gives each pipeline a meaningful name, making the pipeline list and notifications far easier to scan.

What it is. workflow:name sets the pipeline's name. It can include CI/CD variables, and it is often combined with workflow:rules:variables to name pipelines per situation. The name appears in the pipeline list, on the pipeline page and in the API (name field).

variables:
  PIPELINE_NAME: "Branch pipeline: $CI_COMMIT_BRANCH"     # default

workflow:
  name: $PIPELINE_NAME
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      variables:
        PIPELINE_NAME: "MR !$CI_MERGE_REQUEST_IID: $CI_MERGE_REQUEST_TITLE"
    - if: $CI_PIPELINE_SOURCE == "schedule"
      variables:
        PIPELINE_NAME: "Nightly: $SCHEDULE_TYPE"          # SCHEDULE_TYPE set on the schedule
    - if: $CI_COMMIT_TAG
      variables:
        PIPELINE_NAME: "Release $CI_COMMIT_TAG"
    - if: $CI_COMMIT_BRANCH

Pipelines list
  #9812  Release v2.4.0                         passed
  #9809  Nightly: security                      failed
  #9807  MR !341: Rate-limit the login endpoint running
  #9801  Branch pipeline: main                  passed

The pipelines API can filter by name (GET /projects/:id/pipelines?name=...), so consistent names also make automation and reporting easier.

Key takeaways

• workflow:name gives pipelines a descriptive name using variables.
• Combine it with workflow:rules:variables to name pipelines by type.
• Names show up in the UI and API and can be used for filtering.
• Pick a consistent naming scheme so names are useful in dashboards and scripts.

Practice

1. (Beginner) Set workflow:name to include the branch name and pipeline source, and check the pipeline list.
2. (Intermediate) Name scheduled pipelines after a SCHEDULE_TYPE variable set on each schedule, then list only Nightly: security pipelines with the API.
3. Interview question: "Why is naming pipelines by purpose helpful during incident response?" Hint: imagine finding which pipeline last deployed production among hundreds of runs.

Auto-Canceling Redundant Pipelines

Why it matters. When a developer pushes three commits in five minutes, only the newest pipeline's result matters. Letting the two older pipelines finish wastes runner capacity and delays everyone's feedback. Auto-cancellation stops redundant work automatically, and its configuration decides how aggressive that is.

What it is. Two layers control it:

1. Project setting. Settings > CI/CD > General pipelines > Auto-cancel redundant pipelines (enabled by default). When a new pipeline starts on a branch, older pipelines on the same branch become candidates for cancellation.
2. workflow:auto_cancel in .gitlab-ci.yml refines the behavior:

Option	Values	Meaning
on_new_commit	conservative (default)	Cancel the older pipeline only if none of its non-interruptible jobs has started
	interruptible	Cancel only the jobs marked interruptible: true. Other jobs keep running
	none	Do not auto-cancel on new commits
on_job_failure	none (default)	Keep running other jobs when one fails
	all	Cancel the whole pipeline as soon as any job fails

workflow:
  auto_cancel:
    on_new_commit: interruptible       # cancel interruptible jobs even if a deploy has started
    on_job_failure: all                # fail fast: stop everything on the first failure
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      auto_cancel:
        on_new_commit: none            # never cancel pipelines on main: every commit deploys
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_TAG

default:
  interruptible: true

deploy:
  script: [./deploy.sh]
  interruptible: false
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

on_new_commit: conservative     older pipeline: build (ok), test (running), deploy (running, non-interruptible)
                                -> nothing canceled
on_new_commit: interruptible    -> test canceled, deploy keeps running
on_new_commit: none             -> nothing canceled

Auto-cancellation only targets pipelines on the same ref. Pipelines on different branches never cancel each other.

Key takeaways

• The project setting enables auto-cancellation. workflow:auto_cancel controls how it behaves.
• conservative cancels only if no non-interruptible job has started. interruptible cancels interruptible jobs regardless.
• on_job_failure: all makes the pipeline fail fast on the first failed job.
• workflow:rules:auto_cancel lets the default branch use different behavior from feature branches.

Practice

1. (Beginner) Confirm that auto-cancel is enabled in your project, mark tests as interruptible, and push two commits in quick succession.
2. (Intermediate) Configure on_new_commit: interruptible for MR pipelines and none for the default branch. Demonstrate the difference with a long-running job.
3. Interview question: "Your team enables on_job_failure: all to save runner minutes. Developers complain they now see fewer failures per pipeline. Is that a problem?" Hint: compare fast feedback on the first error with full feedback on all errors, and consider where each matters.

Avoiding Duplicate Pipelines

Why it matters. A frequent surprise for teams adopting MR pipelines is that every push to a branch with an open MR runs two pipelines: a branch pipeline and an MR pipeline for the same commit. That doubles runner usage, and the two pipelines can disagree, which confuses reviewers.

What it is. A push to a branch with an open MR generates two events, and both can create a pipeline:

git push origin feature/login     (an MR from feature/login is open)
   │
   ├── push event               ──> branch pipeline (CI_PIPELINE_SOURCE = push)
   └── merge request update     ──> MR pipeline     (CI_PIPELINE_SOURCE = merge_request_event)

Duplicates happen when at least one job matches both pipeline types. Typical causes:

Rule	Why it duplicates
- when: on_success (catch-all)	Matches in every pipeline type
- changes: [src/**/*] alone	changes without if matches in both
- if: $CI_PIPELINE_SOURCE == "merge_request_event" and - if: $CI_COMMIT_BRANCH in one job	Each pipeline type matches one rule
only: [branches, merge_requests]	Legacy equivalent of the above

Ways to avoid them:

1. Choose one pipeline type per situation in workflow:rules, so the other is never created (next topic).
2. Restrict job rules so each job matches only one pipeline type, for example if: $CI_PIPELINE_SOURCE == "merge_request_event" for MR-only jobs and if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH for main-only jobs.
3. Avoid bare catch-alls like - when: on_success unless workflow:rules already prevents duplicates.

# Duplicate-prone: this job runs in both the branch and the MR pipeline
test:
  script: [make test]
  rules:
    - changes: [src/**/*]

# Fixed at job level: run once, in the MR pipeline or on main
test:
  script: [make test]
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes: [src/**/*]
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Fixing duplicates per job works, but every new job must follow the same discipline. Fixing them in workflow:rules is more robust, because the duplicate pipeline is never created.

Key takeaways

• Pushing to a branch with an open MR can create both a branch pipeline and an MR pipeline.
• Duplicates come from jobs whose rules match both pipeline types, including catch-alls and bare changes.
• Fix duplicates centrally with workflow:rules, or carefully per job.
• After fixing, each commit should be tested by exactly one pipeline.

Practice

1. (Beginner) Create a job with only - when: on_success, open an MR and push a commit. Count the pipelines.
2. (Intermediate) Fix the duplication twice: once by changing job rules only, once by adding workflow:rules. List the advantages of each.
3. Interview question: "Why can't you prevent duplicate pipelines just by making every job MR-only?" Hint: think about what runs after a merge to the default branch and about tag pipelines.

Switching Between Branch and MR Pipelines

Why it matters. The best of both worlds is usually: branch pipelines while a branch has no MR (so early pushes are still tested), and MR pipelines as soon as an MR exists (with MR context, merged results and merge trains). GitLab supports this switch with one well-known workflow:rules pattern that every team should recognize.

What it is. The pattern uses CI_OPEN_MERGE_REQUESTS, a variable set in branch pipelines when the branch is the source of at least one open MR:

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"    # 1. MR exists: run MR pipelines
    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS     # 2. branch push with an open MR:
      when: never                                         #    skip, the MR pipeline covers it
    - if: $CI_COMMIT_BRANCH                               # 3. branch push without MR: branch pipeline
    - if: $CI_COMMIT_TAG                                  # 4. tags: tag pipeline

Timeline for branch feature/login

push #1 (no MR yet)      -> rule 3 -> branch pipeline
open MR                  -> rule 1 -> MR pipeline
push #2 (MR open)        -> push event:  rule 2 -> no branch pipeline
                            MR event:    rule 1 -> MR pipeline
merge into main          -> push to main (no open MR from main) -> rule 3 -> branch pipeline on main
push v1.0.0 tag          -> rule 4 -> tag pipeline

Because the workflow guarantees one pipeline per commit, job rules become simple. One detail remains: jobs without any rules still default to branch and tag pipelines only. A job that should run in MR pipelines too needs a rule that matches them, and a catch-all is now safe:

test:
  script: [make test]
  rules:
    - when: on_success                         # safe: the workflow already removed duplicate pipelines

deploy-review:
  script: [./deploy-review.sh]
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

deploy-production:
  script: [./deploy.sh production]
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Caveats:

• CI_OPEN_MERGE_REQUESTS is only set in branch and MR pipelines, and counts open MRs (including drafts) whose source is the pushed branch.
• On the push that happens before the MR is opened, a branch pipeline runs. Opening the MR then starts an MR pipeline for the same commit. That one-time overlap is expected.
• Protected variables and runners are available in MR pipelines only when the source branch is protected (and for merged results, also the target). Deployment jobs that need secrets usually belong to default-branch or tag pipelines.

Key takeaways

• The $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS → when: never rule is the core of the switch.
• Branches without an MR get branch pipelines. Branches with an MR get only MR pipelines.
• Jobs still need a rule that matches MR pipelines, but a catch-all becomes safe with this workflow.
• Expect a one-time overlap when the MR is first opened.

Practice

1. (Beginner) Apply the workflow above, push to a new branch, then open an MR and push again. Record which pipeline type each push created.
2. (Intermediate) Extend the workflow to also create pipelines for schedules and the Run pipeline button, without reintroducing duplicates.
3. Interview question: "After adopting this workflow, a team's test job stopped running in MR pipelines, but still runs on main. What is the likely cause?" Hint: recall the default for jobs that have no rules.

<a id="45-parallelism-and-concurrency"></a>

4.5 Parallelism and Concurrency

Pipelines get faster when independent work runs side by side, and safer when work that must not overlap is serialized. This subchapter covers both directions: parallel and parallel:matrix to fan a job out into many copies, techniques to split a test suite across them, and resource_group to make sure only one job at a time touches a shared resource.

parallel Keyword

Why it matters. A 40-minute test suite on one runner gives slow feedback, even if ten runners sit idle. Splitting the job into N identical copies, each handling a share of the work, can cut wall-clock time almost N-fold. parallel creates those copies without duplicating YAML.

What it is. parallel: N (2 to 200) creates N instances of the job, which run at the same time if enough runners are available. Each instance receives two predefined variables:

Variable	Value
CI_NODE_TOTAL	Total number of instances (N)
CI_NODE_INDEX	This instance's position, from 1 to N

The instances are named job 1/N, job 2/N, ... and are grouped in the pipeline graph. The job as a whole succeeds only if all instances succeed.

rspec:
  image: ruby:3.3
  parallel: 4                                   # creates "rspec 1/4" ... "rspec 4/4"
  script:
    - echo "I am node $CI_NODE_INDEX of $CI_NODE_TOTAL"
    - bundle exec rspec $(./scripts/select-specs.sh "$CI_NODE_INDEX" "$CI_NODE_TOTAL")

                 ┌─ rspec 1/4  (specs 1, 5, 9, ...)
build ─> rspec ──┼─ rspec 2/4  (specs 2, 6, 10, ...)
                 ├─ rspec 3/4  (specs 3, 7, 11, ...)
                 └─ rspec 4/4  (specs 4, 8, 12, ...)

parallel only creates copies. Dividing the work is the job's responsibility, through the two variables (see "Splitting Tests Across Parallel Jobs"). Parallel jobs also multiply fixed costs: each instance pulls its image, fetches the repository and restores caches. Past a certain point, adding instances makes the pipeline slower and more expensive.

Key takeaways

• parallel: N creates N copies of a job (2 to 200) that can run at the same time.
• Each copy gets CI_NODE_INDEX (1-based) and CI_NODE_TOTAL.
• The job must use those variables to split its work. parallel does not split anything itself.
• Per-instance overhead limits how far parallelism pays off.

Practice

1. (Beginner) Create a job with parallel: 3 that prints CI_NODE_INDEX and CI_NODE_TOTAL. Find the three instances in the pipeline graph.
2. (Intermediate) Measure a test job's duration with parallel set to 1, 2, 4 and 8. Plot duration against N and identify where returns diminish.
3. Interview question: "Doubling parallel from 8 to 16 did not make the test job faster. Give three possible reasons." Hint: runner availability, per-job setup overhead and uneven work distribution.

parallel:matrix

Why it matters. Libraries must be tested against several language versions, and deployments often target several regions or clouds. Writing one job per combination is tedious and error-prone. parallel:matrix generates one job per combination of variable values from a single definition.

What it is. parallel:matrix is a list of maps. Each map defines variables and their possible values. GitLab creates one job for every combination within each map (a Cartesian product), and the maps' results are added together. A matrix can generate at most 200 jobs.

test:
  image: python:$PYTHON_VERSION                 # matrix variables can be used in image
  script: [pytest]
  parallel:
    matrix:
      - PYTHON_VERSION: ["3.11", "3.12", "3.13"]  # quote versions so they stay strings
        DB: [postgres, mysql]                     # 3 x 2 = 6 jobs
      - PYTHON_VERSION: "3.13"
        DB: sqlite                                # + 1 extra combination = 7 jobs

deploy:
  stage: deploy
  script: [./deploy.sh "$PROVIDER" "$REGION"]
  tags: [$PROVIDER]                               # route each job to a provider-specific runner
  environment: production/$PROVIDER/$REGION
  parallel:
    matrix:
      - PROVIDER: aws
        REGION: [us-east-1, eu-west-1]
      - PROVIDER: gcp
        REGION: [europe-west4]

Generated job names list the variable values:

test: [3.11, postgres]   test: [3.12, postgres]   test: [3.13, postgres]
test: [3.11, mysql]      test: [3.12, mysql]      test: [3.13, mysql]
test: [3.13, sqlite]
deploy: [aws, us-east-1] deploy: [aws, eu-west-1] deploy: [gcp, europe-west4]

A job can depend on one specific matrix instance with needs:parallel:matrix:

smoke-aws-eu:
  stage: verify
  script: [./smoke.sh aws eu-west-1]
  needs:
    - job: deploy
      parallel:
        matrix:
          - PROVIDER: aws
            REGION: eu-west-1                  # wait only for this instance, not all of deploy

	parallel: N	parallel:matrix
Instances are	Identical, distinguished by index	Different, distinguished by variables
Typical use	Splitting one workload	Testing or deploying many configurations
Variables provided	CI_NODE_INDEX, CI_NODE_TOTAL	Your matrix variables (plus the node variables)
Limit	200 instances	200 generated jobs

Key takeaways

• parallel:matrix creates one job per combination of variable values in each map.
• Several maps let you add specific combinations without a full Cartesian product.
• Matrix variables work in script, image, tags, environment and other expandable keywords.
• needs:parallel:matrix lets other jobs depend on a single instance.

Practice

1. (Beginner) Build a matrix that tests Node.js 20 and 22 on two operating system images. Count and name the generated jobs.
2. (Intermediate) Create a regional deployment matrix, then add a per-region smoke test that depends only on its own region's deployment.
3. Interview question: "Your matrix grows to 3 languages x 4 databases x 3 OS images. What problems will appear, and how can you keep coverage while reducing jobs?" Hint: the 200-job limit, runner cost, and testing full combinations only for important pairs.

Splitting Tests Across Parallel Jobs

Why it matters. Parallel instances only help if each one gets a fair share of the work. Splitting naively by file count can give one instance all the slow tests while others finish in seconds, so the job is only as fast as its slowest instance. Good splitting is what turns parallel into real speed.

What it is. There are three common approaches:

Approach	How it divides work	Balance	Effort
Built-in sharding in the test runner	The tool splits by index and total	Usually by file count	Low
Simple script (round-robin by file)	Every Nth file goes to node N	By file count	Low
Timing-based splitting	Uses previous run durations to give each node equal time	Best	Needs timing data

Many test runners support sharding directly:

# Jest: --shard expects index/total (1-based index, like CI_NODE_INDEX)
npx jest --shard="$CI_NODE_INDEX/$CI_NODE_TOTAL"

# Playwright
npx playwright test --shard="$CI_NODE_INDEX/$CI_NODE_TOTAL"

# pytest with the pytest-split plugin (timing-based if .test_durations exists)
pytest --splits "$CI_NODE_TOTAL" --group "$CI_NODE_INDEX"

A portable fallback for any tool, using round-robin over a stable, sorted file list:

#!/usr/bin/env bash
set -euo pipefail
# Select every CI_NODE_TOTAL-th test file, starting at this node's offset
find tests -name 'test_*.py' | sort \
  | awk -v total="$CI_NODE_TOTAL" -v node="$CI_NODE_INDEX" \
        '(NR - 1) % total == node - 1'       # NR and CI_NODE_INDEX are both 1-based

Putting it together, with JUnit reports so GitLab merges the results of all instances in the MR widget:

unit-tests:
  image: node:22
  parallel: 4
  script:
    - npm ci
    - npx jest --shard="$CI_NODE_INDEX/$CI_NODE_TOTAL" --ci --reporters=default --reporters=jest-junit
  variables:
    JEST_JUNIT_OUTPUT_FILE: "reports/junit-$CI_NODE_INDEX.xml"   # unique file per instance
  artifacts:
    when: always                                                 # keep reports even when tests fail
    reports:
      junit: reports/junit-*.xml

File-count split (unbalanced):   node1 ████████████████ 16m   node2 ███ 3m   node3 ███ 3m
Timing split (balanced):         node1 ███████ 7m            node2 ███████ 7m node3 ████████ 8m
Job duration = slowest node:     16m  vs  8m

Splitting requires the selection to be deterministic: every instance must compute the same full list in the same order, or tests will be skipped or run twice. Sorting the file list ensures this.

Key takeaways

• Parallel speedup depends on balanced splitting. The slowest instance sets the job's duration.
• Use the test runner's built-in sharding when available, or a deterministic round-robin script.
• Timing-based splitting gives the best balance, using durations from earlier runs.
• Give each instance a unique report filename and use artifacts:reports:junit to merge results.

Practice

1. (Beginner) Use the round-robin script with parallel: 3 and print the files each instance selects. Confirm every file appears exactly once across the instances.
2. (Intermediate) Split a real test suite with your test runner's sharding option, collect JUnit reports from every instance and check that the MR test summary shows the full total.
3. Interview question: "Each of four parallel test instances reports 'all passed', yet a broken test reached main. How could the splitting be responsible?" Hint: non-deterministic file ordering, and files that match no instance.

resource_group

Why it matters. Parallelism is not always welcome. Two pipelines deploying to the same production environment at the same time can interleave, leaving a mix of versions or a corrupted state. Shared resources such as a physical test device, a Terraform state or a rate-limited external API need mutual exclusion. resource_group provides it without custom locking scripts.

What it is. resource_group: <name> makes a job acquire a lock on a named resource before it runs. Only one job per resource group runs at a time within a project, across all pipelines. Other jobs that use the same group wait with the status waiting for resource.

Think of it as the single key to a shared meeting room: whoever holds the key uses the room, and everyone else queues at the door.

deploy-production:
  stage: deploy
  script: [./deploy.sh production]
  environment: production
  resource_group: production                  # one production deployment at a time

terraform-apply:
  stage: deploy
  script: [terraform apply -auto-approve plan.tfplan]
  resource_group: terraform-$TF_STATE_NAME    # names can include variables: one lock per state

hardware-tests:
  stage: test
  script: [./run-on-device.sh]
  resource_group: test-device-01              # a single physical device

Pipeline #101: deploy-production  ── running ──────────────┐
Pipeline #102: deploy-production  ── waiting for resource ─┴─> running ──┐
Pipeline #103: deploy-production  ── waiting for resource ───────────────┴─> running

Resource groups are project-scoped. To serialize deployments across projects, put the resource_group on a trigger job that starts a downstream pipeline. The lock is then held until the downstream pipeline finishes, when the trigger uses strategy: depend:

deploy-shared-infra:
  trigger:
    project: ops/infrastructure
    strategy: depend                           # lock covers the whole downstream pipeline
  resource_group: shared-infra

Mechanism	Prevents
resource_group	Two jobs using the same resource at the same time
interruptible: false	A running job being canceled by a newer pipeline
"Prevent outdated deployment jobs" setting	An older pipeline deploying over a newer deployment

Key takeaways

• resource_group serializes jobs that share a name, across all pipelines of a project.
• Waiting jobs show the status "waiting for resource".
• Group names can include variables, giving one lock per environment, state or device.
• For cross-project serialization, put the group on a trigger job with strategy: depend.

Practice

1. (Beginner) Add resource_group: demo to a job that sleeps for 60 seconds. Start three pipelines quickly and watch them queue.
2. (Intermediate) Use a variable in the group name so deployments to staging and production lock independently. Verify that a staging deploy does not wait for production.
3. Interview question: "Is resource_group alone enough to guarantee production always ends up on the newest version?" Hint: consider the order in which waiting jobs are picked, which is the next topic.

Process Modes for Resource Groups

Why it matters. When several jobs wait on the same resource, the order in which they are served matters. For deployments, you might want strict commit order, or you might want only the newest version to deploy as soon as possible. The process mode chooses this order.

What it is. Each resource group has a process mode:

Mode	Next job to get the resource	Typical use
unordered (default)	Any waiting job, in no guaranteed order	Mutual exclusion where order does not matter, such as a test device
oldest_first	The job from the oldest pipeline	Deploy every version in commit order, such as migrations that must apply sequentially
newest_first	The job from the newest pipeline	Get the latest version deployed fastest

Recent GitLab versions add further modes, such as newest_ready_first. Check the documentation for your version.

Queue while #100 deploys:   #101 (older)  #102  #103 (newest)

unordered:     next = any of #101, #102, #103
oldest_first:  #101 -> #102 -> #103
newest_first:  #103 first; with "Prevent outdated deployment jobs" enabled,
               #101 and #102 then fail as outdated instead of overwriting #103

The process mode is not set in .gitlab-ci.yml. A resource group is created the first time a job uses it, and its mode is changed with the API:

# Switch the "production" resource group of project 12345 to oldest_first
curl --silent --fail --request PUT \
  --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  --data "process_mode=oldest_first" \
  "https://gitlab.example.com/api/v4/projects/12345/resource_groups/production"

# Inspect the group and the jobs waiting for it
curl --silent --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "https://gitlab.example.com/api/v4/projects/12345/resource_groups/production" | jq '{key, process_mode}'
curl --silent --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "https://gitlab.example.com/api/v4/projects/12345/resource_groups/production/upcoming_jobs" \
  | jq -r '.[] | [.id, .name, .pipeline.id] | @tsv'

Choosing a mode:

• If every version must reach the environment in order (for example, sequential database migrations), use oldest_first.
• If only the latest state matters (most stateless web deployments), use newest_first together with "Prevent outdated deployment jobs".
• If jobs are independent and only need exclusive access, keep unordered.

Key takeaways

• Process modes decide which waiting job gets the resource next.
• unordered is the default. oldest_first preserves order. newest_first prioritizes the latest pipeline.
• Process modes are set through the Resource Groups API, not in the YAML.
• Combine newest_first with "Prevent outdated deployment jobs" so older pipelines do not overwrite newer deployments.

Practice

1. (Beginner) Create a resource group, then use the API to read its process mode and list its upcoming jobs while three pipelines wait.
2. (Intermediate) Switch the group to oldest_first, start three pipelines with distinguishable commits, and confirm they deploy in order. Repeat with newest_first and compare.
3. Interview question: "Your team deploys a stateless web app from main many times a day, and deployments queue up behind each other. Which process mode and settings would you choose, and why?" Hint: consider whether intermediate versions ever need to be deployed at all.

---

<a id="5-variables-and-secrets"></a>