Fix GitLab Runner Timeout Issue: Step-by-Step Guide

1. Introduction

A GitLab runner timeout kills your job before it finishes and typically logs: ERROR: Job failed: execution took longer than 1h0m0s seconds. The job doesn't fail because of a bug — it fails because it ran out of time. But the underlying cause could be a genuinely slow job, a stuck process, or a misconfigured timeout that's too low for what you're doing.

This guide explains GitLab's three-level timeout hierarchy, how to tune each level, and more importantly, how to make your jobs faster so you don't need to raise timeouts as a workaround.

2. How GitLab CI Timeouts Work

GitLab CI has three timeout levels, applied in order from most to least restrictive:

• Runner-level timeout — set in config.toml on the runner host. Hard maximum. Default: no limit for self-hosted; 1 hour for GitLab.com shared runners.
• Project CI/CD timeout — set per project in Settings → CI/CD → General pipelines → Timeout. Default: 60 minutes.
• Job-level timeout — set in .gitlab-ci.yml with the timeout: keyword. Cannot exceed the project timeout.

3. Common Causes

• Project CI/CD timeout is set to 60 minutes and the job legitimately needs more time
• Docker image pull takes too long (large base image, slow registry)
• Dependency installation (npm install, pip install) without caching
• Test suite has grown without being parallelised
• Job is hanging on user input, a stuck network call, or a zombie process
• Docker build without layer caching rebuilds every layer from scratch
• Large artifact upload to GitLab or S3
• kubectl rollout status without a timeout waits indefinitely

4. Step-by-Step Fix

Step 1: Identify which timeout fired and which step is slow

# The error message tells you the timeout value that fired:
# "execution took longer than 1h0m0s seconds" → 60 min = default project timeout
# "execution took longer than 3600 seconds"  → same
# "execution took longer than 2h0m0s seconds" → custom 2h project timeout

# To find which step is slow, add timing to your jobs:
build:
  script:
    - date && echo "Starting build..."
    - npm ci
    - date && echo "Build complete."
    - npm run build
    - date && echo "Build artifact ready."

# Or use the built-in GitLab timestamps on each log line

Step 2: Increase project CI/CD timeout (if the job legitimately needs more time)

# In GitLab UI:
# Project → Settings → CI/CD → General pipelines → Timeout
# Change from 1h to 2h or 3h as needed

# Then set a job-level timeout in .gitlab-ci.yml:
build:
  timeout: 90 minutes
  script:
    - npm ci
    - npm run build

# Or for a specific deployment job that might take longer:
deploy-production:
  timeout: 2 hours
  script:
    - ./deploy.sh

Step 3: Fix slow dependency installation with caching

# Cache node_modules between pipeline runs
build:
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/
    policy: pull-push
  script:
    - npm ci --prefer-offline  # uses cache; fails if package not cached

# Cache Python packages
test:
  variables:
    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
  cache:
    key: "$CI_JOB_IMAGE"
    paths:
      - .cache/pip/
  script:
    - pip install --cache-dir .cache/pip -r requirements.txt

Step 4: Fix slow Docker builds with layer caching

# Pull the previous image to use as cache source
build-image:
  script:
    # Pull existing image for cache (ignore failure if image doesn't exist yet)
    - docker pull $CI_REGISTRY_IMAGE:latest || true
    # Build using the pulled image as cache
    - docker build
        --cache-from $CI_REGISTRY_IMAGE:latest
        --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
        --tag $CI_REGISTRY_IMAGE:latest
        .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

Step 5: Fix jobs hanging due to missing timeout on commands

If a job appears to hang (shows no log output for many minutes), a command is waiting indefinitely. See also Fix Jenkins Pipeline Stuck for the same pattern in Jenkins.

# Always add --timeout to kubectl commands
kubectl rollout status deployment/my-app --timeout=5m
kubectl wait --for=condition=Ready pods -l app=my-app --timeout=120s

# Add timeouts to curl calls
curl --connect-timeout 10 --max-time 30 https://api.example.com

# Add timeouts to AWS CLI calls
aws --cli-read-timeout 30 --cli-connect-timeout 10   ecs wait services-stable --cluster my-cluster --services my-service

Step 6: Parallelise slow test suites

# GitLab CI parallel jobs
test:
  parallel: 4
  script:
    # Use CI_NODE_INDEX and CI_NODE_TOTAL to split tests
    - pytest tests/ --split-by $CI_NODE_TOTAL --split-index $CI_NODE_INDEX

# Or use matrix for different test groups
test:
  parallel:
    matrix:
      - TEST_SUITE: [unit, integration, e2e]
  script:
    - pytest tests/$TEST_SUITE

5. Verification Steps

# After fixing, check the pipeline duration in GitLab:
# CI/CD → Pipelines → click the pipeline → check each job duration

# Confirm the timeout is now correct relative to expected duration
# A job that normally takes 8 minutes shouldn't have a 60-minute timeout
# Set it to ~15-20 minutes to catch actual hangs without being too tight

6. Common Mistakes

• Raising the timeout without fixing the underlying slowness — this just delays the timeout from catching real hangs
• Setting a job-level timeout higher than the project timeout — GitLab silently uses the project limit
• Not caching dependencies — every job reinstalls from scratch and takes 5-10x longer than necessary
• Using docker pull without || true — if the image doesn't exist yet, the cache pull fails and blocks the build
• Not adding --timeout to kubectl rollout status — if the deployment fails, the job hangs indefinitely

7. Prevention Tips

• Set job-level timeouts explicitly for every job — don't rely on the project default
• Monitor pipeline duration trends — if a job grows from 5 min to 45 min over time, investigate before it starts timing out
• Use distributed caching with a shared S3 backend for large dependency sets shared across many runners
• Profile your Docker builds with --progress=plain to identify slow layers
• Consider using GitLab's built-in CI analytics to identify which jobs are consistently slow
• For deploy jobs, always specify a rollout timeout that reflects your deployment SLO

8. FAQ

9. Summary