Running Dagger?

Dagger pipelines stalling or caches not hitting? We sort it out.

Dagger is a strong idea once it is up and a frustrating one when it is not. An engine container that refuses to start on a self hosted runner, cache volumes that only work on one machine, or a module graph that takes four minutes to initialize before a single step runs are all familiar calls. We get the engine, the modules and the cache layout producing short predictable runs.

  • Since 2015Fixing technology
  • FreeEstimates
  • US teamReal engineers

dagger-run.log

still failing

What it is

Dagger at a glance

Dagger is a programmable CI runtime that lets engineers express pipelines in Go, Python, TypeScript or a growing set of languages, running them against a BuildKit backed engine. Solomon Hykes, who previously founded Docker, started Dagger in 2022, and the project shipped its module system in 2024 along with Dagger Cloud for shared cache and telemetry.

The design decouples pipelines from any particular CI product: the same code runs locally, inside GitHub Actions, GitLab CI, Jenkins or on a self hosted runner, because the engine is just a container that speaks to BuildKit. The win is reproducible builds and portable caching, and the tradeoff is a new runtime and module format to run, upgrade and debug alongside your existing CI.

Type
Portable CI pipelines as code with a BuildKit engine
First released
2022 initial release, modules shipped in 2024
License
Apache 2.0, open source core, Dagger Cloud as a paid service
Maintained by
Dagger, Inc. and community contributors
Typically used for
CI pipelines, container builds, test matrices, release automation, local to CI parity

The honest version

Dagger pros and cons

No technology is all upside. This is where Dagger earns its place and where it tends to cause trouble.

Where Dagger is strong

  • Pipelines are real codeGo, Python or TypeScript with types, tests and IDE support beat a 400 line YAML file every time, especially when a pipeline grows conditionals and loops.
  • The same pipeline runs locally and in CIdagger run executes against a local engine or a remote one, so you can reproduce a red build on a laptop rather than pushing debug commits.
  • BuildKit caching is powerfulLayer and mount caches cut rerun times aggressively, and Dagger Cloud lets you share them across runners and branches.
  • Portable across CI providersA pipeline that works in GitHub Actions also works in GitLab, Jenkins or a self hosted runner, so a CI migration stops being a rewrite.
  • Module ecosystem is growingPublished modules for Node, Go, Python and container tasks mean you lean on shared code rather than reinventing the same lint job on every repo.
  • Honest about what it isDagger does not try to be the whole CI product. It is the runtime; your CI provider still runs the triggers and surfaces the logs.

Where Dagger bites

  • Engine startup time can hurt short pipelinesThe engine container has to boot, load modules and warm caches before anything runs. On a very short pipeline that overhead is visible and sometimes unpleasant.
  • Cache portability is not freeLocal caches do not always port cleanly to a different runner, and cross version engines can invalidate caches silently. Dagger Cloud helps but is a paid service.
  • Module graph errors are opaqueA typo in a module parameter or a mismatched schema usually surfaces as a generic failure during load, with the real cause two layers deeper in the engine logs.
  • You still need a CI providerDagger replaces the pipeline definition but not the trigger, UI, approvals or secret management. Teams expecting a full CI replacement are often surprised.
  • Debugging is a new skillBuildKit errors, cache mount lock issues and engine restarts need a mental model that is different from a YAML runner, and the docs are catching up with reality.
  • Ecosystem is youngFewer community modules than GitHub Actions marketplace, fewer blog posts, and smaller hiring pool for engineers who have run it in anger.

Sound familiar?

Common Dagger problems we fix

The Dagger problems that bring people to us, and what you have once each one is fixed.

failed to connect to engine

The Dagger engine will not start

A runner without permission to run privileged containers, a Docker socket that is not mounted, or an engine image version mismatch between the CLI and the container.

Engine starts, pipelines run

cache mount locked by another build

Parallel runs are fighting over a cache

Two pipelines sharing a cache volume name and running concurrently, so one waits for the other to release the lock instead of using independent caches.

Caches scoped sensibly per pipeline

engine OOMKilled during export

The engine runs out of memory on large exports

A pipeline that assembles large container images or tarballs in a single step, exceeding the engine container's memory limit mid export.

Exports fit the resource budget

cache invalidated after engine restart

Cache hits dropped after an engine update

An engine bump that changed the cache format subtly, invalidating prior entries and making the first build after upgrade look slow for no obvious reason.

Cache strategy survives upgrades

module not found: dagger.json missing

A module cannot be resolved

A pipeline referencing a module by git URL that does not have dagger.json at the referenced path, or a tag that was deleted upstream.

Modules load reliably

context canceled while loading module

Module loading is timing out

A module graph that pulls many transitive modules at startup, hitting network timeouts on a constrained runner.

Module graph loads inside limits

buildkit solve: context canceled

Pipeline steps are being canceled mid run

A CI job timeout tripping while Dagger is still resolving, or a flaky connection to a remote engine that cuts the session.

Runs complete inside the CI budget

remote runner connection reset

A remote runner keeps dropping the session

A network path between the CLI and a remote engine that cannot hold a long running session, often an idle proxy or a load balancer timeout.

Sessions stay connected

What we do

Dagger help, from engine boot to cached runs

A single red pipeline or a whole CI move to Dagger, both land in our inbox regularly.

Rescue
An engine that will not start, a pipeline that times out, or a cache that stopped hitting after an upgrade. We get runs green before we tidy the code.
Pipeline design
Module layout, step boundaries and cache scope worked out so pipelines are short, honest and easy for another engineer to read.
CI migration
Porting from Jenkins, CircleCI or GitHub Actions YAML into Dagger modules, keeping triggers and approvals on the current provider.
Cache and performance
Mount cache design, Dagger Cloud wiring and engine sizing so the median build drops without sacrificing reproducibility.
Self hosted runners
Engine deployment on your own runners, including Docker in Docker, Kubernetes and Nomad topologies, with the gotchas handled.
Managed Tech
Engine upgrades, module version hygiene and an on call CI engineer when a cache goes cold on the morning of a release.

How it works

From first message to fixed

  1. Look Free estimate

    Tell us what's going wrong. We look at it, then tell you plainly what's wrong and what it will cost.

  2. Fix

    We fix the cause, not the symptom. If something is down, getting it back up comes first.

  3. Harden

    Backups, monitoring, updates and access, so the same problem can't quietly come back.

  4. Hand back or stay on

    You get a plain-language write-up. Or we stay on for hosting, upkeep and new work.

Often found together

Running Dagger alongside something else?

We work on the whole stack, so the problem can be wherever it actually is.

Questions

Dagger questions

Something else? Call+1 737 727 7327

Does Dagger replace our CI provider?

No, and it is honest about that. Dagger replaces the pipeline definition and the execution runtime. The trigger on a pull request, the UI that shows the build result, the secret store and the approval workflow still live in GitHub Actions, GitLab, Jenkins or whichever CI you already have. Dagger runs inside those as a step, and the win is that the step can run identically on a laptop.

Why is our first Dagger run so slow?

Three usual causes. The engine container has to boot and that first cold start costs several seconds. The module graph has to load, and a large graph with many transitive modules pulls them all on first run. And the BuildKit cache is empty, so every layer gets built from scratch. Once the cache warms and the modules are resolved, the median run is much shorter.

Can we run Dagger in Kubernetes based runners?

Yes, and it is a common topology. The engine runs as a Pod with enough privileges to drive BuildKit, the CLI connects to it over a Unix socket or TCP, and caches live on a persistent volume or on Dagger Cloud. The gotchas are around privileged containers and image pull policy, both of which can be tightened down rather than left open.

How do we share Dagger caches between runners?

Local caches are per host, which is a problem for autoscaling runners. Dagger Cloud is the simplest answer, as it stores the cache off box and shares it across any engine that authenticates to the same account. Self hosted options exist but need registry and storage work you have to maintain.

Is Dagger worth it for a small CI pipeline?

Honestly, often no. A short YAML on GitHub Actions is fine for a repo with three jobs and a single language. Dagger starts paying off when you have many repos sharing patterns, when local reproducibility matters, or when the YAML has grown enough conditionals that a real programming language would be clearer.

How much does Dagger help cost?

Our quotes are no cost. Describe the pipeline and what is stalling, and a price arrives before any engineer touches your CI.

Free estimate

Dagger pipelines stalling?

Send over a failing run link or a summary of the pipeline, and a CI engineer gets back with a plan and a price.

  • No charge to find out what it will take
  • If it's urgent, say so

Replies come from a person. By sending, you authorize us to email you back. No marketing.