Running Playwright?

Playwright flaking in CI you cannot reproduce? We stabilise it.

Most Playwright suites that reach us pass locally and fail in CI with a different test each run. We read the traces, pin down what is actually flaky versus what is the app being honest, and leave the suite with the kind of pass rate that makes merging a release feel safe again.

  • Since 2015Fixing technology
  • FreeEstimates
  • US teamReal engineers

playwright.log

still failing

What it is

Playwright at a glance

Playwright is an end-to-end testing framework for web applications, released by Microsoft in 2020. It drives Chromium, Firefox and WebKit through a single API, supports TypeScript, JavaScript, Python, Java and .NET, and ships with a test runner, trace viewer, codegen and visual testing. The project is maintained by the same team that originally built Puppeteer and reuses many of the same engineering patterns.

Its design assumptions, auto-waiting on locators, strict mode for ambiguous selectors, isolated browser contexts per test, trace capture on failure, cut a lot of the flakiness that older tools inherited. Playwright is now the default E2E choice for a majority of new JavaScript projects, overtaking Cypress on download numbers in 2023.

Type
End-to-end browser testing framework
First released
2020
License
Apache 2.0, open source
Maintained by
Microsoft and the Playwright contributors
Typically used for
End-to-end web testing, visual regression, authenticated user flows, cross-browser verification, CI smoke tests, API plus UI mixed suites

The honest version

Playwright pros and cons

No technology is all upside. This is where Playwright earns its place and where it tends to cause trouble.

Where Playwright is strong

  • Three real browser enginesChromium, Firefox and WebKit all drive through the same API, which means a Safari-specific bug can be caught in CI instead of on a staging URL opened on an iPad.
  • Auto-waiting cuts flakiness at the sourceLocators retry assertions and interactions until the element is actionable, so most of the sleep calls that polluted older suites disappear.
  • Trace viewer is a step changeA failing test produces a timeline with DOM snapshots, network, console and action logs you can click through. Debugging a CI failure feels almost like being there.
  • Codegen for new testsnpx playwright codegen records a session into a working test, which is a useful starting point for anyone writing their first scenario.
  • Browser contexts isolate tests cleanlyEach test gets a fresh context, cookies and storage, so flakes caused by shared state between tests largely disappear by design.
  • Network interception and API testingThe same test can stub a backend response, hit the real API and verify the UI, which keeps complex flows readable in one file.

Where Playwright bites

  • CI setup is more involved than it looksBrowser binaries, fonts, system libraries and Docker images all have to agree. A test that passes on a laptop can fail on CI because a shared library is missing from the runner image.
  • Timeouts cascade in confusing waysTest timeout, action timeout, expect timeout and navigation timeout each exist and interact. A failure that looks like one can really be any of them.
  • Strict mode catches sloppy selectorsA locator resolving to multiple elements now throws rather than picking the first match, which breaks older suites that relied on implicit behaviour.
  • Parallel execution is a double edgeThe default parallelism is fast until shared infrastructure, databases or auth tokens are not safe to use from multiple contexts at once.
  • Visual tests are pickyAntialiasing, cursor rendering and font smoothing differ between the dev machine and the CI container, which produces diffs that are visually real but functionally meaningless.
  • The test budget still needs disciplinePlaywright is fast, but it is not free. A suite that runs in minutes locally can take half an hour in CI, and nobody notices until a release is being held up.

Sound familiar?

Common Playwright problems we fix

The Playwright problems that bring people to us, and what you have once each one is fixed.

TimeoutError: page.click: Timeout 30000ms exceeded

Tests flake on timeouts in CI

Shared staging data changed, an animation delay only shows up on slower CI hardware, or a modal is covering the target element intermittently.

Causes pinned, waits replaced with locators

Executable doesn't exist at /ms-playwright

Browsers are not installed in CI

A pipeline runs npm ci but never runs npx playwright install, or the CI cache is restored without the browser binaries, so every test fails before the first assertion.

Install step wired into CI cache

StrictModeViolation

Locators match more than one element

A selector like text=Save matches three buttons after a UI change, and strict mode throws rather than quietly picking the first one.

Selectors unambiguous, tests honest

net::ERR_CONNECTION_REFUSED at http://localhost:3000

The test server never finishes starting

The webServer config does not wait long enough on cold boot, or a database migration inside the dev server pushes startup past the configured timeout.

Server ready before the first test

same 3 tests retried 5 times

The same tests are always the flaky ones

A specific flow, usually auth or checkout, relies on timing or external state that is not quite stable. Retry hides it; a real fix stabilises it.

Flaky tests rewritten or removed

visual diff: 142 pixels different

Visual snapshots do not match CI

A font fallback, a cursor render, or a subpixel antialiasing difference between the dev machine and the CI browser produces a diff that is not meaningful.

Baselines generated in CI, tolerance tuned

Test is slow: 82.4s

The suite has slowed to a crawl

Sequential setup per test, reused fixtures that are expensive, or a slow backend in the test environment are stretching every test individually.

Fixtures shared where safe, time recovered

EADDRINUSE: port 3000

Port conflicts break the webServer

A stale Playwright process is still holding the port from a crashed run, or local development is already using it, and the new run cannot bind.

Dedicated ports or dynamic allocation

What we do

Playwright help, from stabilisation to CI

Call us for one flaky suite, or let us own the whole E2E setup.

Rescue
A suite that passes locally and fails in CI with a different test each run. We read the traces and stabilise what is actually broken before anything else.
First-time E2E setup
A Playwright project set up correctly from scratch: fixtures, auth, test data, parallelism, CI wiring and a sensible baseline for the first few flows.
Migration from Cypress
Porting an existing Cypress suite to Playwright with equivalent coverage, with the bits of the old suite that were not pulling their weight quietly retired.
CI pipeline
GitHub Actions, GitLab CI or CircleCI configured with correct Playwright containers, sharding, retries, HTML reports and failure traces uploaded for review.
Visual and accessibility
Snapshot baselines, @axe-core integration, and the tooling to triage diffs without burning out the person on review duty.
Managed Tech
Monthly maintenance of the suite, Playwright minor upgrades, and a pair of eyes when a release-blocking flake needs to be understood today.

How it works

From first message to fixed

  1. Look Free estimate

    Tell us what's going wrong. We look at it, then tell you plainly what's wrong and what it will cost.

  2. Fix

    We fix the cause, not the symptom. If something is down, getting it back up comes first.

  3. Harden

    Backups, monitoring, updates and access, so the same problem can't quietly come back.

  4. Hand back or stay on

    You get a plain-language write-up. Or we stay on for hosting, upkeep and new work.

Often found together

Running Playwright alongside something else?

We work on the whole stack, so the problem can be wherever it actually is.

Questions

Playwright questions

Something else? Call+1 737 727 7327

Why does our Playwright suite pass locally and fail in CI?

Usually a handful of causes: different timing on slower CI hardware, system fonts missing from the CI image, test data that was pre-populated locally but not in the pipeline, or a webServer config that races the first test. We read the trace from a failing run and the real cause is almost always visible there.

Can you stabilise a flaky Playwright suite without rewriting it?

In most cases yes. The flakes concentrate in a small number of tests, and the fix is often moving away from CSS selectors, swapping sleeps for locator assertions, or adding one deterministic setup step. A full rewrite is rarely the right first move.

Should we migrate from Cypress to Playwright?

For many teams yes, especially for cross-browser coverage, better debugging with the trace viewer, and genuinely faster parallel execution. For teams happy with a stable Cypress suite covering Chromium only, the rewrite does not always pay for itself. We give you an honest comparison before any work begins.

Can you add Playwright to our existing CI without breaking it?

Yes. We add the install step, the browser cache, correct Docker containers or system packages, a sharded job, and HTML report uploads. The pipeline keeps running everything it did before, with Playwright as an additional stage rather than a replacement.

Why do our visual snapshots keep diffing?

Font rendering, antialiasing, cursor state and tiny animation timings differ between a developer laptop and a CI container, which produces diffs that look real and are not. We generate the baseline in CI itself, set tolerances where they belong, and mask the regions that are genuinely nondeterministic.

How much does Playwright help cost?

The quote is free. Send over the failing CI log and a recent trace zip, and a written price lands in your inbox the same day.

Free estimate

Playwright suite driving you mad?

Tell us what you're running and what's going wrong. A person reads it and comes back with an estimate.

  • No charge to find out what it will take
  • If it's urgent, say so

Replies come from a person. By sending, you authorize us to email you back. No marketing.