Comma

Share a Playwright HTML report

Every team that runs Playwright in CI knows the ritual. A test fails on main. Someone posts the CI link in Slack. Whoever clicks it logs into the CI provider, finds the right workflow run, downloads playwright-report.zip, unzips it, opens index.html locally — and then screenshots the failure back into Slack anyway, because there's no way to point at it otherwise.

Three structural problems, none of them Playwright's fault:

  1. The report is buried. It lives inside an artifact zip, behind CI login, N clicks from the Slack message that mentioned it.
  2. The link expires. CI providers retain artifacts on a schedule — the run you want to reference from last month's incident review is often already gone.
  3. There's nowhere to discuss it. A failing trace deserves a thread anchored to that failure. Instead the conversation happens in Slack, detached from the artifact, and evaporates into scrollback.

The fix is the same one-curl pattern Comma uses for coverage and eval reports: publish the run's HTML to a stable URL, let reviewers comment on the failures in place, and append a revision per run.

The pattern

Store a scoped token (just reports:write) in your CI secret store. Create the report once, save its id, and PATCH it on every run so there is one URL per suite, not one per run:

curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html run-summary.html '{html: $html}')"

Each push appends a revision. Reviewers can diff any two runs — "which tests changed state since the last green build?" is a diff, not a memory exercise — and the bookmarked link never goes stale.

In GitHub Actions:

- name: Publish Playwright summary to Comma
  if: always()
  env:
    COMMA_API_TOKEN: ${{ secrets.COMMA_API_TOKEN }}
    REPORT_ID: ${{ vars.COMMA_E2E_REPORT_ID }}
  run: |
    curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
      -H "Authorization: Bearer $COMMA_API_TOKEN" \
      -H "Content-Type: application/json" \
      -d "$(jq -n --rawfile html run-summary.html '{html: $html}')"

The if: always() matters — the runs you most want to share are the red ones. The same two lines work in GitLab CI, CircleCI, or Buildkite; it's plain HTTPS.

The honest part: it depends on how big the run is

Comma stores report HTML verbatim and renders it inside a sandboxed iframe (sandbox="allow-scripts", no allow-same-origin). Scripts run — the frame just can't reach the app's DOM, cookies, or storage. So Playwright's own report, which inlines its run data into index.html as a base64 zip, publishes and stays interactive: filtering works, steps expand.

Where it stops working is size and siblings. The report body is capped at 5 MB, and a few hundred tests with inline attachments clears that easily. Screenshots, videos and traces live in playwright-report/data/; drop the whole directory in and those relative src/href references are rewritten to the uploaded assets, but a report that fetches its data over XHR at view time has an opaque origin and nothing to resolve the path against.

So: publish index.html directly when the suite is small. For everything else — and for the review experience, which is better anyway:

  • Publish a static digest as the report body. A plain HTML table of tests, statuses, durations, and error text, generated from Playwright's JSON reporter output (--reporter=json) by a small script in your pipeline. A ~30-line script covers it, and the result is exactly the surface a reviewer wants to comment on. Failure screenshots can be inlined as images.
  • Attach the heavy artifacts as assets. Trace zips, videos, and the full report archive go in as report assets — 25 MB per file, 250 MB per report — so they live next to the discussion instead of in an expiring CI artifact.
  • Step through traces in the trace viewer. When a failure needs frame-by-frame investigation, download the attached trace and open it in Playwright's trace viewer. Comma is where the discussion lives, not a trace debugger — and it doesn't pretend to be.

If you need the full interactive report preserved verbatim, a static host plus the artifact zip still does that job. What it won't give you is the next section.

Comments land on the failure, not near it

The published digest renders inside Comma's sandbox with an anchored comment layer on top. A reviewer highlights the row for checkout.spec.ts › applies discount code and pins a thread to it: "selector changed in Tuesday's release — fix in #482." The thread stays attached as revisions accumulate, so when the same test flakes again in three weeks, the prior discussion is one click away instead of buried in Slack.

The loop closes with agents, too: a Claude Code or Cursor agent attached via Comma's MCP server can read those threads with list_comments, fix the selector, and reply on the thread — same scoped token as the CI publish.

Per-run hygiene

  • One report per suite, one revision per run — PATCH, don't POST, and title revisions by commit ("E2E — a1b2c3d") so the history reads like a log.
  • Rate limits are a non-issue — per-token limits default to 60 requests/minute; one publish per build is nowhere near it.
  • Announce red runs — add a webhook notification on revision.created and the new revision posts itself to Slack or Discord. The Slack message goes back to being a notification; the report is the artifact.
  • Keep it team-visible — CI-published reports follow the same sharing model as everything else. E2E results usually want team or private, not public-by-default.

Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision history. Take the next red run you would have zipped, publish the digest, and send the link instead.

Create your first report →

Related