Comma

Share a Buildkite HTML report

Buildkite's artifact story is deliberate: artifact_paths uploads the file, and Buildkite serves it back as a download. That is exactly right for a .zip, a core dump, or a binary. It is exactly wrong for index.html.

Click the coverage report in the Artifacts tab and you get a file in your Downloads folder. Open that file and the stylesheet is missing, because the CSS it references was a sibling in a directory that no longer exists around it. The reviewer sees unstyled text and gives up.

The usual fallback is an annotation:

buildkite-agent annotate --style error --context tests < summary.md

Annotations are genuinely good, and they are not a report. They take a restricted subset of markup — no scripts, no external stylesheets — and they are size-capped. A failure summary fits. A Playwright HTML report, a coverage tree, or a pytest --html page does not.

Publish the report from the step

Mint a scoped token with reports:write and nothing else, expose it to the agent the way you already expose secrets, create the report once, then PATCH that id every build:

steps:
  - label: ":test_tube: tests"
    command: .buildkite/test.sh
    artifact_paths:
      - "playwright-report/**/*"

#!/bin/bash
# .buildkite/test.sh
set -uo pipefail

npx playwright test
EXIT=$?

curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$COMMA_REPORT_ID" \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html playwright-report/index.html \
        --arg title "Tests — ${BUILDKITE_BRANCH} @ ${BUILDKITE_COMMIT:0:7}" \
        '{title: $title, html: $html}')"

buildkite-agent annotate --style info \
  "Report → https://commareports.com/p/$COMMA_REPORT_ID"

exit $EXIT

Two details carry the weight:

  • Capture the exit code, publish, then re-exit. Buildkite halts a command list at the first non-zero exit, so a naive && chain publishes only the green builds — the ones nobody needed to read. A pre-exit agent hook does the same job pipeline-wide if you would rather not touch each script.
  • Annotate with the link, not the report. The annotation is now one line and always fits, and it is the fastest path from the build page to something a human can read.

artifact_paths stays. Keeping the artifact costs nothing and engineers who are already inside Buildkite may prefer it; the published report is for everyone who is not.

What the published copy adds

  • It renders. Report HTML is stored verbatim and served inside a sandboxed iframe with scripts enabled, so an interactive report stays interactive instead of arriving as a download prompt.
  • One URL across every build. Revisions accumulate at the same address and any two can be diffed, which turns "what changed since the last green build" into a question with an answer.
  • Readers with no Buildkite seat. Visibility is private, team, domain-gated, or link, decided per report and independent of your pipeline's ACL.
  • Comments anchored to the content. The reviewer highlights the assertion that regressed and pins a thread to it, and the thread survives the next hundred builds — see commenting on HTML.

Limits

  • HTML body: 5 MB. Traces, videos and archives go in as assets at 25 MB per file, 250 MB per report.
  • Self-contained reports stay interactive. A report that fetches sibling data files at view time can't, because the frame has an opaque origin — publish a static digest and attach the archive, as in sharing an Allure report.
  • Rate limit: 60 requests/minute per token. One publish per build is nowhere near it.

Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision history.

Create your first report →

Related