Share a pytest HTML report

pytest --html=report.html is one of the most-used lines in Python CI, and the file it produces is one of the least-read documents in software. It lands in an artifact bucket, expires on a retention schedule, and requires a CI login and a zip download from anyone who wants to see why the build went red. So nobody looks. Someone screenshots the traceback into Slack instead, and the report might as well not exist.

The report is fine. The distribution is broken.

Publish it to a URL instead

Generate a self-contained report, then push it to Comma:

pytest --html=report.html --self-contained-html

curl -fsS -X POST https://commareports.com/api/v1/reports \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html report.html \
        '{title: "API suite", html: $html}')"

The response carries the report id and its share URL. Save the id — that's what turns a pile of one-off reports into a single living document.

One URL per suite, one revision per run

PATCH the same id on every run and the link stops rotting:

- name: Publish pytest report to Comma
  if: always()
  env:
    COMMA_API_TOKEN: ${{ secrets.COMMA_API_TOKEN }}
    REPORT_ID: ${{ vars.COMMA_PYTEST_REPORT_ID }}
  run: |
    curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
      -H "Authorization: Bearer $COMMA_API_TOKEN" \
      -H "Content-Type: application/json" \
      -d "$(jq -n --rawfile html report.html '{html: $html}')"

if: always() is the whole point — the runs worth sharing are the failing ones. Use a scoped token with only reports:write in your CI secret store; it's plain HTTPS, so GitLab CI, CircleCI, Jenkins, and Buildkite all take the same two lines. The general shape is documented in publish from CI.

Every push appends a revision at the same URL. Two runs can be diffed against each other, which turns "did anything change since the last green build?" into a question with an answer instead of a memory exercise.

What renders, and what doesn't

pytest-html's output is mostly a static table, which is why it survives the trip well:

  • Renders faithfully — the results table, pass/fail/xfail/error statuses, durations, captured stdout and stderr, and full tracebacks.
  • Inert — the sortable column headers and the "show/hide passed" checkboxes are script-driven, and Comma strips <script> on write as defense-in-depth on an endpoint that accepts arbitrary HTML.
  • Better with --self-contained-html — one file, no sibling assets/ directory to lose. If you'd rather keep the assets folder, drop the whole directory (or a zip of it) into Comma: relative src and href references are rewritten to uploaded assets, so the report renders intact.

The part CI artifacts can't do

A shared link is table stakes. What the artifact bucket never had is a place to put the sentence you were going to type in Slack.

A reviewer highlights the traceback for test_checkout.py::test_applies_discount and pins a thread to it: "this started after the pricing refactor — see #482." The thread stays anchored to that failure as revisions pile up, so when the same test flakes three weeks later, the prior investigation is one click away rather than lost in scrollback. That's the anchored comment model in one paragraph.

The loop closes on the automation side too. An agent in Claude Code or Cursor attached through Comma's MCP server can read those threads with list_comments, push a fix, and reply on the thread — same scoped token as the CI publish.

Housekeeping

  • One report per suite. Unit, integration, and E2E each get their own id and their own URL. Title revisions by commit ("API suite — a1b2c3d") so the history reads like a log.
  • Attach the heavy stuff. Screenshots, logs, and profiles go in as assets — 25 MB per file, 250 MB per report.
  • Announce red runs. A webhook on revision.created posts the new revision to Slack or Discord. The Slack message goes back to being a notification; the report is the artifact.
  • Rate limits are a non-issue. Per-token limits default to 60 requests/minute — one publish per build is nowhere near it.
  • Pick the right visibility. Test results usually want team or private, not public. See the sharing model.

Try it

Comma is free: unlimited reports, unlimited commenters, unlimited revision history. Take the next red build you would have zipped, publish report.html, and send the link instead.

Create your first report →