Share a pytest HTML report
pytest --html=report.html is one of the most-used lines in Python CI, and
the file it produces is one of the least-read documents in software. It
lands in an artifact bucket, expires on a retention schedule, and requires
a CI login and a zip download from anyone who wants to see why the build
went red. So nobody looks. Someone screenshots the traceback into Slack
instead, and the report might as well not exist.
The report is fine. The distribution is broken.
Publish it to a URL instead
Generate a self-contained report, then push it to Comma:
pytest --html=report.html --self-contained-html
curl -fsS -X POST https://commareports.com/api/v1/reports \
-H "Authorization: Bearer $COMMA_API_TOKEN" \
-H "Content-Type: application/json" \
-d "$(jq -n --rawfile html report.html \
'{title: "API suite", html: $html}')"
The response carries the report id and its share URL. Save the id — that's what turns a pile of one-off reports into a single living document.
One URL per suite, one revision per run
PATCH the same id on every run and the link stops rotting:
- name: Publish pytest report to Comma
if: always()
env:
COMMA_API_TOKEN: ${{ secrets.COMMA_API_TOKEN }}
REPORT_ID: ${{ vars.COMMA_PYTEST_REPORT_ID }}
run: |
curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
-H "Authorization: Bearer $COMMA_API_TOKEN" \
-H "Content-Type: application/json" \
-d "$(jq -n --rawfile html report.html '{html: $html}')"
if: always() is the whole point — the runs worth sharing are the failing
ones. Use a scoped token with only reports:write in
your CI secret store; it's plain HTTPS, so GitLab CI, CircleCI, Jenkins,
and Buildkite all take the same two lines. The general shape is documented
in publish from CI.
Every push appends a revision at the same URL. Two runs can be diffed against each other, which turns "did anything change since the last green build?" into a question with an answer instead of a memory exercise.
What renders, and what doesn't
pytest-html's output is mostly a static table, which is why it survives the trip well:
- Renders faithfully — the results table, pass/fail/xfail/error statuses, durations, captured stdout and stderr, and full tracebacks.
- Inert — the sortable column headers and the "show/hide passed"
checkboxes are script-driven, and Comma strips
<script>on write as defense-in-depth on an endpoint that accepts arbitrary HTML. - Better with
--self-contained-html— one file, no siblingassets/directory to lose. If you'd rather keep the assets folder, drop the whole directory (or a zip of it) into Comma: relativesrcandhrefreferences are rewritten to uploaded assets, so the report renders intact.
The part CI artifacts can't do
A shared link is table stakes. What the artifact bucket never had is a place to put the sentence you were going to type in Slack.
A reviewer highlights the traceback for
test_checkout.py::test_applies_discount and pins a thread to it: "this
started after the pricing refactor — see #482." The thread stays anchored
to that failure as revisions pile up, so when the same test flakes three
weeks later, the prior investigation is one click away rather than lost in
scrollback. That's the anchored comment model in one
paragraph.
The loop closes on the automation side too. An agent in Claude Code or
Cursor attached through Comma's MCP server can read those threads
with list_comments, push a fix, and reply on the thread — same scoped
token as the CI publish.
Housekeeping
- One report per suite. Unit, integration, and E2E each get their own
id and their own URL. Title revisions by commit (
"API suite — a1b2c3d") so the history reads like a log. - Attach the heavy stuff. Screenshots, logs, and profiles go in as assets — 25 MB per file, 250 MB per report.
- Announce red runs. A webhook on
revision.createdposts the new revision to Slack or Discord. The Slack message goes back to being a notification; the report is the artifact. - Rate limits are a non-issue. Per-token limits default to 60 requests/minute — one publish per build is nowhere near it.
- Pick the right visibility. Test results usually want
teamorprivate, not public. See the sharing model.
Try it
Comma is free: unlimited reports, unlimited commenters, unlimited revision
history. Take the next red build you would have zipped, publish
report.html, and send the link instead.