Share a Schemathesis report

Schemathesis generates requests your OpenAPI schema claims are legal and finds the ones your server disagrees with. The result is almost always a surprise to somebody, and that somebody is usually not the person who ran it.

Which makes the delivery format the bottleneck, not the tool.

Run it and export JUnit

pip install schemathesis junit2html

schemathesis run https://api.example.com/openapi.json \
  --report junit || true
junit2html schemathesis-report/junit.xml schemathesis.html

On Schemathesis 3.x the flag is --junit-xml=report.xml instead; the conversion step is identical either way. || true keeps the pipeline alive long enough to publish — keep a separate unmuted run as the gate.

Publish it

Drag schemathesis.html into the app, or from CI:

curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html schemathesis.html \
        --arg title "Schemathesis — $GITHUB_REF_NAME" '{title: $title, html: $html}')"

One report id per API. Every run PATCHes it, so the URL you put in the API channel stays correct.

Why the failing input needs a page

A Schemathesis failure is a generated request: a string with a zero-width space in it, an integer one past the documented maximum, a null where the schema said nullable: false was enforced. Pasted into chat, that input goes through a markdown renderer and stops reproducing.

On a published page it is inside the report, byte for byte, next to the reproduction command — and someone can put a thread on it:

  • Anchored threads on the specific failing case. See commenting on HTML.
  • Revisions, so "did we already fix this one?" has an answer.
  • Access per report — API contract findings are usually internal; keep the report private or team-scoped. See the sharing model.

Limits

  • Entry HTML: 5 MB. Assets: 25 MB per file, 250 MB and 500 files total.
  • 60 requests/minute per token.

Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision history.

Publish an API report →

Related