How do I share a Criterion benchmark report?

`cargo bench` writes `target/criterion/report/index.html` with the violin plots, the PDF charts and the regression estimates. Drag the `target/criterion/` folder into Comma, or PATCH the index HTML through the API — you get one browsable URL instead of a directory only you can open.

Does this work for pytest-benchmark and JMH too?

Yes. pytest-benchmark's `--benchmark-histogram` output and the JMH Visualizer's exported HTML publish the same way — a single entry page plus whatever assets it references. Any tool that produces an HTML page and some images is a bundle upload.

Do the SVG plots survive?

Yes. Criterion's plots are SVG files referenced from the report page; they upload as assets and their relative references are rewritten, so the charts render at the URL exactly as they do locally.

How do I compare two runs?

PATCH the same report id after each run. Each run appends a revision at one stable URL, so a regression is a diff between two revisions instead of two folders and a memory. Criterion's own baseline comparison still works inside a single run; the revision history is what makes the comparison durable and shareable.

Why publish a benchmark rather than paste the numbers?

Because a pasted number invites the wrong argument. The report has the distribution, the outliers and the confidence interval, which is what separates a real 12% regression from noise on a loaded CI runner — and a link lets a reviewer check that themselves.

Share a benchmark report

Every benchmarking tool worth using produces more than a number. Criterion gives you the sample distribution, the outlier count and a confidence interval. pytest-benchmark gives you a histogram. JMH gives you per-fork variance. All of it exists so you can tell a regression apart from a noisy CI runner.

And all of it gets thrown away at the moment of sharing, when someone pastes "12% slower" into a channel and the team argues about whether that is real.

Publish the report

cargo bench                      # → target/criterion/report/index.html

Drag target/criterion/ (or a zip of it) into the app: report/index.html becomes the report body, the per-benchmark pages and the SVG plots upload as assets, and the relative references are rewritten so the drill-down from the summary into a single benchmark still works.

Same shape for the others:

pytest-benchmark — pytest --benchmark-histogram=bench writes SVGs; wrap them in a page or publish the generated output directly.
JMH — export from the JMH Visualizer and publish the HTML.
hyperfine — --export-markdown is one table; render it and publish the page with the sentence explaining what changed.

From CI, one URL per suite

curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html target/criterion/report/index.html \
        --arg title "Benchmarks — $GIT_SHA" '{title: $title, html: $html}')"

Every run appends a revision at the same URL. That is the part that changes the conversation: performance work is inherently a comparison over time, and a stable URL with history is the smallest possible version of a performance dashboard — no service to run, no metrics pipeline to own. See publishing from CI.

What the review layer adds

"This regressed with the allocator change, and we accepted it — see the memory numbers." Anchored to the benchmark it concerns, still there six months later. See commenting on HTML.
"The runner was shared for this build; rerun before you believe it." Recorded next to the result, not lost in a thread.
Access per report — private, team, domain-gated, or public if the numbers are a selling point. See the sharing model.

Limits

Entry HTML: 5 MB. Assets: 25 MB per file, 250 MB and 500 files total. A large Criterion suite generates a lot of SVGs — publish per benchmark group if you approach the file count.
Scripts run, sandboxed: allow-scripts, no allow-same-origin.
60 requests/minute per token.

Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision history.

Publish a benchmark report →