Can hyperfine export HTML?

Not directly. It exports JSON, CSV, Markdown and AsciiDoc, and its repository ships Python plotting scripts. The usual routes are Markdown through pandoc, or the JSON through a plotting script into an interactive chart.

Which export should I use?

`--export-markdown` for a quick comparison table — it is already the format people paste into pull requests. `--export-json` when you want a whisker plot or a histogram, because the per-run timings are only in the JSON.

Why do error bars matter here?

Because a 4% improvement with an 8% standard deviation is not an improvement. hyperfine reports the spread; a screenshot of a mean hides exactly the number that decides whether the change is real.

Can teammates comment on a specific comparison?

Yes. Threads anchor to the rendered content, so 'this one is dominated by process startup, not the change' sits on that row.

Share a hyperfine benchmark

hyperfine is the tool that makes "it's faster now" a checkable claim: warmup runs, statistical outlier detection, and a standard deviation next to every mean. Then the result gets pasted into a pull request as a screenshot of a terminal, and the standard deviation — the number that decides whether the change is real — is the first thing nobody reads.

Markdown → HTML

hyperfine --warmup 3 \
  './old-binary --input big.json' \
  './new-binary --input big.json' \
  --export-markdown bench.md

pandoc bench.md -s -o bench.html

The Markdown table hyperfine emits is already the comparison, with relative speedup and the ± spread. pandoc -s makes it a standalone page. See sharing pandoc HTML.

JSON → a plot

hyperfine --warmup 3 -m 30 \
  './old-binary' './new-binary' \
  --export-json bench.json

python scripts/plot_whisker.py bench.json -o bench.png

The plotting scripts in hyperfine's repository (plot_whisker.py, plot_histogram.py) read the JSON. If you want something interactive rather than a PNG, feed the same JSON into a Plotly or Vega-Lite figure and publish that — the per-run timings are all in the file.

Publish it

Drag bench.html into the app, or from CI:

curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html bench.html \
        --arg title "Benchmark — $GITHUB_REF_NAME" '{title: $title, html: $html}')"

Benchmarks want a schedule, not a commit hook

CI runners are shared, throttled and noisy; per-commit benchmarks mostly measure the neighbours. A nightly run on a consistent machine PATCHing one report id gives you a revision history where a real regression stands out against the noise floor instead of drowning in it.

A routine does the re-running and the publishing.
Anchored threads on a specific row when the number is disputed. See commenting on HTML.

Limits

Entry HTML: 5 MB. Assets: 25 MB per file, 250 MB and 500 files total.
60 requests/minute per token.

Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision history.

Publish a benchmark →