How do I share JMH benchmark results?

Run JMH with `-rf json -rff result.json`, render it with jmh-visualizer (which produces a static HTML bundle), and publish that to Comma. You get a URL with the score, the error bars and the per-fork detail, instead of a pasted score that lost its confidence interval.

Why does JMH not produce HTML itself?

JMH's job is measurement, not presentation — it emits text, CSV, JSON or SCSV. The community viewers (jmh-visualizer, JMH Visual Chart) read the JSON, which is why `-rf json` is the format to keep even if you also print the table.

Why do the error bars matter so much for a microbenchmark?

Because a 3% difference with a ±5% confidence interval is not a difference. JMH reports the interval precisely so you can tell a real improvement from noise, and quoting the score alone in a pull request throws that away — which is how benchmarks end up justifying changes that did nothing.

Can I publish results from CI?

Yes, with a caveat: shared CI runners are noisy, so treat CI numbers as regression detection with a wide threshold, not as the numbers you quote. PATCH one report id per benchmark suite so the run-to-run variance is itself visible.

Can teammates comment on a specific benchmark?

Yes. Threads anchor to the rendered content, so 'this one is allocation-dominated, check with -prof gc' stays attached to the benchmark it refers to.

Share a JMH benchmark report

JMH is careful in a way that gets discarded the moment results are shared. It runs warmup iterations, forks JVMs to defeat profile pollution, and reports a score with an error bound:

Benchmark                Mode  Cnt   Score   Error  Units
SerdeBench.jackson      thrpt   15  482.113 ± 9.244  ops/ms
SerdeBench.handRolled   thrpt   15  501.882 ± 22.71  ops/ms

And then somebody writes "hand-rolled is 4% faster" in a pull request. Look at the error bars: the intervals overlap. The benchmark does not say that.

Sharing the rendered artifact rather than a remembered number is most of the fix.

Get JSON, then render it

# from the benchmarks jar
java -jar target/benchmarks.jar -rf json -rff result.json

# or via the Gradle/Maven plugin, then render
npx jmh-visualizer result.json -o report/

jmh-visualizer writes a static folder — the chart bundle plus your data — with the score, the error bars, the per-fork breakdown and the secondary metrics (-prof gc allocation rates, if you collected them).

Publish the folder

Drag report/ (or a zip of it) into the app:

index.html becomes the report body.
The chart JS and your result JSON upload alongside it, with relative references rewritten to the uploaded copies.
Scripts run inside a sandboxed iframe (allow-scripts, no allow-same-origin), so the charts render and the per-benchmark drilldown works.

Or send the entry file to the API:

curl -fsS -X POST "https://commareports.com/api/v1/reports" \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html report/index.html \
        --arg t "JMH — serialization, JDK 21" '{title: $t, html: $html}')"

Collect the profilers while you are there

A score tells you that something changed. The profilers tell you why, and they cost one flag:

java -jar target/benchmarks.jar -rf json -rff result.json \
  -prof gc \
  -prof perfasm    # Linux, needs perf

-prof gc in particular turns "faster" into "allocates 40% less per op", which is the version of the claim that survives review.

From CI, carefully

Shared CI runners share CPU, so absolute JMH numbers from CI are not publication-grade. They are still useful as a regression tripwire with a generous threshold. PATCH one report id per suite:

curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html report/index.html \
        --arg t "JMH — serialization @ $GITHUB_SHA" '{title: $t, html: $html}')"

The revision list then shows the run-to-run spread, which is the honest way to decide whether a 6% drop is a regression or the runner.

What review adds

Anchored threads on the benchmark under debate — see commenting on HTML.
Revisions, so the noise floor is visible rather than assumed.
Access per report — private, team, domain-gated or named reviewers. See the sharing model.

Limits

Entry HTML: 5 MB. Assets: 25 MB per file, 250 MB and 500 files total. A large parameterized sweep is the JSON to watch.
60 requests/minute per token.

Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision history.

Publish a benchmark →