Comma

Publish an HTML report from Azure Pipelines

Azure Pipelines runs the job, the job produces an HTML report, and then Azure DevOps offers you an artifact download. PublishPipelineArtifact@1 stores the folder; a reviewer opens the run, finds Artifacts, downloads, unzips, and opens index.html from ~/Downloads, where half the relative asset paths work.

The alternatives inside the product don't close the gap:

  • The run summary accepts markdown, not HTML. Useful for a table of numbers; useless for a rendered report.
  • PublishCodeCoverageResults gives you the Code Coverage tab, but only from Cobertura or JaCoCo XML, and only for coverage.
  • The marketplace extensions that add an HTML tab need an organization-level install. In a lot of enterprise tenants that's a ticket, an approval, and a wait.
  • Retention deletes the run. Pipeline artifacts live and die with the run, and retention policies clear older runs on a schedule. The link in the incident review goes dead.

One task, one URL

Store a scoped token (reports:write is enough) as a secret variable or in a variable group, create the Comma report once, and put its id in a pipeline variable:

- script: |
    curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$(commaReportId)" \
      -H "Authorization: Bearer $COMMA_API_TOKEN" \
      -H "Content-Type: application/json" \
      -d "$(jq -n --rawfile html coverage/index.html \
            --arg title "Coverage — $(Build.SourceVersion)" \
            '{title: $title, html: $html}')"
  displayName: Publish report to Comma
  condition: always()
  env:
    COMMA_API_TOKEN: $(COMMA_API_TOKEN)

Three details do the work:

  • PATCH, not POST. Every run appends a revision at the same URL. One bookmark per report, forever — and any two runs can be diffed.
  • condition: always(). The runs worth sharing are the red ones. Without it, the publish step is skipped exactly when it matters.
  • A commit-stamped title. Revisions titled by Build.SourceVersion make the history read like a log instead of a pile of timestamps.

$(COMMA_API_TOKEN) must be mapped through env: — Azure Pipelines does not expose secret variables to the script environment automatically, which is the single most common reason this step fails with a 401 on the first try.

Posting the link on the pull request

The report URL is stable, so the PR comment stays one line, permanently valid:

- script: |
    echo "##vso[task.setvariable variable=reportUrl]https://commareports.com/p/$(commaReportId)"
  displayName: Expose report URL

Use it in a PR comment step, a status check, or the run summary — every one of them now carries a link that opens a rendered report instead of starting a download.

What you get that an artifact can't give you

  • Comments anchored to the report. A reviewer highlights the module that lost 8% coverage and pins a thread to it; the thread survives the next twelve runs. See commenting on HTML.
  • Revision history with diffs. "What changed since the last green build?" is a diff, not a memory exercise. Artifacts have no notion of a previous version.
  • Readers without a seat. Azure DevOps artifacts need project access. A Comma report can be team-visible, domain-gated, or shared with a named reviewer who will never have a license — see the sharing model.
  • No retention clock. Revisions stay.
  • Announcements for free. A webhook on revision.created posts each run to Teams, Slack or Discord.

Limits worth knowing before you wire it up

  • HTML body: 5 MB. Screenshots, trace zips and JS bundles go in as assets — 25 MB per file, 250 MB per report.
  • Scripts run, sandboxed. Report HTML is stored verbatim and rendered in an iframe with sandbox="allow-scripts" and no allow-same-origin, so an interactive report keeps working. A report that loads sibling files at view time (Allure, JMeter's dashboard) needs those files uploaded as assets alongside the HTML.
  • Rate limits are per token, 60/minute by default. One publish per build is nowhere near it.
  • Use a self-hosted agent's network egress rules. If your agents run in a locked-down VNet, commareports.com needs to be allowed like any other HTTPS endpoint.

Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision history. Add the task to one pipeline and send a link instead of an artifact.

Create your first report →

Related