# Share a DeepEval Report — LLM Test Results as a Link

Canonical: https://commareports.com/share-deepeval-report
Published: 2026-09-15

> DeepEval runs as pytest, so pytest-html gives you a page. Publish it to Comma and the LLM test run becomes a URL the product owner can read and comment on.

# Share a DeepEval report

DeepEval makes LLM evaluation look like unit testing, which is a genuinely
good idea: `assert_test(test_case, [AnswerRelevancyMetric(threshold=0.7)])`
either passes or it doesn't, and it runs in the same pipeline as everything
else.

The catch is that a failure here is not like a failing unit test. It is a
judgment call rendered as a number, and the person qualified to review it is
usually not the person who can read pytest output.

## pytest-html on a DeepEval suite

```bash
pip install deepeval pytest-html

deepeval test run test_llm.py || true
# or, for the HTML artifact:
pytest test_llm.py --html=deepeval.html --self-contained-html || true
```

`--self-contained-html` inlines the CSS and JS so the result is one file. The
metric `reason` strings — DeepEval's explanation of why a score landed where
it did — come through in the captured output, which is the part that makes the
report readable to a non-engineer.

## Publish it

Drag `deepeval.html` into [the app](https://commareports.com/), or from CI:

```bash
curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html deepeval.html \
        --arg title "LLM eval — $GITHUB_REF_NAME" '{title: $title, html: $html}')"
```

Keep the hard regression cases as a separate unmuted gate step; publish the
full run with `if: always()` either way.

## Thresholds are opinions

`threshold=0.7` is a choice somebody made on a Tuesday. Whether it is right
depends on the product, and the argument about it should happen where the
failing cases are visible:

- **Anchored threads** on the failing case — "this one is fine, the metric is
  penalising the disclaimer we're required to include." See
  [commenting on HTML](/comment-on-html).
- **Revisions**, so raising a threshold shows exactly which cases it newly
  catches.
- **Access per report** — test cases often contain real prompts and customer
  phrasing; keep them private or team-scoped. See the
  [sharing model](/docs/sharing).

## Limits

- **Entry HTML: 5 MB.** Assets: 25 MB per file, 250 MB and 500 files total.
  Long generations in the captured output are what grows the file — truncate
  or sample for the shareable view.
- **60 requests/minute per token.**

## Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision
history.

**[Publish an eval report →](https://commareports.com/)**

### Related

- [Share a Ragas evaluation](/share-ragas-report) · [Share a promptfoo report](/share-promptfoo-report)
- [Share an Inspect AI log](/share-inspect-ai-log) · [Share a Giskard report](/share-giskard-report)
- [Share a pytest report](/share-pytest-report) · [Publish from CI](/docs/ci)