How do I turn a Ragas result into a shareable report?

Call `.to_pandas()` on the `EvaluationResult`, then `.to_html()` on the DataFrame, and publish that file to Comma. You get a single URL with the per-sample scores, and comments anchored to the rows that failed.

Does Ragas have its own dashboard?

Ragas app is hosted and account-gated, and `evaluate()` itself returns an object in your process. The HTML export is the artifact you can hand to someone who does not have the notebook or the account.

Can I compare two retrieval configurations?

PATCH one report id per eval suite. Each run appends a revision, so the URL shows the current scores and keeps every earlier run — which is the before/after you actually wanted when you changed the chunk size.

Which metrics should be on the page?

Whatever you gate on. Faithfulness and context precision are the two that move when retrieval changes; answer relevancy moves when the prompt changes. Put the per-sample column next to the aggregate so a reviewer can see which sample dragged the mean.

Is a failing eval worth publishing?

It is the one worth publishing. Run the publish step unconditionally so a run that dropped below threshold still produces a link someone can read.

Share a Ragas evaluation report

evaluate() returns an object. You print it, get four means, and the four means are the least interesting thing about the run — the question is always which sample scored 0.3 on faithfulness and why.

That answer lives in a DataFrame, in your process, and dies when the kernel restarts.

Export the per-sample table

from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_precision

result = evaluate(dataset, metrics=[faithfulness, answer_relevancy, context_precision])
df = result.to_pandas()

df.to_html("ragas.html", index=False, float_format="%.3f")

Drag ragas.html into the app, or publish it from the same job that ran the eval:

curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
  -H "Authorization: Bearer $COMMA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile html ragas.html \
        --arg title "RAG eval — $GIT_SHA" '{title: $title, html: $html}')"

Sort the frame by the metric you gate on before exporting. The reviewer's attention goes to the top of the table whether you earned it or not.

Why review beats a score

A faithfulness score of 0.62 is not a bug report. It becomes one when someone reads the retrieved context next to the generated answer and says the chunker split the table header off the rows. So put the disagreement on the row:

Anchored threads — the objection sits on the sample, not in a thread in Slack that loses which row it meant. See commenting on HTML.
Revisions — one report id per suite. Change the retriever, PATCH again, and the URL carries both runs.
Access per report — eval sets are usually built from real customer questions. Keep the report private, team-only, or domain-gated; see the sharing model.

Close the loop with the agent

If an agent is the thing editing your retrieval config, it can read the comments back through the MCP server and act on them — see let an agent respond to comments. A routine can re-run the suite nightly so the link is never showing last week's index.

Limits

Entry HTML: 5 MB. A large eval with full retrieved contexts per row gets there fast — truncate the context column, or split by metric.
Assets: 25 MB per file, 250 MB and 500 files per report.
60 requests/minute per token.

Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision history.

Publish a Ragas report →