Share a Braintrust eval report
Braintrust is good at the part that is genuinely hard: scoring runs, diffing experiments, keeping the trace. What it cannot do is get a domain expert who does not work here to look at eleven specific rows.
That is a sharing problem, and it is solved by a link rather than by another invite.
Export the rows you need reviewed
import braintrust, pandas as pd
experiment = braintrust.init(project="support-answers", open=True, experiment="run-1042")
rows = [
{
"input": r["input"]["question"],
"output": r["output"],
"expected": r.get("expected"),
**{k: v for k, v in (r.get("scores") or {}).items()},
}
for r in experiment.fetch()
]
df = pd.DataFrame(rows).sort_values("factuality")
df.to_html("braintrust.html", index=False, float_format="%.3f", escape=True)
Sort ascending on the score you care about so the worst cases are at the top of the page. Then publish:
curl -fsS -X PATCH "https://commareports.com/api/v1/reports/$REPORT_ID" \
-H "Authorization: Bearer $COMMA_API_TOKEN" \
-H "Content-Type: application/json" \
-d "$(jq -n --rawfile html braintrust.html \
--arg title "Support answers — run 1042" '{title: $title, html: $html}')"
Why the review layer is the missing half
Scores tell you something moved. They do not tell you whether the reference answer was right in the first place — and the person who knows that is rarely the person with the eval tooling open.
- Anchored threads — "expected is wrong, the model is right" sits on the row. See commenting on HTML.
- Revisions — one report id per experiment family; each publish appends a run rather than replacing it.
- Access per report — private, team-only, or domain-gated. Eval sets carry customer text; see the sharing model.
Close the loop with the agent
The corrections are structured feedback, and an agent can read them back through the MCP server to fix the eval set — give an agent feedback covers the shape of it.
Limits
- Entry HTML: 5 MB. Full model outputs per row add up; truncate the output column or split the export by scorer.
- Assets: 25 MB per file, 250 MB and 500 files per report.
- 60 requests/minute per token.
Try it
Comma is free — unlimited reports, unlimited commenters, unlimited revision history.