Share a Langfuse eval result
You self-host Langfuse for a good reason: the traces contain prompts, customer messages and retrieved documents, and you would rather they never left your network. That decision is correct and it has a consequence — the person whose judgement you need is on the other side of the VPN.
The people who should be reading eval results are rarely the people with a cluster login. The support lead who knows which answers are actually wrong. The clinician, the lawyer, the domain expert you are paying for exactly this. The exec who asked whether the model is good enough to ship. None of them are getting a Langfuse project role, and none of them should need one.
So the results get exported to a spreadsheet, the spreadsheet gets emailed, and the feedback comes back as row numbers in a reply-all.
Publish the scored table, not the traces
Pull the dataset run out with the SDK, keep the columns a reviewer needs, and publish:
import os
import pandas as pd
import requests
from langfuse import Langfuse
lf = Langfuse() # LANGFUSE_HOST points at your own install
run = lf.get_dataset_run(dataset_name="qa-golden-set", dataset_run_name="prompt-v7")
df = pd.DataFrame(
[
{
"question": item.input["question"],
"answer": item.output["answer"],
"expected": item.expected_output["answer"],
"score": next((s.value for s in item.scores if s.name == "correctness"), None),
}
for item in run.dataset_run_items
]
)
html = f"""<!doctype html><meta charset="utf-8">
<title>prompt-v7 — golden set</title>
<style>body{{font:15px/1.6 system-ui;margin:2rem;max-width:70rem}}
table{{border-collapse:collapse;width:100%}}
td,th{{border:1px solid #ddd;padding:.5rem;vertical-align:top}}</style>
<h1>prompt-v7 — golden set</h1>
<p>Mean correctness: {df["score"].mean():.2f} over {len(df)} examples.</p>
{df.to_html(index=False, escape=True)}"""
requests.patch(
f"https://commareports.com/api/v1/reports/{os.environ['COMMA_REPORT_ID']}",
headers={"Authorization": f"Bearer {os.environ['COMMA_API_TOKEN']}"},
json={"title": "Golden set — prompt-v7", "html": html},
timeout=30,
).raise_for_status()
Note what did not leave: the system prompt, the retrieved chunks, the spans, the latency traces, every other dataset. The report is the four columns you chose. Langfuse keeps everything else, on your cluster, where you put it.
What the published copy adds
- Reviewers outside the network. Visibility is private, team, domain-gated, or link, set per report — no VPN, no project role, no seat.
- Judgement anchored to the example. A reviewer highlights the generation and pins "this is confidently wrong about the refund window" to it. Those threads read back through the API or MCP server, so the labels can feed your next dataset — see commenting on HTML.
- One URL as the eval re-runs. Revisions accumulate at the same address and any two can be diffed, which is how a regression gets dated.
- A deliberate boundary. Every field in the report is one you wrote into it — which is a stronger privacy story than "we gave them read access and hoped they only looked at the one project".
Worth knowing
- HTML body: 5 MB. For a large eval, publish a sampled or aggregated view and attach the full export as an asset — 25 MB per file.
- Scope the token to
reports:write. The eval job needs to publish, not to read your workspace back — see scoped tokens for AI agents. - Nothing re-executes. A published report is a snapshot; a routine is what refreshes one nightly.
Try it
Comma is free — unlimited reports, unlimited commenters, unlimited revision history.