Share a TruLens evaluation

You instrumented the RAG app, ran the feedback functions, and run_dashboard() opened a Streamlit leaderboard showing groundedness at 0.71 and context relevance at 0.58.

The retrieval problem is obvious from that pairing. It is also visible only to you, on localhost:8501.

localhost is not shareable

TruLens's dashboard is a local Streamlit app. The URL it prints works on your machine and nowhere else — see why a localhost link doesn't work for others.

You could deploy the dashboard, which means standing up and securing a Streamlit service holding your eval data, so that a PM can look at three numbers once a sprint. That is a lot of infrastructure for a reporting problem.

Export the leaderboard and the bad records

TruLens hands you dataframes:

from trulens.core import TruSession

session = TruSession()
board = session.get_leaderboard()                   # per-app aggregate scores
records, feedback = session.get_records_and_feedback()

worst = records.nsmallest(10, "Groundedness")
# render leaderboard + worst records to HTML, then publish

Structure the page in that order — aggregate first, failures second:

  1. The leaderboard — each app version against the RAG triad.
  2. What it implies — one sentence. Low context relevance with decent groundedness means retrieval, not generation.
  3. The ten worst records — question, retrieved context, generated answer, the three scores.

The triad is the diagnosis

Showing all three feedback functions together is what makes the page useful rather than decorative:

  • Context relevance low, groundedness high — the model is faithfully answering from the wrong documents. Fix retrieval.
  • Context relevance high, groundedness low — the right documents are arriving and the model is inventing anyway. Fix the prompt or the model.
  • Answer relevance low with both others high — you are answering a different question than the one asked.

A single "quality score" hides all of this. Three numbers per version, on one page, tells the reader what to do next.

Records are what convince people

An aggregate of 0.58 is an argument nobody can check. Ten records showing the retriever pulling the 2023 pricing page for a question about current pricing is an argument that ends the discussion.

And when someone disagrees — "that one's actually fine, the user's question was ambiguous" — the objection lands on that record, per commenting on HTML, instead of on the score.

In CI

Evaluate on every prompt or retriever change and publish, updating one report id per app. The link in the ticket then always shows current quality, and the comparison against last week is the comment history right there on the page.

Worth knowing

  • Include the eval set version. Scores across different eval sets are not comparable and will be compared anyway unless you say so.
  • Restrict the report — records contain real user questions and retrieved document text. See security.
  • 5 MB per report body. Ten records with context fits easily.

Try it

Free — unlimited reports, commenters and revisions.

Read the API reference →

Related