Comma

Share an Evidently drift report

Model monitoring has a distribution problem shaped exactly like the notebook problem, because it usually is a notebook. Evidently produces a genuinely good self-contained HTML report — drift per feature, distribution plots, test results — and then it lives in the ML engineer's reports/ directory, where the people who could explain the drift never see it.

And drift is almost always explained by someone outside the model. The upstream vendor changed units. Marketing ran a campaign in a new region. A form field became optional. None of that is visible in the data; all of it is one sentence from someone who isn't running the notebook.

Publish it

import os, requests
from evidently.report import Report
from evidently.metric_preset import DataDriftPreset

report = Report(metrics=[DataDriftPreset()])
report.run(reference_data=ref, current_data=cur)
report.save_html("drift.html")

requests.patch(
    f"https://commareports.com/api/v1/reports/{os.environ['COMMA_REPORT_ID']}",
    headers={"Authorization": f"Bearer {os.environ['COMMA_API_TOKEN']}"},
    json={"title": "Churn model — drift", "html": open("drift.html").read()},
    timeout=30,
)

One report id per model, PATCHed on every run: one URL, a revision per run, and diffs between any two. Use a scoped token with reports:write.

What the URL adds

  • Explanations attached to the feature. A thread pinned to the drifted column, still there next month — see commenting on HTML.
  • A trend, not a snapshot. Revisions accumulate at one address, which is what makes "is this new?" answerable.
  • Readers without the repo. Data owners, the vendor's contact, the risk reviewer. Sharing is per report.
  • Unattended runs. A routine refreshes it on a schedule; a webhook on revision.created announces each run in Slack.

Limits

  • HTML body: 5 MB. Wide reports over many features approach it — split by feature group into separate reports, which also gives each group its own audience and history.
  • Scripts run, sandboxed: allow-scripts, no allow-same-origin.

Try it

Comma is free — unlimited reports, unlimited commenters, unlimited revision history.

Create your first report →

Related