Share a Giskard scan report
Giskard's scan is one of the few LLM evaluation tools that produces something
already shaped like a document. scan_results.to_html("report.html") gives you
a real report: findings grouped by category, with examples.
Then you close the notebook and the report is a file in a working directory.
The audience is not in the notebook
A Giskard scan surfaces prompt injection, harmfulness, robustness and performance-bias findings. Every one of those is a risk acceptance decision, and risk acceptance is not made by the ML engineer who ran the scan.
It is made by some combination of product, legal, security and whoever signs off on the launch. None of them will:
- run a notebook,
- open an HTML attachment their laptop policy blocks, or
- read a summary that says "the scan found 3 issues, mostly minor".
Which is how model risk ends up accepted by default rather than on purpose.
Publish what the scan already produced
The nice thing is you do not have to build anything:
scan_results = giskard.scan(giskard_model, giskard_dataset)
html = scan_results.to_html()
httpx.post(
"https://commareports.com/api/v1/reports",
headers={"Authorization": f"Bearer {os.environ['COMMA_API_TOKEN']}"},
json={"title": f"Giskard scan — {model_name} {version}", "html": html},
)
One call turns the notebook artifact into a URL. The report keeps its structure, its examples and its severity grouping, and the recipient needs a browser.
Report HTML renders in a sandboxed iframe — allow-scripts without
allow-same-origin — so the scan report's interactivity works while having
access to nothing else.
Findings need threads, not a meeting
The value of publishing rather than presenting is that each finding gets its own thread. "This injection vector is acceptable — the tool has no write access" is a decision, and it belongs attached to the finding it is about, with a name and a date on it. See commenting on HTML.
Six months later, when someone asks why a known finding was shipped, the answer is on the page rather than in somebody's memory of a Thursday call.
Run it on every model change
- run: python scripts/scan.py --out scan.html
- run: python scripts/publish_scan.py scan.html
Update the same report id so the risk owner's bookmark shows the current state and the previous review's comments are still there. New findings since the last sign-off are then visible as new, which is the only question a re-reviewer actually has.
Worth knowing
- Restrict the report. A scan report is a catalogue of your model's weaknesses, including working injection strings — see security.
reports:writeis the scope the scan job needs — see scoped tokens for AI agents.- 5 MB per report body. Trim the number of examples per finding if a large scan exceeds it.
Try it
Free — unlimited reports, commenters and revisions.