Share Vertex AI results

The evaluation finished. Vertex AI has the metrics, the confusion matrix, the feature attributions — everything needed to decide whether this model replaces the one in production.

To look at any of it, a person needs an IAM principal on the GCP project.

IAM is the wrong lever for this

Granting project access so someone can read a number has costs that outlast the decision:

  • Cloud IAM grants are rarely revoked once made.
  • Viewer on a project is considerably more than "can see one evaluation".
  • Your security team will ask, correctly, why a product manager has a role on the training project.
  • And the person still has to navigate the Vertex console to find the run, which they will not.

The fallback — screenshotting the metrics panel into Slack — loses every number's precision and any chance of a follow-up question landing usefully.

Publish the evaluation

The SDK has what the console shows:

from google.cloud import aiplatform

aiplatform.init(project=PROJECT, location=REGION)
model = aiplatform.Model(model_name=MODEL)
evals = model.list_model_evaluations()
metrics = dict(evals[0].to_dict()["metrics"])

Render a page that answers the decision:

  1. Recommendation — ship, hold, or retrain, in one sentence.
  2. Candidate vs production — the same metrics, side by side.
  3. Where it gets worse — the slice that regressed, stated plainly.
  4. Appendix — dataset version, pipeline run id, model resource name.

Point four is what keeps the console authoritative. Anyone who does have IAM can jump from the claim to the run; everyone else reads the claim.

Pipelines should publish their own summary

Add a final component to the Kubeflow pipeline that renders and publishes:

@component
def publish_summary(metrics: Input[Metrics], report_id: str):
    ...  # build HTML, PATCH the existing report

Now every scheduled retrain updates one URL. The stakeholder's bookmark shows the current model's evaluation, and the comment history from previous rounds is still attached — see publishing from CI for the same pattern in a build pipeline.

Restricted, not public

Model evaluations frequently reveal what the model was trained on, including slice names that are themselves sensitive. Publish to a restricted report — see security.

Worth knowing

  • Name the slice that regressed. A summary that only shows improvement gets discovered as incomplete at the worst moment.
  • Comments anchor to the metric, so "what's the threshold at this recall?" lands on the number — see commenting on HTML.
  • 5 MB per report body.

Try it

Free — unlimited reports, commenters and revisions.

Read the API reference →

Related