Comma

How to give an AI agent feedback

The hardest part of running an AI agent in production isn't the prompt. It's the feedback loop. The agent runs, produces something useful but imperfect, and a teammate notices the imperfection. What happens next is where most agent systems break down.

Where feedback usually goes

Pick the failure mode you've personally seen:

  • A Slack reply. "@amir this row in the eval looks wrong." The agent doesn't read Slack. The reply is feedback for the human; the agent doesn't even know it happened.
  • A DM to the agent's owner. "Hey, the report you set up has been off since Tuesday." The owner makes a mental note. The mental note expires.
  • A reply to the emailed report. Nobody reads inbox replies.
  • A spreadsheet of corrections. Someone keeps a list of "things the agent gets wrong" in a Google Sheet. The list grows. The agent never sees it.
  • A retry without specifics. "Try again, but better." The agent tries again, similarly.

In all of these, the agent produced output, a human had a specific critique of a specific part of the output, and the critique never made it back to a place where the agent could act on it. The loop doesn't close.

What anchored feedback looks like

A different shape: feedback is anchored — pinned to the specific content it's about — and lives on the artifact the agent produced.

When a human reviewer reads the agent's HTML report and notices a problem, they select the offending text or table cell and leave an anchored comment. The comment exists alongside the artifact, not in a separate channel.

Two structural properties matter:

  1. The reviewer's feedback is now bound to the content. Not "the third paragraph" — this exact sentence the reviewer highlighted.
  2. The feedback is on the artifact itself, not on a screenshot or a reply in a channel. When the report refreshes next week, the feedback travels with it.

This is the model Google Docs uses for human collaboration. Comma applies it to AI agent collaboration.

How the agent reads the feedback

Comma's MCP server exposes the comment primitives as typed tools. From Claude Code, Cursor, or any MCP-capable client:

list_comments(report_id: "rpt_…")

Returns each thread on the report, including:

  • The anchored text the comment is attached to.
  • The comment body — what the reviewer actually said.
  • The author — who left it.
  • The status — open or resolved.
  • Any replies in the thread.

The agent now has structured access to feedback on its own output. It can read the anchored text and the comment side by side, decide what to change, and either:

  • Reply to the thread with reply_to_comment — explaining what changed or asking for clarification.
  • Update the report with update_report — shipping a new revision that addresses the feedback.
  • Mark the thread resolved with set_comment_status.

A worked example

Friday: a routine fires and posts the new weekly analytics digest to a Comma report. PM lead opens it Monday morning, scrolls to the funnel table, sees that the "Q3 conversion" row looks off, selects that row, and pins a comment:

Anchored to: "Q3 conversion: 8.2% (-2.1pp WoW)"

"This looks like it includes the pricing test cohort even though the test was rolled back on Thursday. Can the digest filter that out?"

Tuesday: the agent's next routine run begins. Before generating the new HTML, the skill does:

comments = list_comments(report_id="rpt_weekly_analytics")

It sees the open thread, including the anchored text. The skill incorporates the request: filter the cohort. Generates the new HTML. Posts it as the next revision.

The skill also does:

reply_to_comment(
  thread_id="thr_…",
  body="Filtered the pricing-test cohort. Q3 conversion is now 9.1% in this revision; thread closed.",
)
set_comment_status(thread_id="thr_…", status="resolved")

Monday's reviewer opens the link Wednesday morning, sees the thread is resolved, sees the new number, sees the reply explaining what changed. The loop closed.

This is not a hypothetical — it's the shape of every well-run agent report loop. The infrastructure to support it is the part most teams have to build themselves.

What "good" feedback looks like

Reviewers leaving anchored comments tend to follow a few patterns:

  • Specific over general. "The Q3 conversion row looks wrong" is better than "the report looks off."
  • Anchored to data, not opinion. Pin the comment on the row, not on the executive summary.
  • Actionable when possible. "Filter the pricing-test cohort" beats "this isn't right."
  • Resolved when addressed. Closed threads keep the surface clean.

The agent benefits from all four. So do the humans. The comment discipline that makes Comma useful for human-only teams is the same discipline that makes it useful for AI-human teams.

What's not the right shape

To be explicit about what doesn't close the loop:

  • A Slack reply to a screenshot. No anchor, no read path back.
  • A "try again" prompt. No specifics, no diff target.
  • A fine-tune dataset built quarterly from spreadsheet entries. Too slow, too disconnected from the artifact.
  • A "comment" that lives in the agent's own dashboard. The whole point is that the comment lives on the artifact the team already opens, not on a separate UI.

The default is broken. Anchored comments on the artifact, exposed through MCP, is the working alternative.

Try it

The fastest end-to-end test:

  1. Have an agent (Claude Code, Cursor, a custom script) publish an HTML report to Comma.
  2. Open the report in the web UI.
  3. Select a paragraph or table cell and leave an anchored comment.
  4. Ask the agent (on its next turn) to read the comments and respond.

Free tier supports up to three commenters per report and full MCP access. No card.

Install the plugin → or create your first report →

Related