Skip to content

Say Blocked about a refusal the audit page had been calling Allowed - #302

Merged
davidmckayv merged 3 commits into
CopilotKit:mainfrom
kevin9327:fix/audit-refusals-not-allowed
Aug 31, 2026
Merged

Say Blocked about a refusal the audit page had been calling Allowed#302
davidmckayv merged 3 commits into
CopilotKit:mainfrom
kevin9327:fix/audit-refusals-not-allowed

Conversation

@kevin9327

Copy link
Copy Markdown
Contributor

What this changes

The audit page asks "what kind of row is this" twice, from two hand-written lists:

  • app/src/routes/_authed/admin/audit.tsx:146-158 and :168 — the refused / failed predicate,
    behind the label at :276-278 and the colour at :269-274.
  • app/src/routes/_authed/admin/audit.tsx:36-59 — the eventType query behind the Blocked and
    Did not happen saved views.

Anything on neither list falls through to Allowed. Six event types are on neither, and all six
are written by reachable code today:

event type written at drawn as in Blocked
agent.handoff_refused server/src/agents/handoff.ts:108 Allowed no
agent.dial_refused server/src/index.ts:585 Allowed no
credential.rotation_refused server/src/credentials.ts:567 Allowed no
session.refused server/src/auth/index.ts:227, :269 Allowed no
agent.handoff_failed server/src/agents/handoff-runner.ts:403 Allowed (should be Did not happen)
agent.escalation_failed server/src/agents/escalation.ts:123 Allowed (should be Did not happen)

The comment beside the predicate already states the rule this breaks:

the fallback below calls anything it does not recognise "Allowed", which for a refusal is the one
wrong answer. A trail that is confidently wrong is worse than a silent one.

And server/src/audit.ts:341-347 says of the first row in that table:

BOTH OUTCOMES, and the refused one is the more important of the pair. A hop that happened is
visible in the transcript anyway; a hop that was refused is invisible everywhere else.

It was invisible on the one screen built to show it, and affirmatively labelled Allowed. Same shape
for session.refused, which audit.ts:306-308 calls "the one somebody investigating actually
reaches for"
, and credential.rotation_refused, of which audit.ts:41-46 says "the refusals are
the interesting ones"
.

routines.dispatch_refused — the most recently added refusal — was wired into both places, which
is the pattern this follows and the reason I read the omission as an oversight rather than a choice.

The fix

The two lists become one module, app/src/lib/audit/outcome.ts, and the page derives both the row
and the saved view from it. That is the part worth more than the six additions: a refusal on one list
and not the other is either a row drawn as Allowed or a view that is not empty, just short — and the
second is the harder one to notice.

Deliberately unchanged: mcp.call_failed ("The server did not answer"), component.function_failed
("Could not be read") and bot.declined ("The Bot declined") already carry accurate labels through
DECISIONS, so only their colour is muted; that is a judgement about emphasis rather than a wrong
sentence, and I left it to you. The fallback also stays open, so a row type a build has never heard
of is still not called a refusal.

Where it runs

  • New state that outlives a request? None. One pure module with two frozen lists and a
    lookup; no component state, no cache, no store.
  • What happens on the second replica? The same. This is browser code deciding how to draw a
    row it was handed; the classification is a constant compiled into the bundle.
  • Anything serialised? Nothing. The saved views are still a ?eventType= query string
    against the existing read endpoint, now generated from the list instead of retyped.
  • Anything fanned out to a browser? No.
  • New listener, port, or schedule? None.

Boundary and audit

  • Every acting call still goes through the gateway. Nothing on the acting path is touched — this
    is the read surface for rows the server already writes.
  • New refusals and new failures each write a row. No new events; six existing ones are now read
    correctly.
  • Nothing new is trusted from the client.

Changelog

  • CHANGELOG.md, under Unreleased.

Proof

Before. main's predicate and Blocked list, lifted verbatim and run over the six types:

agent.handoff_refused          -> Allowed         in Blocked view: false
agent.dial_refused             -> Allowed         in Blocked view: false
credential.rotation_refused    -> Allowed         in Blocked view: false
session.refused                -> Allowed         in Blocked view: false
agent.handoff_failed           -> Allowed         in Blocked view: false
agent.escalation_failed        -> Allowed         in Blocked view: false

After. app/tests/audit-outcome.test.ts, following the silenceOf / audit-silence.test.ts
pattern — the pure part extracted and tested, no renderHook. It pins each of the six, keeps the
six that were already right, keeps the ordinary rows "allowed", holds the fallback open for an
unknown type, and asserts that each saved view filters by exactly the family it draws (which is the
drift the module exists to stop).

bun run format:check   -> Checked 489 files. No fixes applied.
bun run lint           -> Checked 492 files. No fixes applied.
bun run typecheck      -> app / server / worker all exit 0
bun test app/tests     -> 187 pass, 0 fail (30 files)

The audit page decides what a row was twice: once for the label and the colour, and once for the
`eventType` list behind the Blocked and Did not happen saved views. Both were hand-written lists, and
the fallback for anything neither list recognises is "Allowed".

Six event types were on neither. A hop one Bot was not allowed to make, an endpoint this deployment
refused to dial, a rotation the vault turned down and a sign-in it turned away were each drawn as
Allowed, in the muted colour every ordinary row uses, on the screen an administrator opens to find
out what this deployment refused. Clicking Blocked showed none of them. `agent.handoff_failed` and
`agent.escalation_failed` read as Allowed too, where they are "Did not happen": nothing was refused,
and nothing came of it either.

The comment beside the predicate already says why this matters — "the fallback below calls anything
it does not recognise Allowed, which for a refusal is the one wrong answer" — and the trail's own
notes say `agent.handoff_refused` is the more important of its pair, because a hop that happened is
visible in the transcript and one that was refused is invisible everywhere else. It was invisible
here, and worse than invisible: affirmatively wrong.

Two lists in one module now, and the page derives both the row and the saved view from them, so the
next refusal is added in one place or in none. Nothing else changes: the types already recognised
keep their labels, `mcp.call_failed`, `component.function_failed` and `bot.declined` keep the
accurate labels they already had, and the fallback stays open so a row type this build has never
heard of is still not called a refusal.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

davidmckayv
davidmckayv previously approved these changes Aug 31, 2026

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified + CI verify green; template-appropriate correctness fix.

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI verify green pre-rebase; CHANGELOG-only rebase; template-appropriate.

@davidmckayv
davidmckayv merged commit 91d8ae8 into CopilotKit:main Aug 31, 2026
davidmckayv added a commit that referenced this pull request Aug 31, 2026
…one that landed (#306)

The Audit screen's Bot column reads `payload.bot` and nothing else, and renders a dash without it.
`agent.handoff_offered` says so in its own comment, added when the accepted pair was fixed:

    without this the two handoff rows are the only Bot actions on a screen headed "Every action a
    Bot took" that name no Bot.

The pair got the key. The four rows either side of it did not: `agent.handoff_refused`, both
`agent.handoff_retried` rows, and `agent.handoff_failed` carry `from` and no `bot`, so every one of
them draws a dash.

Those are the rows the trail exists for. `server/src/audit.ts` says it outright of the first one —
"a hop that happened is visible in the transcript anyway; a hop that was refused is invisible
everywhere else" — and the same holds for one that ran out of attempts, which is where a person's
unanswered question ends. So the screen named the Bot on the outcome you could already see, and drew
a dash on the three you could not.

`agent.escalated` and `agent.escalation_failed` one file over share a single payload and get this
right on both outcomes, which is the shape being restored here.

Pairs with #302, which stops that same screen calling `agent.handoff_refused` "Allowed". Together a
refused hop reads as a refusal and says whose it was; either alone leaves half the row wrong.

Co-authored-by: kevin9327 <kevin9327@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: David McKay <davidmckayv@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants