Skip to content

fix(anthropic): emit prompt cache usage attributes - #1450

Open
jstar0 wants to merge 1 commit into
AgentOps-AI:mainfrom
jstar0:fix/1445-anthropic-cache-usage
Open

jstar0 wants to merge 1 commit into
AgentOps-AI:mainfrom
jstar0:fix/1445-anthropic-cache-usage

Conversation

@jstar0

@jstar0 jstar0 commented Sep 9, 2026

Copy link
Copy Markdown

Summary

  • Extract Anthropic prompt-cache read and creation token counts from non-streaming messages and raw streaming metadata.
  • Keep the existing prompt, completion, and total attributes based on Anthropic native input/output values.
  • Cover synchronous and asynchronous stream final-message paths with fixture-based regressions.

Why

Anthropic reports cache-read and cache-creation tokens as fields separate from input_tokens. The instrumentation previously dropped those fields, making cache activity invisible to telemetry consumers. This change exposes both fields independently while preserving the existing native input/output total, so consumers can choose their accounting policy without double-counting cache tokens.

Validation

  • uv run pytest -q tests/unit/instrumentation/anthropic/test_attributes.py tests/unit/instrumentation/anthropic/test_stream_wrapper.py — 22 passed
  • uv run ruff check on the four changed files — passed
  • uv run ruff format --check on the four changed files — passed

Fixes #1445

@roy-tong roy-tong left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — the semantics are right and the docstring states them well: Anthropic's input_tokens excludes cached buckets, so exposing cache_read_input_tokens / cache_creation_input_tokens as independent facets without touching the native totals is the correct shape, and is not None correctly keeps measured zeros distinct from absent. Approving with one cross-provider consistency point worth settling here rather than later:

TOTAL_TOKENS = input + output is a third grain, not a shared one. On the OpenAI path (and in the dual-emit from #1453), input_tokens already includes cached tokens, so total = input + output is the inclusive sum. On this path it is not: Anthropic-exclusive input plus output omits both cache buckets. A collector summing gen_ai.usage.total_tokens across a mixed OpenAI+Anthropic workload is aggregating two different grains — the same cross-SDK hazard #1447/#1453 exist to close, one attribute earlier than anyone looks. Either:

  • make this path's total inclusive too (input + cache_read + cache_creation + output, matching the OpenAI-side grain), or
  • keep it native and document in the docstring that total_tokens here excludes cache buckets and consumers must add their own policy.

I'd lean inclusive for consistency, but either is defensible if the docstring names the basis. Non-blocking either way since the previous code had the same total.

One forward note (already on record in my #1453 review): these flat cache attribute names are the same legacy generation that will eventually need dual-emitting alongside the current nested gen_ai.usage.cache_read.input_tokens form — worth keeping in the same transition tracking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic prompt caching: cache tokens not emitted on the Anthropic paths — is session total_tokens systematically low?

2 participants