Conversation
roy-tong
left a comment
There was a problem hiding this comment.
Reviewed — the semantics are right and the docstring states them well: Anthropic's input_tokens excludes cached buckets, so exposing cache_read_input_tokens / cache_creation_input_tokens as independent facets without touching the native totals is the correct shape, and is not None correctly keeps measured zeros distinct from absent. Approving with one cross-provider consistency point worth settling here rather than later:
TOTAL_TOKENS = input + output is a third grain, not a shared one. On the OpenAI path (and in the dual-emit from #1453), input_tokens already includes cached tokens, so total = input + output is the inclusive sum. On this path it is not: Anthropic-exclusive input plus output omits both cache buckets. A collector summing gen_ai.usage.total_tokens across a mixed OpenAI+Anthropic workload is aggregating two different grains — the same cross-SDK hazard #1447/#1453 exist to close, one attribute earlier than anyone looks. Either:
- make this path's total inclusive too (
input + cache_read + cache_creation + output, matching the OpenAI-side grain), or - keep it native and document in the docstring that
total_tokenshere excludes cache buckets and consumers must add their own policy.
I'd lean inclusive for consistency, but either is defensible if the docstring names the basis. Non-blocking either way since the previous code had the same total.
One forward note (already on record in my #1453 review): these flat cache attribute names are the same legacy generation that will eventually need dual-emitting alongside the current nested gen_ai.usage.cache_read.input_tokens form — worth keeping in the same transition tracking.
Summary
Why
Anthropic reports cache-read and cache-creation tokens as fields separate from input_tokens. The instrumentation previously dropped those fields, making cache activity invisible to telemetry consumers. This change exposes both fields independently while preserving the existing native input/output total, so consumers can choose their accounting policy without double-counting cache tokens.
Validation
Fixes #1445