This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.
The bug
Sentry's JavaScript SDK auto-instruments the Vercel AI SDK, so every generateText or streamText call shows up as a gen_ai span with token usage attached. When the model is a Gemini reasoning model, that usage was wrong: the span undercounted the output and the total because it silently dropped Gemini's reasoning tokens.
Gemini reports its reasoning ("thoughts") tokens separately from the visible answer. The AI SDK's outputTokens covers only the candidate answer and exposes the reasoning count through providerMetadata.google.usageMetadata.thoughtsTokenCount. Sentry's getProviderMetadataAttributes() handled OpenAI, Anthropic, Bedrock and DeepSeek metadata, but it never looked at the Google or Vertex block, so the reasoning tokens vanished from the span and the total was computed as input plus candidate-only output.
On a real Gemini call this is not a rounding error. The model returned usageMetadata { promptTokenCount: 14, candidatesTokenCount: 1, thoughtsTokenCount: 100, totalTokenCount: 115 }. Sentry recorded output_tokens: 1 and total_tokens: 15, hiding 100 reasoning tokens. For a reasoning model the reasoning tokens are most of the cost, so the trace understated real usage by nearly 8x.
Bug Fix or Performance Improvement
The fix adds a Google and Vertex branch to getProviderMetadataAttributes() in packages/server-utils/src/ai/vercel-ai/index.ts. It reads providerMetadata.{google|vertex}.usageMetadata and, when thoughtsTokenCount is greater than zero, sets the output to candidatesTokenCount + thoughtsTokenCount, sets the total from the real totalTokenCount and records the reasoning breakdown under gen_ai.usage.reasoning.output_tokens.
Deriving the output from the raw candidate and thoughts counts, rather than adding reasoning onto the SDK's existing value, is deliberate: it stays correct even if a future AI SDK version folds reasoning into outputTokens itself, so it cannot double count. The branch is gated on thoughtsTokenCount > 0, so a non-reasoning Gemini response is left exactly as it was. Both the OpenTelemetry span path and the ai tracing-channel path go through this one helper, so both emit the corrected shape.
My Improvements
I added a test file with four cases. A Gemini reasoning response now records output_tokens: 101, total_tokens: 115 and gen_ai.usage.reasoning.output_tokens: 100. The vertex metadata variant is handled the same way. A non-reasoning response is unchanged. Run against the current source the three reasoning cases fail (expected 1 to be 101) and the non-reasoning case passes, which pins the bug. With the fix all four pass.
Green on the repo's own gates: the full @sentry/server-utils vitest suite (377 passing across 40 files, my four added), oxlint --type-aware, oxfmt --check and tsc on the source types all clean. I confirmed the fix against the AI SDK's own source: convert-google-usage.ts maps output to candidates plus thoughts and reasoning to thoughts, which is exactly the accounting the fix uses.
Best Use of Sentry
This is a fix to Sentry's own JavaScript SDK, in the Vercel AI instrumentation that feeds the AI monitoring product. Token usage on a gen_ai span is what teams read to track model cost and behavior. For a reasoning model the reasoning tokens are the bulk of the spend, so an output_tokens that omits them makes the trace understate cost in the direction that hurts most. After the fix the span carries the real output and total, so Sentry's usage numbers match what Gemini actually counted.
Best Use of Google AI
I verified this end to end with a real Google Gemini reasoning call, because the whole bug only appears when a model emits thoughtsTokenCount. A real gemini-3.6-flash request returned { promptTokenCount: 14, candidatesTokenCount: 1, thoughtsTokenCount: 100, totalTokenCount: 115 }. Fed through the actual Sentry Vercel AI processor, the emitted span attributes were output_tokens: 1, total_tokens: 15 before the fix and output_tokens: 101, total_tokens: 115, reasoning.output_tokens: 100 after. Google Gemini produced the reasoning tokens. Sentry is where they reappear once the SDK reads the Google usage metadata correctly.
PR: getsentry/sentry-javascript#23433, from branch fix/vercel-ai-gemini-reasoning-tokens. Found by probing the integration with real Gemini traffic, not from a filed issue.
AI disclosure
AI assistance (Claude, Anthropic) was used in developing this change. The design, review and verification were done by the author. Verified locally before submitting: the four new tests fail on the unpatched source and pass with the fix, the @sentry/server-utils suite (377 passing), oxlint --type-aware, oxfmt --check and tsc on the source types, plus a real Gemini reasoning run showing the span output and total tokens corrected.
Top comments (0)