Token dashboards tell you the bill. They do not tell you why the bill is that size. When a coding agent gets slow, expensive, and a little dumb, it is usually because its context window has quietly filled with junk: the same file read six times, a 12k-token tool result that mattered for exactly one turn, tool schemas re-sent on every single step. You can see the total go up. You cannot see where the tokens went, so you cannot delete anything with confidence.
I wanted a profiler for that. Not a chat UI, not a live proxy, just a tool I could point at a session transcript and ask: what is in this context window, and how much of it is avoidable? That is ctxlens.
The core idea
ctxlens treats an agent session the way a CPU profiler treats a program. A profiler does not judge whether your code is good; it tells you where the time went so you know where to look. ctxlens does the same for tokens. It parses a session transcript, attributes every message to a segment, counts tokens per segment and per turn, and then runs rule-based checks to flag the parts that are genuinely wasted.
Every message lands in one of these buckets: system, tool_definitions, user, assistant, thinking, tool_call, tool_result. Once every token has a home, the interesting questions become answerable. Which segment dominates? When did the context spike? What is being paid for on every turn versus once?
It reads Claude Code JSONL sessions (the ones under ~/.claude/projects/*/*.jsonl), OpenAI/Codex rollout sessions, and generic OpenAI chat arrays. The format is auto-detected by sniffing the file, and you can force it with --format if you need to.
How it works
The basic run is one command:
pip install ctxlens-cli
ctxlens analyze session.jsonl
You get a summary panel, a breakdown of context composition by segment, a couple of sparklines for how context grew over the run, and a list of recommendations. The composition view is the part I reach for first, because it immediately answers "what is this window made of":
Context composition by segment
Segment Tokens % Msgs Share
tool result 6,204 49.7 22 ██████████████·······
assistant 2,110 16.9 14 ██████···············
system 1,540 12.3 1 ████·················
Tool results eating half the window is the single most common thing I see. Which leads to the second half of the tool: the waste report.
waste_ratio = total_waste / total_tokens, and total waste is the sum of four disjoint sources:
-
Duplicate tokens. The same file or tool result appearing more than once, matched either by reference (for example
Read:file_path=config.py) or by exact body. Every copy after the first is counted as wasted. -
Tool-result bloat. Tokens in a tool result above a per-result cap (
--tool-result-cap, default 400). Only the overage counts, and each unique body is charged once so a repeated giant result is not double-counted here and again as a duplicate. - Stale tool outputs. When the same reference is read more than once and a later read supersedes an earlier one, the older superseded copies are dead weight still sitting in context.
-
Tool-definition overage. Tool schema tokens above a budget (
--tool-def-budget, default 800). This one stings because you pay it on every turn.
Each finding carries a severity and an estimated token saving, so recommendations read like "'Read:file_path=config.py' appears 6 times, ~2,410 tokens" rather than generic advice to "manage your context better." The estimate is exactly the arithmetic above, not a guess.
Because it is all deterministic, it slots into CI. You can fail a build when a captured session wastes too much:
ctxlens analyze session.jsonl --fail-over-ratio 0.30
Exit code 0 is fine, 2 means the threshold was exceeded, 1 is an error. Add --json for machine-readable output, or diff a baseline against a candidate with ctxlens diff before.jsonl after.jsonl to catch regressions when you change a prompt or a tool. There is also an HTML reporter via ctxlens report session.jsonl --html -o report.html for when you want to actually look at it.
On counting: by default ctxlens uses a deterministic heuristic tokenizer with no network calls and no heavy dependencies, which is deliberate. For relative profiling and CI thresholds you mostly care about proportions and trends, and a stable heuristic gives you reproducible numbers everywhere. If you install tiktoken, --tokenizer auto picks it up and you get exact BPE counts. The whole thing has 55 tests covering the parsers, analysis, tokenizers, reporters, and CLI.
One honest limitation
The heuristic tokenizer is an approximation, and it should be treated as one. Its token counts will not match your provider's billing exactly, so the absolute numbers in the summary panel are estimates unless you install tiktoken. What stays reliable without tiktoken is the shape of the picture: which segment dominates, which references repeat, where the spikes are. If you need the reported token figures to line up with an actual invoice, install the extra and use exact counts. I would rather ship a tool that is honest about being a fast approximation by default than one that implies billing-grade precision it does not have.
Closing
ctxlens started because I was tired of guessing which part of a bloated agent session was safe to trim. Having the window broken down by segment, with the duplicates and stale reads called out by name and token count, turned that from a hunch into an edit. If you run agents and your context windows feel heavier than they should, point it at a real session and see what falls out.
Top comments (5)
The CPU-profiler analogy is exactly right — you can't compact what you can't attribute. The segment that consistently surprises people when we run this kind of breakdown is
tool_result: it's usually the biggest bucket and the most avoidable, because 90% of a large result matters for one turn and then just rides along inflating every subsequent request. Two things I'd love to see ctxlens flag beyond raw duplicates: (1) result staleness — a 12k-token file dump that was last referenced 15 turns ago is effectively dead weight and a great compaction candidate; and (2) schema churn — tool definitions re-sent every turn are a fixed tax that's invisible unless you separate "paid once" from "paid every turn," which your bucketing already sets up nicely. Have you thought about emitting a suggested compaction plan (drop these segments, summarize those) rather than just the waste ratio? The gap between "here's the waste" and "here's the safe edit" is where this becomes something you can wire into an agent loop instead of running by hand.The profile I always want is not only token count by source, but token count by usefulness after the run. Tool schemas, old chat history, retrieved docs, and error logs all compete for the same window. If a chunk never affected a decision, it should be a candidate for staged loading instead of permanent context.
Charging tool-definition overage on every turn is the right accounting for attention, and it is worth saying explicitly that billing accounting disagrees: with prompt caching, re-sent schemas sit in the stable prefix, so the invoice barely notices them while the model still attends over them on every step. That gap is an argument for the tool, not against it, since waste_ratio then measures the thing dashboards structurally cannot. The phrase that sold me was "a 12k-token tool result that mattered for exactly one turn", because that is exactly the shape of waste a cost report never surfaces. Every one of those tokens was legitimately billed. Framing waste_ratio as a quality metric first and a cost metric second in the README would defuse the "caching already solved this" objection before it gets raised.
Context profiling is valuable because it turns a vague 'the model forgot' complaint into a budget problem. I would separate durable facts, transient task state, and raw transcript tokens. Those three age differently and should not compete in the same bucket.
This is the kind of measurement more agent users need before they rewrite prompts. In long sessions I keep finding the expensive part is not the clever reasoning. It is the repeated command output, whole file dumps, and leftover tool noise that never gets evicted. Once you can see which tokens are waste, the fix is usually ruthless context hygiene, not a smarter model. Curious what your biggest single bucket of waste was once you broke the transcripts down.