TL;DR: for text-to-CAD, the dominant token cost is not generation — it's context replay, and a 4-agent pipeline with structured state passing removes that term almost entirely. Pan-Chera/Multi-Agent-CAD (MIT, Tsinghua IEI Lab) runs the same 10 prompts and 141 geometry features as the single-agent cad skill baseline (earthtojake/text-to-cad) on the same LLM (Qwen 3.7-max): tokens 103,950,189 → 896,340 (116×), API calls 1,307 → 50 (26×), cost ¥125.69 → ¥9.67 (13×), feature pass rate 138/141 → 140/141 (97.9% → 99.3%). I have not tested or run it; every figure is from the authors' benchmark doc, and the arithmetic reproduces row by row.
The numbers check out. Their pricing: input ¥6/M, cache_read ¥0.6/M, output ¥18/M; cost = (input×6 + cache_read×0.6 + output×18)/1e6 CNY. Spot-check P1 baseline: 212,689×6 + 5,413,760×0.6 + 55,878×18 = ¥5.53 ✓; MAC P6: 157,815×6 + 119,799×18 = ¥3.10 ✓. I re-ran all 20 rows from docs/quantified_quality.md — every row, the totals, and the 116.0×/13.0×/26.1× ratios reproduce exactly.
The mechanism behind 116×. 96,192,896 of the baseline's 103,950,189 tokens — 92.5% — were cache_read: a single agent re-reads its own accumulated conversation on every repair round, so tokens grow superlinearly with iterations. MAC passes only compact structured JSON between stages (CADBrief → ArchitectPlan → code → QA report); cache_read falls 96.2M → 10,496, input 5.97M → 524k, output 1.79M → 362k. Recomposing the old bill: cache_read ¥57.72 (46%) is the single most expensive line. Hallucination propagation is also cut at stage boundaries — each agent starts from structured output, not the previous agent's narrative.
Deterministic translator and dual-engine QA. _plan_to_code (multi_agent_cad/nodes.py:893) translates ArchitectPlan JSON into build123d code at zero token cost for common operations; unsupported step types emit # TODO_AIDER placeholders filled by an Aider pass (nodes.py:460, 524–542). Verification is machine-checked: Engine A runs cadpy.analysis geometry selectors on the .step, Engine B checks the .stl triangulation; runtime guards record typed markers (MISSED_CUT, FILLET_FAILED, …); schemas.py routes errors by type — DIMENSION back to the Coder, TOPOLOGY back to the Architect, FATAL halts.
Read the caveats. 10 prompts, one run each, one LLM; the baseline was rerun on the weaker Qwen 3.7-max (the original cad-skill tests used Claude/ChatGPT and scored higher); "~10× faster" wall-clock is explicitly an order-of-magnitude estimate, not measured; you need your own model API key, and the Web UI executes generated .py server-side (trusted network only). Observable for evaluators: the baseline's 3 failures and MAC's single failure are all fillet operations.
Who should look. Engineers building multi-stage agent pipelines where context cost compounds with repair iterations — structured handoff, deterministic translator, and typed error routing transfer beyond CAD. Skip it if you expect turnkey output without running the pipeline or paying API costs.
Project: https://github.com/Pan-Chera/Multi-Agent-CAD · README: https://github.com/Pan-Chera/Multi-Agent-CAD/blob/main/README.md · Benchmark doc: https://github.com/Pan-Chera/Multi-Agent-CAD/blob/main/docs/quantified_quality.md · Baseline: https://github.com/earthtojake/text-to-cad
Top comments (0)