📝 Originally published (in Japanese) at forge.workstyle.tech.
You've got a code that looks correct when read, but when you open it in the browser, it's slightly different from the mockup - this "visual discrepancy" is the most troublesome part of UI development. A slight CSS specification, nesting of elements, and flex wrapping. Discrepancies that cannot be noticed by statically reading the code together will only appear when actually rendered.
Until now, it was necessary for a human to open the screen in a browser, compare it with the mockup image, and verbally communicate the differences to the AI. This workflow replaces the process of "humans visually seeing and verbalizing" by showing the screen to the AI agent itself via the browser. By combining Claude Code and browser automation extensions (Chrome extensions), we will "see" the screen actually rendered on localhost, compare it with the mockup, identify layout discrepancies, and fix them.
Why is it necessary to "show the actual screen"?
There are limitations to just handing over the code for UI review. It's difficult for both humans and AI to completely reproduce the final rendering result in their minds from the code. In particular, these discrepancies are difficult to detect just by looking at the code.
- Layout skeleton discrepancies - One area is crushed when it's supposed to be a 2-column layout, or the vertical split ratio is different from the mockup, resulting in structural-level discrepancies
- Element placement errors - A preview that should be in the upper right column is wrapped around to the bottom
- Unexpected wrapping and overflow - The component wraps due to insufficient width, changing the impression from the mockup
These discrepancies cannot be determined without seeing the "rendering result" as a fact. That's why we show the actual screen to the AI.
Workflow: Show, Compare, and Fix
1. Provide the mockup as a baseline
First, provide the target mockup image to the AI and share the baseline that "this is the correct appearance". It's particularly helpful to verbalize the layout skeleton to make the comparison more accurate later. For example, describe the structure as "The whole is a 2-column layout. The left column is the adjustment UI. The right column is divided into upper and lower parts, with a radar chart and preview at the top, and voice samples and advanced adjustments at the bottom".
2. Launch the actual screen locally and "show" it to the AI
Start the development server on localhost and have Claude Code open that screen through the browser automation extension. The role of the extension here is to deliver the actually rendered DOM and screenshot to the AI's eyes. The AI no longer imagines the code but judges based on the rendered actual object.
If necessary, operate the screen to a specific state (after recording audio, after moving a slider, etc.) before taking a screenshot. This is because dynamically changing UIs cannot capture all discrepancies just by looking at the static initial screen.
3. Compare the mockup with the actual screen to identify discrepancies
With the baseline (mockup) and the actual object (screenshot) in hand, have the AI list the differences. What works here is that the skeleton was verbalized in step 1. Have the AI specifically point out structural-level discrepancies, such as "The preview that should be in the upper right column is at the bottom" or "The right column is not divided into upper and lower parts and is in one column".
The key is to break down the abstract "something is different" into specifics, such as "which area, which element, where it is in the mockup, and where it is in the actual screen". This becomes the correction instruction as is.
4. Fix and show again
Based on the feedback, have Claude Code fix the code and open it in the browser again for comparison. Check if the fix is actually reflected in the rendering result and if it hasn't created new discrepancies. In reality, layout reconstructions such as 2-column + right column vertical splits often don't work on the first try, and this "show, fix, and show again" loop is essential. If you skip the confirmation and just fix the code, discrepancies that you thought were fixed may remain.
Pitfalls and Learnings
Here are some cautionary notes that have emerged from running this loop.
- Take a new screenshot with each fix - Reusing a previously taken screen won't allow you to evaluate the fixed state. Take a new screenshot of the actual object every time.
- Reproduce the state before showing - Not just the initial display, but also operate to the state assumed by the mockup, such as after recording or slider operation, before taking a screenshot.
- Match the viewport width - Wrapping changes with different display widths. Compare after matching the browser to the width assumed by the mockup.
- Verbalize the skeleton first - Having a structural description like "2-column / right column vertical split" keeps the AI's feedback from becoming abstract and makes it a fixable granularity.
- Code review and screen review are different - Consistency in the code and consistency in the rendering result do not match. Always make the final judgment on the actual screen.
Summary
- Discrepancies in appearance, such as layout skeleton discrepancies and wrapping/overflow, cannot be detected by statically reading the code together
- Use browser automation extensions to "show" the actual screen on localhost to Claude Code, compare it with the mockup image, and identify discrepancies
- Specify discrepancies as specifically as "which area, which element, and how it differs between the mockup and actual screen", and use this as the correction instruction
- After fixing, open it in the browser again for comparison. The loop of showing, fixing, and showing again is essential
- Take a new screenshot every time, match the state and viewport width, and then compare
- Make the final judgment on the actual screen, not the code - the new practice is to show the screen to the AI and have it fix it
Top comments (2)
We chased a "bug" for a while that turned out to be an unpinned window: a flex wrap that only happened at 1280px wide, appearing and disappearing between sessions because default window dimensions drift. So your viewport-width bullet is the one I would print in bold, and the sharper version is pinning the viewport to an explicit size before every capture, not just matching it roughly. One addition from running the same workflow: structural discrepancies (the crushed second column, the preview wrapping below) show up more reliably in the accessibility tree than in screenshots, so having the agent read the DOM tree and compare region nesting catches skeleton breaks even when the screenshot looks plausibly fine at a glance. I use screenshots for spacing and color, tree reads for structure. Do you keep per-breakpoint baseline captures, or only compare at one width?
Really appreciate this — the 1280px flex-wrap that drifts between sessions is exactly the failure mode, and "pin the viewport to an explicit size before every capture" is the version I'd now print in bold. "Roughly matching" the width isn't enough; the drift is the bug.
Your accessibility-tree point is the sharper half, and honestly a step beyond what the post described — we leaned on screenshots for structure too. You're right that skeleton breaks (the crushed second column, the preview wrapping below) surface far more reliably in the tree than in a screenshot that looks plausible at a glance. Screenshots for spacing/color, tree for structure is a clean split I'm adopting. The one thing that made it click for us was writing the intended skeleton in words first ("2-col; right column splits top/bottom, preview top-right…"); pair that stated skeleton with a tree read and the agent can assert region nesting matches the spec instead of eyeballing it.
To your actual question: in this project it was a single-width comparison at the desktop layout — no per-breakpoint baselines, so you're ahead of us there. But thinking it through, the cleaner answer to which widths is to derive them from the breakpoints declared in the CSS/config itself, rather than hand-picking or settling on one — capture at each breakpoint boundary and at boundary ±1px, since the wrap flips exactly at the transition. That keeps the capture set in sync with the source of truth and aims it right where skeleton breaks happen. Storage (frozen fixtures vs regenerated each run) is then a separate axis on top. Do you drive your capture widths off the CSS breakpoints, or set them independently?