Most Solidity security tools have the same failure mode: they cry wolf. You run them on an audited protocol and get 600 "findings," 98% of which are noise. The signal drowns. Worse — send a client a report full of false positives once, and you've burned your credibility.
I've been building a scanner with the opposite goal: report almost nothing, but be right when it does. Zero false positives, verified across the entire OpenZeppelin library. Here's how that actually works, with three real examples from today's run against production protocols.
Example 1 — the "spec violation" that isn't
A naive detector reads a NatSpec comment or a spec doc that says "only the rate manager can update the rate", then flags the function as a violation if it "can't prove" the restriction. On Ember's vaults today, that produced six [real] findings — one of them a CRITICAL:
function pause() external onlyGuardian { ... }
function processWithdrawalRequests(uint256 n) external nonReentrant onlyOperator { ... }
function setMaxTVL(...) external onlyAdmin { ... }
Every one of these is a correctly access-controlled, intended feature. The "spec violation" was the tool listing the protocol's own role design and calling it a bug. All six were false positives.
The fix is deterministic, not AI-guesswork: before emitting, find the affected function and check whether the restriction is actually enforced (onlyX / onlyRole / msg.sender == / a require). If it is — it's the design, not a violation. Suppress it. If there's genuinely no enforcement, it still fires. Safe direction.
Example 2 — fee-on-transfer that's out of scope by design
Another classic: a vault does token.transferFrom(user, address(this), amount) and then uses amount for accounting. Fee-on-transfer tokens send less than amount, so the internal books inflate → the tool screams "insolvency."
Real? Only if the protocol lets users deposit arbitrary tokens. Most don't:
// deposit is gated by a curated registry / whitelist
uint256[] memory types = IRegistry(registry).batchGetAssetTypes(assets);
IRegistry(registry).batchProcessDeposit(creditor, assets, ids, amounts);
If governance curates which tokens can ever enter, a fee-on-transfer token simply isn't listed. It's an accepted design assumption, not an exploit. So the detector now checks for token curation (registry / isAllowedAsset / whitelist) and stays quiet when it's present — but still fires on genuinely permissionless deposit paths.
Example 3 — the unchecked call that's actually checked
(bool success, ) = recipient.call{ value: amount }("");
require(success, "transfer failed");
A regex that only looks at the call line flags this as "unchecked return value." Look one line down and the require(success) is right there. .transfer() gets flagged too — even though it auto-reverts. The fix: read the next few lines, and know the difference between .call and .transfer.
Why bother being this strict?
Because the number that matters isn't how many findings you produce — it's how many are real. A report a founder can trust is worth more than a wall of noise they have to triage themselves. I verified 13 flagged findings across audited protocols today; all 13 were false positives, and the pipeline now suppresses those classes automatically, forever. That's the moat: not more findings, fewer wrong ones.
If you're building something pre-mainnet and want a security first-pass that only tells you about real issues — no noise, hand-verified — I'm happy to take a look. You can find my work at juan23z.github.io.
Written after a long day of teaching a scanner to shut up when it should.
Top comments (3)
Ha, love this. You ran a scan on the scanner guy's own site and found real issues - fair play, and exactly the signal-over-noise spirit the post is about.
Just pushed fixes for the easy wins: meta description, robots.txt and sitemap.xml are live now. You're right about the headers - GitHub Pages won't let me set custom response headers, so a Cloudflare proxy with Transform Rules is the plan once the site carries real weight.
Appreciate you taking the time to actually run the check. Thanks!
Respect the "reports almost nothing and that's the point" honesty — a scanner that cries wolf is worse than none. Out of curiosity I ran my own launch scan on juan23z.github.io: Content-Security-Policy and X-Frame-Options missing, nosniff/Referrer-Policy/Permissions-Policy unset, no meta description, robots.txt and sitemap.xml 404. The headers aren't really your fault — GitHub Pages doesn't let you set custom response headers at all. If the report site starts mattering for credibility, the usual escape hatch is a free Cloudflare proxy in front with headers added via Transform Rules, or a host with a _headers file (Netlify/CF Pages). The meta description you can fix today in the HTML head. Happy to re-run the check if you change the setup.
Confirmed live — robots.txt 200, sitemap.xml 200, and the meta description is in the head (good copy too — "zero-false-positive pipeline" is exactly the right promise from a scanner).
The Cloudflare Transform Rules route is the right call when the site earns it. One gotcha for when you set it up: stage HSTS with a short max-age (say 300s) before going to a year — a bad HSTS value on an apex is painful to roll back, and browsers cache it aggressively.
Happy to re-run the scan free once the proxy is in front so you get a verified before/after — just drop a note here.