Hallucinated Audits: AI Invents Bugs, Misses Real Ones
It found four vulnerabilities. Three don't exist. The fourth was the fix.
There is a badge going around. Clean font, small shield, three words: audited by AI. It costs nothing, it arrives in ninety seconds, and this year it has been sitting on landing pages that raised real money. Underneath it is a language model that read your contract, produced a confident report, and — as the OWASP Top 10 for LLM Applications now states out loud under LLM09 Misinformation — has no mechanism whatsoever for knowing whether a word of it is true. The academic arithmetic is unkind: tested against 52 already-audited DeFi protocols, GPT-4 and Claude could be pushed to name most of the genuine vulnerabilities only at settings that buried them under invented ones, roughly thirty-nine phantoms for every real finding.
Which gives us the shape of the thing, and it is a beautifully stupid shape. The bot invents a bug. The developer, being conscientious, 'fixes' the invention — sometimes by deleting a guard that was doing actual work. The fix ships. And the real hole, the one the model never mentioned because nobody ever wrote a Stack Overflow answer about it, walks into production wearing a security badge and a very nice suit. Nobody lied. Everybody was polite. The money still left.
The Scheme In One Paragraph
Three parties, one document. The scam project buys the badge because it is cheaper than an audit and looks identical on a screenshot. The honest developer runs the tool because it is fast and free, then patches phantom findings and burns his review budget on nothing. The attacker reads the same report — same model, same prompt, same public contract — and pays attention to the one thing everybody else skipped: the sections the tool marked green. A machine that is wrong in a predictable direction is not a defence. It is a map of where nobody is looking.
An audit is not a document. It is a person putting their name on a risk. The badge sells the paperwork and quietly loses the person.
A Very Confident Wrongness
The report always looks superb. Severity labels, line references, a remediation snippet, an executive summary written with the calm of a man who has never lost money. That polish is the product. Fluency and correctness are separate features, and only one of them was trained for.
The failure modes are specific, not random. Models invent vulnerability classes that do not apply to the language version in front of them. They cite CVEs that were never issued. They recommend libraries that have never been published — which is precisely why slopsquatting now exists as a supply-chain technique: register the hallucinated package name, wait for the next developer to paste the same suggestion. The attacker does not have to guess what you will trust. He asks the same model you did.
And then the omissions, which cost more. Trail of Bits found the tooling stalls exactly where audits earn their fee: multi-contract interactions, economic assumptions, incentive design, the state of the protocol three blocks into an attack. Reentrancy in a tutorial is easy. A broken invariant across four contracts and an oracle is where the money actually leaves.
It reads like an audit, prices like a coffee, and covers you like a photograph of an umbrella.
Why The Model Invents Bugs
A language model is not searching your code for defects. It is predicting the most plausible continuation of a document that begins 'security audit report'. Plausible audit reports contain findings. So findings appear — the format demands them, and there is no internal step that checks whether the finding survives contact with your bytecode.
OWASP files this under LLM09 Misinformation and pairs it with overreliance for a reason: the danger is not the wrong sentence, it is the wrong sentence delivered with the tone of a specialist. The NIST adversarial ML taxonomy makes the same point in colder language — confidence in generative systems is not calibrated to correctness, and treating output as evidence is a control failure, not a model failure.
Then add the incentive layer. Vendors sell coverage, and coverage is easiest to demonstrate with volume. A tool that reports twelve issues feels stronger than one that reports two, and nobody markets a scanner on the number of things it correctly ignored. Sensitivity sells. Specificity is what saves you, and it does not photograph well.
We asked a machine that writes fiction for a living to write a document whose only value is that it is not fiction.
Vocabulary Decoded
Six terms. Learn them now, or learn them from a post-mortem thread later.
Hallucinated finding
What it sounds like:
A small mistake in an otherwise useful report.
What it actually is:
A vulnerability the model generated because the format expected one. It has a severity, a line number and a fix, and none of them refer to anything real. OWASP LLM09 calls this class misinformation: plausible, confident, wrong.
False positive rate
What it sounds like:
An acceptable engineering trade-off.
What it actually is:
Your review budget, spent by a stranger. In the 52-protocol study, tuning the models to catch most real vulnerabilities pushed false positives to roughly thirty-nine noise findings for every genuine one — a ratio that quietly converts your senior engineer into a triage clerk.
Slopsquatting
What it sounds like:
Typosquatting with a rude name.
What it actually is:
Registering package names that language models invent, so the next developer who follows the suggestion installs the attacker's code. The USENIX study counted about 205,000 unique hallucinated package names and found the same names recur across runs — hallucinations are repeatable enough to be a supply chain.
AI-audited badge
What it sounds like:
Modern, automated assurance.
What it actually is:
A graphic. There is no firm, no scope, no methodology, no signature and no liability behind it. On a scam token's landing page it performs exactly one job: converting your hesitation into a deposit.
Remediation-induced bug
What it sounds like:
A patch that needed a second pass.
What it actually is:
A real, exploitable defect introduced by applying an AI's fix for an imaginary problem — a guard removed, a check reordered, an access modifier loosened to satisfy an issue that never existed. The audit did not find the bug. It shipped one.
Scope
What it sounds like:
Boring contractual language at the top of a PDF.
What it actually is:
The only part of any audit that tells you what was not examined — commit hash, contracts in scope, assumptions, out-of-scope components. Hallucinated reports never have one, because a scope is a promise and a model cannot make promises.
Every one of these is free, instant and looks professional. That combination has never once been good news.
How The Con Runs
Five stages. Only the last one has a block explorer link.
Stage 1: Generate the paperwork
Paste the contract into a chat window, ask for a security audit in the style of a well-known firm, and receive twelve pages with severity ratings, a scoring table and a logo-shaped hole at the top. Cost: nothing. Time: under two minutes.
The output is not a forgery of a specific firm's report. It is a forgery of the genre.
Stage 2: Dress the front page
A shield graphic, a green 'PASSED' line, a fake reference number and a date. On a token launch page it sits next to the liquidity-lock claim and the roadmap, doing the same job as both: replacing verification with decoration.
Nobody clicks the badge. That is the entire commercial insight behind it.
Stage 3: Let honest teams do the same thing sincerely
This is the part that scales. A real developer runs an AI review, gets a list of findings, and starts patching. Hours go into phantom issues; the genuine risk in the economic logic is never raised because the model was never shown the incentives, only the syntax.
The scam project fakes the report. The honest one believes it. Both end up equally uncovered.
Stage 4: Poison the fix
Some remediations actively hurt. Removing a re-entrancy guard the model called redundant, widening a modifier to silence an access-control 'finding', replacing a checked call with an unchecked one for gas. The diff is small, reviewed quickly, and merged with the confidence of a resolved ticket.
The most dangerous line in any repository is one added to satisfy an issue that did not exist.
Stage 5: Cash the badge
Investors see an audit. Listing forms accept an audit. Aggregators display an audit. Nothing in that chain verifies the firm, the scope or the commit hash, and by the time anyone checks, the deposit has been in the contract for a fortnight.
The badge does not need to survive scrutiny. It needs to survive the ten seconds before a purchase.
The Timeline: How We Got Here
Documented milestones, in order.
March 2023: the first honest benchmark
Trail of Bits publishes its assessment of Codex and GPT-4 for smart-contract review and reports the pattern everyone would rediscover later: usable on textbook bug classes, unreliable on the multi-contract and economic reasoning that real audits exist to cover.
The conclusion was not 'useless'. It was 'not a substitute', which is a much harder thing to sell.
June 2023: the false-positive number arrives
Researchers test GPT-4 and Claude against 52 real DeFi protocols with known, previously audited vulnerabilities. The models can be coaxed into naming most of the real issues, but only at settings that generate a torrent of invented ones alongside.
Recall you can buy. Precision is the bill that arrives afterwards, paid in engineer-hours.
2023 to 2024: the 'AI audit' badge becomes a product
Automated review services multiply, and their output starts appearing as a trust signal on token launch pages — often with no named auditor, no scope, no commit hash and no way to verify that any tool was ever run.
The badge industry did not wait for the technology to be ready. It never has.
January 2024: maintainers meet the slop
curl's Daniel Stenberg documents AI-generated vulnerability reports arriving through its bug bounty — detailed, well-formatted, entirely fabricated — and by July 2025 writes about the volume problem directly: teams now spend real time disproving machine-written findings.
Defenders are being denial-of-serviced by documents that took nine seconds to produce.
2024 to 2025: hallucination becomes a supply chain
The USENIX Security 2025 package-hallucination study measures 576,000 generated code samples, finds roughly one in five recommended dependencies does not exist, and identifies about 205,000 unique invented names — a ready-made target list now known as slopsquatting.
The same imagination that invents your bug invents the library that will fix it, and someone else registers it first.
2025: the standards catch up
OWASP's 2025 Top 10 for LLM Applications names LLM09 Misinformation explicitly, describing confidently wrong output and human overreliance as a security risk in its own right rather than a quality complaint.
When the risk gets an official number, the excuse 'we didn't know it does that' expires.
Three years, one direction: better prose, same epistemics, wider distribution.
Signals Worth Acting On
- A badge with no firm, no report and no commit hashA real audit names who did it, what version they saw and what they excluded. Anything without those three is decoration.
- The report is dated the same day the contract was deployedHuman review runs for days or weeks and produces a fix-and-retest cycle. Same-day assurance is a document, not a process.
- Findings are generic and quote no line of your codeReentrancy, integer overflow, unchecked call — copied from a curriculum. A genuine finding is annoyingly specific to your architecture.
- A 'critical' issue you cannot reproduceAsk for the transaction sequence that exploits it. Hallucinations do not survive the request for a proof of concept.
- Recommended libraries that do not existCheck the registry before installing anything a model suggested. An invented name that resolves is worse than one that fails.
- No verifiable link between the audit and this deploymentThe report must map to the deployed bytecode. Audited testnet code and deployed mainnet code are two different products.
Not one of these requires you to read Solidity. Every one of them requires you to read the first page.
The Numbers
Published figures only. No estimates of our own.
~39 false positives per real finding
In the 52-protocol study, the configuration that let GPT-4 and Claude surface most real vulnerabilities produced roughly thirty-nine invented findings for every genuine one.
19.7% of recommended packages do not exist
Across 576,000 LLM-generated code samples in the USENIX Security 2025 study, nearly a fifth of suggested dependencies were hallucinated.
205,000 unique invented package names
The same study catalogued them, and found hallucinations repeat across runs — which is exactly what makes registering them profitable.
LLM09 — an official risk, not an opinion
OWASP's 2025 Top 10 for LLM Applications lists misinformation and overreliance as a named security risk with documented impact.
The Second Act: Who Is Liable For A Robot's Signature
The interesting question is not technical. When the contract fails, everyone reaches for the audit — and discovers there is nobody at the other end of it.
There is no firm to sue
An audit is a professional engagement: named auditors, a scope, an agreed methodology and, occasionally, insurance. A generated PDF has an author who cannot be deposed and a vendor whose terms disclaim everything.
The exchange sees a formality
Listing checklists ask whether an audit exists, rarely who performed it. That gap is the whole business model of the badge, and it is closing far too slowly.
The insurer reads the scope
Cover for smart-contract failure depends on a documented review of the deployed code. 'We ran a model over it' is not a control, and underwriters have started saying so in writing.
Use AI review the way you use a smoke alarm: cheap, always on, useful for the obvious. Do not let it sign the building inspection.
Assurance without accountability is theatre. The costume is very good now, which is the only thing that changed.
Why Competent Teams Still Get Hit
None of the usual mistakes are required here, which is the uncomfortable part.
The output is graded on fluency
Humans judge documents by structure and tone. A generated report has better structure and tone than most real ones, because that is the exact axis it was optimised on.
Negative results are invisible
You can count the bugs a tool found. You cannot count the bugs it silently declined to mention, and that number never appears in a procurement comparison.
The deadline wants a green tick
Launch dates need an artefact, not a conversation. A tool that returns a clean report in ninety seconds is, organisationally, extremely difficult to argue with.
Triage exhaustion
After thirty phantom findings, the thirty-first gets a shorter look. Alarm fatigue is not carelessness; it is the predictable output of a bad signal-to-noise ratio.
The lesson is not that AI review is worthless. It is that it is a linter with excellent handwriting, and a linter has never been allowed to sign off a release.
What To Actually Do
In order of how much it helps.
- Verify the auditor before you read the audit. Named firm, named engineers, a report published on the firm's own domain, and a commit hash that matches the deployed bytecode. Two of three is a fail.
- Demand a proof of concept for every finding. A real vulnerability comes with a reproducible transaction sequence or a failing test. Hallucinations cannot produce one, which makes this the cheapest filter you own.
- Never merge an AI remediation without a human review of the diff. Treat a suggested fix as an untrusted pull request from a stranger who is confident, fast and occasionally inventing the problem it solves.
- Check every recommended dependency in the registry. Existence, ownership, publish date, download history. A brand-new package with the exact name your model suggested is the slopsquatting attack, not a coincidence.
- Use AI review as a pre-filter, never as the last gate. Run it before the humans, to clear the obvious. The order matters: a machine ahead of a specialist saves time, a machine instead of one buys silence.
Checks Worth Running This Week
None of these take long, and all of them are skipped.
Open the audit badge on the last three projects you funded
Follow the link. If it does not lead to a report on an auditor's own site, you invested against a picture.
Match the report's commit hash to the deployed code
Verified source on the block explorer, compared with the hash in the report. Different bytecode means an unaudited contract holding audited-contract money.
Re-check every dependency an assistant recommended this month
Registry page, maintainer history, first publish date. Anything under a few weeks old with a suspiciously perfect name gets removed today.
Ask your AI reviewer for a proof of concept on its own top finding
Watch what happens when it has to produce an exploit path instead of a paragraph. That is the reliability test, and it takes five minutes.
Diff every 'security fix' merged in the last quarter
Look specifically for guards removed, modifiers widened and checks reordered. Remediation-induced bugs hide in commits that sound reassuring.
An audit you did not verify is a rumour with a font.
AI Audit Verification Checklist
Worth keeping next to the investment memo, not in a bookmark you never open.
Identify the named firm and the named engineers; anonymous assurance is not assurance.
Find the report on the auditor's own domain, not a PDF hosted by the project.
Match the audited commit hash to the verified bytecode actually deployed on chain.
Read the scope section first: what was excluded matters more than what passed.
Require a reproducible proof of concept for every critical and high finding.
Review every AI-suggested remediation as untrusted code before it is merged.
Verify each recommended package exists, with real maintainers and real history.
Confirm a human specialist reviewed the economic and multi-contract logic, not just the syntax.
Got a Suspicious Message?
Use our AI-powered detector to analyze potential scams instantly.
Key Takeaways
- 1AI review generates findings because the format expects findings; OWASP names this LLM09 Misinformation and treats overreliance on it as a security risk.
- 2The published false-positive ratio is brutal — around thirty-nine invented findings per real one at the settings that catch most genuine bugs.
- 3Omissions cost more than inventions: economic logic and multi-contract interactions are exactly where automated review is weakest and where funds actually leave.
- 4Applying a fix for an imaginary bug can create a real one; every AI remediation is an untrusted diff until a human reads it.
- 5Hallucinated dependencies are now a supply chain — 19.7% of suggested packages did not exist and about 205,000 invented names are there to be registered.
- 6An 'AI audited' badge with no firm, no scope and no commit hash is marketing; verification means a named human accepting a named risk.
A badge is not a bodyguard. If nobody signs their name under the audit, the only thing being audited is your patience — and the only thing being drained is your wallet.
Frequently Asked Questions
Sources & Citations
Research for this investigation compiled from publicly available blockchain data, security reports, and community documentation.
blog.openzeppelin.com
blog.trailofbits.com
docs.code4rena.com
Verification: All blockchain transactions and addresses referenced in this article can be independently verified through the linked blockchain explorers. We encourage readers to conduct their own verification.
Methodology: Every case requires at least three independent sources plus verifiable on-chain evidence before publication. Full standards: /methodology
Legal notice: This assessment is based on publicly available data, including on-chain records, official statements and reported incidents. It is journalistic and educational analysis, not legal advice, an accusation of criminal conduct or a court finding. Named companies, projects, domains, wallets and individuals are described as reported by the cited sources; a company name may appear because fraudsters impersonated it, not because the company did anything wrong. If you believe something is inaccurate or out of date, write to cryptostrapon@proton.me and we will correct it and log the change. Editorial policy