If autonomous agents are going to hire each other, pay each other and rely on each other's output, they need some way to answer a basic question: has this counterparty done this well before? The leading answer is an on-chain reputation registry. The leading implementation now has enough data to check whether it works. It does not.
What is actually at stake
ERC-8004, "Trustless Agents," defines how autonomous agents publish identity, capability and reputation without a central operator. It is not a proposal on a whiteboard. It is deployed, with reported adoption of 170,000+ registered agents and 150,000+ feedback records across Ethereum, BNB Smart Chain and Base, and it sits underneath a broader vision — Circle's open agentic economy paper, x402 payment rails — of agents as a kind of labour market, discovered on merit and paid in stablecoins.
The whole design rests on one load-bearing idea: reputation derived from demonstrated behaviour rather than self-description. An agent should not be trusted because it claims to be good. It should be trusted because its record says so.
The measurement
A peer-reviewed empirical study of the deployed ecosystem — Can Trustless Agents Be Trusted? An Empirical Study of the ERC-8004 Decentralized AI Agent Ecosystem (arXiv 2606.26028) — examined the live registries rather than the specification. Its findings are blunt:
- Feedback values are not commensurable — one agent's 5 is not another's 5.
- Feedback records are rarely grounded in verifiable interactions. Most reviews do not correspond to a demonstrable event.
- Reputation is manipulable at minimal cost.
- Coordinated Sybil behaviour among 73.5% of reviewers on Ethereum, 59.2% on BNB Smart Chain, and 90.6% on Base.
On Base, roughly nine out of ten reviewers show coordinated fake behaviour. Read against 150,000+ feedback records, the registry is not a weak trust signal. It is closer to noise wearing the costume of a trust signal, which is more dangerous than no signal at all — an empty reputation field invites caution, whereas a populated one invites reliance.
The interesting part: this is not a broken standard
The easy version of this story is "the registry is broken." That version is wrong, and worth correcting because it is the difference between a cheap dunk and a real problem.
The specification anticipated exactly this. Its own text warns, inline, that "results without filtering by clientAddresses are subject to Sybil/spam attacks," and its Security Considerations section states the design intent plainly:
"We expect reputation systems around reviewers/clientAddresses to emerge. While simple filtering by reviewer and by tag are enabled on-chain, more complex reputation aggregation will happen off-chain."
So the registry was never meant to be the trust layer. It is a data layer. Trust computation was deliberately delegated to whoever consumes it, by filtering to a set of reviewers you consider credible.
Which leaves one question the ecosystem has not answered: who supplies a trustworthy list of reviewers to filter to?
The standard said such systems were expected to emerge. The Sybil measurements are what it looks like when they have not. The plumbing shipped and scaled; the layer that makes the plumbing mean something is still missing, and the gap is now quantified rather than suspected.
Where we stand on this, honestly
We have not built anything here and we are not selling anything here. We found this while researching adjacent ground, checked whether someone had already solved it, and concluded the gap looked real. So we did the obvious next thing: rather than assume, we posted the finding as a question to the people actually building on the standard, in the Ethereum Magicians thread where ERC-8004 is discussed — asking whether anyone is building that off-chain reviewer-reputation layer, what production consumers filter on today, and what would make a reviewer allowlist credible.
If the answer comes back "someone already does this, here is the link," that closes the question and costs nothing. That is a good outcome. We would rather be corrected in public than build against an assumption.
Why this rhymes with everything else we do
The same question keeps appearing in unrelated domains: how do you actually know, versus assume?
In patent due diligence, a high similarity score between a patent claim and a product description feels like evidence and is not — resemblance is a screening signal, not proof of claim practice. Our calibration benchmark grades our own analysis engine against court-adjudicated outcomes, carrying real losses as well as real wins. Twice this month it scored full marks on verdicts and then refused to certify itself, because one of its evidence sources had been degraded during the run. A pass obtained on unreliable evidence is not a pass, even when the score says it is.
In database security, we recently found that a log query returning zero results looked exactly like proof nothing had happened — until we proved the query itself was silently failing to scan.
And in agent reputation, 150,000 feedback records look like a track record until someone checks whether the reviews correspond to anything real.
Same failure in three costumes: a number that is present, plausible and unearned. The discipline that catches it is unglamorous and always the same — ask what the number is grounded in, and refuse to accept it when the honest answer is nothing.