Bitsec Outperforms Claude Fable 5 in Code Vulnerability Detection

A Bittensor subnet beat a jailbroken Claude Fable 5 on vulnerability detection — raising questions about specialized AI versus general-purpose models in security work.

Share
Bitsec Outperforms Claude Fable 5 in Code Vulnerability Detection

The breakthrough

Bitsec, running as Subnet 60 on the Bittensor network, found more vulnerabilities than a jailbroken instance of Anthropic’s Claude Fable 5 when both were run against the same client codebase. The test was posted to X by a developer affiliated with Bitsec, with the client’s permission.

How the two systems work

Bittensor is a decentralized network that rewards specialized AI development through a modular subnet architecture, where multiple specialized agents combine to perform specific tasks.

Bitsec runs a network of AI agents trained to process code and smart contracts, combining ML, fine-tuned detection models, autonomous agents, and traditional static analysis. Agent outputs are validated against a wide range of real-world samples, creating competition that sharpens detection accuracy over time. Bitsec returns audit feedback within 24 hours and has launched its paid service, which includes repository scanning and bug bounty administration currently in development.

Anthropic’s Claude Fable 5, released June 9, 2026, is a Mythos-class model built for complex, long-horizon reasoning — primarily engineering and agentic workflows. It performs well across coding benchmarks, knowledge work, and extended autonomous projects. According to Anthropic’s launch documentation, when Fable 5’s classifiers detect a cybersecurity-related request, the response is automatically handled by Claude Opus 4.8 instead, and users are informed when this occurs. Those classifiers are what required bypassing before the comparison could run.

On June 12, three days after launch, the US government issued an export control directive forcing Anthropic to disable Fable 5 and Mythos 5 for all users globally, citing national security concerns tied to jailbreak reports. Anthropic complied but disputed the severity, describing the government’s basis as a narrow, non-universal technique amounting to asking the model to read a codebase and fix software flaws. The Bitsec comparison was conducted before that shutdown.

The evaluation

The test used a live paying client’s codebase. According to a post from Bitsec’s account, a Bitsec-affiliated developer jailbroke Fable 5 across multiple agent instances — bypassing its cybersecurity classifiers entirely — then ran the Bitsec subnet against the same code. According to a post from @markjeffrey on X, Fable 5 returned roughly 60 findings — none critical, five rated high or medium severity. Bitsec found more than 160 vulnerabilities in total, including five critical issues and ten high-severity problems Fable 5 had not flagged. Bitsec also caught every issue Fable 5 found.

The jailbreak itself took 3–4 hours across three agent instances. Even operating outside its safety constraints, Fable 5’s write-ups of identified exploits were clearer than Bitsec’s, and its false positive rate was near zero — it was faster but more expensive per audit.

What this suggests

The result carries a methodological caveat: the Fable 5 tested here was not Anthropic’s shipped product. It was a jailbroken instance operating outside its intended safety constraints. Whether an unjailbroken Fable 5 — or a version purpose-built for security tooling — would perform differently is an open question this test does not answer.

Within those limits, the comparison shows that Bitsec’s network of competing agents can achieve wider vulnerability coverage than a single general-purpose model running without restrictions. Bitsec has completed contract audits for clients including liquidity vault projects, suggesting those capabilities are translating into practical use.

What’s next

Bitsec has said it plans to publish supporting evidence for independent verification. The developer who ran the test noted interest in repeating it against Mythos 5, the same underlying model as Fable 5 but with cybersecurity classifiers lifted, available to vetted organizations through Project Glasswing. That comparison would isolate whether the classifier constraints — rather than the model’s underlying capability — account for the gap.

Cyberpunk Background