AI Pulse · Under the Radar
The play
Discount one-shot demos by at least 50 percent when setting internal capability benchmarks or client expectations.
A developer named Matt Shumer asked Claude to build a first-person shooter in one prompt. The result, a Three.js browser game called “Claude of Duty,” racked up nearly a thousand GitHub stars in two days. Social feeds lit up with claims that AI can now crank out commercial-grade games from a single instruction.
Then someone ran a blind test. Independent reviewers scored the demo against actual Call of Duty footage on a ten-point scale. Claude of Duty landed between 3.6 and 5.05 out of ten. Every tester identified the real game correctly. Words like “amateur” came up often. The AI version works, but it looks and feels like a student project, not a shipping title.
This matters because one-shot demos travel fast and set unrealistic expectations. A GitHub star count tells you the idea is interesting, not that the output is production-ready. When you’re deciding whether to invest in AI tooling for your business, the gap between “it generated something” and “it generated something I can actually use” is the entire budget conversation.
We see this pattern constantly. A model produces a flashy prototype, the demo goes viral, and three months later the same teams are hiring humans to rebuild it properly. The repository is transparent about what Claude did and didn’t handle, which is useful. The hype around it is less so.
If you’re building an AI system that needs to deliver consistent quality, not just impress people once, you need guardrails, feedback loops, and humans in the review chain. That’s the kind of thing we build into an AI command centre, where the goal is repeatable business value, not a viral moment. The corrective here is simple: test against the real thing before you believe the claim.
Free daily email
Get this every morning.
This brief is one item from today's AI Pulse, the short daily read we run for ourselves on what is actually happening in AI. Subscribe free and it lands in your inbox each morning.
Free daily email
Subscribe to the daily AI Pulse
One short read every morning on what is actually happening in AI. Free.
You are in
Your first AI Pulse lands tomorrow morning. Keep an eye on your inbox.