AI Pulse · Under the Radar

A four-person team called Offloop just posted benchmark numbers that have developers paying attention. They claim 84.9% on GDPval and 67.2% on JobBench, two tests that measure how well AI handles real coding work people actually pay for, not toy problems. That puts them ahead of Claude Code and Codex on those specific tasks.

The cost claim is harder to verify. Multiple accounts on X are passing around a figure of $363 for Offloop to complete a 220-task benchmark run, compared to roughly $3,100 for Claude Code on the same set. That spread is wide enough to matter if you’re running dozens of these tasks a week, but the numbers come from fan testing, not an official audit. The original thread shows at least four independent accounts posting within 24 hours, so this isn’t one person’s hype cycle.

What matters for a business owner is whether the tool can handle the boring, repetitive code work that eats up contractor hours. Benchmarks like JobBench try to simulate that, pulling from tasks clients actually post on freelance boards. If a small team can deliver those results at a fraction of the cost, it changes the math on whether you hire a developer for three weeks or spin up an AI agent for three days. The caveat is that real-world performance always lags benchmarks, especially when your codebase is messy or your requirements shift mid-task.

This is exactly the kind of signal we track when we build tools like the Omni Command Centre, where you can test multiple models against your own tasks and see which one actually delivers before you commit budget. A four-person startup beating the big labs on specific metrics is interesting. Whether it holds up in your environment is the only question that matters.