Magnitude launches self-tuning agent engine

Magnitude has released an open-source inference engine designed to make locally run AI models fit the machine they are running on. Instead of relying on precompiled kernels built for broad hardware categories, it compiles and tunes those kernels on the user’s own device before the model runs. The project says this can make open models run up to twice as fast as llama.cpp, with support for Apple Silicon, NVIDIA and AMD hardware, or CPU-only machines. It runs on macOS, Windows and Linux, and connects with agents including Codex, Claude Code, Cline and others. The project’s GitHub page also says prompts, files and models stay on the machine once a model has been downloaded.

For a business owner, the practical point is not the benchmark alone. Faster local models can make private AI tools more usable for work that involves internal documents, customer information or repeatable staff tasks, without sending that material to an external model provider. The speed claim will depend on the specific hardware and model, so treat it as a reason to test rather than a promised outcome.

This is the kind of thing we build into an AI command centre, where the question is which AI work should stay inside your business, which tools staff can use, and how you measure whether the setup is actually saving time.