AI Pulse · Under the Radar

A thread showing how to run Kimi K3, a frontier-scale open model, on just 29GB of RAM at half a token per second hit the front page of Hacker News this week. It’s one of at least three similar threads in the last few days, all focused on cramming enormous mixture-of-experts models onto hardware you can buy at a store. The speed is barely usable, but the sheer volume of interest tells you something: a lot of people would rather own the inference, even if it’s slow, than rent it by the token.

This isn’t about performance. It’s about control and cost predictability. If you’re running a few hundred queries a month through an API, paying per token is fine. But if you’re prototyping something that might scale, or you want to experiment without watching a meter, or you just don’t want your prompts leaving your network, local inference starts to make sense. The fact that hobbyists are now squeezing models that would have needed a data centre two years ago onto a gaming rig changes the calculation for small operators.

What it means for you

You don’t need to run models on your own hardware today. But you should know the option exists and is getting cheaper fast. If you’re building something that depends on an API, ask what happens if usage spikes or if you want to move it in-house later. The gap between cloud and local is narrowing, and that changes how you think about vendor lock-in.

This is exactly the kind of trade-off we model when we build an AI command centre: when does it make sense to own the stack, and when do you rent? The answer isn’t always obvious, but it’s worth asking before you’re three months into a contract.

The thread is on Hacker News if you want to see the technical details.