Large model runs on RAM

A project called llama-modchip says it can run a 284-billion-parameter DeepSeek model on a 64GB system using only the CPU, with no GPU. It reports that a compressed version of the model uses 56.56 GiB of storage and reaches 9.1 to 14.7 tokens per second for generation. The project also reports 44.4 tokens per second for prompt processing on a 16-core Zen 5 system. Those are the project’s figures, not a general promise for every 64GB computer. The setup depends on custom model files and software, and the repository says standard llama.cpp, Ollama and LM Studio builds won’t load its custom model format. The project details and benchmarks are here.

The practical point isn’t that every business should run a model this large on a desktop. It’s that local AI may be possible on less specialized hardware than many operators assume, if they accept the project’s compression and performance trade-offs and use its specific tools. That could matter for teams weighing local inference against sending work to a hosted service, but this example doesn’t establish that it’ll be faster, cheaper or better for a particular workload. The project also describes smaller models that run on 16 to 32GB of RAM, with reported speeds ranging from 25 to 60.4 tokens per second.

For an owner, the useful next step is to test a real task on the hardware you already have, rather than planning around a headline model size. This is the kind of thing we build into an AI command centre, so teams can keep track of which tools and models are practical for their work.