Maple targets small-GPU inference

A project called MAPLE says it can shrink Gemma 4 26B-A4B, a mixture-of-experts model, to a 4.75 GB file that runs entirely on a 6 GB GPU. It does this by profiling which experts activate on a specific kind of work, then removing experts it judges less relevant. The published version is aimed at Python, algorithm and data-structure tasks, and STEM reasoning.

The project’s author reports a speed of 45.4 tokens per second on an RTX 4050 laptop GPU. The repository also reports strong results on four small tests, including code generation and physics reasoning. Those are the author’s own benchmarks, so they don’t establish how the model performs across everyday business tasks. The Hacker News discussion was modest, with 19 points and 5 comments. Still, if the approach holds up, it could make local experiments with larger models more practical for teams with limited GPU memory. That’s the kind of trade-off worth tracking in an AI command centre, where teams can compare tools against their own needs.

For an operator, the useful question isn’t just whether a model fits on a small GPU. It’s whether the compressed model remains accurate on the work you’d give it. MAPLE provides scripts to profile and prune a model for a chosen domain, but testing it on your own tasks would be essential before relying on it. The project details and reported benchmarks are available on GitHub.