Running Gemma-4 26B at 124 tokens/SEC on a CPU, no GPU

Show HN: Running Gemma-4 26B at 124 tokens/SEC on a CPU, no GPU

Running Gemma-4 26B at 124 tokens/SEC on a CPU, no GPU is a business in show hn launch.

Profile

I wanted to know how fast a 26B mixture-of-experts model could run on a desktop CPU with no GPU. Got ~40 tok/s single-stream (lossless) and ~124 batched. The surprising part was the byte budget: for this model you compress the output head (32% of per-token bytes), not the experts (16%). The writeup has the bandwidth roofline and the dead-ends; the repo has the reproducible recipe. Happy to answer questions.Repo: https://github.com/arun-prasath2005/gemma4-cpu-moe

As a business in show hn launch, Running Gemma-4 26B at 124 tokens/SEC on a CPU, no GPU is listed here with the public details available about it.

It is recorded at the Show HN launch stage.

Website: apeg.dev/writing/running-gemma4-26b-on-a-cpu