About

I run open-weight AI models on cheap hardware and write down what actually happens.

That is the whole idea here. I have a couple of Nvidia GPU's for training and a thin, fan-less laptop with a Snapdragon X Plus inside it, where CPU, GPU, and NPU share a single memory bus, and I want to know what these models really do on a machine like that. So I measure. I run things through stacks like llama.cpp, watch where each phase of inference lands, and report the numbers I get.

How it works

I pick a model and a stack, then I measure where each phase of inference belongs across the chip. The benchmarks are meant to be reproducible: I tell you the hardware, the settings, and how I timed things, and most posts ship with open code so you can rerun the same test and check me.

I also state the catch. An NPU usually buys you efficiency and fit, and raw speed is a separate question, so I will tell you plainly where the hardware loses and what I could not verify. When a model loses, the post says it loses.

You will also see me poke at newer efficient designs, like state-space and constant-state models such as Mamba and Gated DeltaNet, to see what they actually buy on a small device.

If you like measured, traceable, slightly stubborn write-ups about real on-device AI, follow along. I would be glad to have you reading.