Row 3 of VERIFICATION.md. With no CUDA, core/gpu_tier.classify() returns NONE and the box runs one model on CPU, no draft, no speculative decoding (running both on CPU costs more than the draft saves). We say that path is usable. Nobody has measured it on a Pi.
To settle it: tokens/sec for the 2B on a Pi 4 8GB, plus which quantisation and how long the first token took.
If it is unusably slow, that is the answer and it goes in the README.
Row 3 of VERIFICATION.md. With no CUDA,
core/gpu_tier.classify()returns NONE and the box runs one model on CPU, no draft, no speculative decoding (running both on CPU costs more than the draft saves). We say that path is usable. Nobody has measured it on a Pi.To settle it: tokens/sec for the 2B on a Pi 4 8GB, plus which quantisation and how long the first token took.
If it is unusably slow, that is the answer and it goes in the README.