One 0.5B model proposes a block of tokens past the frontier and verifies every one of them exactly. Step through a run and watch where each accepted token came from — and what happens when the verifier says no.
sbd_generate call on belumind/sbd-sql-0.5b,
executed on an A100 in bf16 and logged forward pass by forward pass; the tracer was checked
against the shipped decoder and reproduces it token for token. The live version runs in Colab —
demo/demo_colab.ipynb.
| Prompt | Sources | τ | tok/fwd | forwards | rollbacks | MATP | n-gram |
|---|
Single runs scatter. Averaged results over 20 held-out documents, and the corpus comparison, are on the model card, together with what the fine-tune costs in generation quality.
Every forward pass proposes and verifies at the same time. The block of 8 mask tokens appended after the frontier gives the MATP guesses; the same pass recomputes the true conditionals for the tokens proposed last round, which is what verification costs — nothing extra.
The first proposed token is always exact. It is the ordinary next-token prediction at a purely causal frontier, so the decoder can never fall below one token per forward.
A rollback is not an error. It is the verifier refusing a guess. Output stays identical to greedy decoding — measured at 100% agreement in bf16 and fp32 — and τ only trades rollbacks against speed. Loosen it and you will see more crossed-out tokens here, and a higher tok/fwd anyway.
tokens/forward is not wall-clock. This reference decoder keeps no
KV cache and recomputes the whole sequence every step. It still beat
generate(use_cache=True) by 1.58× end to end on the same A100 — a
transformers baseline, not an optimised server.