Test results
Compare speed, quality and configuration across runs on one desktop. The quality tests differ between rows; open Notes for the conditions.
Read the comparisons: graph capture and agent time · running a model across two GPUs.
11 results
Measured results
Select a column heading to sort; select it again to reverse the order. Eight-stream speed is total throughput. Configurations and quality tests differ between runs.
Swipe the table sideways for more columns →
| 2026-10-01 | Two B60s · official FP8 · host-staged evaluationSetup and notesQwen3.8-27B Candidate, not adopted: Qwen's pre-quantized FP8 release vs. our own online (at-load) FP8 of the same BF16 weights. Roughly tied on speed and quality; switch rule (adopt only if it beats production on both LCB and speed within 5%) was not met. |
53 | 227 | 10/10correctness gate |
| 2026-10-01 | Two B60s · AutoRound Int4 · host-staged evaluationSetup and notesQwen3.8-27B Best-scoring int4 candidate found in the int4 quant search (Frozenlock/Qwen3.8-27B-int4-AutoRound). Did not match the FP8 reference on quality at greedy decoding, so it was kept as the int4 rollback reference rather than switched to. |
73 | 317 | 6/9 (vs FP8 7/9) / 10/12 / 10/10LiveCodeBench-9 (greedy) / exact-answer(12) / gate |
| 2026-10-01 | Two B60s · FP8 · vision tower disabledSetup and notesQwen3.8-27B The checkpoint carries an unused(?) vision tower that vLLM profiles and warms up on every start. --language-model-only cut engine start from ~230s to 131s and freed KV capacity; adopted (production start since is ~142s). Whether anything actually sent this model images could not be confirmed (gateway logs bodies, not metadata, only). |
Not measured | Not measured | 226-239s / 396,601 tokens -> 131s / 452,706 tokens (+14%)engine start time / KV pool (vision tower on vs. off) |
| 2026-09-30 | Two B60s · online FP8 · host-staged fallbackSetup and notesQwen3.8-27B Automatic fallback path after a real GPU engine-reset fault tripped the P2P watchdog, which restarts serving on the validated host-staged config. Same checkpoint as the P2P FP8 row below; P2P vs host-staged is the clean comparison here. |
37 | 155 | Not recorded |
| 2026-09-30 | Two B60s · online FP8 · P2PSetup and notesQwen3.8-27B Selected for production on 2026-09-30. Chosen over the int4+int4-head checkpoint above: FP8 solved a strict superset of int4's LiveCodeBench problems (14/24 vs 12/24) at about half int4's tokens/turn, at the cost of slower decode and roughly half the KV pool (395K vs 757K tokens). |
52.9 | 251 | 14/24 / 75 / 9 pass, 0 fail, 1 n/aLiveCodeBench-24 (medium thinking) / websites (/75) / readiness (10 tasks) |
| 2026-09-29 | Two B60s · Int4 weights, head & MTP layer · P2PSetup and notesQwen3.8-27B lm_head (2.54 GB FP16) is read k+1 times per decode step (~40% of bytes moved); quantizing it plus the MTP layer to int4 was the lever left once a graph-capturable 2-rank allreduce (vLLM PR #54768, measured +1.7% by its author) wasn't worth building. 55% of the 456 GB/s spec bandwidth roofline / 63% of the achievable 399 GB/s. |
86.3 | 342 | 156/164 / 240 (n.s. vs 243) / 10/10 / 158HumanEval / GSM8K(250) / gate / exact-recall(160) |
| 2026-09-29 | Two B60s · Int4 · P2P · MTP k=4Setup and notesQwen3.8-27B k=4 vs. production k=3 (70.2 single / 319 at 8 on this same checkpoint). k=3 was kept. k=2 was also tested and rejected (along with full-graph decode and an in-graph drafter) but no speed number was recorded for it in the source. |
67 | 311 | k=4MTP speculative tokens (k) |
| 2026-09-28 | One B60 · Int4 · eagerSetup and notesQwen3.8-27B Single-card baseline. Eager mode; measured at 85% of the card's 29.4 tok/s bandwidth roofline. |
23.9 | 160.6 | 10/10correctness gate |
| 2026-09-28 | Two B60s · Int4 · host-staged · eagerSetup and notesQwen3.8-27B Naive TP=2, eager, no graph capture. Slower than one card: the Alder Lake host bridge (8086:4650) is not on the kernel's p2pdma whitelist, so the allreduce is host-staged and launch overhead on half-size kernels cancels the split. |
13.7 | 102 | Not recorded |
| 2026-09-28 | Two B60s · Int4 · host-staged · graph + MTP k=3Setup and notesQwen3.8-27B PIECEWISE CUDA-graph-style capture with vllm::all_reduce as a split point, plus MTP k=3 with the drafter forced eager (in-graph drafter collapses acceptance to 1.01). Validated run 253-validate-20260928-082853; 900s stress 64/0 fail. |
42.3 | 172.5 | 10/10 / passcorrectness gate / tool call |
| 2026-09-28 | Two B60s · Int4 · P2P · graph + MTP k=3Setup and notesQwen3.8-27B Same checkpoint and graph config as the row above, with GPU-to-GPU P2P enabled via a kernel whitelist override for the Alder Lake host bridge (out-of-tree kretprobe module). Soak run under a concurrent ZFS scrub on the SATA mirror behind the PCH. |
56 | 301.8 | 10/10 / pass / 312 ok, 0 fail (900s)correctness gate / tool call / stress |
Context is the configured --max-model-len. Speeds are
greedy decode tokens/second, 1 stream and 8 concurrent. Public raw logs
for these older rows are not yet available. Reviewed notebook results
are in the JSON feed.