01Recipe notes
Calibrated DFlash2 draft model (NVFP4 GGUF, Q8_0 & Q4_K_M variants) with n_draft=7 and p_min=0.75 — more accepted draft tokens per verification step, which is where most of the decode speed comes from.
Target: unsloth Qwen3.8-27B UD-Q4_K_XL GGUF (17.9 GB), all layers on GPU, F16 KV cache, FA auto
256K ctx configured (16K active slot).
Perceived stability on longer reasoning and large context has been good so far — single-agent serving profile, no concurrency.
02Decode throughput
Per-request decode rate and aggregate throughput at each concurrency level. Output capped at 384 tokens, temperature 0, thinking disabled.
single-stream decode
56.47 tok/s
c1 · mean of 3 runs
best aggregate throughput
54.86 tok/s
c1 · all batches
decode budget
384 tok
per request, temp 0
| concurrency | per-req decode | min – max | ttft mean | aggregate |
|---|---|---|---|---|
| c13 reqs · 3 batch(es) | 56.47 tok/s | 54.37 – 58.50 | 210.1 ms | 54.86 tok/s |
03Prefill & TTFT by prompt size
Time to first token as the prompt grows. Cold = first request for that size; warm = repeated identical prompt (prefix-cached prefill).
| prompt size | ttft cold | ttft warm | prefill rate |
|---|---|---|---|
| ~256actual 192 tokens | 1,638.9 ms | 2,025.4 ms | 95 tok/s |
| ~1,000actual 719 tokens | 437.4 ms | 131.7 ms | 5,459 tok/s |
| ~4,000actual 2,765 tokens | 1,270.0 ms | 124.6 ms | 22,191 tok/s |
| ~8,000actual 5,691 tokens | 2,815.5 ms | 149.2 ms | 38,143 tok/s |
| ~12,000actual 8,531 tokens | 4,349.0 ms | 140.6 ms | 60,676 tok/s |
04Prefix cache effect
Identical ~8000-token prompt sent repeatedly. The ratio shows how much the serving stack saves on cache hits.
ttft cold
2,661.5 ms
first request, ~5,691 tokens
ttft repeated
127.1 ms
identical prompt, prefix cache hit
cache speedup
20.94×
cold / repeated TTFT