G-Bench

── PERFORMANCE AND CAPABILITY BENCHMARK FOR LOCAL AND HOSTED MODELS ──

G-Bench puts the same four tests to every model: how fast it answers, whether it can find one fact buried in a quarter-million tokens of context, whether it can finish a multi-step job with tools, and whether it answers from the source or from memory. The four results fold into a single score out of 100. Self-hosted stacks and hosted endpoints are measured with the same prompts and the same scale — a run on a GPU in the next room is directly comparable to a run against an API.

// SCOREBOARD

The composite G-Bench score: all four tests weighted and added up to a single number out of 100. Taller is better. Only runs that completed the full suite are ranked here.

RUN_001–003 measured serving speed only (decode, prefill, cache) and carry no composite score, so they are not ranked on this chart.

// BENCH RUNS

NEW RUNS ARE APPENDED AT THE TOP. LATEST RESULTS FIRST.

RUN_005 · 2026-09-16 · LATEST

Union Alpha

stealth/union-alpha · OpenRouter (Stealth provider) · 262k advertised ctx

score: 91.30 / 100
e2e tok/s: 17.6 @131k
needle recall: 0.97

OPEN REPORT ///

RUN_004 · 2026-09-15

Ornith-1.5-35B

Ornith-1.5-35B-A3B · UD-Q4_K_S · llama.cpp :8082 · 262k ctx

decode c1: 174.7 tok/s
aggregate: 524.4 tok/s @c3
score: 76.55 / 100

OPEN REPORT ///

RUN_003 · 2026-09-14

Qwen3.8-27B

Qwen3.8-27B-NVFP4 · Gustav's Recipe (DFlash2) · RTX 5090

decode c1: 56.5 tok/s
aggregate: 54.9 tok/s
cache ×: 20.9×

OPEN REPORT ///

RUN_002 · 2026-09-14

DeepSeek V4.1 Flash

DeepSeek-v4.1-Flash-EXL3 · MiaAI Lab TR3 4bpw · 2× DGX Spark

decode c1: 29.3 tok/s
aggregate: 41.7 tok/s @c2
cache ×: 0.99×

OPEN REPORT ///

RUN_001 · 2026-09-14

GLM 5.3 Flash

GLM-5.3-Flash-EXL3 · MiaAI Lab TR3 4bpw · 2× DGX Spark

decode c1: 22.8 tok/s
aggregate: 31.5 tok/s @c4
cache ×: 16.9×

OPEN REPORT ///

// METHOD

Four tests, each asking a different question of the same model. Every test is graded on a fixed scale and weighted, and the four results add up to one G-Bench score out of 100. The same prompts and scenarios are replayed for every model, so a difference in the score is a difference in the model — not in the test.

01 · WEIGHT 25%

SPEED

Can it answer fast enough to use? A model that is right but slow is still unusable in a live pipeline. This test times how long it takes to start answering when the prompt is a paragraph and when it is a whole document, how many words per second it produces once it gets going, and what happens when several requests arrive at once. Sending the same prompt twice also shows whether the serving stack remembers its own work instead of starting over.

02 · WEIGHT 30%

LONG CONTEXT

Can it find one sentence in a very large pile? A big context window is only real if the model can actually read it. One specific line is hidden inside documents that grow from a few thousand words up to a quarter of a million tokens, placed near the start, the middle and the very end, and the model is asked to quote it back. Some versions plant convincing but wrong lines nearby, to see whether it reports the fact it was given or the one that merely looks right.

03 · WEIGHT 25%

TOOL USE

Can it finish a real job, not just describe one? The model is given a small set of tools and a task too big for one step: it has to look something up, use that answer in the next call, and arrive at a result. This test watches whether it picks the right tool, passes the right arguments, works through the whole chain — and avoids inventing tools or extra calls nobody asked for.

04 · WEIGHT 20%

GROUNDING

Does it answer from the source, or from memory? The model is handed a long, deliberately obscure reference text and asked questions that can only be answered from it. The same questions are then asked with no reference at all. A model that scores just as well without the source material was answering from what it memorized in training — the quiet failure that makes an answer sound authoritative and be wrong.

Answers are checked against fixed expected results; free-form answers are read and graded by a judge model. A run is only comparable if it is asked exactly what the previous runs were asked, so the prompts never change between models.