── CONTEXT COMPRESSION FOR CODING AGENTS · 2.7 GB · LOCAL · OPEN WEIGHTS ──
A small, local model that compresses a coding-agent session into structured state an agent can resume from. An agent that has worked for an hour carries an enormous, mostly stale history — build logs, abandoned approaches, diffs superseded twenty turns ago — and every new turn re-reads it. Viscer is the compression step: transcript in, structured state out. Exact paths, symbols, commands, and error signatures survive; unresolved work stays unresolved.
The output is not for a human to read — it is for a machine to act on. A summary that reads well but loses a file path, an error signature, or the distinction between resolved and unresolved work is worse than useless: the agent will act on it confidently. Everything Viscer emits is judged by whether the work can be picked back up.
Most compression is judged by how a summary reads. Viscer is judged by whether an agent can pick the work back up: every measurement in the project is ledger-grounded and exact-match, not vibes.
File paths, symbol names, commands, versions, and error signatures are preserved character-for-character — a median of 37 verbatim anchors per held-out session, against 14 for the few-shot base model.
~184 tok/s on one consumer GPU in the deployment configuration — a 167K-token session compresses end-to-end in about 90 seconds, locally.
2.7 GB quantised. No API, no data leaving the machine. Compression runs every turn — a local 4B beats a round-trip to a datacentre.
Apache-2.0. The technical report covers model selection, synthetic data, training, and quantization — including five measurement failures that shaped the result. Read the paper ///
Everything published is measured and reproducible, including the parts that did not work. Schema validity is 12/16, recall is ~0.71 median, and the limitations are stated plainly on this page.
Input: a 78 KB coding-agent transcript — a GCC migration session the model has never seen in training. Output: this state object, in ~20 seconds. Trimmed for length; the full object is 11 KB.
{
"handling": "compressed",
"handling_reason": "cpp build session with actionable work",
"objective": "Migrate libtelemetry from GCC 11.4 to GCC 12 while preserving functionality, fixing link/RPATH issues, resolving ODR violations, and validating cross-compilation.",
"hard_constraints": [
"Do not enable -ffast-math anywhere.",
"Do not silence AddressSanitizer ODR checks with detect_odr_violation=0."
],
"current_state": [
"Baseline clean tree at tag v2.4.1 built successfully on GCC 11.4: 312/312 unit tests pass, libtelemetry.a = 51065632 bytes (48.7 MB)",
"Shared library target telemetry_shared added with -fPIC; SONAME libtelemetry.so.2, RPATH $ORIGIN",
"ASan ODR violation resolved by removing out-of-line FrameHeader::kMagic definition from src/codec/frame_v2.cpp line 23",
"Ch9 ASan odr-violation (0x000000c1a2b8) for telemetry::codec::FrameHeader::kMagic remains unresolved",
"Ch10 aarch64 cross-link failure unresolved: cannot find -lstdc++"
],
"errors": [
{"exact_signature": "undefined reference to `telemetry::codec::detail::zigzag_decode(unsigned long)'",
"hypotheses": ["zigzag_decode defined in .cpp with external linkage but no declaration visible to other TUs"],
"resolved": true},
{"exact_signature": "… (8 more)", "resolved": false}
],
"open_tasks": ["…", "…", "…"],
"rejected_approaches": ["…", "…"],
"verbatim_anchors": [
"/src/telemetry/CMakeLists.txt",
"/src/telemetry/include/telemetry/codec/detail/zigzag.hpp",
"/src/telemetry/build-baseline/libtelemetry.a",
"… (34 more exact literals)"
]
}
Note what is in there: file paths, symbol names, byte counts, an ASan violation address, and which of two similar errors is resolved and which is not. And what is not in there: no invented facts, and unresolved work left unresolved.
16 held-out sessions across 14 domains, never seen in training. Deployment configuration (UD-Q4_K_XL, MTP depth 2), single seed. Every number below is measured and reproducible from the technical report.
| Metric | Viscer-4B | Base Qwen3.5-4B (few-shot) |
|---|---|---|
| parseable output | 15/16 (94%) | 14/16 (88%) |
| critical-literal recall — median | 0.712 | 0.337 |
| critical-literal recall — mean | 0.656 | 0.311 |
| verbatim anchors — median | 37 | 14 |
| state delivered — median tokens | 2,924 | 1,172 |
| decode speed | 184 tok/s | 181 tok/s |
Long context (same model, one RTX 5090): a 144K-token session compresses validly at 0.514 recall in 29 s of prompt processing; a 167K-token session at 0.280 — against 0.032 for the base model at that length. No OOM, 12.5 GB peak. Speed: ~90 seconds end-to-end to compress a 167K-token session locally.
Median recall 0.712 means roughly one in four designated literals is lost on a typical session. Treat the output as a high-quality index, not a lossless record — see limitations below.
Any GGUF runtime works. With llama.cpp:
llama-cli -m viscer-4b-v0.1-UD-Q4_K_XL.gguf \
-f your_session_prompt.txt -n 8192 -c 32768 -ngl 99 --temp 0 \
--repeat-penalty 1.1 --repeat-last-n 256 --jinja -st --simple-io \
--chat-template-kwargs '{"enable_thinking": false}'
Three settings are required, not stylistic: the repeat-penalty pair
(without it the model can loop without closing the JSON), thinking disabled, and an
output budget matched to ~5× compression — a 12K-token session needs 4–6K tokens of
output. Sizing context to the session rather than the model's 262K maximum roughly
doubles prefill speed. The prompt format — system rules plus the transcript between
documented markers — is in PROMPTS.md in the model repository.
| File | Size | When to use it |
|---|---|---|
viscer-4b-v0.1-UD-Q4_K_XL.gguf |
2.68 GiB | default — dynamic quant, MTP head kept at Q8_0 |
viscer-4b-v0.1-Q4_K_M.gguf |
2.59 GiB | uniform quant, for comparison |
viscer-4b-v0.1-f16.gguf |
8.07 GiB | reference precision |
imatrix.gguf + quant-recipe.txt |
— | reproduce the quantization exactly |
All files ship from
huggingface.co/TheWickedGustav/viscer-4b ↗
together with the model card, license, and CITATION.cff.
How it was made, in short:
The remaining failures are JSON syntax slips — a missing colon inside otherwise complete output — not semantic errors. Retry on parse failure if you need guaranteed validity.
Roughly one in four designated literals is lost on a typical session. Treat the output as a high-quality index, not a lossless record, and keep the transcript until the agent confirms the state.
Generated by a larger model with programmatically verified fact ledgers; there has been no independent human audit. Unusual domains may degrade.
Literal-complete compression saturates there; 16× at full fidelity is not achievable by this model.
+18% at depth 2 — the draft head is not fine-tuned. Two attempts to retrain it in isolation made it worse; both are documented in the report.
Is it a summariser?
No. It is built for a machine to resume from — exact literals preserved, unresolved work kept unresolved, superseded decisions marked rather than deleted.
Is it available now?
Yes — v0.1 is out. Weights, model card, corpus and technical report are live at huggingface.co/TheWickedGustav/viscer-4b ↗.
What will I need to run it?
A GGUF-compatible runtime (llama.cpp, LM Studio, Ollama). The default quant is 2.68 GiB; measured peak was 12.5 GB of VRAM at 167K context. Shorter sessions fit smaller cards.
Why a 4B model instead of an API call?
Compression runs every turn. A local model at ~184 tok/s beats a round-trip, and nothing leaves the machine.
Does it work on any session?
It is trained on coding-agent sessions specifically.
Non-coding text is out of scope (the schema has a refused state for
it).
How is this different from just truncating the history?
Truncation loses the objective, the constraints, and any error signature older than the window. Viscer preserves exactly those.
Can I fine-tune it further?
Yes — Apache-2.0, and the training recipe, corpus and tooling are published alongside the weights.
YOUR AGENT SPENT 160K TOKENS GETTING HERE. VISCER KEEPS THE PART THAT STILL MATTERS.