Token compression guide
LLM token compression before inference
LLM token compression removes waste before the model spends compute on it. The goal is to preserve the evidence needed for the answer while cutting everything else.
What is LLM token compression?
LLM token compression reduces the number of tokens sent to a language model during inference while preserving the information needed to answer the current query.
In many RAG queries, much of the retrieved context is irrelevant to the specific question. Those tokens still cost money and add latency.
Token compression methods compared
| Method | Mechanism | Query-aware | Extra LLM call |
|---|---|---|---|
| SuperCompress | Learned query-aware line selection: keeps original lines chosen against the question | Yes | No |
| Summarization | Rewrites context with another model | Varies | Yes |
| Truncation (FIFO) | Keeps head and tail; drops the middle regardless of the question | No | No |
Integrating with your LLM pipeline
curl -X POST https://www.supercompress.dev/api/v1/compress \
-H "X-API-Key: $SUPERCOMPRESS_API_KEY" \
-H "Content-Type: application/json" \
-d '{"context":"...","query":"..."}'Frequently asked questions
Does token compression require a model download?
No. SuperCompress runs on the hosted API, so there is nothing to download or run yourself.
Does it work with local models?
Yes. Compression is model-agnostic and works before any model call.