How to compress an AI to 2 bits: the trellis
A model is made of billions of numbers. To fit one on a graphics card you have to write them with fewer bits. The trellis is one of the cleverest ways: instead of storing the values, it stores a path to find them again.
A number in 16 bits
Take one of the model's weights, say 0.734. The computer writes it in fp16: sixteen on/off switches. The first gives the sign, the next five the scale (a power of two), the last ten the factor that multiplies it, a number between 1 and 2. Change the number and watch the bits.
- Scale
- 0.5
- Factor
- × 1.4678
- Stored value
- 0.733887
- Error
- 0.00011
With 16 bits you are off by about one ten-thousandth. But a model has billions of weights, and at 16 bits each it doesn't fit in a graphics card's memory. So we use two. For simplicity the weights of a block are divided by a common scale and sit between −2 and 2: here the scale is 1. Press "Reduce to 2 bits".
Why one weight at a time is not enough
At 2 bits every weight has only four possible values and goes to the nearest. That's deciding each weight on its own: two weights together land on a rigid grid of sixteen points. But real weights sit mostly near zero, so the points in the corners are almost never used.
Better to put the points where the weights are and write them in a list, a codebook: for each group of weights you only store which row to use. The more weights you take together, the smaller the error. The problem is that the list grows far too fast.
| Weights together | Bits per weight | Codebook rows |
|---|---|---|
| 2 | 2 | 16 |
| 8 | 2 | 65,536 |
| 8 | 3 | 16,777,216 |
| 256 | 2 | a 155-digit number |
A list like that can't be written down. The trellis does it without writing it.
The trellis
Start with the simplest case: one bit per weight. The trellis reads the bits one after another and remembers the last three: that's the state, and with three bits there are eight. A fixed formula assigns each state a value, known to both the one compressing and the one reading. The values look random, but they are distributed like the real weights: many near zero, few large.
Two arrows leave each state, because a bit is 0 or 1. Trellis means lattice-work in English. Choosing the bits means choosing a path, and the values along the path are the weights. Try it: the four weights are 0.734, −1.2, 0.3 and 1.5. Change the bits and see where the path takes you.
Chosen bits
| Weight | Value | Error² |
|---|---|---|
| +0.73 | −0.42 | 1.332 |
| −1.2 | −0.42 | 0.608 |
| +0.3 | −0.42 | 0.518 |
| +1.5 | −0.42 | 3.686 |
| Total | 6.145 | |
Rounding one weight at a time (1 bit: ±0.8)
+0.8 −0.8 +0.8 +0.8 → total 0.904
Each weight goes to the nearest of −0.8 and +0.8, knowing nothing about the others.
Each bit chooses the current weight and also where the next one starts from: that's why weights aren't chosen one at a time but all together. "Find the best path" tries all sixteen. In the example, with a single bit per weight, the best path is off by 0.584 against 0.904 for classic rounding.
In the real trellis the numbers are bigger: QTIP uses 16 bits of state (65,536 states), blocks of 256 weights and values computed by a formula of a few instructions, which the graphics card recomputes while the model answers. To find the best path without trying them all there is the Viterbi algorithm: for each state it keeps only the best way to get there, because from there on what matters is where you are, not how you got there.
The real numbers
Two measurements, from two different sources. The first is from the QTIP paper: the rounding error at 2 bits on test numbers.
Mean squared error at 2 bits (QTIP, table 1)
| Method | Error |
|---|---|
| One number at a time (Lloyd-Max) | 0.118 |
| E8 lattice | 0.089 |
| Trellis | 0.069 |
| Theoretical limit | 0.063 |
From 0.118 to 0.069: 42% less, one step from the theoretical limit.
Qwen 3.6 35B-A3B, with the EXL3 variant at 2.08 bits per weight
| Size | Perplexity | |
|---|---|---|
| bf16 | 66.97 GiB | 6.7802 |
| EXL3 2.08 bit | 10.50 GiB | 7.2418 (+6.8%) |
Perplexity measures the model's uncertainty about the next word. EXL3 is a slimmed-down variant of QTIP, as the exllamav3 README says.
The trellis inside Mango Code
TOQ, Mango Code's model, is Qwen3.8-27B compressed with a trellis from the same family as QTIP and EXL3: 64 layers at 2 bits per weight, and the layer that picks the word at 3. On the graphics card the weights take 6.17 GiB, plus 176 MiB for a correction learned from the original model; an 8 GB card holds the weights and about 14,000 tokens of context. The trellis state is 14 bits instead of 16. On the multiple-choice questions of the MMLU test it answers 77.8% right; the same model at 4 bits, a 13.3 GiB file, 82.6%.
Go to Mango CodeWhat is not verified
- The 42% is about rounding error on test numbers, not the quality of an AI's answers.
- That one weight at a time isn't enough at 2 bits is measured on some methods, not all: in the OmniQuant paper Llama-2-7B goes from 5.47 perplexity to about 4,200 with plain rounding, to 36.77 with GPTQ and to 11.06 with OmniQuant.
- In the example to try, the values of the eight states are there to show the idea: they are not QTIP's values.
- TOQ hasn't been measured as an agent: MMLU is a multiple-choice test and doesn't say whether the model finishes a job that takes several steps.