Natural language tends to contain quite a bit of repetition and redundancy.1 This isn’t inherently bad; indeed, it’s quite useful for making language robust to noise and the general realities of communication. However, redundancy always comes with tradeoffs, and in the era of language models, redundancy costs tokens. When working with substantial knowledge content in agent sessions, repetition takes up space in context windows that might be better spent on documentation about other topics. That raises the question: Can documentation be made less repetitive? Can documentation be compressed?
To explore this, we’ve prototyped a system called Knowledge Compressor, which accepts articles as input and iteratively reduces their token counts while ensuring the knowledge in the source2 remains usable. The takeaway is that typical technical documentation can have its token count cut in half without substantially reducing its usefulness to language models. Here’s a quick before-and-after glance at one such evaluation article, cut from 996 tokens to 480:
Ferry Rotation in the Ferryline Build Cache
Ferryline is a distributed build cache that stores compiled artifacts keyed by a hash of their inputs. Each stored artifact is written to exactly three storage nodes, chosen by rendezvous hashing over the artifact key. A read succeeds as soon as any one replica responds, so the cache stays available when a node is draining or restarting.
The cache is divided into two tiers. The hot tier holds artifacts requested at least twice in the preceding six hours and is backed entirely by NVMe. The cold tier holds everything else on spinning disks. Promotion between tiers happens asynchronously: a background sweeper records access counts in a ring buffer and promotes an artifact only after the count crosses the threshold twice in separate windows, which prevents a single burst of parallel builds from flooding the hot tier.
Ferryline does not evict by least-recently-used order. Instead it evicts by projected rebuild cost, which is the product of an artifact's measured compilation time and its access frequency. An artifact that takes four minutes to rebuild is retained over one that takes eight seconds, even when the cheap artifact was touched more recently. Eviction runs whenever a tier exceeds eighty-five percent capacity and stops at seventy percent, and this gap exists so that eviction does not run continuously near the threshold.
Writes use a protocol called ferry rotation. A client uploads an artifact to a single node, designated the ferry, which acknowledges the write immediately and then replicates to the other two nodes in the background. The ferry role rotates per artifact rather than per node, so no single machine absorbs all inbound traffic for a popular key. If the ferry fails before replication completes, the write is lost and the client rebuilds; Ferryline deliberately accepts this because a lost cache entry costs one rebuild, while synchronous replication would add latency to every write.
Consistency is scoped to the key. Because keys are content hashes, two clients that produce the same artifact produce the same key, and a conflicting write is therefore byte-identical to the value already stored. Ferryline exploits this by skipping conflict resolution entirely: a write to an existing key is discarded without comparing contents. This assumption breaks for non-deterministic compilers, which is why Ferryline refuses to cache any target whose build rule declares itself non-hermetic.
Garbage collection is generational. Artifacts are stamped with the build graph revision that produced them, and revisions older than the fiftieth most recent are deleted wholesale rather than traversed. The team chose the number fifty after observing that bisecting a regression rarely reaches further back, and that traversal-based collection spent more time walking the graph than it recovered in space.
Membership changes are deliberately lazy. The controller publishes a numbered membership epoch whenever nodes join or leave, but artifacts already on disk are not copied immediately to match the new rendezvous-hash placement. A read first checks the current three owners and then, on a miss, checks the owners from the preceding two epochs. Finding an artifact on an old owner schedules a background copy to the current owners. Operators may drain a node only after it has remained in read-only mode for two complete membership epochs; removing it earlier can strand cold artifacts that have not been read and migrated.
Integrity repair is separate from membership migration. Every artifact carries a BLAKE3 digest, and a scrubber verifies one percent of each node's resident bytes per hour. When one replica is corrupt, the scrubber copies a valid replica over it. When all available replicas disagree with the digest, the key is deleted and the next requesting client rebuilds it. Repair traffic is capped at five percent of a node's outbound bandwidth so that a damaged disk cannot displace normal cache reads.
Clients also enforce admission limits before ferry rotation begins. Artifacts larger than two gigabytes are never uploaded, and each tenant may have at most two hundred pending ferry writes. At the limit, the client continues the build without caching the artifact rather than waiting. These limits protect the shared service from oversized debug bundles and from retry storms after a large worker pool reconnects.
Ferryline exposes two metrics that operators are told to watch together. Hit rate alone is misleading, because a cache that stores only trivially cheap artifacts can report a high hit rate while saving no time. The second metric, saved seconds per gigabyte, divides total avoided compilation time by bytes resident, and a healthy deployment holds it above one hundred. When saved seconds per gigabyte falls while hit rate stays flat, the usual cause is a build rule that began emitting large artifacts that were cheap to produce.
Content compression is nothing new, of course. Compressing knowledge for language models is broadly similar to compressing a WAV audio file into an MP3 or a PNG image into a JPEG. However, those processes aim to maximize data reduction while minimizing human-perceptible degradation. In this case, we want to minimize LLM-perceptible degradation — but what do language models perceive, exactly? We could ask them, but there’s a good chance they’ll just make something up. Instead, we can test them.
Open-book quizzes
Roughly speaking, the purpose of putting knowledge content into language model context windows is to allow models to answer questions that aren’t covered by their inherent world knowledge. One way to test the fidelity of documentation compression, then, is to see whether language models can accurately answer questions using the compressed content. To this end, the system starts by extracting a set of questions and answers from the input content. It can adaptively choose the number of questions based on the content’s length and density, or the number can be set manually. These extracted questions can be thought of as spiritually similar to unit tests.
Next, the system ensures that every extracted question can, in fact, be answered — each in a fresh context window — based on the original article. Questions that can’t actually be answered are filtered out. This testing system is admittedly not perfect: Even if a question is not strictly answerable from the content, a language model might nonetheless respond using its built-in world knowledge, or even just guess. To mitigate this, we instruct the model to use only the context provided — a helpful, though incomplete, technique — and focus our evaluation runs on facts that can’t possibly be in current models’ world knowledge, either because the information is newer than the training cutoff or because it’s entirely synthetic.3
Verbosity descent
Once the test questions are confirmed, the system begins the agentic compression process. An agent receives instructions and technical advice, along with tools for reading the original content, counting tokens, applying diffs, and running test questions. It iterates on the content until it concludes that no further compression is possible, then exits. The system can then review the run against various completion criteria and decide whether to send the draft back for further revision.
One risk with this agentic process is that the agent might “play it safe” and barely compress the content at all. To guard against this, the system can require at least one test question to fail before the run completes. If the compression agent hasn’t shortened the source article enough to lose at least some meaningful content, it likely hasn’t reached the lower bound. When a question does fail, that result is fed back to the agent, which can restore the necessary material to get the tests passing again.
Token accounting
Compressing documentation lets us include more information in a single context window, which is great for increasing the knowledge available in a particular session. Beyond that, though, compressing documentation also reduces the recurring cost of including the content, even when context space isn’t saturated. However, the compression process comes with its own cost: the system consumes tokens generating questions, testing drafts, and iteratively producing the final shortened article. This lets us ask a concrete question after each run: How many times would the compressed content need to be used before its savings repay the compression cost?
For the evaluation run shown above, compressing the 996-token source to 480 tokens cost about $2. At current pricing for the same model,4 each uncached inclusion saves about $0.001, putting the break-even point at roughly 2,000 uses. The exact result depends on model pricing, caching, source material, and the agent’s decisions, of course — but for knowledge that is expected to be needed repeatedly, the compression can effectively pay for itself.
Read it in action
When you stack it all together, it looks something like this5:
Draft
Ferry Rotation in the Ferryline Build Cache
Ferryline is a distributed build cache that stores compiled artifacts keyed by a hash of their inputs. Each stored artifact is written to exactly three storage nodes, chosen by rendezvous hashing over the artifact key. A read succeeds as soon as any one replica responds, so the cache stays available when a node is draining or restarting.
The cache is divided into two tiers. The hot tier holds artifacts requested at least twice in the preceding six hours and is backed entirely by NVMe. The cold tier holds everything else on spinning disks. Promotion between tiers happens asynchronously: a background sweeper records access counts in a ring buffer and promotes an artifact only after the count crosses the threshold twice in separate windows, which prevents a single burst of parallel builds from flooding the hot tier.
Ferryline does not evict by least-recently-used order. Instead it evicts by projected rebuild cost, which is the product of an artifact's measured compilation time and its access frequency. An artifact that takes four minutes to rebuild is retained over one that takes eight seconds, even when the cheap artifact was touched more recently. Eviction runs whenever a tier exceeds eighty-five percent capacity and stops at seventy percent, and this gap exists so that eviction does not run continuously near the threshold.
Writes use a protocol called ferry rotation. A client uploads an artifact to a single node, designated the ferry, which acknowledges the write immediately and then replicates to the other two nodes in the background. The ferry role rotates per artifact rather than per node, so no single machine absorbs all inbound traffic for a popular key. If the ferry fails before replication completes, the write is lost and the client rebuilds; Ferryline deliberately accepts this because a lost cache entry costs one rebuild, while synchronous replication would add latency to every write.
Consistency is scoped to the key. Because keys are content hashes, two clients that produce the same artifact produce the same key, and a conflicting write is therefore byte-identical to the value already stored. Ferryline exploits this by skipping conflict resolution entirely: a write to an existing key is discarded without comparing contents. This assumption breaks for non-deterministic compilers, which is why Ferryline refuses to cache any target whose build rule declares itself non-hermetic.
Garbage collection is generational. Artifacts are stamped with the build graph revision that produced them, and revisions older than the fiftieth most recent are deleted wholesale rather than traversed. The team chose the number fifty after observing that bisecting a regression rarely reaches further back, and that traversal-based collection spent more time walking the graph than it recovered in space.
Membership changes are deliberately lazy. The controller publishes a numbered membership epoch whenever nodes join or leave, but artifacts already on disk are not copied immediately to match the new rendezvous-hash placement. A read first checks the current three owners and then, on a miss, checks the owners from the preceding two epochs. Finding an artifact on an old owner schedules a background copy to the current owners. Operators may drain a node only after it has remained in read-only mode for two complete membership epochs; removing it earlier can strand cold artifacts that have not been read and migrated.
Integrity repair is separate from membership migration. Every artifact carries a BLAKE3 digest, and a scrubber verifies one percent of each node's resident bytes per hour. When one replica is corrupt, the scrubber copies a valid replica over it. When all available replicas disagree with the digest, the key is deleted and the next requesting client rebuilds it. Repair traffic is capped at five percent of a node's outbound bandwidth so that a damaged disk cannot displace normal cache reads.
Clients also enforce admission limits before ferry rotation begins. Artifacts larger than two gigabytes are never uploaded, and each tenant may have at most two hundred pending ferry writes. At the limit, the client continues the build without caching the artifact rather than waiting. These limits protect the shared service from oversized debug bundles and from retry storms after a large worker pool reconnects.
Ferryline exposes two metrics that operators are told to watch together. Hit rate alone is misleading, because a cache that stores only trivially cheap artifacts can report a high hit rate while saving no time. The second metric, saved seconds per gigabyte, divides total avoided compilation time by bytes resident, and a healthy deployment holds it above one hundred. When saved seconds per gigabyte falls while hit rate stays flat, the usual cause is a build rule that began emitting large artifacts that were cheap to produce.
Questions
0/24 tested
…How are artifact replicas placed, and what is required for a read to succeed?
Expected: Each artifact is stored on exactly three nodes selected by rendezvous hashing over its key, and a read succeeds when any one replica responds.
…Which artifacts belong in each cache tier, and what storage backs the tiers?
Expected: The hot tier contains artifacts requested at least twice in the preceding six hours and uses NVMe; all other artifacts are in the cold tier on spinning disks.
…How does asynchronous promotion prevent a burst of parallel builds from flooding the hot tier?
Expected: A sweeper tracks access counts in a ring buffer and promotes an artifact only after its count crosses the threshold in two separate windows, so one burst is insufficient.
…How does Ferryline prioritize artifacts for eviction instead of using LRU order?
Expected: It uses projected rebuild cost—measured compilation time multiplied by access frequency—and preferentially retains higher-cost artifacts. Thus a four-minute artifact can be retained over a more recently used eight-second artifact.
…When does tier eviction start and stop, and why are those thresholds separated?
Expected: Eviction starts above 85% capacity and stops at 70%. The gap prevents eviction from running continuously near a single threshold.
…What sequence occurs during a ferry-rotation write?
Expected: The client uploads to one designated ferry node, which acknowledges immediately and then replicates the artifact to the other two nodes in the background.
…How is the ferry role assigned, and what traffic problem is this intended to avoid?
Expected: The ferry role rotates per artifact rather than being fixed per node, preventing one machine from absorbing all inbound traffic.
…What happens if a ferry fails before replication finishes, and why does Ferryline accept that outcome?
Expected: The write is lost and the client rebuilds the artifact. Ferryline accepts one possible rebuild rather than imposing synchronous-replication latency on every write.
…Why are writes by two clients for the same content key expected not to conflict semantically?
Expected: Keys are content hashes, so clients producing the same artifact generate the same key, and their values should be byte-identical.
…How does Ferryline handle a write to a key that already exists?
Expected: It discards the new write without comparing its contents or performing conflict resolution.
…Which build targets does Ferryline refuse to cache, and what assumption makes that restriction necessary?
Expected: It refuses targets whose build rules declare them non-hermetic because non-deterministic compilers can violate the assumption that a content key always maps to byte-identical output.
…How does generational garbage collection decide what to delete?
Expected: Artifacts are stamped with their producing build-graph revision, and those older than the fiftieth most recent revision are deleted wholesale rather than found by graph traversal.
…Why did the team choose its revision-retention policy instead of traversal-based collection?
Expected: Regression bisects rarely need revisions farther back, while traversal-based collection spent more time walking the graph than the space it recovered justified.
…What happens to existing artifacts when storage membership changes?
Expected: The controller publishes a numbered membership epoch, but existing artifacts are not immediately copied to their new rendezvous-hash owners.
…How does a read locate and migrate an artifact whose placement is stale?
Expected: It checks the three current owners first, then owners from the preceding two epochs. If found on an old owner, it schedules a background copy to the current owners.
…What condition must be met before operators drain a node, and what can happen if it is removed sooner?
Expected: The node must remain read-only for two complete membership epochs. Earlier removal can strand unread cold artifacts that have not yet migrated.
…How are stored artifacts checked for corruption?
Expected: Every artifact has a BLAKE3 digest, and a scrubber verifies 1% of each node's resident bytes per hour.
…How does integrity repair respond to a single corrupt replica versus no available replica matching the digest?
Expected: A single corrupt replica is overwritten from a valid one. If all available replicas disagree with the digest, the key is deleted and rebuilt by the next requesting client.
…How is repair traffic limited, and why?
Expected: It is capped at 5% of a node's outbound bandwidth so repair from a damaged disk cannot displace ordinary cache reads.
…What size and concurrency limits do clients apply before beginning ferry rotation?
Expected: Artifacts larger than 2 GB are never uploaded, and each tenant may have at most 200 pending ferry writes.
…What does a client do upon reaching the pending-write limit, and what service risks do the admission limits address?
Expected: It continues the build without caching rather than waiting. The limits protect the service from oversized debug bundles and retry storms after large worker pools reconnect.
…Which two metrics should operators monitor together, and why is one of them insufficient alone?
Expected: They should monitor hit rate and saved seconds per gigabyte together. Hit rate alone can look high when the cache contains only cheap-to-rebuild artifacts and therefore saves little time.
…How is saved seconds per gigabyte calculated, and what level indicates a healthy deployment?
Expected: It is total avoided compilation time divided by resident bytes, expressed per gigabyte, and a healthy deployment keeps it above 100.
…What is the usual explanation when saved seconds per gigabyte declines while hit rate remains flat?
Expected: A build rule has usually begun producing large artifacts that are inexpensive to compile.
Activity
Newest first · 2 events
Screening
Discarding questions the model can answer without the article
Extracting
Mapping the knowledge in the article (24 questions)
01Extract the questions
Twenty-four unique test questions are generated based on the source material.
Compressing onward
So, the basic answer seems to be yes: Documentation can be compressed. But this exploration has only scratched the surface, and there are many questions left to answer: How well do different types of source content compress? Do different genres benefit from different compression strategies? Can the compression process itself be made more token-efficient? We’ll just have to keep compressing docs to find out!
