---
schema_version: "1"
id: 50bff54214496c0c4c7c56a7cfb463372e00a23b5530d45abab07a51eb033105
title: Model Proxy Request Compression Policy
collection: mulesoft
version: current
locale: en-us
portal_url: https://docs.salesforce.com/mulesoft/current/en-us/gateway/latest/policies-included-llm-proxy-request-compression.md
canonical_url: https://docs.mulesoft.com/gateway/latest/policies-included-llm-proxy-request-compression
source_site: docs.mulesoft.com
content_type: documentation
content_hash: 75fd02b87797ed6f9443ddacafbf9a02edcf645ed3136c24d32c859775d41ee7
description: MuleSoft Documentation Site
release: "1.14"
latest: true
---
---
product: Omni Gateway
version: 1.14
is-latest-version: true
keywords: api gateway, flex gateway, gateway, policy, llm, model proxy, compression, tokens, toon
---

# Model Proxy Request Compression Policy

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

<table><colgroup><col> <col></colgroup><tbody><tr><td><p><strong>Policy Name</strong></p></td><td><p>Model Proxy Request Compression Policy</p></td></tr><tr><td><p><strong>Summary</strong></p></td><td><p>Compresses message content and replayed tool results in Model Proxy request bodies before they reach the upstream provider, reducing billed tokens</p></td></tr><tr><td><p><strong>Category</strong></p></td><td><p>LLM</p></td></tr><tr><td><p><strong>First Omni Gateway version available</strong></p></td><td><p>v1.13.0</p></td></tr><tr><td><p><strong>Release Notes</strong></p></td><td><p><a href="https://docs.mulesoft.com/release-notes/gateway/policies/llm-proxy-request-compression-release-notes">Model Proxy Request Compression</a></p></td></tr><tr><td><p><strong>Returned Status Codes</strong></p></td><td><p>No return codes exist for this policy.</p></td></tr></tbody></table>

## Summary

The Model Proxy Request Compression policy reduces token consumption by compressing the strings that
agents carry in an LLM request body before that body reaches the upstream provider.

In an ongoing agent conversation, each message and each tool result is added to the conversation
history. To keep context, a client or runtime might resend the whole history with each new request.
Every time it does, the provider charges input tokens for that history again. Compressing those
repeated strings reduces the input tokens.

The policy scans the entire request body and rewrites each scope-selected string, keeping the
rewritten value only when it is strictly smaller than the original. It supports both the Chat
Completions and Responses API shapes.

Two independent **compression scopes** control which strings are eligible:

- **Compress Tool Results** - Compresses every supplied tool result, including replayed history.
  This covers Chat Completions tool-message content and outputs from Responses function and
  custom-tool calls. Enabled by default to preserve the policy’s original behavior.

- **Compress Messages** - Compresses the Responses `instructions` field and the textual content of
  every supplied system, user, and assistant message, including history. Roles, tool calls,
  metadata, cache breakpoints, and non-text content are preserved. Disabled by default because
  message content might contain formatting-sensitive instructions or data.

Both scopes traverse the whole request rather than only the latest message. When a runtime resends
the raw history in the request body, the policy rewrites it the same way every time, which keeps
prompt-cache prefixes stable. History that the provider manages server-side is never part of the
request body, so the policy neither transforms nor reconstructs it.

The transforms applied within those scopes are organized into two independent features:

1.  **Clean Payload** - Compresses each eligible string using a configurable set of transforms:
    minify (removes only whitespace, so no data is lost), remove null values, remove empty values,
    and HTML to Markdown. Enabled by default.

2.  **TOON Encoding** - Re-encodes uniform JSON structures as Token-Oriented Object Notation (TOON),
    a compact tabular format that reduces tokens for repeated structures. Disabled by default.

The policy is **fail-open** throughout: any gate miss, parse failure, or result that doesn’t get
smaller forwards the original bytes unchanged. When compression does engage, the policy measures the
tokens and bytes saved and feeds the token/cost savings to Cost Management telemetry.

## Placement

Apply this policy to the inbound Model Proxy chain:

- **After** the Model Proxy core processing.

- **Before** any provider transcoding policy, so it operates on the request before it is rewritten
  for a specific provider.

## Configuring Policy Parameters

### Omni Gateway Local Mode

The Model Proxy Request Compression policy isn’t supported in Local Mode.

### Managed Omni Gateway and Omni Gateway Connected Mode

When you apply the policy from the UI, the following parameters are displayed:

The policy configuration is organized into two compression scopes that select which strings are
eligible, plus two transform features. Each feature has a switch that enables the fields below it
and an **Agent IDs** allowlist that controls which agents the feature applies to.

### Compression Scopes

<table><colgroup><col> <col> <col></colgroup><thead><tr><th>Element</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td><div><div><p>Compress Tool Results</p></div></div></td><td><div><div><p>Applies the enabled transforms to every supplied tool result, including replayed history. Covers Chat Completions tool-message content and Responses function and custom-tool call outputs.</p></div></div></td><td><div><div><p>Enabled</p></div></div></td></tr><tr><td><div><div><p>Compress Messages</p></div></div></td><td><div><div><p>Applies the enabled transforms to the Responses <code>instructions</code> field and the textual content of every supplied system, user, and assistant message, including history. Roles, tool calls, metadata, cache breakpoints, and non-text content are preserved.</p></div></div></td><td><div><div><p>Disabled</p></div></div></td></tr></tbody></table>

### Clean Payload

<table><colgroup><col> <col> <col></colgroup><thead><tr><th>Element</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td><div><div><p>Enable Clean Payload</p></div></div></td><td><div><div><p>Enablement switch for the Clean Payload feature.</p></div></div></td><td><div><div><p>Enabled</p></div></div></td></tr><tr><td><div><div><p>Minify</p></div></div></td><td><div><div><p>Reserializes eligible JSON content in compact form, dropping only insignificant whitespace such as indentation and spaces between tokens. The parsed data is unchanged, so no information is lost.</p></div></div></td><td><div><div><p>Enabled</p></div></div></td></tr><tr><td><div><div><p>Remove Null Values</p></div></div></td><td><div><div><p>Lossy. Recursively drops object keys with <code>null</code> values from eligible JSON content. Null array elements are preserved.</p></div></div></td><td><div><div><p>Disabled</p></div></div></td></tr><tr><td><div><div><p>Remove Empty Values</p></div></div></td><td><div><div><p>Drops object keys whose value is an empty string, array, or object, applied bottom-up until no more can be removed.</p></div></div></td><td><div><div><p>Disabled</p></div></div></td></tr><tr><td><div><div><p>HTML to Markdown</p></div></div></td><td><div><div><p>Converts eligible HTML content to Markdown after stripping <code>&lt;script&gt;</code> and <code>&lt;style&gt;</code> blocks. If the conversion fails, the policy falls back to that stripped HTML instead of erroring.</p></div></div></td><td><div><div><p>Disabled</p></div></div></td></tr><tr><td><div><div><p>Clean Payload — Agent IDs</p></div></div></td><td><div><div><p>Allowlist of agent IDs opted into the Clean Payload feature. For details on allowlist behavior, see <a href="#agent-id-gating">Per-Agent Gating</a>.</p></div></div></td><td><div><div><p><code>[]</code> (empty)</p></div></div></td></tr></tbody></table>

### TOON Encoding

<table><colgroup><col> <col> <col></colgroup><thead><tr><th>Element</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td><div><div><p>Enable TOON Encoding</p></div></div></td><td><div><div><p>Enablement switch for the TOON feature.</p></div></div></td><td><div><div><p>Disabled</p></div></div></td></tr><tr><td><div><div><p>TOON Minimum Response Bytes</p></div></div></td><td><div><div><p>Apply TOON encoding only to eligible strings that are at least this many bytes long. A value of <code>0</code> disables the threshold.</p></div></div></td><td><div><div><p><code>500</code></p></div></div></td></tr><tr><td><div><div><p>TOON — Agent IDs</p></div></div></td><td><div><div><p>Allowlist of agent IDs opted into the TOON feature. For details on allowlist behavior, see <a href="#agent-id-gating">Per-Agent Gating</a>.</p></div></div></td><td><div><div><p><code>[]</code> (empty)</p></div></div></td></tr></tbody></table>

### Per-Agent Gating

Each feature has its own **Agent IDs** independent allowlist (**Clean Payload — Agent IDs** and
**TOON — Agent IDs**). The policy resolves the agent ID from the request and decides to apply the
feature to:

- Direct requests to the Model Proxy that carry no agent ID. The feature always applies to these
  requests, regardless of the allowlist contents.

- Requests from an identified agent specified in the allowlist.

You must add an agent to the allowlist for the the feature to apply to its request. There is no
allow-all functionality.

In Cost Management, applying the policy to an agent appends the agent’s ID to the allowlist of the
enabled feature.

## How This Policy Works

For each request, the policy processes the body only when these conditions are met:

- The request is a `POST` with a `Content-Type` of `application/json`.

- The request has been normalized to OpenAI shape by the upstream Model Proxy core processing.

When those conditions are met, the policy:

1.  Determines which scopes and features are engaged and if the agent is in a per-agent allowlist.

2.  Scans the full request body and locates the scope-selected strings, in document order, across
    the complete supplied history.

3.  Rewrites each located string.

4.  Keeps a rewritten string only when it is strictly smaller than the original.

5.  Removes the stale `Content-Length` header and reserializes the body.

Because the policy is fail-open, an unparseable body, a missing gate, or a result that does not
shrink causes the original request to pass through unchanged, and no savings are reported.

### TOON Encoding

Token-Oriented Object Notation (TOON) is a compact, JSON-equivalent text format that reduces token
consumption for structured data. When **Enable TOON Encoding** is enabled, the policy re-encodes
eligible JSON content as TOON, prefixing the result with \`TOON: \`.

TOON encoding:

- Applies only to eligible strings that are at least **TOON Minimum Response Bytes**.

- Applies only to the shapes that TOON can shrink, such as uniform arrays of objects and objects
  with repeated structure.

- Is lossless but format-changing: the data is preserved, but the model receives it as TOON rather
  than JSON.

- Preserves or reduces payload size: the re-encoded value is kept only when it is strictly smaller
  than the original.

## See Also

- [MCP Payload Optimization Policy](https://docs.mulesoft.com/gateway/latest/policies-included-mcp-payload-optimization)

- [Creating and Managing Model Proxies](https://docs.mulesoft.com/general/model-proxy) - Overview of Model Proxy and
  routing