---
schema_version: "1"
id: c40b4fc2bd38e4083ab18fb7915ee921a94d51c48e03ee22a89688a025aaf913
title: LLM Token Based Rate Limit Policy
collection: mulesoft
version: current
locale: en-us
portal_url: https://docs.salesforce.com/mulesoft/current/en-us/gateway/latest/policies-included-llm-token-rate-limit.md
canonical_url: https://docs.mulesoft.com/gateway/latest/policies-included-llm-token-rate-limit
source_site: docs.mulesoft.com
content_type: documentation
content_hash: f50a915bac472259f24271cb0bba5be41ae9268b74d4e8705162d9852bc1fe84
description: MuleSoft Documentation Site
release: "1.14"
latest: true
---
---
product: Omni Gateway
version: 1.14
is-latest-version: true
keywords: api gateway, flex gateway, gateway, policy, rate limiting, token, llm, openai
---

# LLM Token Based Rate Limit Policy

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

<table><colgroup><col> <col></colgroup><tbody><tr><td><p><strong>Policy Name</strong></p></td><td><p>LLM Token Based Rate Limit</p></td></tr><tr><td><p><strong>Summary</strong></p></td><td><p>Rate limits Model Proxy usage based on token consumption</p></td></tr><tr><td><p><strong>Category</strong></p></td><td><p>LLM</p></td></tr><tr><td><p><strong>First Omni Gateway version available</strong></p></td><td><p>v1.11.0</p></td></tr><tr><td><p><strong>Release Notes</strong></p></td><td><p><a href="https://docs.mulesoft.com/release-notes/gateway/policies/llm-token-rate-limit-release-notes">LLM Token Based Rate Limit</a></p></td></tr><tr><td><p><strong>Returned Status Codes</strong></p></td><td><p>429 - Too Many Requests: Token limit exceeded, requests are blocked until the current window finishes</p></td></tr></tbody></table>

## Summary

The LLM Token Based Rate Limit policy enforces rate limiting on Model Proxy requests based on the
number of tokens consumed providing control over API usage based on actual token consumption
reported by the upstream LLM provider.

It counts request, response, and reasoning token (if applicable). When the token limit is exceeded,
the policy blocks the call and returns a `429` status code.

The policy supports both streaming and non-streaming responses.

## Configuring Policy Parameters

### Omni Gateway Local Mode

The LLM Token Based Rate Limit policy isn’t supported in Local Mode.

### Managed Omni Gateway and Omni Gateway Connected Mode

When you apply the policy from the UI, the following parameters are displayed:

<table><colgroup><col> <col> <col></colgroup><thead><tr><th>Element</th><th>Required</th><th>Description</th></tr></thead><tbody><tr><td><div><div><p>Maximum Tokens</p></div></div></td><td><div><div><p>Yes</p></div></div></td><td><div><div><p>Maximum number of tokens allowed within the specified time period. Must be a positive integer of one or greater.</p></div></div></td></tr><tr><td><div><div><p>Time Period (ms)</p></div></div></td><td><div><div><p>Yes</p></div></div></td><td><div><div><p>Time period in milliseconds for the rate limit window. Minimum time period is 1000 milliseconds.</p></div></div></td></tr><tr><td><div><div><p>Key Selector</p></div></div></td><td><div><div><p>Yes</p></div></div></td><td><div><div><p>DataWeave expression that selects the key for rate limiting. This creates independent rate limit counters for each unique value resolved by the expression. For example:</p></div><div><ul><li><p><code>#[attributes.headers['client_id']]</code>: Rate limit per client ID</p></li><li><p><code>#[attributes.principal]</code>: Rate limit per user</p></li></ul></div></div></td></tr></tbody></table>

## How This Policy Works

The LLM Token Based Rate Limit policy monitors token consumption in the current window, allowing
requests to reach the backend only if the available token quota is greater than zero.

The policy uses a fixed-window rate limiting algorithm. The window starts after it receives the
first request. When the quota is exhausted, requests are rejected with a `429` status code until the
time window resets.

The `keySelector` parameter enables you to create independent rate limit counters for different
groups of requests. Each unique value resolved by the DataWeave expression has its own token quota
and time window. For example, if you configure `keySelector: "#[attributes.headers['client_id']]"`
with a limit of 1000 tokens per 60 seconds, each client has its own 1000-token quota in a 60-second
window.

### Response Headers

When a request is processed, the policy adds the following headers to the response:

- `x-token-limit`: The maximum number of tokens allowed per window

- `x-token-remaining`: The number of tokens remaining in the current window

- `x-token-reset`: The remaining time, in milliseconds, until a new window starts

## See Also

- [Creating and Managing Model Proxies](https://docs.mulesoft.com/general/model-proxy) - Overview of Model Proxy and
  routing

- [Viewing Token Usage and Model Proxy Metrics](https://docs.mulesoft.com/general/model-proxy-token-reports) - Token
  usage reports and limiting usage

- [A2A Token Based Rate Limit](https://docs.mulesoft.com/gateway/latest/policies-included-a2a-token-rate-limit) - Token-based rate limiting
  for A2A agent traffic