---
schema_version: "1"
id: 07ae0ee7c9ce6b1f97563f3560f5dd8f1bbced802639f298ed3144abf1e9396a
title: Creating and Managing Model Proxies
collection: mulesoft
version: current
locale: en-us
portal_url: https://docs.salesforce.com/mulesoft/current/en-us/general/model-proxy.md
canonical_url: https://docs.mulesoft.com/general/model-proxy
source_site: docs.mulesoft.com
content_type: documentation
content_hash: f5f19b47ffe0c6c02ece724c98f4bfd92a93477918d2d5399ecd90955d02721d
description: MuleSoft Documentation Site
latest: true
---
---
product: Getting Started
---

# Creating and Managing Model Proxies

> For the full documentation index, see: https://docs.mulesoft.com/llms.txt

Model Proxy provides a unified access layer for multiple Large Language Model (LLM) providers. Model
Proxies are deployed to Omni Gateway to enable governance, intelligent routing, and cost management
for AI applications.

Model Proxy is supported on Managed Omni Gateway and Self-Managed Omni Gateway running in Connected
Mode.

By creating a proxy, the user defines a singular LLM service that can receive requests for multiple
LLM providers. This simplifies the developer experience. You can seamlessly add new models to the
service without changing the endpoint.

![A Model Proxy with a single endpoint routing requests to multiple LLM providers](/mulesoft/current/en-us/general/_images/model-proxy.png)

## Supported Endpoint Formats

Model Proxy supports these endpoint formats that are selected when creating a Model Proxy and can’t
be changed later:

- **OpenAI**: The OpenAI API format works with all supported LLM providers and supports
  multi-routing and fallback mechanisms.

- **Gemini**: The native Gemini API format for direct Google Gemini model requests. Supports
  model-based routing only.

- **Anthropic**: The native Anthropic API format for direct Anthropic Claude model requests.
  Supports model-based routing only.

Depending on configuration, the proxy then sends the request to the model defined by the user or
dynamically sends the request to the provider that best matches the request:

- [Model-Based Routing](#model-based-routing): Static routing. The user specifies what model the
  Model Proxy should send the request to.

- [Semantic Routing](#semantic-routing): Dynamic routing. The Model Proxy chooses which model to
  send the request to based on the request content.

## Model-Based Routing

Model-based routing is static routing. In the request, the user specifies what model the Model Proxy
should send the request to. By specifying a target model, Model Proxy can override the model version
provided by the user.

## Semantic Routing

Semantic routing is dynamic routing where the Model Proxy chooses which model to send the request
to. For Sematic Routing, the user creates prompt topics for each route. When a request is sent to
the Model Proxy, a semantic service compares the request to the define topic utterances and sends
the request to the route that best matches it.

## Semantic Caching

Semantic caching stores and reuses LLM responses based on semantic similarity. When an incoming
request is semantically similar to a previous request, Model Proxy returns the cached response
instead of calling the LLM provider again, which reduces latency and cost. To use semantic caching,
configure a semantic caching service and enable it on the Model Proxy. Semantic caching supports
Azure AI Search as the vector database only. For more information, see
[Configuring Semantic Caching Services](https://docs.mulesoft.com/general/model-proxy-semantic-caching-service).

## Connections and Model Proxies

Model Proxy includes these types:

- **Proxy**: A proxy configuration that can route requests to one or multiple LLM providers by using
  model-based routing and semantic routing strategies. The enhanced experience manages proxy
  configurations.

- **Connection**: A connection configuration that stores credentials and metadata for connecting to
  a specific LLM provider. The enhanced experience displays connections as a single-route proxy.
  Connection configurations are managed by external projects, such as an Agent Network project.

## Model Proxy Limits

Up to 50 Model Proxies are supported per Large Omni Gateway.

## Supported LLM Providers and Models

Model Proxy supports these LLM providers and models:

- OpenAI

  - GPT 5.2

  - GPT 5.2 Pro

  - GPT 5 Mini

  - GPT 5.2 Codex

  - GPT 5 Nano

  - GPT 5

  - GPT 4.1

  - GPT 4o Mini

  - Chat Latest

  - ChatGPT-4o

  - codex-mini-latest

  - computer-use-preview

  - Daybreak Blue

  - Daybreak Red

  - GPT-3.5 Turbo

  - GPT-4

  - GPT-4 Turbo

  - GPT-4 Turbo Preview

  - GPT-4.1 Mini

  - GPT-4.1 nano

  - GPT-4.5 Preview

  - GPT-4o

  - GPT-4o Search Preview

  - GPT-4o Mini Search Preview

  - GPT-5 Chat

  - GPT-5 Pro

  - GPT-5-Codex

  - GPT-5.1

  - GPT-5.1 Chat

  - GPT-5.1-Codex

  - GPT-5.1-Codex Mini

  - GPT-5.1-Codex-Max

  - GPT-5.2 Chat

  - GPT-5.3 Chat

  - GPT-5.3-Codex

  - GPT-5.4

  - GPT-5.4 Mini

  - GPT-5.4 nano

  - GPT-5.4 Pro

  - GPT-5.5

  - GPT-5.5 Pro

  - GPT-5.6 Cyber

  - GPT-5.6 Luna

  - GPT-5.6 Sol

  - GPT-5.6 Terra

  - GPT-6 Astra

  - GPT-6 Sol

  - GPT-6 Luna

  - gpt-oss-120b

  - gpt-oss-20b

  - o1

  - o1 Preview

  - o1-mini

  - o1-pro

  - o3

  - o3-deep-research

  - o3-mini

  - o3-pro

  - o4-mini

  - o4-mini-deep-research

- Azure OpenAI

  - GPT-6 Astra

  - GPT-6 Sol

  - GPT-6 Luna

  - GPT-5.6 Sol

  - GPT-5.6 Terra

  - GPT-5.6 Luna

  - GPT-5.5

  - GPT-5.4

  - GPT-5.4 Mini

  - GPT-5.4 Nano

  - GPT-5.4 Pro

  - GPT-5.3-Codex

  - GPT 5.2

  - GPT 5.2 Pro

  - GPT 5 Mini

  - GPT 5.2 Codex

  - GPT 5 Nano

  - GPT 5

  - GPT 4.1

  - GPT 4o Mini

- Bedrock OpenAI

  - GPT-6 Astra (Bedrock)

  - GPT-6 Sol (Bedrock)

  - GPT-6 Luna (Bedrock)

  - GPT-5.6 Sol (Bedrock)

  - GPT-5.6 Terra (Bedrock)

  - GPT-5.6 Cyber (Bedrock)

  - GPT-5.6 Luna (Bedrock)

  - GPT-5.5 (Bedrock)

  - GPT-5.4 (Bedrock)

  - GPT-OSS 120B (Bedrock)

  - GPT-OSS 20B (Bedrock)

- Gemini

  - Gemini 3 Flash Preview

  - Gemini 2.5 Flash

  - Gemini 2.5 Flash Preview 09-2025

  - Gemini 2.5 Flash Lite

  - Gemini 3 Flash

  - Gemini 3.1 Pro

  - Gemini 3.5

  - Gemini 3.8 Flash

  - Gemma 4 26B A4B

  - Gemma 4 31B

- Anthropic

  - Claude Opus 5.5

  - Claude Opus 5

  - Claude Sonnet 5

  - Claude Fable 5.1

  - Claude Opus 4.8

  - Claude Opus 4.7

  - Claude Opus 4.6

  - Claude Sonnet 4.6

  - Claude Haiku 4.5

  - Claude Fable 5

- Bedrock Anthropic

  - Claude Opus 5.5 (Bedrock)

  - Claude Opus 5 (Bedrock)

  - Claude Opus 4.8 (Bedrock)

  - Claude Opus 4.7 (Bedrock)

  - Claude Opus 4.6 (Bedrock)

  - Claude Sonnet 5 (Bedrock)

  - Claude Sonnet 4.6 (Bedrock)

  - Claude Haiku 4.5 (Bedrock)

  - Claude Fable 5.1 (Bedrock)

  - Claude Fable 5 (Bedrock)

  - Claude Sonnet 4.5 (Bedrock)

- NVIDIA

  - Nemotron 3 Nano 30B A3B

  - Nemotron 3 Super 120B A12B

  - Llama Nemotron Ultra 253B

  - Nemotron 3 Ultra 550B A55B

  - Nemotron 3.5 Lightning 30B A3B

  - Llama Nemotron Super 49B

  - Llama Nemotron Super 49B v1.5

  - Llama Nemotron Nano 8B v1

  - Nemotron Nano 9B v2

  - Llama Nemotron 70B Instruct

- xAI

  - Grok 4.6

  - Grok 4.5

  - Grok 4.3

  - Grok 4.20 (Reasoning)

  - Grok 4.20 (Non-Reasoning)

  - Grok 4.20 Multi-Agent Beta

  - Grok Build 0.1

- Mistral

  - Mistral Large 3

  - Mistral Medium 3.5

  - Mistral Small

  - Magistral Medium

  - Magistral Small

  - Ministral 8B

  - Ministral 3B

  - Codestral

  - Devstral Medium

  - Devstral Small

  - Pixtral Large