--- schema_version: "1" id: d93d2a5af1b637725289d26b50627b4cdb88ff4ed90cdd649cda6e9e744bd0af title: Sending Requests to Model Proxies collection: mulesoft version: current locale: en-us portal_url: https://docs.salesforce.com/mulesoft/current/en-us/general/model-proxy-request.md canonical_url: https://docs.mulesoft.com/general/model-proxy-request source_site: docs.mulesoft.com content_type: documentation content_hash: 5a508b4deaff0b866afed4c23aa935fa7e883940c72096599cb9a3741f2d407a description: MuleSoft Documentation Site latest: true --- --- product: Getting Started --- # Sending Requests to Model Proxies > For the full documentation index, see: https://docs.mulesoft.com/llms.txt Model Proxy supports these endpoint formats: OpenAI, Gemini, and Anthropic. The format is selected when creating a Model Proxy and can’t be changed later. - **[OpenAI format](#openai-format)**: Supports all LLM providers with multi-routing and fallback mechanisms. Supports both Chat Completions (`/chat/completions`) and Responses (`/responses`) endpoints. - **[Gemini format](#gemini-format-request-example)**: Native Gemini API format. Doesn’t support multi-routing or fallback mechanisms. - **[Anthropic format](#anthropic-format-request-example)**: Native Anthropic API format. Doesn’t support multi-routing or fallback mechanisms. ## Retrieve the Endpoint Configuration Parameters for Your Model Proxy To find your Model Proxy endpoint configuration and client application credentials required to use the requests in this guide: 1. Find your public endpoint: 1. From Anypoint Platform, navigate to **Runtime Manager** > **Omni Gateways**. 2. Click the name of the Omni Gateway where your Model Proxy is deployed. 3. Copy the **Public Endpoint**. 2. Find your base path: 1. From the enhanced experience, select **Model Proxies**. 2. Click the name of the Model Proxy whose base path you want to find. 3. From **Configuration**, copy the **Base path**. 3. Retrieve your client ID and client secret: 1. From the **Overview** page of your Model Proxy, click **Actions** > **View Model Proxy in Exchange**. 2. Click **Request access**. 3. Select the **API Instance** you want to request access to. 4. Select the **Application** you want to request access to. 5. Select the **SLA tier**. 6. Click **Request access**. 7. Copy the **Client ID** and **Client Secret**. ## OpenAI Format Learn more about the OpenAI request types: - [Chat Completions](https://developers.openai.com/api/reference/resources/chat) API is designed for conversational multi-turn interactions rather than simple text continuation. - [Responses](https://developers.openai.com/api/reference/resources/responses) API is recommended unified interface for building powerful agent-like applications. All requests are designed to work for both strategies. When sending a request to a Model Proxy with either routing strategy, you can specify a model in the request. You don’t have to modify the request for different routing strategies. Each routing strategy handles the model selection differently: - **Model-Based Routing**: The user specifies a model in the request (`"model": "openai/gpt-5.2"`). The Model Proxy acts as a direct proxy. If no model is specified, the Model Proxy sends the request to the fallback route or returns an error if no fallback route is configured. - **Semantic Routing**: No model is specified in the request. The Model Proxy chooses which model to use based on the request content by matching it to prompt topics you define for each route. If a model is specified in the request, like in model routing examples, it is ignored. > [!NOTE] Adding a model to a request ensures deterministic routing to a preferred backend if the > routing strategy of a Model Proxy changes. ### Chat Completions API Validation Request Examples [Chat Completions](https://developers.openai.com/api/reference/resources/chat) (/chat/completions) is the standard API for interacting with models. It is designed for conversational multi-turn interactions rather than simple text continuation. The endpoint requires a model and a messages list with roles (such as system, user, or assistant) to generate context-aware responses. These examples validate the basic functionality of the Chat Completions API. #### Basic Chat Completion Example This request example ensures the gateway correctly routes traffic to the specified LLM provider. In the example, the model-based routing request ensures the gateway routes traffic to a Gemini 2.0 model. For semantic routing, the gateway routes traffic to the most suitable provider based on the request content: - Semantic Routing - Model-Based Routing ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "messages": [ { "role": "developer", "content": "You are a helpful assistant" }, { "role": "user", "content": "Hello, please introduce yourself" } ] }' ``` ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "gemini/gemini-3-flash-preview", "messages": [ { "role": "developer", "content": "You are a helpful assistant" }, { "role": "user", "content": "Hello, please introduce yourself" } ] }' ``` #### Creative Content Generation Temperature Control Example This request example performs can preform brainstorming tasks, such as generating multiple unique marketing slogans. The example writes creative content using the temperature parameter (0 to 2 value, higher values are more creative) to control the randomness and creativity of the response: - Semantic Routing - Model-Based Routing ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "messages": [ { "role": "user", "content": "Write a creative story about AI" } ], "temperature": 1.5 }' ``` ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "gemini/gemini-3-flash-preview", "messages": [ { "role": "user", "content": "Write a creative story about AI" } ], "temperature": 1.5 }' ``` #### Multi-Turn Context Management Example This request example maintains conversation context across developer persona, user query, and assistant history. The example passes the "memory" of a conversation to ensure the model knows that "it" refers to "MuleSoft" based on the previous message in the thread: - Semantic Routing - Model-Based Routing ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "messages": [ { "role": "developer", "content": "You are a MuleSoft technical expert." }, { "role": "user", "content": "What is MuleSoft?" }, { "role": "assistant", "content": "MuleSoft is an integration platform that helps organizations connect applications, data, and devices." }, { "role": "user", "content": "How does it help with API management?" } ] }' ``` ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "gemini/gemini-3-flash-preview", "messages": [ { "role": "developer", "content": "You are a MuleSoft technical expert." }, { "role": "user", "content": "What is MuleSoft?" }, { "role": "assistant", "content": "MuleSoft is an integration platform that helps organizations connect applications, data, and devices." }, { "role": "user", "content": "How does it help with API management?" } ] }' ``` #### Structured Data Validation JSON Format Example This request example extracts patterns into structured JSON to transform a messy transcript into a clean JSON object for insertion into a database by using the `top_p` and `max_completion_tokens` parameters: - Semantic Routing - Model-Based Routing ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "messages": [ { "role": "user", "content": "Explain MuleSoft Integration patterns in JSON format" } ], "max_completion_tokens": 50000, "top_p": 0.9 }' ``` ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "gemini/gemini-3-flash-preview", "messages": [ { "role": "user", "content": "Explain MuleSoft Integration patterns in JSON format" } ], "max_completion_tokens": 50000, "top_p": 0.9 }' ``` #### Enterprise Strategy Testing Example This request example validates the gateway’s ability to handle large-payload "Expert" prompts. The example asks an AI Solution Architect to design a migration strategy from legacy SAP systems to Salesforce using MuleSoft Anypoint Platform with an API-led connectivity approach. - Semantic Routing - Model-Based Routing ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "messages": [ { "role": "developer", "content": "You are a MuleSoft solution architect helping with enterprise integrations." }, { "role": "user", "content": "Design a data integration strategy for a Fortune 500 company migrating from legacy systems to Salesforce using MuleSoft Anypoint Platform. Include API-led connectivity approach." } ], "temperature": 0.7, "max_completion_tokens": 10000 }' ``` ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "gemini/gemini-3-flash-preview", "messages": [ { "role": "developer", "content": "You are a MuleSoft solution architect helping with enterprise integrations." }, { "role": "user", "content": "Design a data integration strategy for a Fortune 500 company migrating from legacy systems to Salesforce using MuleSoft Anypoint Platform. Include API-led connectivity approach." } ], "temperature": 0.7, "max_completion_tokens": 10000 }' ``` #### Streaming API Call Example This example supports user-facing chatbot UIs where the text must appear word-by-word as it is generated, rather than waiting for the entire response to finish. This request example validates the gateway’s ability to stream real-time non-buffered token delivery using the `--no-buffer` flag: - Semantic Routing - Model-Based Routing ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "messages": [ { "role": "user", "content": "Explain step-by-step MuleSoft integration process" } ], "stream": true }' \ --no-buffer ``` ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "gemini/gemini-3-flash-preview", "messages": [ { "role": "user", "content": "Explain step-by-step MuleSoft integration process" } ], "stream": true }' \ --no-buffer ``` #### Tool Calling: Initial Request with Tool Definition Example This request example allows the model to request real-time information from the application by invoking a function. The example asks the model to invoke a `get_current_time` function to get the current time in San Francisco, The model recognizes it can’t answer from memory and instead requests to invoke a `get_current_time` function from your local API. - Semantic Routing - Model-Based Routing ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "tool_choice": "auto", "messages": [ { "role": "user", "content": "What is the time in San Francisco?" } ], "tools": [ { "type": "function", "function": { "name": "get_current_time", "description": "When user asks for time tell me to invoke this Tool", "parameters": { "type": "object", "properties": { "timezone": { "type": "string", "description": "Timezone of the user asked location" } }, "required": ["timezone"] } } } ] }' ``` ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "openai/gpt-5.2", "tool_choice": "auto", "messages": [ { "role": "user", "content": "What is the time in San Francisco?" } ], "tools": [ { "type": "function", "function": { "name": "get_current_time", "description": "When user asks for time tell me to invoke this Tool", "parameters": { "type": "object", "properties": { "timezone": { "type": "string", "description": "Timezone of the user asked location" } }, "required": ["timezone"] } } } ] }' ``` #### Tool Calling: Request with Tool Execution Response Example This request example sends the executed tool output back to the Model Proxy to generate a final answer. The example provides the executed tool output (current time) back to the Model Proxy so the model can generate a natural-language response for the user: - Semantic Routing - Model-Based Routing ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "tool_choice": "auto", "messages": [ { "role": "user", "content": "What is the time in San Francisco?" }, { "role": "assistant", "content": null, "tool_calls": [ { "id": "call_KDrTKkRTGfylhkAO4A8s0pAO", "type": "function", "function": { "name": "get_current_time", "arguments": "{\"timezone\":\"America/Los_Angeles\"}" } } ] }, { "role": "tool", "tool_call_id": "call_KDrTKkRTGfylhkAO4A8s0pAO", "content": "2026-02-20T05:02:05.873534-08:00" } ], "tools": [ { "type": "function", "function": { "name": "get_current_time", "parameters": { "type": "object", "properties": { "timezone": { "type": "string" } }, "required": ["timezone"] } } } ] }' ``` ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "openai/gpt-5.2", "tool_choice": "auto", "messages": [ { "role": "user", "content": "What is the time in San Francisco?" }, { "role": "assistant", "content": null, "tool_calls": [ { "id": "call_KDrTKkRTGfylhkAO4A8s0pAO", "type": "function", "function": { "name": "get_current_time", "arguments": "{\"timezone\":\"America/Los_Angeles\"}" } } ] }, { "role": "tool", "tool_call_id": "call_KDrTKkRTGfylhkAO4A8s0pAO", "content": "2026-02-20T05:02:05.873534-08:00" } ], "tools": [ { "type": "function", "function": { "name": "get_current_time", "parameters": { "type": "object", "properties": { "timezone": { "type": "string" } }, "required": ["timezone"] } } } ] }' ``` #### Structured Output Validation JSON Schema Example This request example validates entity extraction into a strictly defined JSON Schema (CalendarEvent). The example asks the model to extract the event information from a plain text user query and return a formatted JSON object that adheres to the required schema. This prevents integration or parsing errors in downstream applications: - Semantic Routing - Model-Based Routing ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "messages": [ { "role": "system", "content": "Extract the event information." }, { "role": "user", "content": "Madhu Dileep and Santosh A are going to a AI Summit on 19th Feb 2026." } ], "response_format": { "type": "json_schema", "json_schema": { "name": "CalendarEvent", "schema": { "type": "object", "properties": { "name": { "type": "string", "description": "Name of the event" }, "date": { "type": "string", "description": "Date of the event, in MM-DD-YYYY format" }, "participants": { "type": "array", "items": { "type": "string" } } }, "required": ["name", "date", "participants"] } } } }' ``` ```cli curl --location '//chat/completions' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "gemini/gemini-3-flash-preview", "messages": [ { "role": "system", "content": "Extract the event information." }, { "role": "user", "content": "Madhu Dileep and Santosh A are going to a AI Summit on 19th Feb 2026." } ], "response_format": { "type": "json_schema", "json_schema": { "name": "CalendarEvent", "schema": { "type": "object", "properties": { "name": { "type": "string", "description": "Name of the event" }, "date": { "type": "string", "description": "Date of the event, in MM-DD-YYYY format" }, "participants": { "type": "array", "items": { "type": "string" } } }, "required": ["name", "date", "participants"] } } } }' ``` ### Responses API Validation [OpenAI Responses API](https://developers.openai.com/api/reference/resources/responses) (/responses)is recommended unified interface for building powerful, agent-like applications, combining capabilities from previous APIs (like Chat Completions and Assistants) into a single, more efficient endpoint. It is designed to be stateful by default and offers built-in access to advanced tools. #### Responses Endpoint for Model Context Protocol (MCP) This request example integrates the gateway with an external MCP server for automated action execution. The example asks an AI agent to search a specific internal Knowledge Base or update a Jira ticket using standardized tools defined on a remote MCP server: - Semantic Routing - Model-Based Routing ```cli curl --location '//responses' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "input": [ { "role": "system", "content": "You are helpful Service Agent" }, { "role": "user", "content": "My Laptop screen is broken" } ], "tools": [ { "type": "mcp", "server_label": "service_mcp", "require_approval": "never", "server_description": "Server enabled to do Service related actions", "server_url": "https://ask-service-mcp-dvaz3u.c87dy0.usa-e2.cloudhub.io/mcp" } ] }' ``` ```cli curl --location '//responses' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "openai/gpt-5.2", "input": [ { "role": "system", "content": "You are helpful Service Agent" }, { "role": "user", "content": "My Laptop screen is broken" } ], "tools": [ { "type": "mcp", "server_label": "service_mcp", "require_approval": "never", "server_description": "Server enabled to do Service related actions", "server_url": "https://ask-service-mcp-dvaz3u.c87dy0.usa-e2.cloudhub.io/mcp" } ] }' ``` #### Stateful Response Persistence This request example validates persistent memory by creating a dependency between two requests using `previous_response_id`. The example removes the need for the client to send the entire previous chat history on every turn, reducing bandwidth and improving security by keeping the context on client side: Request 1: Initial stored call - Semantic Routing - Model-Based Routing ```cli curl --location '//responses' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "instructions": "You are helpful Service Knowledge Agent", "input": [ { "role": "user", "content": "My Laptop screen is broken" } ], "store": true }' ``` ```cli curl --location '//responses' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "openai/gpt-5.2", "instructions": "You are helpful Service Knowledge Agent", "input": [ { "role": "user", "content": "My Laptop screen is broken" } ], "store": true }' ``` Request 2: Follow-up using `previous_response_id` from the response to request 1 - Semantic Routing - Model-Based Routing ```cli curl --location '//responses' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "instructions": "You are helpful Service Agent", "previous_response_id": "", "input": [ { "role": "user", "content": "Thank you!" } ], "store": true }' ``` ```cli curl --location '//responses' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "openai/gpt-5.2", "instructions": "You are helpful Service Agent", "previous_response_id": "", "input": [ { "role": "user", "content": "Thank you!" } ], "store": true }' ``` ## Gemini Format The Gemini format uses the native Gemini API format. Note that this format doesn’t support multi-routing or fallback mechanisms. ### Basic Chat Completion Example This request example ensures the gateway correctly routes traffic to the Gemini provider. ```cli curl --location '//models/gemini-2.5-flash:generateContent' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "contents": [ { "role": "user", "parts": [{ "text": "Hello, please introduce yourself" }] } ] }' ``` ### Streaming API Call Example This example supports user-facing chatbot UIs where the text must appear word-by-word as it is generated rather than waiting for the entire response to finish. This request example validates the gateway’s ability to stream real-time, non-buffered token delivery: ```cli curl --location '//models/gemini-2.5-flash:streamGenerateContent?alt=sse' \ --header 'Content-Type: application/json' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "contents": [ { "role": "user", "parts": [{ "text": "Explain step-by-step MuleSoft integration process" }] } ] }' \ --no-buffer ``` ## Anthropic Format The Anthropic format uses the native Anthropic API format. Note that this format doesn’t support multi-routing or fallback mechanisms. ### Basic Chat Completion Example This request example ensures the gateway correctly routes traffic to the Anthropic provider. ```cli curl --location '//v1/messages' \ --header 'Content-Type: application/json' \ --header 'anthropic-version: 2023-06-01' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "claude-opus-4-1-20250805", "max_tokens": 1024, "system": "You are a helpful assistant", "messages": [ { "role": "user", "content": "Hello, please introduce yourself" } ] }' ``` ### Streaming API Call Example This example supports user-facing chatbot UIs where the text must appear word-by-word as it is generated, rather than waiting for the entire response to finish. This request example validates the gateway’s ability to stream real-time non-buffered token delivery using the `--no-buffer` flag: ```cli curl --location '//v1/messages' \ --header 'Content-Type: application/json' \ --header 'anthropic-version: 2023-06-01' \ --header 'client_id: ' \ --header 'client_secret: ' \ --data '{ "model": "claude-opus-4-1-20250805", "max_tokens": 2048, "stream": true, "messages": [ { "role": "user", "content": "Explain step-by-step MuleSoft integration process" } ] }' \ --no-buffer ``` ## Specify Amazon Bedrock Model Names in Requests Amazon Bedrock Claude models must be specified in Amazon Resource Name (ARN) format, for example: bedrockanthropic/us.anthropic.claude-sonnet-4-5-20250929-v1:0 Formatted as: bedrockanthropic/.anthropic. To find your region and model name for your Claude model, see [Supported Regions and models for inference profiles](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html).